Important Solaris Commands

extended device statistics

r/s    w/s   kr/s   kw/s wait actv wsvc_t asvc_t  %w  %b device

8.2    6.2  163.8   90.0  0.5  0.2   35.4   13.1   8  10 c3d0

1.4   12.2   30.0   81.4  0.1  0.2    8.9   13.0   3   7 c6t0d0

Look at the wait: This is the number of IO operations that are waiting to be serviced. In other words: “wait” tells you the waiting queue length. If the wait time is in the single digit range, then your problem may be elsewhere.

$ fsstat -F 5 new  name   name  attr  attr lookup rddir  read read  write write file remov  chng   get   set    ops   ops   ops bytes   ops bytes 0     0     0     0     0      0     0     0     0     0     0 ufsv 0     0     0 1.05K     0  2.22K     4 1.04K  406K     2    32 proc 0     0     0     0     0      0     0     0     0     0     0 nfs 1     1     0 3.89K     5  12.6K     4  9.7K 1.33M 8.19K  568K zfs 0     0     0    41     0      0     0     0     0     0     0 lofs 8     0     6    33     0     34     0     2     0     6 2.41K tmpfs 0     0     0     0     0      0     0     0     0     0     0 mntfs 0     0     0     0     0      0     0     0     0     0     0 nfs3 0     0     0     0     0      0     0     0     0     0     0 nfs4

Are your users creating lots of files? Or are they modifying/removing/changing attributes a lot? What filesystems are causing the most IO load? How much IO goes through NFS and how much is local? All these questions can be easily answered with fsstat and a few flags.

$ kstat cpu_info | grep core_id | uniq | wc -l

32

Divide the total number of cores by the output of “psrinfo -p” to get the cores per physical processor

$ psrinfo -p

4

So 32/4 = 8 cores per physical processor

$ psrinfo -pv

prstat -a -s size (MEMORY CONSUMING PROCESS IN DESCENDING ORDER)

ps -efo pmem,rss,pid,pcpu,args | sort -r | head -20

prstat -t  (It provides complete users resource utilization)

prstat -L (This shows thread per line instead of one process per line)

PID:

the process ID of the process.

USERNAME:

the real user (login) name or real user ID.

SIZE:

the total virtual memory size of the process, including all mapped files and devices, in kilobytes (K), megabytes (M), or gigabytes (G).

RSS:

the resident set size of the process (RSS), in kilobytes (K), megabytes (M), or gigabytes (G).

STATE:

the state of the process (cpuN/sleep/wait/run/zombie/stop).

PRI:

the priority of the process. Larger numbers mean higher priority.

NICE:

nice value used in priority computation. Only processes in certain scheduling classes have a nice value.

TIME:

the cumulative execution time for the process.

CPU:

The percentage of recent CPU time used by the process. If executing in a non-global zone and the pools facility is active, the percentage will be that of the processors in the processor set in use by the pool to which the zone is bound.

PROCESS:

the name of the process (name of executed file).

NLWP:

the number of lwps in the process.

—————————–

load average: 0.65 0.42 0.36

0.65 is the average over the last minute, 0.42 is the average over the last five minutes, and 0.36 is the average over the last 15 minutes.

prstat ouput can be sorted using set of sub-options .sub options are cpu,pri,rss,size,time

(Ex:prstat -s cpu)

cpu – sort by process cpu usage on the system

pri – sort the process by priority

rss – sort by resident set size (i.e physical memory)

size – sort by size of the process (i.e virtual memory)

time – sort by process execution time

This command is very useful to find out where is the actual issue on the system.The blow output shows the percentage of time spent in a given microstate over the last sampling period (default is 5 seconds), beginning with the USR column, up to and including the LAT column. The values in the columns USR through LAT should total 100, accounting for 100% of the threads time for the last sampling period.

# prstat -mL

System Bottleneck determination using prstat -mL

Description        Fields

Other users and application (Like top command usr field)                                     USR

Kernel (Like top command sys fields)                                                       SYS

Monitor the amount of time threads wait for a processor (CPU bottleneck identification)  LAT

Memory paging waits (Memory bottleneck identification)                                     TFL and DTL

Disk and Network waits (I/O bottlenect identification )                                     SLP

prstat -c -n 3  (it will print only top 3 rows)

With the option -c to prstat, new reports are printed below previous ones, instead of overprinting them. This is especially useful when gathering information to a file (i.e. prstat -c 2 > prstat.txt). The option -n <number of output lines> can be used to set the maximum length of a report.

Or

vmstat 5 (Collect virtual memory statistics with a time interval in 5 seconds)

EXAMPLE

$ vmstat 5

kthr      memory            page            disk          faults      cpu

r b w   swap  free  re  mf pi po fr de sr cd s0 s2 —   in   sy   cs us sy id

0 0 0 446144 130076 23 100  0  1  3  0 12  7 -0 13  0  465 1352 1137  6 12 82

0 0 0 405376 90808  33  41  0  0  0  0  0 39  0  3  0  514  500  571  4 11 85

0 0 0 405296 90536   0   0  0  0  0  0  0 29  0  1  0  502  778  551  4 10 86

…

(Remember to ignore the first line of the output as it may contain accumulated data from an unknown sample size.)

FOR CPU

Now watch the rightmost column, which is the system idle time in percent. Is it bigger than 0 most of the time? Then you have enough CPU power. It’s that simple. If idle time is 0 most of the time, buy a bigger CPU, if not, look elsewhere.

FOR MEMORY

Look at three values: swap, free and sr (or: scan rate):

swap: This is the amount of free virtual memory.

free: This is the amount of free physical memory.

scan rate (sr): This is the number of times that the memory page scanner is cleaning up memory pages, freeing the lesser used memory pages to make room for data that needs to be allocated from physical memory. The bigger this value, the more stressed our memory subsystem is, because the OS is more and more busy scanning memory pages for expendable chunks so it can fulfill a high demand in fresh memory. If the scan rate is a single digit value most of the time, you’re ok. If it shows large values over extended periods of time, you’ll likely benefit from some extra RAM in your system.