prstat – a great tool for process monitoring

Solaris administrators with solid Linux experience are usually installing top on their systems because of convenience. Quite a few administrators are aware of prstat but don't see benefits of using its format which somewhat differs from top. And a really small number of Solaris sysadmins really know how to use the prstat command properly. I consider myself a power user, so won't even be claiming to be a prstat guru, but these command examples will hopefully teach you a thing or two.

 

prstat advantages at glance

I really like prstat because it gives me access to the following information:

  • microstate accounting with LOTS of CPU info
  • CPU usage stats across global and non-global zones
  • provide reports (multiple screens of stats taken at specific intervals) forwarded to a file
  • do really useful summaries about top users consuming your OS resources

Default mode of running prstat

When you simply type prstat in your command line, you will see a screen similar to the one shown below, refreshing every few seconds and sorting all the information by the CPU column:

 

   PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP
24382 greys 11M 3236K cpu0 59 0 0:00:00 0.0% prstat/1
24360 greys 18M 4788K sleep 59 0 0:00:00 0.0% sshd/1
24361 greys 10M 2188K sleep 59 0 0:00:00 0.0% bash/1
584 root 13M 3832K sleep 59 0 0:01:59 0.0% nscd/51
154 root 13M 2068K sleep 59 0 0:00:00 0.0% syseventd/18
183 root 1772K 776K sleep 59 0 0:00:13 0.0% utmpd/1
538 root 11M 2572K sleep 59 0 0:00:00 0.0% picld/4
655 root 7316K 4288K sleep 59 0 0:01:53 0.0% hald/4
163 root 3220K 1244K sleep 59 0 0:00:00 0.0% vbiosd/3
964 noaccess 10M 1760K sleep 59 0 0:00:01 0.0% asr-notify/2
565 root 14M 1776K sleep 59 0 0:00:00 0.0% cupsd/1
475 root 11M 808K sleep 59 0 0:00:00 0.0% net-physical/1
105 root 9672K 868K sleep 59 0 0:01:04 0.0% in.mpathd/1
48 netcfg 3784K 1740K sleep 59 0 0:00:43 0.0% netcfgd/4
114 root 2124K 1164K sleep 59 0 0:00:00 0.0% pfexecd/3
89 netadm 4240K 2124K sleep 59 0 0:00:00 0.0% ipmgmtd/5
257 daemon 14M 3400K sleep 59 0 0:00:00 0.0% kcfd/3
657 root 4076K 1164K sleep 59 0 0:00:00 0.0% hald-runner/1
43 root 15M 3424K sleep 59 0 0:00:41 0.0% dlmgmtd/5
824 root 1980K 1220K sleep 59 0 0:00:00 0.0% vtdaemon/3
13 root 27M 25M sleep 59 0 0:03:00 0.0% svc.configd/22
11 root 14M 11M sleep 59 0 0:00:42 0.0% svc.startd/13
606 root 8784K 856K sleep 59 0 0:00:00 0.0% iscsid/2
8 root 0K 0K sleep 99 -20 0:00:00 0.0% vmtasks/1
7 root 0K 0K sleep 60 - 0:00:26 0.0% intrd/1
Total: 62 processes, 391 lwps, load averages: 0.00, 0.00, 0.00

 

This looks good, doesn't it? Quite useful already as you can easily see top consumers of CPU. But what if you wanted to get a different view of the same info, like a summary of what users own these CPU consuming processes? After all, each user is likely to have more than one process, so why not make sure we know the whole story?

 

prstat with additional report about users

 

If you run prstat with the -a option (prstat -a) you will get an output similar to the default one, but the last few lines of it will be used for providing a really useful report of the users consuming top system resources.

 

In this example, I'm one of the top users as you can see (it's a tiny Solaris 11 VM, hence high percentages of resources consumed):

 

   PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP
24383 greys 11M 3236K cpu0 59 0 0:00:00 0.0% prstat/1
24360 greys 18M 4788K sleep 59 0 0:00:00 0.0% sshd/1
24361 greys 10M 2192K sleep 59 0 0:00:00 0.0% bash/1
584 root 13M 3832K sleep 59 0 0:01:59 0.0% nscd/51
475 root 11M 808K sleep 59 0 0:00:00 0.0% net-physical/1
105 root 9672K 868K sleep 59 0 0:01:04 0.0% in.mpathd/1
48 netcfg 3784K 1740K sleep 59 0 0:00:43 0.0% netcfgd/4
114 root 2124K 1164K sleep 59 0 0:00:00 0.0% pfexecd/3
89 netadm 4240K 2124K sleep 59 0 0:00:00 0.0% ipmgmtd/5
257 daemon 14M 3400K sleep 59 0 0:00:00 0.0% kcfd/3
657 root 4076K 1164K sleep 59 0 0:00:00 0.0% hald-runner/1
43 root 15M 3424K sleep 59 0 0:00:41 0.0% dlmgmtd/5
824 root 1980K 1220K sleep 59 0 0:00:00 0.0% vtdaemon/3
13 root 27M 25M sleep 59 0 0:03:00 0.0% svc.configd/22
11 root 14M 11M sleep 59 0 0:00:42 0.0% svc.startd/13
606 root 8784K 856K sleep 59 0 0:00:00 0.0% iscsid/2
8 root 0K 0K sleep 99 -20 0:00:00 0.0% vmtasks/1
7 root 0K 0K sleep 60 - 0:00:26 0.0% intrd/1
NPROC USERNAME SWAP RSS MEMORY TIME CPU
5 greys 52M 13M 1.3% 0:00:00 0.0%
50 root 841M 571M 56% 0:22:22 0.0%
2 noaccess 20M 3776K 0.4% 0:00:43 0.0%
1 netcfg 3784K 1740K 0.2% 0:00:43 0.0%
1 netadm 4240K 2124K 0.2% 0:00:00 0.0%
2 daemon 17M 4520K 0.4% 0:00:04 0.0%
Total: 62 processes, 391 lwps, load averages: 0.00, 0.00, 0.00

prstat and microstate accounting

If you want to get down to really low level details of your system's wellbeing in terms of CPU and memory usage, you'll love the microstate accounting support in prstate.

Activated by the -m option (prstat -m), this option will show you lots of columns with percentage numbers confirming how and what exactly each process (or LWP thread) is doing.

Useful applications of microstate accounting include confirming how long a thread spends in a sleep mode and whether it's lacking CPU resources (shown by CPU latency).

Here's how the output will look:

PID USERNAME USR SYS TRP TFL DFL LCK SLP LAT VCX ICX SCL SIG PROCESS/NLWP
24410 greys 25 74 0.0 0.0 0.0 0.0 0.0 0.9 0 2K .2M 0 prstat/1
24405 greys 0.3 0.1 0.0 0.0 0.0 0.0 99 0.3 126 7 1K 0 sshd/1
24406 greys 0.1 0.1 0.0 0.0 0.0 0.0 99 0.4 12 14 745 3 bash/1
24404 root 0.1 0.1 0.0 0.0 0.0 0.0 100 0.2 15 1 391 0 sshd/1
916 root 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 1 0 5 0 dhcpagent/1
665 root 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 0 0 4 0 hald-addon-a/1
154 root 0.0 0.0 0.0 0.0 0.0 76 24 0.0 0 0 0 0 syseventd/18
183 root 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 0 0 0 0 utmpd/1
538 root 0.0 0.0 0.0 0.0 0.0 25 75 0.0 0 0 0 0 picld/4
655 root 0.0 0.0 0.0 0.0 0.0 25 75 0.0 0 0 0 0 hald/4
163 root 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 0 0 0 0 vbiosd/3
964 noaccess 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 0 0 0 0 asr-notify/2
565 root 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 0 0 0 0 cupsd/1
475 root 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 0 0 0 0 net-physical/1
584 root 0.0 0.0 0.0 0.0 0.0 2.0 98 0.0 0 0 0 0 nscd/51
105 root 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 0 0 5 0 in.mpathd/1
48 netcfg 0.0 0.0 0.0 0.0 0.0 25 75 0.0 0 0 0 0 netcfgd/4
114 root 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 0 0 0 0 pfexecd/3
89 netadm 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 0 0 0 0 ipmgmtd/5
257 daemon 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 0 0 0 0 kcfd/3
657 root 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 0 0 0 0 hald-runner/1
43 root 0.0 0.0 0.0 0.0 0.0 20 80 0.0 0 0 0 0 dlmgmtd/5
824 root 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 0 0 0 0 vtdaemon/3
13 root 0.0 0.0 0.0 0.0 0.0 9.1 91 0.0 0 0 0 0 svc.configd/22
11 root 0.0 0.0 0.0 0.0 0.0 38 62 0.0 0 0 0 0 svc.startd/13
Total: 62 processes, 391 lwps, load averages: 0.00, 0.00, 0.00

That's it for today, hope you enjoyed the post!

Comments

  1. Yusuf Yak says:

    Nice Post, Really enjoyed reading it.

  2. On a multi-cored machine, is the CPU percentage calculated as an aggregate for all cores? Or is it only for the cpu that’s holding that process?

  3. Raghavendra says:

    How do we change the refresh interval to 1 second ?

  4. it’s really useful for me !!!

  5. Rodrigo says:

    How do we change the refresh interval to less than 1 second?

Speak Your Mind

*