vmstat - virtual memory statistics is a great command to give a rough idea of how your system performs as a whole. For CPU specific output
r: number of current runnable processes. Ideally this value should match with the number of CPUs available.
b: number of processes blocked and waiting for I/O to complete
in: number of interrupt occurring on the system
cs: number of context switches happening on the system
us: % cpu time on user processes
sy: % cpu time spent in system code
wa: cpu time waiting for I/O
id: idle
vmstat -s : memory, swap, pages paged in, paged out, etc..
2059580 total memory
1976900 used memory
1265408 active memory
499676 inactive memory
82680 free memory
133088 buffer memory
599228 swap cache
4095992 total swap
90796 used swap
4005196 free swap
33206 non-nice user cpu ticks
6098 nice user cpu ticks
18493 system cpu ticks
6244845 idle cpu ticks
158150 IO-wait cpu ticks
1346 IRQ cpu ticks
1145 softirq cpu ticks
0 stolen cpu ticks
1398931 pages paged in
8113635 pages paged out
0 pages swapped in
24 pages swapped out
7631737 interrupts
141139062 CPU context switches
1329539948 boot time
138095 forks
top: The system wide performance statistics. It shows the load average of the system over the past 1,5,15 minutes, CPUs states, and processes state (sleeping, running, etc.). While top is running, you can press F while top command is running. Pressing F key then indicate A for PID, B for PPID, etc..) to display the desired statistics.
top - 13:57:06 up 17:17, 2 users, load average: 1.18, 0.49, 0.54
Tasks: 194 total, 2 running, 192 sleeping, 0 stopped, 0 zombie
Cpu(s): 1.3%us, 0.3%sy, 0.0%ni, 98.0%id, 0.3%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 2059580k total, 1967912k used, 91668k free, 136012k buffers
Swap: 4095992k total, 90888k used, 4005104k free, 589284k cached
mpstat: shows you how your processes are behaving based on time. The most advantages is that it shows the time new to statistics, so you can look for a correlation between CPU uage and time of day. In addition, mpstat can be used to determine whether the CPUs are fully utilized and relatively balanced. By observing the number of interrupts each CPUs is handling, you can find an imbalance.
mpstat -P 0 1 10 (-P option tells mpstat which CPUs to monitor. cpu is the number between 0 and total CPUs minus 1.)
mpstat -P ALL 1 10
sar: system activity reporter
c: how many processes are being created per second
w: number of context switches
q: run queues and load average
sar -w -c -q 1 2
Linux 2.6.18-194.el5 (si01.an.com) 02/18/2009
02:20:18 PM proc/s
02:20:19 PM 0.00
02:20:18 PM cswch/s
02:20:19 PM 2242.16
02:20:18 PM runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15
02:20:19 PM 1 554 0.86 0.57 0.42
02:20:19 PM proc/s
02:20:20 PM 0.00
02:20:19 PM cswch/s
02:20:20 PM 2225.74
02:20:19 PM runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15
02:20:20 PM 1 554 0.79 0.56 0.42
runq-sz: run queue
plist-sz: number of processes (sleeping, running, or waiting for I/O)
ldavg-1 ldavg-5 ldavg-15: average load of the last minute, past 5 minutes, past 15 minutes
cswch: context swithes per second
to be continued..
