Linux System Performance Utility dstat

 

 

Dstat provides valuable insights into Linux system performance, pretty much replacing a collection of older tools such as vmstat, netstat, iostat, and ifstat with a flexible and powerful command that combines their features. With this one command, you can look at virtual memory, network connections and interfaces, CPU activity, input/output devices and more.

 

- Options -

 

First, let's start with a fairly simple command. With the dstat -c (CPU) option, dstat displays CPU stats. In the example below, we're asking for two-second intervals and six reports.

 

$ dstat -c 2 6

--total-cpu-usage--

usr sys idl wai stl

  1   3  96   0   0

 33  67   0   0   0

 34  66   0   0   0

 35  66   0   0   0

 37  63   0   0   0

 36  64   0   0   0

 

Note that the first line of data in this report, which looks very different than the others, gives you the averages since the system was last booted and is returned immediately regardless of the specified interval. In this example, we see that the system on average has been largely idle (96%), but is now quite busy working between user and system processing tasks.

 

If you don't supply any options with dstat, the command will use a default set (-cdngy) set of options. These include:

 

    c -- cpu

    d -- disk

    n -- network

    g -- paging stats

    y -- system stats

 

The output of this command will look something what you see below.

 

$ dstat 2 10

You did not select any stats, using -cdngy by default.

--total-cpu-usage-- -dsk/total- -net/total- ---paging-- ---system--

usr sys idl wai stl| read  writ| recv  send|  in   out | int   csw

  1   2  96   0   0|5568B 4040B|   0     0 |   0     0 |  58    63

 34  66   0   0   0|   0     0 | 174B  700B|   0     0 | 679   371

 34  66   0   0   0|   0     0 | 174B  407B|   0     0 | 680   377

 36  64   0   0   0|   0     0 |  64B  407B|   0     0 | 678   430

 35  65   0   0   0|   0     0 | 283B  407B|   0     0 | 680   374

 32  68   0   0   0|   0     0 | 238B  407B|   0     0 | 679   376

 33  67   0   0   0|   0     0 | 128B  407B|   0     0 | 680   374

 32  68   0   0   0|   0     0 | 251B  407B|   0     0 | 679   374

 33  67   0   0   0|   0     0 | 238B  407B|   0     0 | 676   376

 34  66   0   0   0|   0     0 | 173B  407B|   0     0 | 680   372

 

You probably noticed the "You did not select any stats" message near the top of the output displayed above.  To overcome this with little effort, simply add the -a option. It will select the default options and omit the warning message.

 

$ dstat -a 2 5

--total-cpu-usage-- -dsk/total- -net/total- ---paging-- ---system--

usr sys idl wai stl| read  writ| recv  send|  in   out | int   csw

  4  10  86   0   0|5921B   12k|   0     0 |   0     0 | 127    99

 15  35  50   0   0|   0     0 | 302B  838B|   0     0 | 369   220

 15  35  50   0   0|   0    14k|  96B  407B|   0     0 | 365   216

 15  35  50   0   0|   0     0 | 246B  407B|   0     0 | 372   229

 18  32  50   0   0|   0     0 | 286B  407B|   0     0 | 359   208

 

In this "no options" approach, you can still the timing for each interval in seconds and the number of intervals you want to see reported. If you don't specify the number of intervals, the command will continue running until you stop it with a ^c.

 

- What it means -

 

In the output shown above, we saw evidence that the system being queried was fairly busy. No idle time was being reported; the CPU was spending all of its time between user and system tasks. Compare this with report which shows the system is idle half the time.

 

 

$ dstat -a 2

--total-cpu-usage-- -dsk/total- -net/total- ---paging-- ---system--

usr sys idl wai stl| read  writ| recv  send|  in   out | int   csw

  2   4  94   0   0|7024B 5297B|   0     0 |   0     0 |  72    70

 14  36  50   0   0|   0     0 | 160B  809B|   0     0 | 381   229

 15  35  50   0   0|   0     0 | 238B  407B|   0     0 | 375   215

 16  34  50   0   0|   0     0 | 128B  346B|   0     0 | 369   204

 

The disks, on the other hand, are not busy at all with zero reads and writes.

 

One key to becoming adept at evaluating system performance is to run commands like these periodically -- even when you don't see the need to question how well a system is running. If  you come to know what normal performance looks like for a server, you will have a much easier time spotting problems.

 

Here's another example, this one with some disk activity:

 

$ dstat -a 2 5

--total-cpu-usage-- -dsk/total- -net/total- ---paging-- ---system--

usr sys idl wai stl| read  writ| recv  send|  in   out | int   csw

  3   6  92   0   0|6631B 5133B|   0     0 |   0     0 |  89    79

 16  34  50   0   0|   0    16k| 270B  809B|   0     0 | 384   256

 16  34  50   0   0|   0     0 | 141B  407B|   0     0 | 358   207

 16  34  50   0   0|   0     0 | 238B  407B|   0     0 | 364   222

 15  35  50   0   0|2048B   18k| 350B  415B|   0     0 | 379   253

 

In all these samples, we're not seeing any paging (loading executable images into a process's virtual memory) activity. There is a fairly constant amount of interrupts and context switching going on, but the numbers are all quite modest.

 

In the command below, we're looking at a memory usage report. Notice the amount of free memory compared to the memory in use. This system is not being challenged.

 

$ dstat -m 2 3

------memory-usage-----

 used  free  buff  cach

 372M 4659M  145M  681M

 373M 4659M  145M  681M

 373M 4659M  145M  681M

 

In the next command, we're looking at an advanced memory usage report. Some additional memory statistics are provided.

 

$ dstat --mem-adv

-------------advanced-memory-usage-------------

total  used  free  buff  cach dirty shmem  recl

5960M  372M 4660M  144M  681M    0  1616k  104M

5960M  372M 4660M  144M  681M    0  1616k  104M

5960M  372M 4660M  144M  681M    0  1616k  104M

5960M  372M 4660M  144M  681M    0  1616k  104M

 

In this next command, we're looking at open files and inodes in use.

 

$ dstat --fs

--filesystem-

files  inodes

 4704  73925

 4704  73925

 4704  73925

 4704  73925

 

In this last example, we're generating the standard report, but adding one thing. We're also writing the report to a .csv file so that it can be used in other tools such as Excel.

 

$ dstat --output /tmp/stats.csv -a 2 5

--total-cpu-usage-- -dsk/total- -net/total- ---paging-- ---system--

usr sys idl wai stl| read  writ| recv  send|  in   out | int   csw

  4  10  86   0   0|5918B   12k|   0     0 |   0     0 | 128    99

 18  32  50   0   0|   0     0 | 504B  923B|   0     0 | 377   237

 19  31  50   0   0|   0     0 | 355B  407B|   0     0 | 368   224

 15  36  50   0   0|   0    14k| 160B  407B|   0     0 | 372   227

 18  32  50   0   0|   0     0 | 270B  407B|   0     0 | 366   221

 

 

Here's what the csv file looks like:

 

$ cat /tmp/stats.csv

$ cat /tmp/stats.csv

"Dstat 0.7.3 CSV output"

"Author:","Dag Wieers <dag@wieers.com>",,,,"URL:","http://dag.wieers.com/home-made/dstat/"

"Host:","vswHostway",,,,"User:","root"

"Cmdline:","dstat --output /tmp/stats.csv -a 2 5",,,,"Date:","23 Jul 2018 10:36:31 EDT"

"total cpu usage",,,,,"dsk/total",,"net/total",,"paging",,"system",

"usr","sys","idl","wai","stl","read","writ","recv","send","in","out","int","csw"

6.081,1.059,92.759,0.101,0,204075.232,17856457.461,0,0,0,0,835.422,2975.869

12.344,1.122,86.534,0,0,0,4096,347,125,0,0,794.500,2372.500

11.602,0.631,87.642,0.126,0,0,0,243,122,0,0,680.500,2274

12.281,0.627,87.093,0,0,0,38912,88,260,0,0,691,2228.500

13.585,1.132,85.283,0,0,0,221184,98,66,0,0,1527.500,2861