Sar Command Sysstat Reports Good For Your Career Advancement

Sar command is used to collect sysstat reports on various elements System Loads, Memory/Paging, CPU, Disk I/O usage and network information in Linux system. SAR command can also save the reports in the log files. Collecting sar command output and reports are not real time, which are older as specified in configuration file.

Sar command will write to standard output the contents of selected cumulative activities counters in the operating system. Sar command will display average statistics for the time sing the system was started.

Features of SAR :

  • CPU utilization
  • Process Report
  • Load average utilization
  • Memory and swap space Utilization
  • Block Device Statistics
  • I/O Related activities
  • System Buffer activity
  • Context Switch Related Information
  • Network Related Statistics
  • Memory Allocation
  • Semaphore Allocation
  • kernel activities and TTY statistics
  • Report sar data from a specific time

Installing sysstat reports (sar command) package:

By default sar(sysstat) package is not installed in Linux machine. so you need to download the systat package and install. Follow below steps for install and configure systat.

[root@server ~]# yum install -y sysstat*

(or)

[root@server ~]# wget http://pagesperso-orange.fr/sebastien.godard/sysstat-10.0.0.tar.bz2
[root@server ~]# tar -xzvf sysstat-10.0.0.tar.bz2
[root@server ~]# cd sysstat-10.0.0 
[root@server ~]# make 
[root@server ~]# make install

How to check sysstat version and Enable stat collection?

[root@server ~]# sar -V
sysstat version 10.1.5

when you have installed sysstat package in new machines, it will enables stat collections by default.

[root@server ~]# cat /etc/default/sysstat

In case it is not enabled you can do manually.

change ENABLED=”false” to ENABLED=”true” in below path.

[root@server ~]# vi /etct/default/sysstat

How long to keep log files History of sysstat ?

By default 7 days in a month. You can modify days of history in configuration file mentioned below

[root@server ~]# vi /etc/sysconfig/sysstat

HISTORY=7           ========>   by default 7 days.

:wq

Now restart the sysstat service.

[root@server ~]# systemctl restart sysstat.service

(or)

[root@server ~]# /etct/init.d/sysstat restart

when you restart this service, it will start collecting data.

SAR generates  report and collect data daily basis by default it will collects  data every 10 min . You can change it.

[root@server ~]# cat /etc/cron.d/sysstat 

# Run system activity accounting tool every 10 minutes
*/10 * * * * root /usr/lib64/sa/sa1 1 1
# 0 * * * * root /usr/lib64/sa/sa1 600 6 &
# Generate a daily summary of process accounting at 23:53
53 23 * * * root /usr/lib64/sa/sa2 -A

CPU Usage Statistics

Generating CPU usage reports. Using sar command we can generate each CPU utilization and all the CPU Utilization as cumulative. Use sar command along with -P option to generate followed by 0. 0 is the first CPU increase numbers how many CPUs you have. Below is the command which shows your first CPU utilization.

[root@ArkIT-Serv ~]# sar -P 0
Linux 3.10.0-229.el7.x86_64 (ArkIT-Serv) 11/20/2016 _x86_64_ (2 CPU)

05:05:52 PM LINUX RESTART

05:10:01 PM CPU %user %nice %system %iowait %steal %idle
05:20:01 PM 0 0.86 0.00 0.14 0.13 0.00 98.87
05:30:01 PM 0 0.03 0.00 0.05 0.00 0.00 99.92

Average: 0 0.22 0.00 0.11 0.10 0.00 99.57

10:08:53 PM LINUX RESTART

All CPUs usage using single command sar -P ALL

  • %user – Percentage of CPU utilization that occurred while executing at the user level.
  • %nice – CPU Utilization occurred with nice priority
  • %system – Utilization that Occurred while executing Kernel Level (System Level)
  • %iowait–  CPUs were idle during which the system had an outstanding disk I/O request
  • %steal – Show the percentage of time spent in involuntary wait by the virtual CPU
  • %idle –  CPUs were idle and the system did not have an outstanding disk I/O request
[root@ArkIT-Serv ~]# sar -P ALL
Linux 3.10.0-229.el7.x86_64 (ArkIT-Serv) 11/20/2016 _x86_64_ (2 CPU)

05:05:52 PM LINUX RESTART

05:10:01 PM CPU %user %nice %system %iowait %steal %idle
05:20:01 PM all 0.85 0.00 0.19 0.11 0.00 98.86
05:20:01 PM 0 0.86 0.00 0.14 0.13 0.00 98.87
05:20:01 PM 1 0.83 0.00 0.24 0.09 0.00 98.84

Average: CPU %user %nice %system %iowait %steal %idle
Average: all 0.23 0.00 0.15 0.09 0.00 99.54
Average: 0 0.22 0.00 0.11 0.10 0.00 99.57
Average: 1 0.23 0.00 0.18 0.07 0.00 99.51

Disk I/O Transfer Stats

Report of Disk transfer rate statistics, Along with sar command we have to use -b option list disk I/O statistics.

  • tps – Total number of transfers per second that were issued to Physical devices.
  • rtps – Total read requests per second
  • wtps – Total write requests per second
  • bread/s – Read blocks per second from the devices
  • bwrtn/s – Write blocks per second to devices
[root@ArkIT-Serv ~]# sar -b
Linux 2.6.18-371.3.1.el5 (ArkIT-Serv)         11/23/2016

12:00:01 AM       tps      rtps      wtps   bread/s   bwrtn/s
12:10:01 AM     68.57      0.00     68.57      0.00    670.24
12:20:01 AM     76.61      0.00     76.61      0.00    739.28
12:30:01 AM     77.12      0.00     77.12      0.00    744.20
12:40:01 AM     77.14      0.00     77.14      0.00    745.83
12:50:01 AM     68.29      0.00     68.29      0.00    656.81
Average:        70.79      0.01     70.78      0.12    690.69

Disk utilization activity for each block device. Note that device activity report based on sadc options “-S DISK” or “-S XDISK”.  Use -d along with sar command.

As stated earlier, the sadc command collects system utilization data and writes it to a file for later analysis. By default, the data is written to files in the /var/log/sa/ directory. The files are named sa<dd>, where <dd> is the current day’s two-digit date.

  • rd_sec/s – Read sectors from the device. Sector size is 512 bytes
  • wr_sec/s – Sectors written to the device. Sector size is 512 bytes
  • avgrq-sz – Size of the average requests issued to device
  • avgqu-sz– Average queue length were issued to device
  • await – Average time I/O issued to device (in milliseconds)
  • svctm – The average service time (in milliseconds) for I/O requests that were issued to the device.
  • %util –  Percentage of elapsed time during which I/O requests were issued to the device
[root@ArkIT-Serv ~]# sar -d
Linux 3.10.0-229.el7.x86_64 (ArkIT-Serv) 11/23/2016 _x86_64_ (2 CPU)

07:32:36 PM LINUX RESTART

07:40:01 PM DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util
07:50:01 PM dev8-16 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
07:50:01 PM dev8-0 0.09 0.07 0.84 9.77 0.00 1.25 0.96 0.01
07:50:01 PM dev253-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
07:50:01 PM dev253-1 0.10 0.07 0.84 9.27 0.00 1.20 0.92 0.01
Average: dev8-16 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average: dev8-0 1.09 15.33 4.75 18.49 0.01 4.84 4.56 0.50
Average: dev253-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average: dev253-1 1.12 15.31 4.75 17.88 0.01 4.77 4.45 0.50

Paging Information Report

Paging information, Paging is a memory management scheme by which a computer stores and retrieves data from secondary storage for use in main memory.

  • pgpgin/s – Kilobytes paged in from disk per second
  • pgpgout/s – Kilobytes paged out to disk per second
  • fault/s – Number of page faults made per second
  • majflt/s – Major faults made per second
  • pgfree/s – Pages placed on free list per second
  • pgscank/s – pages scanned by the kswapd daemon per second
  • pgscand/s – pages scanned directly per second
  • pgsteal/s – Pages reclaimed from cache per second
[root@ArkIT-Serv ~]# sar -B
Linux 3.10.0-229.el7.x86_64 (ArkIT-Serv) 11/23/2016 _x86_64_ (2 CPU)

07:32:36 PM LINUX RESTART

07:40:01 PM pgpgin/s pgpgout/s fault/s majflt/s pgfree/s pgscank/s pgscand/s pgsteal/s %vmeff
07:50:01 PM 0.03 0.42 71.75 0.00 53.66 0.00 0.00 0.00 0.00
08:00:01 PM 0.00 0.36 68.16 0.00 50.06 0.00 0.00 0.00 0.00
08:10:01 PM 30.63 8.50 110.56 0.07 69.46 0.00 0.00 0.00 0.00
08:20:01 PM 0.00 0.21 68.17 0.00 50.09 0.00 0.00 0.00 0.00
Average: 7.67 2.38 79.66 0.02 55.82 0.00 0.00 0.00 0.00

Memory utilization report 

[root@ArkIT-Serv ~]# sar -r
Linux 3.10.0-229.el7.x86_64 (ArkIT-Serv) 11/23/2016 _x86_64_ (2 CPU)

07:32:36 PM LINUX RESTART

07:40:01 PM kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kbdirty
07:50:01 PM 809248 1061264 56.74 876 407028 2102716 53.00 514364 337388 0
08:00:01 PM 809272 1061240 56.74 876 407040 2102712 53.00 514336 337376 0
08:10:01 PM 766148 1104364 59.04 876 412028 2102712 53.00 518388 338416 0
08:20:01 PM 766092 1104420 59.04 876 412028 2102712 53.00 518456 338420 0
Average: 787690 1082822 57.89 876 409531 2102713 53.00 516386 337900 0

Swap space Utilization statistics

sar command along with -S option. Refer swap partition creation guide

[root@ArkIT-Serv ~]# sar -S
Linux 3.10.0-229.el7.x86_64 (ArkIT-Serv) 11/23/2016 _x86_64_ (2 CPU)

07:32:36 PM LINUX RESTART

07:40:01 PM kbswpfree kbswpused %swpused kbswpcad %swpcad
07:50:01 PM 2097148 0 0.00 0 0.00
08:00:01 PM 2097148 0 0.00 0 0.00
08:10:01 PM 2097148 0 0.00 0 0.00
08:20:01 PM 2097148 0 0.00 0 0.00
Average: 2097148 0 0.00 0 0.00

Huge Page Usage statistics 

[root@ArkIT-Serv ~]# sar -H
Linux 3.10.0-229.el7.x86_64 (ArkIT-Serv) 11/23/2016 _x86_64_ (2 CPU)

07:32:36 PM LINUX RESTART

07:40:01 PM kbhugfree kbhugused %hugused
07:50:01 PM 0 0 0.00
08:00:01 PM 0 0 0.00
08:10:01 PM 0 0 0.00
08:20:01 PM 0 0 0.00
Average: 0 0 0.00

Network Related Statistics

Possible keywords are DEV, EDEV, NFS, NFSD, SOCK, IP, EIP, ICMP, EICMP, TCP, ETCP, UDP, SOCK6, IP6, EIP6, ICMP6, EICMP6 and UDP6. Network related statistics using -n option.

[root@ArkIT-Serv ~]# sar -n <Name>
[root@ArkIT-Serv ~]# sar -n DEV
Linux 3.10.0-229.el7.x86_64 (ArkIT-Serv) 11/23/2016 _x86_64_ (2 CPU)

07:32:36 PM LINUX RESTART

07:40:01 PM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s
07:50:01 PM eno16777736 0.18 0.14 0.01 0.01 0.00 0.00 0.00
07:50:01 PM lo 0.03 0.03 0.00 0.00 0.00 0.00 0.00

[root@ArkIT-Serv ~]# sar -n NFS
Linux 3.10.0-229.el7.x86_64 (ArkIT-Serv) 11/23/2016 _x86_64_ (2 CPU)

07:32:36 PM LINUX RESTART

07:40:01 PM call/s retrans/s read/s write/s access/s getatt/s
07:50:01 PM 0.00 0.00 0.00 0.00 0.00 0.00
08:00:01 PM 0.00 0.00 0.00 0.00 0.00 0.00

[root@ArkIT-Serv ~]# sar -n SOCK
Linux 3.10.0-229.el7.x86_64 (ArkIT-Serv) 11/23/2016 _x86_64_ (2 CPU)

07:32:36 PM LINUX RESTART

07:40:01 PM totsck tcpsck udpsck rawsck ip-frag tcp-tw
07:50:01 PM 1155 10 11 0 0 0
08:00:01 PM 1155 10 11 0 0 0

Queue length and load average report using -q option

[root@ArkIT-Serv ~]# sar -q
Linux 3.10.0-229.el7.x86_64 (ArkIT-Serv) 11/23/2016 _x86_64_ (2 CPU)

07:32:36 PM LINUX RESTART

07:40:01 PM runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15 blocked
07:50:01 PM 0 648 0.00 0.03 0.10 0
08:00:01 PM 0 649 0.00 0.01 0.05 0

Fetch Custom Report from file

Everyday one file will be created in /var/log/sa/ path with sa’date’  (ex. sa23 ). later fetch the details from stored file with starting and ending time. -s starting time hh:mm:ss (Hours:Minutes:Seconds) -e end time hh:mm:ss. 

Queue length with load average report below

[root@ArkIT-Serv sa]# sar -q -s 12:00:00 -e 23:00:00 -f /var/log/sa/sa22
Linux 3.10.0-229.el7.x86_64 (ArkIT-Serv) 11/22/2016 _x86_64_ (2 CPU)

08:57:26 PM LINUX RESTART

09:00:01 PM runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15 blocked
09:10:02 PM 3 566 0.00 0.06 0.10 0
09:20:01 PM 0 643 0.25 0.26 0.21 0
09:30:01 PM 0 656 0.00 0.08 0.15 0
09:40:01 PM 0 657 0.01 0.05 0.11 0
09:50:01 PM 0 660 0.09 0.05 0.07 0
Average: 1 636 0.07 0.10 0.13 0

CPU Utilization fetch from stored sar file

[root@ArkIT-Serv sa]# sar -P ALL -s 12:00:00 -e 23:00:00 -f /var/log/sa/sa21
Linux 3.10.0-229.el7.x86_64 (localhost.localdomain) 10/21/2016 _x86_64_ (2 CPU)

10:21:01 PM LINUX RESTART

10:30:01 PM CPU %user %nice %system %iowait %steal %idle
10:40:01 PM all 5.21 0.00 0.73 0.02 0.00 94.05
10:40:01 PM 0 5.51 0.00 0.71 0.01 0.00 93.77
10:40:01 PM 1 4.91 0.00 0.74 0.02 0.00 94.33

Read sar file using less 

[root@ArkIT-Serv ~]# sar -f /var/log/sar20 |less
[root@ArkIT-Serv sa]# sar -r -n DEV -f /var/log/sa/sa21
Linux 3.10.0-229.el7.x86_64 (localhost.localdomain) 10/21/2016 _x86_64_ (2 CPU)

10:21:01 PM LINUX RESTART

10:30:01 PM kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kbdirty
10:40:01 PM 556708 1313804 70.24 928 400236 2205668 55.59 790828 322332 0
10:50:01 PM 417208 1453304 77.70 928 406932 2243548 56.55 916944 325404 28
Average: 486958 1383554 73.97 928 403584 2224608 56.07 853886 323868 14

10:30:01 PM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s
10:40:01 PM eno16777736 0.29 0.09 0.02 0.01 0.00 0.00 0.00
10:40:01 PM lo 0.02 0.02 0.00 0.00 0.00 0.00 0.00
10:50:01 PM eno16777736 0.29 0.10 0.02 0.01 0.00 0.00 0.00
10:50:01 PM lo 0.04 0.04 0.00 0.00 0.00 0.00 0.00
Average: eno16777736 0.29 0.10 0.02 0.01 0.00 0.00 0.00
Average: lo 0.03 0.03 0.00 0.00 0.00 0.00 0.00

Generate the Utilization report based on the custom seconds and number of times

[root@ArkIT-Serv sa]# sar -u 4 5
Linux 3.10.0-229.el7.x86_64 (ArkIT-Serv) 11/23/2016 _x86_64_ (2 CPU)

10:03:35 PM CPU %user %nice %system %iowait %steal %idle
10:03:39 PM all 0.00 0.00 0.12 0.00 0.00 99.88
10:03:43 PM all 0.12 0.00 0.25 0.00 0.00 99.62
10:03:47 PM all 0.00 0.00 0.00 0.00 0.00 100.00
10:03:51 PM all 0.00 0.00 0.25 0.00 0.00 99.75
10:03:55 PM all 0.00 0.00 0.00 0.00 0.00 100.00
Average: all 0.03 0.00 0.13 0.00 0.00 99.85

Store sar output to mentioned file

Outputfile will be created on the same location, outputfile can’t be read directly we have to use sar -f outputfile

[root@ArkIT-Serv sa]# sar -o outputfile 2 5
[root@ArkIT-Serv sa]# sar -f outputfile
Linux 3.10.0-229.el7.x86_64 (ArkIT-Serv) 11/23/2016 _x86_64_ (2 CPU)

10:06:43 PM CPU %user %nice %system %iowait %steal %idle
10:06:45 PM all 0.00 0.00 0.50 0.00 0.00 99.50
10:06:47 PM all 0.00 0.00 0.00 0.00 0.00 100.00
10:06:49 PM all 0.00 0.00 0.25 0.00 0.00 99.75
10:06:51 PM all 0.00 0.00 0.00 0.00 0.00 100.00
10:06:53 PM all 0.00 0.00 0.25 0.00 0.00 99.75
Average: all 0.00 0.00 0.20 0.00 0.00 99.80

That’s about sar command and sysstat reports.

Related Articles

htop command to see system utilization

Ultimate 25 Linux Commands

Thanks for your wonderful Support and Encouragement

Ravi Kumar Ankam

My Name is ARK. Expert in grasping any new technology, Interested in Sharing the knowledge. Learn more & Earn More

2 Responses

  1. cygnux says:

    Hi… Excellent article. I think you may have a typo in one of the lines… Look for “etct” and fix it. You meant “etc”.

Leave a Reply

Your email address will not be published. Required fields are marked *