Sunday, March 12, 2017

Linux Resource Control and allocation cgroups with dd process example. users having different resource cap.


Imagine a client want dd process to be limited. Any OS user that runs dd except root user should be limited to read of 2MiB/s. IO rate is specified in bytes per second. root can run dd commands without limit.
1A.
-bash-4.2# cat /usr/lib/systemd/system/path_cgroup.service
[Unit]
Description=creates resource control paths, this is a oneshot command. testing throttling with dd.
After=multi-user.target
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStartPre=/bin/bash -c '/usr/bin/sleep 120'
ExecStartPre=/bin/bash -c '/usr/bin/mkdir -p /sys/fs/cgroup/blkio/dd_unlimit'
ExecStartPre=/bin/bash -c '/usr/bin/mkdir -p /sys/fs/cgroup/blkio/dd_limit'
ExecStartPre=/usr/bin/sleep 2
ExecStart=/bin/bash -c '/bin/echo "8:0"  2097152 > /sys/fs/cgroup/blkio/dd_limit/blkio.throttle.read_bps_device'
ExecStart=/bin/bash -c '/bin/echo "8:0"  0 > /sys/fs/cgroup/blkio/dd_unlimit/blkio.throttle.read_bps_device'

1B. Use this preferred method instead 1A. /etc/cgconfig.conf  for persistent in resource controls

vi /etc/cgconfig.conf
group dd_limit {
       blkio {
               blkio.throttle.read_bps_device="8:0  2097152";
               }
       }
group dd_unlimit{
      blkio {
              blkio.throttle.read_bps_device="8:0  0";
              }
       }

2. Load the new configuration. Reload units with
systemctl daemon-reload

3. Confirm status of service and path
-bash-4.2# systemctl status path_cgroup.service
● path_cgroup.service - creates resource control paths, this is a oneshot command. testing throttling with dd.
   Loaded: loaded (/usr/lib/systemd/system/path_cgroup.service; enabled; vendor preset: disabled)
   Active: active (exited) since Sat 2017-03-11 11:13:22 EST; 1h 0min ago
  Process: 2086 ExecStart=/bin/bash -c /bin/echo "8:0"  0 > /sys/fs/cgroup/blkio/dd_unlimit/blkio.throttle.read_bps_device (code=exited, status=0/SUCCESS)
  Process: 2082 ExecStart=/bin/bash -c /bin/echo "8:0"  2097152 > /sys/fs/cgroup/blkio/dd_limit/blkio.throttle.read_bps_device (code=exited, status=0/SUCCESS)
  Process: 2079 ExecStartPre=/usr/bin/sleep 2 (code=exited, status=0/SUCCESS)
  Process: 2076 ExecStartPre=/bin/bash -c /usr/bin/mkdir -p /sys/fs/cgroup/blkio/dd_limit (code=exited, status=0/SUCCESS)
  Process: 2073 ExecStartPre=/bin/bash -c /usr/bin/mkdir -p /sys/fs/cgroup/blkio/dd_unlimit (code=exited, status=0/SUCCESS)
  Process: 1556 ExecStartPre=/bin/bash -c /usr/bin/sleep 120 (code=exited, status=0/SUCCESS)
 Main PID: 2086 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/path_cgroup.service

Mar 10 12:00:33 teits.net systemd[1]: Starting creates resource control paths, this is a oneshot command. testing throttling with dd....
Mar 11 11:13:22 teits.net systemd[1]: Started creates resource control paths, this is a oneshot command. testing throttling with dd..

4. Make the service configuration persistent
-bash-4.2#  systemctl enable path_cgroup.service

5. Add a cgrep rules. This search for the pattern process and place them on appropriate group.
-bash-4.2# vi /etc/cgrules.conf
root:dd       blkio         /sys/fs/cgroup/blkio/dd_unlimit
*:dd       blkio         /sys/fs/cgroup/blkio/dd_limit

Note: First rule which matches the criteria will be executed.

5. Stop and start the cgred rules service.
systemctl stop cgred.service
systemctl start cgred.service

6. Testing Phase
Testing with non-root user

[tobi@teits.net ~]$ dd if=readtest if=testfilein count=100k bs=8k
102400+0 records in
102400+0 records out
838860800 bytes (839 MB) copied, 399.279 s, 2.1 MB/s
[tobi@teits.net ~]$

With root user
-bash-4.2# dd if=readtest if=testfilein count=100k bs=8k
102400+0 records in
102400+0 records out

838860800 bytes (839 MB) copied, 25.4341 s, 33.0 MB/s