trivadis techevent 2016 cgroups im einsatz von florian feicht

BASLE BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA

HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH

cgroups im EinsatzResource Management mal andersrum

Florian FeichtConsultant

Florian Feicht

cgroups im Einsatz2 26.09.2016

Consultant and Trainer at Trivadis GmbH

Focus on:

– Oracle Database High Availability

– Database Administration

– Backup/Recovery

– Upgrades/Migrations

– Linux Administration

Agenda


1. Resource Management

2. What are cgroups?

3. cgroups in init.d and systemd environments

4. Configuration

5. Implementation

6. Use Cases

7. Conclusion


Resource Management

Resource Management


Preserved resources

Operational stability

Reduce of costs

Different possibilities:

– Virtualization

– Application side solutions (Resource Manager)

– …

– cgroups


What are cgroups?

What are cgroups?


Linux Kernel feature (since 2.6.24) to limit resources

Process groups

Based on „everything is a file“ unix paradigma

sched_setaffinity Check cgroup

root Unlimited

cgroupCheck cpuset

configuration

mbind,

set_mempolicyCheck cgroup

root Unlimited

cgroupsCheck cpuset

configuration

What are cgroups?


Consists of resource controller

Base technology for Linux Container systems

Resource

Controller

cgroups


cgroups and init.d

Structure


Process orientated

No cgroup mount per default

libcgroup is available for administration

[root@doagdb-ol6 ~]# lssubsys -am

cpu

memory

devices

net_cls

cpuset,cpuacct,freezer,blkio,perf_event /mnt/cgroup

Creation


libcgroup

– cgconfig

– cgcreate

– etc

Manual configuration

mount {

cpuset = /cgroup/cpuset;

cpu = /cgroup/cpu;

}

[root@doagdb-ol6 cpuset]# mkdir db1

[root@doagdb-ol6 cpuset]# ll db1/

-rw-r--r--. 1 root root 0 Apr 19 13:44 cpuset.cpu_exclusive

-rw-r--r--. 1 root root 0 Apr 19 13:44 cpuset.cpus

-rw-r--r--. 1 root root 0 Apr 19 13:44 cpuset.mems

-rw-r--r--. 1 root root 0 Apr 19 13:44 tasks


cgroups and systemd

Structure


Essential component

Moved from process to application layer

Administration with built-in systemd tools

libcgroup is deprecated

Structure


system.slice

ora_db2.service ora_pmon_DB2

sshd.service

user.slice

user_<id>.slice session_1.scope

sshd: root

-bash

user_id<id>.slice session_3.scope

sshd: oracle

ora_pmon_DB1

systemd service configuration


root@doagdb-cgrp-ol7 ~]# systemctl status oracle_db2.service

● oracle_db1.service - Oracle services started by Trivadis

BasEnv ...

CGroup: /system.slice/ora_db2.service

system.slice ora_db2.service ora_pmon_DB2

systemctl daemon-reload

systemctl enable oracle_db2.service

systemctl start oracle_db2.service

Creation



– start in user.slice

– Check startup

systemd

– No further configuration necessary

– Implementation is not complete

user.slice user_id<id>.slice session_3.scope ora_pmon_DB1



cgroups configuration

libcgroup


Check existing subsystems

– lssubsys -am

Configuration:

– cgcreate

– cgset

– cgsnapshot

Resource Controller cgroup

cgset -r cpuset.cpus=1 db1

zugeordnete CPU



Limitation of resources

Creation of subgroups under depending resource controller

– „Files“ will be created automatically

mkdir /sys/fs/cgroup/cpuset/db1

echo 0 > /sys/fs/cgroup/cpuset/db1/cpuset.cpus

echo 0 > /sys/fs/cgroup/cpuset/db1/cpuset.mems

systemd


Resource Controller:

– CPU

– Memory

– I/O

– Devices

Configuration using systemctl

systemctl set-property unit-name property=value


Monitoring I


/proc/

ps

ps -ax --format args,cgroup

ora_pmon_DB2 1:name=systemd:/system.slice/oracle_db2.service

ora_psp0_DB2 1:name=systemd:/system.slice/oracle_db2.service

cat /proc/2911/cgroup

10:perf_event,blkio,freezer,cpuacct,cpuset:/tvd12cdb

Monitoring II


Only in systemd environments:

– systemd-cgtop

– systemd-cgls

Path Tasks %CPU Memory Input/s Output/s

/system.slice/oracle_db2.service 42 3.2 1.6G 190.8K 31.8K

├─1 /usr/lib/systemd/systemd --switched-root --system --deserialize 21

├─user.slice

│ └─user-54321.slice

│ └─session-1.scope

│ ├─2846 -bash

└─system.slice

├─oracle_db2.service

│ ├─2575 ora_pmon_DB2


How to configure cgroups in Oracle

database environment

Database start in a cgroup


cgexec

cgexec -g cpuset,cpuacct,blkio:tvd12cdb sqlplus / as sysdba

Redirect Shell

systemd Service

echo $$ > /mnt/cgroup/tvd12cdb/tasks

init.ora parameter

– Only with manual configuration

Configuration proposal by Oracle I


MOS: Doc ID 1585184.1 and Doc ID 1928328.1

Provided configuration script: setup_processor_group.sh

– manual cgroup mount will be done

– Resource controller: cpuset

– Persistence is ensured

Configuration proposal by Oracle II


Change of init.ora Parameter

– Restart necessary

Only on CDB level

Entry in alert.log

Instance started in processor group tvd12cdb (NUMA Nodes: 0

CPUs: 0)

ALTER SYSTEM SET processor_group_name='tvd12cdb' SCOPE=SPFILE;

Configuration proposal by Oracle III


Advantages

– Easy naming

– Same functionality in init.d and

systemd environments

– cgroup usage entry in alert.log

– cpuset is enabled by default

– Permission for oracle user is set

Disadvantages

– cgroup necessary for DB startup

– Further resource controller have to be

mounted manually

– Check of the script is very limited

Further configuration possibilities


systemd configuration

– No need to set init parameter

– No cpuset controller possible


– Init parameter necessary

– Flexible resource controller

– Persistence has to be configured manually

Necessary DB parameter


No special settings necessary

Ensure correct cgroup for server processes

One listener per database is recommended

– Check startup configuration

– DEDICATED_THROUGH_BROKER_listener_name

ALTER SYSTEM SET dedicated_server=TRUE;


Implementation

Manual configuration: test database


Requirements

– Usage of just one CPU

– Maximum of 10MB/s IO read und write

echo 1 > cpuset.cpus

echo "252:0 10485760" > blkio.throttle.read_bps_device

echo "252:0 10485760" > blkio.throttle.write_bps_device

Manual configuration: test database


Comparison of AWR reports

– Unlimited:

– Limited:

systemd


BlkIO controller has to be activated

Resource controller cpuset is not available

systemctl set-property oracle_db2.service CPUAccounting=TRUE

systemctl set-property oracle_db2.service CPUShares=500

systemctl set-property oracle_db2.service BlockIOAccounting=TRUE

systemctl set-property oracle_db2.service

BlockIOWriteBandwidth="/dev/sda 10M"


Use Cases

Use Cases


Database cloud environments

– Service classes by defined cgroups

– Increased security level by device mapping

Test and production database on the same hardware

Resource management in Standard Edition

Assigment of CPU and memory segments (NUMA Systeme)

I/O limitation


Conclusion

Conclusion


cgroups are:

– Easy to configure

– flexible

– „Licensefree“

Be careful:

– Concept necessary

– Ensure cgroup availability

– Transparent for the database

– Oracle license

Simple resource management for

Oracle databases on Linux

Any Question…?Florian Feicht

Consultant

[email protected]

26.09.2016 SSD in database environments38

@FlorianFeicht

Session Feedback – now

TechEvent September 201639 09.09.2016

Please use the Trivadis Events Mobile App to give session feedback

Use "My schedule" if you registered for this session

Otherwise use "Agenda" and the search function

If the mobile App does not work (or if you have a Windows Phone) use your Mobile

Browser

– URL: http://trivadis.quickmobileplatform.eu/

– Username: <your_loginname> (like svv)

– Password: sent by mail...

trivadis techevent 2016 cgroups im einsatz von florian feicht

Technology