How to Cgroups

HowTo: CGroups

Well, as you know, GNU & Linux are awesome. Why? Because we have Control Groups (CGroups)! In this post you’ll learn how to:

  • Protect critical system processes for both memory and CPU.
  • A walkthrough video implementing cgroups on a live running server
  • Set up an example implementation of CGroups or SystemD slices.

Keeping Everything under Control

You can use CGroups to constraint the the amount of resources a process or a group of them is allowed to consume. Yep. That’s right. No more out of memory or overloaded CPUs once you put that resource hungry process under CGroups.

In CloudSigma, we love them. In fact, we love them so much that we use them! Everywhere! We use CGroups for our coffee pots! Nah, seriously, we use them to limit the resources available for the processes running in our servers. This way, we keep them in check.

Even better than that, we’re using them to guarantee resources to the most important subsystems on our physical servers which means stability for customer virtual machines.

For example, our system slice is allowed 4 GiB only! More than enough for any GNU & Linux system to run, eh?… it used to be 640 kiB though… 🙁

How do we do this? How can we go as far as guaranteeing resources to, say, our distributed storage system?

Easy! We just constraint everything! And leave enough resources available for the distributed storage CGroup which, at the same time, have their own constraints! MWAHAHAHA!!

What? You don’t know how to do CGroups? OK, let me teach you the basics…

There are different types of CGroups. We have memory CGroups, CPU CGroups, Block Device CGroups, etc!

Watch me constrain a process on a running system in this video:

Getting Started

First things first, install the tools you need to manipulate CGroups. It’s far easier this way.

I am a Funtoo/Gentoo and Fedora user so the examples are outlined for these OSs but are equally achievable across all Linux distros:

And now, we’re ready to start creating a few CGroups.

So, let’s say you have a user in your server that is using waaaay more memory that you want him/her to. Let’s suppose his name is Blagovart. He’s been using that litecoin mining thing and you want to limit the memory and CPU he’s using on it.

So, we create CGroups for CPU and memory:

This command will create a directory at: /sys/fs/CGroup/cpu/blagovart and /sys/fs/CGroup/memory/blagovart. We could’ve just created them with mkdir; which would’ve given us the same result. It is better however, in general, to use the wrapper tools provided by libCGroup. These tools take care of some of the requirements to make the CGroups work; as well as provide us with meaningful errors… well, sometimes at least.

So, we’ve created Blagovart’s CPU and memory CGroups! Now, let’s create a container for his script:

And put his script there:

Now, he’s script is contained but we haven’t set the limits yet. Let’s set some limits:

Oh, now Blagovart will not be a resource hog anymore and the rest of our users will be happier!
Now, we can go on doing this across the system.

SystemD Slices

If you’re using a SystemD system like Fedora, you should read about slices. One can easily configure resource constraints in slices; as well as group processes in them and limit their resource usage.

For example:

Done! This is now persistent. Then again, there are some changes and some things you will need to reboot for so that they’re applied.

This command will create the following files:

If you want the settings gone, just delete those and reboot.

On systems that partially use SystemD; like Ubuntu 14.04; or no SystemD; like Funtoo, well, you have other options.

One of them is using:

The downside of this is that I am not putting his script on the CGroups just yet. This merely creates the CGroup structure in order to allow me to assign processes to it. In this case, I could just:

And then, there’s Ubuntu. AFAIK, Ubuntu 14.04, the latest LTS at the time of writing, has problems with it and you need an alternative approach; like an init script:

And that’s that! You now have CGroups configured. As you can see, it is easy to manage them, but it’s much nicer to have a SystemD-compliant system to manage them all; thanks to slices.

CGroups is an awesome resource from the Linux Kernel. You can use it practically anywhere you want to control resource usage. For example, we have a CGroups library called: cgroupspy which you can freely use from within Python to do as you please with CGroups. As usual, our license is as free as possible: New BSD.

References

00fdaa19dc42fd452f5fdd9cc5cd3087?s=80&r=g

About Renich

DevOps @ CloudSigma during the day, Creative Commons artist and producer on my free time... Yeah, that means going to play my guitar or piano on the streets sometimes. You can listen to my music in my personal project: Renich or my Rock project: introbella. And I'm sure I have a cover or two @ YouTube. I am, also, a Fedora and Funtoo maintainer and contributor. In fact, you can just google me "Renich" and you'll find my website and other stuff. I have a blog somewhere; where I write technical stuff as well. I am sure you can't imagine the blog's title 😉 On other matters, I've met Richard Stallman, started the local PHP and Ruby groups and contribute continuously to LinuxCabal.