Introduction to Containers on Linux using LXC

In this episode, I wanted to give you an Introduction to Containers on Linux using LXC. We will look at what Containers are as a concept, why they are useful, and then move onto a live demo of what a Container looks like on a real system using LXC.

Before we jump into the meat of the discussion, lets briefly talk about traditional virtualization for a minute. You start out with the physical hardware, next you have the operating system, then the hypervisor. Finally you have virtual machine images which interface with the emulated hardware provided by the hypervisor, thus allowing you to create many self contained systems. This is well understood, so lets move on to how this compares to containers.

Containers have a similar foundation, starting with the hardware and operating system, but this is where things take a turn. Containers actually make use of kernel features called namespaces, cgroups, and chroots, to carve off a contained area, and the end result looks much like a virtual machine, just without the hypervisor. The LXC project has said that containers are like, chroot on steroids, basically you are running a minimal operating system, application code, along with all supporting libraries, in a container, but using a shared kernel rather than a hypervisor.

In a nutshell, containers are a method for isolation and resource control, much like traditional virtualization, just without the hypervisor overhead. I think it is a fitting analogy to compare containers to traditional virtualization, because even though the technologies are different, the end results looks vary similar from the users and applications perspective.

There is a term for the functionality which containers provide, called os-level virtualization, and there is a great wikipedia page that talks about it. The page has a table of various operating systems, and their take on it, and you will notice that each operating system has a different name and implementation for it. For example, we have BSD Jails, Solaris Zones, and more recently Containers on Linux. As you can see, the concept of os-level virtualization is not new, as it goes back 30 years.

The features which enable containers are actually part of the Linux Kernel and you use userland tools, like LXC, for the implementation. Although there are many kernel features used to create containment, while doing my research for this episode, I found three keys features, those being namescapes, cgroups, and chroots.

Namespaces allow you to isolate an applications view of the operating environment in various ways. I think of namespaces as a method for creating your own little view of the system for mounts, user ids, networking, and process trees. These are isolated from the rest of the system, similar to how you would create a chroot environment, just apply that concept to mounts, users, and processes. If you are looking for more information on namespaces, check out the series of LWN articles linked to in the episode notes below. There is also an awesome presentation entitled: “Resource management: Linux kernel Namespaces and cgroups” by Rami Rosen, also listed in the episode noted.

Next, containers take advantage of cgroups, which I talked about in episode #14. Basically, you construct an isolated container using namespaces, and then use cgroups to shape how resources are consumed in that container. You can isolate and shape things like cpu usage, what cpu cores are used, memory usage, disk I/O, network I/O, amongst other things. One really cool thing about cgroups, is that you also have resource accounting, so you can see what system resources where consumed by a given container. I highly recommend checking out episode #14 linked to in the episode notes below.

So, at this point, we know that at a high level what containers are in theory, they take advantage of key kernel features to construct an isolated environment for applications, and the end results looks very similar traditional virtualization, but without the hypervisor overhead. So, what is LXC then? Well, LXC is a set of userland tools, which interface with the kernels namespaces, cgroups, and other features to create and manage containers.

I think it is worth noting that containers, the concept, and LXC are not one in the same, although LXC is an acronym for Linux Containers. At least this is my my take on it. I think it is best to give an example of what I am talking about. Lets take databases for instance. We all know what databases are in theory, then over here, you have a bunch of products that implement that theory, concept, or idea of what a database is. With that thought in mind, we have containers, the concept, also called OS-level virtualization, over on this side. Then we have products, tools, and libraries, that implement it, over here. So, LXC is a product that implement the containers theory or concept, and their software is called Linux Containers or LXC for short. Hope this makes sense. I wanted to highlight the distinction, because while researching this episode, I found people using the term: Linux Containers, to either refer to LXC, or the concept of containers as an isolation mechanism. I think it is important to differentiate between containers the concept and container tools.

Now that we know a little about what containers are in theory, why are they useful? Well, compared to traditional virtualization methods, containers will see near native performance because we are not emulating hardware, but we are actually taking advantage of kernel namespaces to provide our own little isolated view of the system. Typically, you can spin up a container in fractions of a second where booting a virtual machine takes much longer.

I should also mention that LXC is not the only game in town, there is actually nothing stopping you from creating your own framework that would implement containers the concept, because after all, these features are available via the kernel. In fact Google has been using containers for many years. There was actually a Wired article that highlights how Google runs Search, Gmail, and their Maps services, from within containers on shared hardware using a massive orchestration system. There was a cool presentation by two Google engineers entitled “Let Me Contain That For You”, listed in the episode notes below, which goes into some technical details. There was also a really cool talk by John Wilkes, of Google, where he talks about containers, and the orchestration system they use. Google is actually developing and open-sourcing their tools via the “Let Me Contain That For You” project on github, but development is still ongoing as Google decouples their framework from the infrastructure. Just like we talked about LXC being a tool that implements the container concept, lmctfy would also be a tool that implements it too.

You might be wondering if container are secure. Well, they are designed to be, but if a bug exists in the container implementation, and its exploited, containers are ultimately using a shared kernel, which could be used to break out of the container. However, this would be considered a bug.

Okay, lets get on with the demo.

For this demo we will using CentOS 6.5 running the stock kernel. I have already installed LXC and configured a bridge device so that our host machine and container can talk to each other. If you are interested in how LXC was installed please see the episode transcript below.

yum update
yum install libcap-devel libcgroup busybox wget bridge-utils
yum groupinstall "Development tools"

chkconfig cgconfig on
service cgconfig start

cat > /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
ONBOOT=yes
TYPE=Ethernet
IPV6INIT=no
USERCTL=no
BRIDGE=br0
    
cat > /etc/sysconfig/network-scripts/ifcfg-br0
DEVICE=br0
TYPE=Bridge
BOOTPROTO=static
DNS1=10.0.2.2
GATEWAY=10.0.2.2
IPADDR=10.0.2.15
NETMASK=255.255.255.0
ONBOOT=yes

# make sure everything comes up as expected
reboot 

# install lxc
wget https://linuxcontainers.org/downloads/lxc-1.0.3.tar.gz
tar zxvf lxc-1.0.3.tar.gz
cd lxc-1.0.3
./configure
make && make install

# ran into missing lib error
lxc-create: error while loading shared libraries: liblxc.so.1: 
cannot open shared object file: No such file or directory

# fix
ln -s /usr/local/lib/liblxc.so.1.0.3 /lib64/liblxc.so.1

After you have installed the LXC package you will see many commands starting with lxc. I am just going to type lxc- and hit tab a couple times so that you can see them. These can be used for creating, starting, stopping, and connecting to containers, amongst other things.

#lxc- (tab tab)
lxc-

Lets run lxc-checkconfig to verify our environment is configured correctly. Things are all green so that is a good start. You will notice that we have a section for namespaces, these are the main features used for isolation, like we talked about earlier in the episode. Then down here, you have control groups, these are used for resource control.

lxc-checkconfig

Now that we have verified our environment, lets move forward with creating our first container. LXC is nice enough to provide templates which can be used for creating common distros. Lets take a peak at /usr/local/share/lxc/templates/ and you will notice that there are quite a few templates.

ls -l /usr/local/share/lxc/templates/

Okay, so lets create a container using one of these templates. Lets run lxc-create -n for the container name, lets call it e24-busybox, then -t for the template name, in our case busybox.

lxc-create -n e24-busybox -t busybox

Since a container is completely isolated, it needs to have a os, supporting libraries, and any applications you need. These templates can be used as a great starting point.

On this system, the container root filesystems is stored under /usr/local/var/lib/lxc/. You can see that we have a folder title with our container name. Lets go in there and have a look. There are two items, a config file for this container, and a directory holding our containers root filesystem.

cd /usr/local/var/lib/lxc/e24-busybox/

Lets open up the config file and have a look. These are basically default settings for how the container will function. I am going to copy a couple networking tweak into here and then review what they do. So, we are creating a virtual network device, connecting it to our bridge br0 device, and turning the device on.

vi config

lxc.network.type = veth
lxc.network.link = br0
lxc.network.flags = up

Now that we have configured networking, lets go ahead and start the container, by running lxc-start -d, this puts the container into the background, -n for the container name, in our case e24-busybox.

lxc-start -d -n e24-busybox

Lets look at the running processes to see if our container is there. Yeah, looks like it worked. Now, you might be wondering, what the heck, how come the containers processes show up on the host OS process listing. This is where traditional virtualization and containers differ, like we talked about earlier. The containers processes are actually running on the host OS, although they are isolated via namespaces, and have their own little view of the world. I will demonstrate this in a minute.

Lets run lxc-info -n and then the container name, e24-busybox. As you can see, it says our container is running, and the process numbers match, the container has a network address, then we are provided with some resource consumption statistics.

lxc-info -n e24-busybox

Lets verify that this container is actually connected to the bridge by running brctl show. So we have the bridge device br0, the hosts eth0 is connected, and our containers virtual network adapter is too.

brctl show

I going to open an additional terminal window, so that we can have one for the host OS, and one connected to the container. Okay, so the top window is our host operating system. Then down here, lets connect to the container, by running lxc-console, -n and then the name of the container, e24-busybox.

lxc-console -n e24-busybox

Success, we are connected. You might remember earlier in the episode the template said the username was root and the password was root as well. So, lets login.

Lets take a look at the process list, by running ps. You will notice that the container cannot view the host operating system, but the host OS can see the containers. The host OS can also manipulate the container live without issue, since these are just processes as far at its concerned. Within the container lets run top. Then up in the top window, lets verify we can see the new top process. Let kill it and see what happens. This works because all container processes are actually running on the host OS just isolated via namespaces. This is where the performance benefits of containers come from.

You can also modify files within the container on the fly. Within the container I am going to create a file called testing123.txt. Then on the host OS, lets add 4 dot dot, then verify within the container to see if it worked.

So, at this point, I think you should have a fairly good understanding of what the concept of containers are, what LXC is, and why containers are so fast compared to traditional virtualization. You get near native performance because the processes are actually running on the host OS.

Before I end this episode, lets just look at the lxc-info command again. There is some nice accounting information, and you could shape these resources using cgoups, check out episode #14. Finally, lets stop the container by running lxc-stop -n, then the container name, e24-busybox.

lxc-info -n e24-busybox
lxc-stop -n e24-busybox

Personally, I think the container ecosystem is about to explode with the rise of technologies like Docker, which acts as a wrapper around LXC to streamline the container workflow. Docker is actually developing their own library called libcontainer which might eventually replace the LXC functionality. OpenStack is also throwing support behind containers. Redhat’s Enterprise Linux 7 is said to support containers by default. The technology for containers has existed for man years, but it is just now that we are finally getting automation software, and tooling, that makes managing the container life cycle easier.

#24 - Introduction to Containers on Linux using LXC

Links, Code, and Transcript

You may also like...