LXC

Doro Wufcwu.tw@gmail.com

2

Who am I

• Software Engineer at Canonical• Skills– Legacy, EFI BIOS– Linux Kernel– Android framework & HAL & Apps– Window Apps with wxPython– Linux Desktop Stack

• Links– Blog, LinkedIn, github

3

In this talk

LXC

Applications Internal

4

Virtualization

• Hardware Virtualization– Full: VirtualBox– Para: Xen, KVM

• Software– Operating system-level virtualization

• LXC• OpenVZ• Linux VServer• FreeBSD Jails• chroot

Containers

Hardware

OS

P0 Pn

P0 Pn

5

LXC

• LXC (LinuX Containers)– Run a Linux system within another Linux system

• Container– a group of processes on a Linux box, put together

in an isolated environment• Inside the box, it looks like a VM• Outside the box, it looks like normal processes

6

Benefit

• Speed - fast– Boots, create VM, deploy tasks

• Footprint - small– aufs or overlayfs

• Virtualization– Own network interface– Own filesystem– Isolation and security– Isolation and resource usage

7

Use Cases

• Continuous Integration– Run 100 tests in 100 VMs

• Escape dependency hell• Do whatever you did in VMs– But faster

8

QUICK START

9

• Ubuntu 12.04.2

$ sudo apt-get install lxc$ sudo lxc-create -t ubuntu -n u1$ sudo lxc-start -n u1 -d$ sudo lxc-console -n u1username/name: ubuntu^aq$ sudo lxc-list$ sudo lxc-info -n u1$ sudo lxc-shutdown -n u1$ sudo lxc-destroy -n u1

10

Cheat Sheet• lxc-create - create system container• lxc-destroy - destroy container• lxc-start - start sys container• lxc-stop - stop sys container• lxc-shutdown - safely shut down a container• lxc-execute - Run command in a app

container• lxc-start-ephemeral - start an one-time

container

• lxc-ls - shorter output than lxc-list• lxc-list - List all containers• lxc-info - Print info on the state of a

container• lxc-monitor - Monitor state• lxc-wait - Wait for a state change

• lxc-restore - restore containers from backups made by lxc-backup

• lxc-backup - back up the root filesystems• lxc-freeze - freeze a running container• lxc-unfreeze - unfreeze a frozen container• lxc-cgroup - View and set container control

group settings

• lxc-netstat - Execute netstat in a running container

• lxc-ps - View process info in a running container

11

create x destroy

• lxc-create -n {ctx-name} -t {template name}– $ sudo lxc-create -n u1 -t ubuntu– $ sudo lxc-create -n u2 -t ubuntu -- -r raring– Templates are in /usr/lib/lxc/lxc-*– When first created, a base filesystem will put in

/var/cache/lxc/– Then copy a instance to /var/lib/lxc/{name}/

• config• fstab• rootfs/

• lxc-destroy -n {name}

12

start x stop x shutdown

• lxc-start -n {name} [-d] [-o logfile] [--logpriority=LEVEL]– Start a system-level container (/sbin/init)

• lxc-shutdown -n name [-w] [-r] [-t timeout]– Cleanly shut down a container.

• Send SIGPWR• If not stopped, call lxc-stop which sends SIGKILL

– -w: wait for shutdown to complete.– -r: reboot (ignore -w).– -t timeout: wait at most timeout seconds (implies -w), then

kill the container.• lxc-stop -n {name}

13

execute x start-ephermal

• lxc-execute -n {NAME} -- {COMMAND}– Run a command in application-level container

• lxc-start-ephemeral [-d] [-u user] [-S key] -o {orig} -- [COMMAND]– Runs an ephemeral (one-off) container– $ sudo lxc-start-ephemeral -u ubuntu -o u1 -- uname -a– Options:

• orig - name of the original container• user - the user to connect to the container as• key - the path to the SSH key to use to connect• -d - run in the background

14

cgroup

• lxc-cgroup -n {name} {subsystem} {value}– View and set container control group settings– $ sudo lxc-cgroup -n u1 memroy.limit_in_bytes

256M– $ lxc-cgroup -n u1 cpu.shares 512• maximum is 1024

– $ lxc-cgroup -n u1 cpuset.cpus 0,3– Configure - /var/lib/lxc/{name}/config, such as• lxc.cgroup.memory.limit_in_bytes = 256M

15

clone x backup x restore

• sudo lxc-clone -o {orig} -n {new}• sudo lxc-backup {name} {number}• sudo lxc-restore {name} {number}

16

APPLICATIONS

17

Docker

Docker can help you easily create lightweight, portable, self-sufficient containers from any

application

18

Deploy Remote Desktop$ sudo apt-get install linux-image-extra-`uname -r`$ sudo add-apt-repository ppa:dotcloud/lxc-docker$ sudo apt-get update$ sudo apt-get install lxc-docker$ docker run -i -t ubuntu /bin/bash$ docker build -t vpsee/docker-desktop git://github.com/rogaha/docker-desktop.git$ docker images$ docker run vpsee/docker-desktop$ docker port a581df505cb9 22$ docker ps$ ssh -XC docker@localhost -p 49153 ./docker-desktop$ xpra --ssh="ssh -p 49153" attach ssh:docker@localhost:10

http://www.vpsee.com/2013/07/use-docker-and-lxc-to-build-a-desktop/

19

Dockerfile allow you to automate the steps you would normally manually take to create an image.

20

Juju

Automate your cloud infrastructureConfigure, manage, maintain, deploy and scale efficiently with

best-practice Charms on any public, private or hybrid cloud from a powerful GUI or the command-line.

21

Deploy WordPress

$ sudo apt-add-repository ppa:juju/stable$ sudo apt-get update$ sudo apt-get install lxc mongodb-server juju juju-core$ juju init $ sed -i ‘s/default: amazon/default: local/’ ~/.juju/environments.yaml$ sudo juju bootstrap$ sudo juju deploy wordpress$ sudo juju deploy mysql$ sudo juju add-relation wordpress mysql$ sudo juju expose wordpress$ sudo juju status

https://juju.ubuntu.com/docs/

22

23

$ sudo juju statusenvironment: localmachines: "0": agent-state: started agent-version: 1.14.1.1 dns-name: 10.0.3.1 instance-id: localhost series: precise "2": agent-state: started agent-version: 1.14.1.1 dns-name: 172.16.0.5 instance-id: doro-local-machine-2 series: precise "3": agent-state: started agent-version: 1.14.1.1 dns-name: 172.16.0.5 instance-id: doro-local-machine-3 series: preciseservices: mysql: charm: cs:precise/mysql-27 exposed: false relations: cluster: - mysql db:

- wordpress units: mysql/0: agent-state: started agent-version: 1.14.1.1 machine: "2" public-address: 10.0.3.162 wordpress: charm: cs:precise/wordpress-18 exposed: false relations: db: - mysql loadbalancer: - wordpress units: wordpress/0: agent-state: error agent-state-info: 'hook failed: "install"' agent-version: 1.14.1.1 machine: "3" public-address: 10.0.3.118

24

$ sudo juju destroy-environment$ sudo apt-get purge juju juju-core mongo-server

25

Vagrant

Development environments made easyCreate and configure lightweight, reproducible,

and portable development environments.

26

Create Ubuntu 12.04 64-bits

$ vagrant box add precise64 http://files.vagrantup.com/precise64.box$ mkdir my_box$ cd my_box$ vagrant init precise64$ vagrant up$ vagrant ssh$ vagrant suspend$ vagrant halt$ vagrant destroy

vagrant-lxc, https://github.com/fgrehm/vagrant-lxc

27

INTERNAL

http://www.slideshare.net/dotCloud/scale11x-lxc-talk-16766275http://lwn.net/Articles/531114/

28

Get Code

• $ apt-get source lxc• configure– /etc/lxc/lxc.conf– /etc/lxc/auto

• init script– /etc/default/lxc– /etc/init/lxc.conf– /etc/init/lxc-net.conf– /etc/dnsmasq.d-available/lxc

29

Namespaces

• Partition essential kernel structures to create virtual environments

• Types– pid– net – ipc– mnt– uts (hostname)– user

30

Create Namespaces

• flags to the system call clone()– mnt: CLONE_NEWNS– uts: CLONE_NEWUTS– ipc: CLONE_NEWIPC– pid: CLONE_NEWPID– net: CLONE_NEWNET– user: CLONE_NEWUSER

• command unshare– unshare [-m] [-u] [-i] [-n] <program> [args...]

31

Create Namespace in Code

32

Network

• Each container has its own interface, routing table, iptables rules…

• Communication between containers via pairs of veth interface• /etc/init/lxc-net.conf: iptables, dnsmasq…

[1] $ sudo unshare --net bash[2] $ echo $$[1] $ sudo ip link add name lxcbr0 type veth peer name vethdoro[1] $ ip link set vethdoro netns <PID>[2] $ ip link set vethdoro name eth0[2] $ ifconfig eth0 192.168.1.2[2] $ ifconfig lo 127.0.0.1[1] $ ifconfig addif vethdoro

33

Mount

• Deluxe chroot() pivot_root()• Filesystems mounted in a mnt namespace are

visible only in this namespace• You need to remount special filesystem– procfs– devpts

• Commands– unshare --mount <program>– mount {--make-[r]shared | --make-[r]slave | -- make --

[r]private | --make-unbindable} <mount-object>http://www.ibm.com/developerworks/linux/library/l-mount-namespaces/index.html

34

cgroup

• Everything exposed through filesystem– cgroup on /sys/fs/cgroup type tmpfs (rw,relatime,mode=755)– cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,relatime,cpuset)– cgroup on /sys/fs/cgroup/cpu type cgroup (rw,relatime,cpu)– cgroup on /sys/fs/cgroup/cpuacct type cgroup (rw,relatime,cpuacct)

• Create a cgroup– mkdir -p /sys/fs/cgroup/cpu/lxc/u1– Add PID to cgroup: echo $PID > /sys/fs/cgroup/cpu/lxc/u1/tasks– Limit: echo 512 > /sys/fs/cgroup/cpu/lxc/u1/cpu.shares

35

Limit & Account

• CPU– cpu.shares– cpustat.usage– cpuset.cpus

• Memory– memory.[soft_]limit_in_bytes– memory.stat

• Block I/O– blkio.throttle.{read,write}.{iops,bps}.device

• RTFM: Documentation/cgroup/*

36

回家吃飯