Date post: | 22-Jan-2018 |
Category: |
Data & Analytics |
Upload: | pivotalopensourcehub |
View: | 242 times |
Download: | 0 times |
PostgreSQL andLinux Containers
Jignesh Shah Founding Team @appOrbit
SF Bay Area PostgreSQL User Group – Jan 5th 2015
Disclaimer
The views expressed in this presentation are my own and do not necessarily reflect the views of appOrbit.
About Jignesh (@jkshah)
• appOrbit• My focus is on managing Application Data running in Containers
• VMware• Lead and manage Postgres and Data Management teams at VMware for various products
embedding PostgreSQL running in virtualized embedded instances
• Sun Microsystems• Team Member of first published SpecJAppServer 2004 benchmark with PostgreSQL• Performance of PostgreSQL on Solaris/Sun Servers
• Working with PostgreSQL community since 2005• http://jkshah.blogspot.com/2005/04/profiling-postgresql-using-dtrace-on_22.html
• Working with Container technologies (Solaris Zones) since 2004• http://jkshah.blogspot.com/2004/08/db2-working-under-solaris-10-zones_30.html
Agenda
• Containers• Definition• Early examples
• Linux Containers• Underlying Technologies • LXC• Systemd-nspawn
• Docker Containers• Installation• Images• Volumes
• PostgreSQL in Docker Container• What it means• Best practices
What are Containers?
• OS Level virtualization where kernel allows for multiple isolated user-space instances
Operating System
Bare Metal Server
OS
Bare Metal Server
Hypervisor
OS
Operating System
Bare Metal Server
C C C C C OS
Bare Metal Server
Hypervisor
OS
C C C C
Advantages of Containers
• Lower footprint• Very Quick Startup and Shutdown time• Density• Nesting
Disadvantages of Containers
• Same Kernel version• Cannot run other OS natively• Security (to be improved)
Where to use container?
• Recreate identical environment (cookie-cutter)• Resource Grouping of specific processes in heavily loaded server• Handling multiple versions of software applications • Ephemeral application instances (Dev/Test)• Many more
Implementations of Containers
• Chroot circa 1982 • FreeBSD Jails circa 2000• Solaris Zones circa 2004• Meiosys – MetaClusters with Checkpoint/Restore 2004-05• Linux OpenVZ circa 2005 (not in mainstream Linux)• AIX WPARs circa 2007• LXC circa 2008 • Systemd-nspawn circa 2010-2013 • Docker circa 2013
What makes containers possible?
• Process Group Isolation• Filesystem Isolation• Network Isolation• CPU Isolation• Memory Isolation
• Example – Solaris Containers• Base implementation provided Process, Filesystem and Network
Isolation• Resource pools consisting of CPU, memory was originally used • Branded Zones – Userland Library Isolation
What makes Linux containers possible ?
• cgroups• Allows limitation and prioritization of resources (CPU, memory, block
I/O, network, etc.) • Namespace isolation
• Mount namespace• PID namespace• Network namespace• UTS (Allows changing hostname, domainname)• IPC Namespace• User namespace
• LXC• Combines kernel’s cgroup and namespaces to provide an isolated
environment
LXC
• CentOS 7 + EPEL Repository set
• Commands available
• Quick Guide to use an LXC based container of busybox
lxc-attach lxc-clone lxc-destroy lxc-ls lxc-stop lxc-usernsexeclxc-autostart lxc-config lxc-execute lxc-monitor lxc-top lxc-waitlxc-cgroup lxc-console lxc-freeze lxc-snapshot lxc-unfreezelxc-checkconfig lxc-create lxc-info lxc-start lxc-unshare
wget https://www.busybox.net/downloads/binaries/busybox-x86_64 -O busyboxchmod a+x busyboxPATH=$(pwd):$PATH lxc-create -t busybox -n mycontainerlxc-start -d -n mycontainerlxc-console –n mycontainer # (Use CTRL-A Q to exit console mode)lxc-stop -n mycontainerlxc-destroy -n mycontainer
yum install epel-releaseyum install bridge-utils libvirt lxc lxc-templates
Systemd-nspawn
• Systemd• Replacement of SysV init scripts
• Systemd-nspawn• Used to run a command or OS in light weight namespace container
• Installed on most newer distros by default
• Commands availablesystemd-analyze systemd-delta systemd-nspawnsystemd-ask-password systemd-detect-virt systemd-runsystemd-cat systemd-cgls systemd-loginctlsystemd-sysv-convert systemd-cgtop systemd-machine-id-setup systemd-coredumpctl systemd-notify systemd-tty-ask-password-agentsystemd-inhibit systemd-stdio-bridge systemd-tmpfilessystemctl machinectl hostnamectl journalctl
yum install systemd
Systemd-nspawn
• Quick guide to a container deployment using systemd-nspawn
• Useful toolsmachinectl status mycontainersystemd-cglssystemd-cgtop
# Create an Imageyum -y --releasever=7 --nogpg --installroot=/mycontainers/centos7 install \systemd passwd yum fedora-release vim-minimal
# Change the root password in the imagesystemd-nspawn -D /mycontainers/centos7passwdexit
# Start the container as if booting into the container imagesystemd-nspawn -D /mycontainers/centos7 –M mycontainer –b
# Get into the containernsenter -m -u -i -n -p –t $PIDmachinectl login mycontainer
Trends of Container Technologies
Disruption of trends by Docker
Docker
• Installation
• Commands available with docker binary
• Quick Guide to use a docker based container
attach build commit cp create diff eventsexec export history images import info inspectkill load login logout logs pause portps pull push rename restart rm rmirun save search start stats stop tagtop unpause version wait
docker run --name mycontainer -e POSTGRES_PASSWORD=mysecretpassword -d postgresdocker exec -ti mycontainer psql -U postgresdocker stop mycontainerdocker rm mycontainerdocker rmi postgres
yum install dockersystemctl start docker
Dockerfile – Custom Recipe
• Container images are created using “Dockerfile”
• Build an image using the recipe file
• Push to a public or private registry (hub account login or private registry needed)
FROM centos:centos7.0.1406RUN yum install -y epel-releaseRUN yum install -y nginxEXPOSE 80 CMD ["bash", "-l", "-c", "/usr/sbin/nginx -g \"daemon off;\""]
docker build –t jkshah/nginx –t .
docker push jkshah/nginx:latest
Docker Images
• Docker Images are layered templates used by containers instances
• Container instance images are layered Copy on Write Images based on Docker images
• You can mutate your local container images (unless you use --read-only flag)• Not good for fast and frequent changing data• Can select a different underlying type using --storage-driver
docker push jkshah/nginx:latest
Do you really want to push to public registry? [y/n]: yThe push refers to a repository [docker.io/jkshah/nginx] (len: 1)d498b5680966: Pushed4b1d16518ce1: Pushed3185bef36db4: Pushed2f7013eef4b8: Pushed539eca37bade: Pushedf1b10cd84249: Pushedlatest: digest: sha256:46208d1f0393946b33c2bdce498168de507b7186a897f332bab0cffc1ea601c7 size: 10608
Docker Volumes
• Persists beyond the life of a Docker container• VOLUME command in Dockerfile or• Using –v using docker run command • Automatically created if not already present during docker run• Not part of docker push/pull operations• Can select a non-local directory using --volume-driver• Third party components required to get multi-host support (NFS, etc )
• On CentOS with SELinux enabled need to set security context
• Different options using –v• -v /hostsrc/data:/opt/data:ro # for read only volumes (default rw)• -v /hostsrc/data:/opt/data:Z # Z – private volume, z – shared volume• -v /etc/nginx.conf:/etc/nginx.conf # for mounting a single file only
• Volumes can be shared from another container using --volumes-from on same host
• Docker 1.9 gives first class status to Docker Volumes
chcon -Rt svirt_sandbox_file_t /hostpath/pgdata
PostgreSQL in Docker container
• Quick Deployment:
• Check Deployment:
• Use inspect command to get more information about the container
• To check PostgreSQL system logs
docker run --name mycontainer -v /hostpath/pgdata:/var/lib/postgresql/data -e POSTGRES_PASSWORD=mysecretpassword -d postgres
docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES8d720b31c1fe postgres "/docker-entrypoint.s" 5 minutes ago Up 5 minutes 5432/tcp mycontainer
docker logs mycontainerThe files belonging to this database system will be owned by user "postgres".This user must also own the server process.…LOG: database system was shut down at 2016-01-04 22:58:40 UTCLOG: MultiXact member wraparound protections are now enabledLOG: database system is ready to accept connectionsLOG: autovacuum launcher started
docker inspect mycontainer
PostgreSQL in docker container
• Part of Postgres Dockerfile has
• Impacts• Port is exposed only to other containers directly linking with this container • Data is persistent only for the life of container (docker rm) but not easily accessible outside the container• -v enables to access the data outside container
• Linking to PostgreSQL Server container
• Ports are exposed externally using –p option• Verify using
docker run --name myapp \--link mycontainer:ds2db \-e POSTGRES_USER=postgres \-e POSTGRES_PASSWORD=mysecretpassword \-p 8080:80 \-d jkshah/dvdstore2
EXPOSE 5432VOLUME /var/lib/postgresql/data
docker port myapp80/tcp -> 0.0.0.0:8080
PostgreSQL in docker container
• Check for statistics using
• Check for top processes in a container
docker stats mycontainer myappCONTAINER CPU % MEM USAGE/LIMIT MEM % NET I/Omyapp 0.01% 51.51 MB/1.924 GB 2.68% 54.42 kB/7.576 MBmycontainer 0.01% 133.3 MB/1.924 GB 6.93% 7.577 MB/54.42 kB
docker top mycontainerUID PID PPID C STIME TTY TIME CMDpolkitd 25313 10698 0 16:58 ? 00:00:00 postgrespolkitd 25385 25313 0 16:58 ? 00:00:00 postgres: checkpointer processpolkitd 25386 25313 0 16:58 ? 00:00:00 postgres: writer processpolkitd 25387 25313 0 16:58 ? 00:00:00 postgres: wal writer processpolkitd 25388 25313 0 16:58 ? 00:00:00 postgres: autovacuum launcher processpolkitd 25389 25313 0 16:58 ? 00:00:00 postgres: stats collector process
docker top myappUID PID PPID C STIME TTY TIME CMDroot 30747 10698 0 18:56 ? 00:00:00 httpd -D FOREGROUND48 30796 30747 0 18:56 ? 00:00:00 httpd -D FOREGROUND48 30797 30747 0 18:56 ? 00:00:00 httpd -D FOREGROUND48 30798 30747 0 18:56 ? 00:00:00 httpd -D FOREGROUND48 30799 30747 0 18:56 ? 00:00:00 httpd -D FOREGROUND48 30800 30747 0 18:56 ? 00:00:00 httpd -D FOREGROUND
PostgreSQL in docker container
• Alternate way to see Process Tree using systemd-cgls tool
systemd-cgls├─1 /usr/lib/systemd/systemd --switched-root --system --deserialize 21├─user.slice│ └─user-0.slice│ └─session-34.scope│ ├─25129 sshd: root@pts/0│ ├─25131 -bash│ └─31397 systemd-cgls└─system.slice
├─docker-9457652d7f6ec24ebd95305e788fe39030b049deb22f240ee2b7383488d0c215.scope│ ├─30747 httpd -D FOREGROUND│ ├─30796 httpd -D FOREGROUND│ ├─30797 httpd -D FOREGROUND│ ├─30798 httpd -D FOREGROUND│ ├─30799 httpd -D FOREGROUND│ └─30800 httpd -D FOREGROUND├─docker-8d720b31c1fe0de77d3cd89942c1a72902b67d466f821ce63ee1271561a36451.scope│ ├─25313 postgres│ ├─25385 postgres: checkpointer process│ ├─25386 postgres: writer process│ ├─25387 postgres: wal writer process│ ├─25388 postgres: autovacuum launcher process│ └─25389 postgres: stats collector process├─docker.service│ ├─10698 /usr/bin/docker daemon --selinux-enabled│ └─30739 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 8080 -contain...
PostgreSQL in Docker container
• Alternate way to see top containers using systemd-cgtopsystemd-cgtopPath Tasks %CPU Memory Input/s Output/s/ 100 0.9 1.5G - -/user.slice 3 0.9 70.7M - -/system.slice - 0.0 268.5M - -/system.slice/docker.service 2 0.0 31.9M - -/system.slice/nimbus.service 5 0.0 20.0K - -/system.slice/tuned.service 1 0.0 36.0K - -/system.slice/dock...49deb22f240ee2b7383488d0c215.scope 6 0.0 49.1M - -/system.slice/auditd.service 1 - 11.9M - -/system.slice/avahi-daemon.service 2 - 12.0K - -/system.slice/crond.service 1 - 92.0K - -/system.slice/dbus.service 1 - - - -/system.slice/dock...7d466f821ce63ee1271561a36451.scope 6 - 127.3M - -/system.slice/gssproxy.service 1 - - - -/system.slice/libvirtd.service 3 - - - -/system.slice/lvm2-lvmetad.service 1 - - - -/system.slice/polkit.service 1 - 16.0K - -/system.slice/postfix.service 3 - 3.5M - -/system.slice/rsyslog.service 1 - 2.1M - -/system.slice/sshd.service 1 - 3.0M - -/system.slice/system-getty.slice/[email protected] 1 - - - -/system.slice/systemd-journald.service 1 - 16.0M - -/system.slice/systemd-logind.service 1 - 20.0K - -/system.slice/systemd-udevd.service 1 - 520.0K - -/system.slice/xe-linux-distribution.service 2 - 1.0M - -/user.slice/user-0.slice/session-34.scope 3 - - - -
PostgreSQL as a database server container
• Maybe you want a database server standalone• Not all database clients will be on docker containers in the same host• Need to limit memory usage• Need different layout of how files are distributed (separage XLOG )
• Use the –p option to make the port available even to non containers clients
• Use –m to limit memory usage by the DB server (by default it can see and use all)• Note this does not set shared buffers automatically with the library image
docker run --name mycontainer \-m 4g \-e POSTGRES_PASSWORD=mysecretpassword \-v /hostpath/pgdata:/var/lib/postgresql/data \-p 5432:5432 -d postgres
PostgreSQL in an enterprise environment
• However for a real production use case we would need• Bigger shared memory configurations • Need different layout of how files are distributed (separage XLOG )• Ability to backup the database• Ability to setup replication • etc
• In short we need a more custom image of PostgreSQL
Best Practices for custom image
• For production install customize the docker image• Allocate proper memory limits - example 8GB
• All pagecache usage shows up as docker container memory usage• Bump up shared buffers and other parameters as required
• Hint: use PostgreSQL 9.3 or later otherwise have to privileged containers• http://jkshah.blogspot.com/2015/09/is-it-privilege-to-run-container-in.html
• Support multiple volumes in your image• Pg_xlog• PITR archives• Tablespaces as required• Full Backup directory
• PostgreSQL Extensions• Setup replication support
• Out of box replication setup• Monitoring Tool
• Your favorite monitoring agent
References
• http://www.freedesktop.org/wiki/Software/systemd/• https://linuxcontainers.org/• http://www.haifux.org/lectures/299/netLec7.pdf• http://haifux.org/lectures/320/netLec8_final.pdf
Revolutionary platform to encapsulate and manage
both legacy and new micro-services based applications
with data and configurations running on containers and
virtual machines in a private, public or hybrid cloud.
Put your DevOps on hyper-drive!
Efficiency and Velocity for Apps & Infrastructure
We areHIRING !!!