PostgreSQL MagazineNews and stories from the Postgres Community

#00May 2011

Performance Tuning :To infinity... and beyond !

Interview : Bruce MomjianCase Study : French Social Security SystemWaiting for 9.1 : ExtensionsOpinion : PostgreSQL vs. NoSQLTips & Tricks : SQL Query Analysis with pgFouine

Special Edition

About PG MagPostgreSQL Magazine is edited by andfor the PostgreSQL Community.Editor: Damien ClochardWriters: Greg Smith, Frank Wiles, HubertLubaczewski, Jim Mlodgenski, Matt Tescher,Josh BerkusReviewers: Alvarro Herrera, Thom Brown,Greg Smith, Raymond O'Donnell, IanBailey‐Leung, Osvaldo Tulini, SivakumarKrishnamurthy, Evan Bennett, BelvedereEditorialTools : Scribus 1.4 / Gimp 2.6License:The articles contained in this magazineare released under the Creative CommonsAttribution‐ShareAlike 3.0 Unportedlicense. This means you may adapt, copy,distribute and transmit the articles butonly under the following conditions: Youmust attribute the work to the originalauthor in some way (at least a name,email or URL) and to this magazine byname ('PostgreSQL Magazine') and the URL(pgmag.org). You cannot attribute thearticle(s) in any way that suggests thatthey endorse you or your use of the work.If you alter, transform, or build upon thiswork, you must distribute the resultingwork under the same, similar or acompatible license.Disclaimer:PostgreSQL Magazine is an independentmedia. The views and opinions in themagazine should in no way be assumed tobe endorsed by the PostgreSQL GlobalDevelopment Group.This magazine is provided with absolutelyno warranty whatsoever; neither thecontributors nor PostgreSQL Magazineaccept any responsibility or liability forloss or damage resulting from readerschoosing to apply this content to theirs orothers computers and equipment.Photo Credits :Front: ©expressmonorail¹ (flickr) / p4:©MagnusHagander / p5: Vardion² (wikipedia) /p6: ©lovelihood.com¹ / p8: ©GregStein / p11:©ChristineMomjian / p13: ©dirkjankraan¹(flickr) / p17: ©GageSkidmore¹ (flickr) / p18:©BULL / p19: ©SebastiaanZwarts (flickr) /p23:©GeekAndPoke³¹ : Creative Commons BY‐SA² : Public Domain³ : Creative Commons BY‐ND

EditorialThis is a test. A release candidate. What youare reading right now is the very first issue ofa print magazine entirely dedicated toPostgreSQL, the world's most advanced opensource database.Launching a print magazine in 2011 may looklike a weird idea... But PostgreSQL is now 25years old and although it is one of the oldestopen source projects alive, there are no printmedia for its large user base...One of the goals of this project is to show youwho are "the people behind the elephant", likeBruce Momjian (one of the PostgreSQLpioneers) who talks about the past and futureof the PostgreSQL project on page 8.And the future is bright for PostgreSQL withthe new 9.1 version coming in a few weeks!This major release will include may stunningfeatures (synchronous replication, unloggedtabled, SQL/MED, etc.). In this issue, we willfocus on extensions (p. 20) which are a veryeasy way to package modules and add‐ons foryour databases.For the first issue, we have tried to answerone of the questions most asked to DBAs: "Canyou deliver the data faster?" Performancetuning is indeed a vast topic! You will findgeneral advice (p. 12), OS oriented tips (p. 16)and a query analysis method (p. 24). We hopethese articles will help you in your day‐to‐dayuse of PostgreSQL.With this magazine, we want to create anopen media aimed at the PostgreSQL userbase. If you like PostgreSQL, we hope you'lllike this magazine too!And of course if you have any comments orideas, just send us an email atcontact@pgmag.org! We can't wait to readyour feedback...

Damien Clochard

3

PG Mag #00

PostgreSQL Magazine #00

AgendaConferences & Events

4

How To Contribute ?This magazine is an open project. You can contribute in variousways: writing, editing, translating, proofreading, etc. Toparticipate, send us a message at contact@pgmag.org

AnnouncementsProduct News

6 InterviewBruce Momjian: 15 years ofPostgreSQL Hacking8

Contents

How ToPerformance Tuning

12 How ToTuning Linux for PostgreSQL

16 Use CasePostgreSQL at CNAF

18

Waiting for 9.1Extensions

20 OpinionOne Database To Rule Them All

22 Tips & TricksQuery Analysis with pgFouine

24

PG Mag #00

4

PGCon 2011May 17‐18 @ Ottawa, CanadaPGCon is the annual international conferencefor developers and users of PostgreSQL.PGCon is the place to meet, discuss, buildrelationships, learn valuable insights, andgenerally chat about the work you are doingwith PostgreSQL.http://www.pgcon.org/2011/

Agenda

PGBR 2011Nov. 3‐4 @ São Paulo, BrazilPGBR (formerly known as PGConBrazil) is the biggest event onPostgreSQL in the Americas: in 2009and 2008, the event brought morethan 300 IT professionals and, in2007, more than 200. In 2011, threerooms will be concurrent withtutorials, lectures and high leveltalks.http://pgbr.postgresql.org.br/

Upcoming Events4 Agenda

Pg Conf Colombia 2011Aug. 4‐5 @ Bucaramanga, ColombiaPG Conf Colombia 2011 will be held from 4to August 5, 2011 at the UniversidadIndustrial de Santander UIS, with privatelessons on 2 and 3 August 2011.http://www.pgconf.org/

Postgres OpenSep. 2011 @ Chicago, USAA summit and conference about thebusiness of PostgreSQL in NorthAmerica, this new conference welcomesusers, vendors, and investors inPostgreSQL database technology.http://postgresopen.org

5

PG Mag #00

China PostgreSQL Conference 2011Jul. 16‐17 @ Guangzhou, ChinaThe Chinese PostgreSQL User Group(CPUG) is organizing a two dayconference in July. Speakers includeDavid Fetter and Tatsuo Ishii, amongothers.For information and registration, contactCPUG2011@gmail.com

CHAR(11)Jul. 11‐12 @ Cambridge, UKCHAR(11) is a conference on Clustering,High Availability and Replication that willbe held in England this summer on 11‐12July 2011. Early bird bookings areavailable now by card or Paypal only for£345 plus VAT.http://www.char11.org/

Agenda

PostgreSQL Conference Europe 2011Oct. 18‐21 @ Amsterdam, Netherlands.The European PostgreSQL Conference(formerly known as PGDay.EU) will be heldat the Casa400 Hotel in Amsterdam. Thefirst day of the conference will be atraining day, and the following three dayswill be regular conference tracks. Theconference will accept talks in English,Dutch, German and French.http://2011.pgconf.eu/

PostgreSQL Session #2Jun. 23 @ Paris, FranceThe PostgreSQL Sessions are free‐as‐beerconferences in Paris. Each session is aday consisting of lectures and workshops,organized around a specific theme and aguest. This second edition will be aboutPostGIS.http://www.postgresql‐sessions.org/en/

For more information on local and internationalPostgreSQL user conferences, check out the wiki !http://wiki.postgresql.org/wiki/Events

PG Mag #00

6

Announcements

pgwatch, a new PostgreSQL MonitorIn February, Cybertec Schönig & SchönigGmbH released pgwatch, a simple tool tomonitor PostgreSQL 9.0 or higher. The goalof the project is to have a professionalmonitoring tool for PostgreSQL which can beused out of the box.It offers a rich set of pre‐defined graphs andfunctionality, such as: easy configuration,large number of ready‐to‐use charts,dashboard for fast system checking included,automated collection of statistics,interactive flash‐charts, integrated SQLworksheet, etc.pgwatch is Open Source and can be usedfreely under the terms of the CreativeCommons License.http://www.cybertec.at/en/pgwatch/

Dumbo 0.50, a new terminal clientDumbo is a new "fullscreen" pgAdmin‐styleeditor for the console. It is written in pythonwith the urwid package, and runs on Linux.https://bitbucket.org/mixmastamyk/dumbo/

repmgr 1.1.0 is outrepmgr is a project developed by2ndQuadrant to build large numbers ofreplicated PostgreSQL 9.0 standby servers.By providing a system daemon outside of thedatabase to help coordinate multi‐nodeactions, repmgr makes putting many nodesinto a cluster easier to setup and manage.repmgr currently targets UNIX‐like systems,requiring access to the PGXS interface forcompiling PostgreSQL related code and thersync utility for efficient nodesynchronization.http://projects.2ndquadrant.com/repmgr

7

PG Mag #00

PL/Proxy 2.2 is outPL/Proxy is a database partitioning systemimplemented as procedural language. The newversion includes a new TARGET statement to specifydifferent functions to call on the remote side.http://pgfoundry.org/projects/plproxy/

German doc translation project startedThe project to translate the PostgreSQLdocumentation into German has started. A smallnumber of volunteers has gathered to work ontools for better maintenance and to maketranslations easier. For now, what is nottranslated yet will be provided in English. TheGerman translations will appear on the websitelittle by little.http://doc.postgres.de

PostgreSQL Query Cache releasedIn March, Satoshi NAGAYASU released a softwarepackage that improves query performance (10x‐100x) by caching query results in front of backends.The PostgreSQL Query Cache waits for connectionsfrom clients on a different port, delegates thequeries to a backend (like a proxy) and cachesSELECT query results. It also manages the lifecycleof the query cache.http://code.google.com/p/pqc/

Helping out Japanand JPUGPostgreSQL Users,Our project has a specialrelationship with Japan, and rightnow the whole country is hurting.Japan PostgreSQL User Group(JPUG) represents the largestnational group of PostgreSQL usersin the world, possibly up to 20% ofour users worldwide. Around 1/4of the code contributors toPostgreSQL come from Japan aswell, including both major andminor contributors, several ofwhom we have still not heard from.As such, we are urging morefortunate supporters of thePostgreSQL project to supportJapan in this disaster. The moreaid Japan gets right now, the fasterthey will be able to recover fromthe current catastrophe. TheJapanese on the coast need food,medical care, housing andrebuilding in excess of what theJapanese government can quicklysupply.We recommend the followingorganizations for donations toJapan relief:International Federation of The RedCross/Crescent:http://pgmag.org/0007aDoctors Without Borders:http://pgmag.org/0007bJosh BerkusPostgreSQL Core Team

Google Summer of Code 2011The PostgreSQL project has been accepted into theGoogle Summer of Code 2011. Students submittedproposals between March 28 and April 8.Development work will run from May 23 throughAugust 15.http://wiki.postgresql.org/wiki/GSoC_2011

News

PG Mag #00

8

9

PG Mag #00

PostgreSQL Magazine: Please tell us a littleabout yourself...Bruce Momjian: I am of Armenian ancestryand have lived in the Philadelphia area allmy life. I went from majoring in history incollege to teaching computer science in highschool to developing database applicationsfor law firms. That's kind of an odd mix, andworking on open source full‐time is an evenodder occupation. But it is a very rewardingone, and I thoroughly enjoy it.PGM: When did you hear of PostgreSQL forthe first time and why did you choose tocontribute to the project?BM: As a database applications developer, Iused Ingres and Informix at work and wanteda database for my Unix machine at home.All the commercial database systems weretoo expensive, and I wanted to know howthe database worked inside. Postgres wasthe ideal database because it was likecommercial database systems, but I couldsee the code.PGM: PostgreSQL has changed a lot sincethen! How do you see the success ofPostgres over the years? Is it somethingthat surprised you or did you expect itfrom the beginning?BM: I really had no idea it would succeed.

Postgres was just an interesting project andI was learning a lot by working with somevery smart people (smarter than me). Inever expected it to go anywhere because,at the time, there were few examples ofopen source software succeeding in the wayPostgres has.In the early years it was a struggle to justifythe many hours I poured into Postgres whileworking at a job where I was paid as anhourly consultant — every hour spent onPostgres was an hour I wasn't paid, but itseemed like a good idea at the time, so I didit. I thought the skills I was learning mightbe useful someday.PGM: PostgreSQL is now one of the oldestopen source projects still actively developed.What is the secret of this longevity?BM : Wow, it does surprise me how Postgreshas taken off. I think one secret is how asmall group of people whose heart isinvested in producing top‐quality softwarecan motivate and influence many volunteersscattered around the world.There are many reasons why our communityshould not be as healthy as it is, but it ishealthy, growing, and productive. Humility,utilizing people's strengths, helpingvolunteers feel their contributions areappreciated — these are all secrets to oursuccess.

Bruce Momjian:15 years of PostgreSQL HackingA true pioneer. Since 1996, Bruce Momjian has been an activemember of the PostgreSQL Global Development Group. His experiencegives him a unique standpoint inside the Postgres community, so we'vedecided to ask him about the past, present and future of the mostadvanced open source database.

Interview

PG Mag #00

10PGM: What is your current job?BM: I tend to do stuff no one wants to do. Iused to apply a lot of patches, but now thatis handled. I am constantly looking forthings that need doing, and doing them, ifonly imperfectly. Practically, I createpatches, track down bugs, improvedocumentation, write presentations, andspeak at conferences.PGM: How many hours do you work a day?BM: Sometimes twelve hours, sometimessix. It all depends on what is happening,and, of course, if I am at a conference I ambusy all the time.If I work all day I try to put down my laptoparound 9 or 10pm and do some leisureactivity, but recently I have been travelingso much I am too backed up on email to dothat. Many days I do some activity with myfamily, and because I work from home, thatis easy to do, and it breaks up the day.This job is a good fit for me because whatthe community (and my employerEnterpriseDB) need me to do is pretty muchwhat I enjoy doing. Plus I like doing thingsthat are useful (even if they are boring), sothat works well for me.And what is work, really? Sometimes Ispend all morning on instant message withcommunity members, going over openitems, bugs, and tasks. Is that work? I thinkso, but it is also fun and interesting — so isimproving documentation or cleaning upcode. Is doing this interview work? I guessit is, but it is also interesting.PGM: What is your current participationon the development group of PostgreSQL?BM: I am a Postgres core member andprimary maintainer of the pg_upgrade

utility, which has become popular recently.I also know most of the Postgres peoplepersonally so I often act as a communityliaison.PGM: According to you, what's thebiggest features in the forthcomingPostgreSQL 9.1 version?BM: Here is my short list:

* Unlogged tables to better handle someNoSQL workloads* Synchronous replication for greaterreliability* SQL/MED (Management of External Data)(flat files, other databases)* Per‐column collation support* Security label, including SE‐Linuxintegration* True serializable isolation with predicatelocking (already had snapshot isolation)* Non‐SELECT statement recursive queries* Nearest‐Neighbor (order‐by‐operator)Indexes* Extensions (plugin) commands* PL/Python overhaul (PL/Perl was done in9.0)PGM: PostgreSQL is often mentioned asbeing "the most advanced open sourcedatabase". Do you think that in the nearfuture it could overcome proprietaryRDBMS and become the most advanceddatabase of the whole market?BM: Yes, I think Postgres is clearly on trackfor that. One of the things I recognizedearly on is that it doesn't matter how youcurrently rate against competitors, butrather how quickly you are improving. Ifyou are improving faster than everyoneelse, you will eventually overtake them,even if it takes decades. Postgres is clearlyincreasing its user base and addingadvanced features faster than any of theproprietary databases, so eventuallyPostgres will be the most advanced

Interview

11

PG Mag #00

database, period. We already excel inseveral areas.PGM: What do you think of the NoSQLmovement? Is it a threat for Postgres?I think the best ideas from NoSQL will beincorporated into existing databasesystems over time, and NoSQL databaseswill lose some of their attraction. Wesaw that with object databases and XMLdatabases in the past, and I think we arealready seeing some of that happeningwith NoSQL.PGM: Is there anything you haven'tdone yet but would like to seehappening in the Postgres community?BM: We had a lot of missing stuff early on,and we just kept chipping away at those.At this point, there isn't much left that ismissing. I think we are going to see moreadvanced stuff being done with Postgresin the coming years, rather than justadding missing features. The good news

is that we are even better at designingand implementing advanced, cutting‐edge stuff than we are with addingmissing features. You can see some ofthat in 9.1, and we already have somecool stuff lined up for 9.2, such as nativeJSON support.PGM: What other things are youinterested in outside of open sourceand PostgreSQL?BM: I like to read, travel, and I enjoydoing things with my family.

Interview 11Interview

More Momjian...You can find more about Bruce on hiswebsite : blog, conference schedule,presentations, webcasts, résumé, pressbook, etc.http://momjian.us

PG Mag #00

12

The problem is that every database is notonly different in its design, but also in itsrequirements. Some systems are used tologging mountains of data that is almostnever queried. Others have essentiallystatic data that is queried constantly,sometimes feverishly. Most systems howeverhave some, usually unequal, level of readsand writes to the database. Add this littlecomplexity on top of your totally uniquetable structure, data, and hardwareconfiguration and hopefully you begin tosee why tuning can be difficult.The default configuration PostgreSQL shipswith is a very solid configuration aimed ateveryone's best guess as to how an"average" database on "average" hardwareshould be setup. This article aims to helpPostgreSQL users of all levels betterunderstand PostgreSQL performance tuning.

Understanding theprocessThe first step to learning how to tune yourPostgreSQL database is to understand thelife cycle of a query. Here are the steps of aquery:1. Transmission of query string to database2. Parsing of query string3. Planning of query4. Retrieval of data from hardware5. Transmission of results to client

The first step is the sending of the querystring (the actual SQL command you typein or your application uses) to thedatabase backend. There isn't much youcan tune about this step; however, if youhave very large queries that cannot beprepared in advance it may help to putthem into the database as a storedprocedure and cut the data transfer downto a minimum.Once the SQL query is inside the databaseserver it is parsed into tokens. This stepcan also be minimized by using storedprocedures.The planning of the query is wherePostgreSQL really starts to do some work.This stage checks to see if the query isalready prepared and if your version ofPostgreSQL and client library support thisfeature. It also analyzes your SQL todetermine what the most efficient way ofretrieving your data is. Should we use anindex and if so which one? Maybe a hashjoin on those two tables is appropriate?These are some of the decisions thedatabase makes at this point of theprocess. This step can be eliminated if thequery is previously prepared.Now that PostgreSQL has a plan of what itbelieves to be the best way to retrievethe data, it is time to actually get it.While there are some tuning options thathelp here, this step is mostly affected byyour hardware configuration.

Performance Tuning PostgreSQLPostgreSQL is the most advanced and flexible open source SQL databasetoday. With this power and flexibility comes a problem. How do thePostgreSQL developers tune the default configuration for everyone?Unfortunately the answer is they can't.

How To

13

PG Mag #00

And finally the last step is to transmit theresults to the client. While there aren'tany real tuning options for this step, youshould be aware that all of the data thatyou are returning is pulled from the diskand sent over the wire to your client.Minimizing the number of rows andcolumns to only those that are necessarycan often increase your performance.

General TuningThere are several postmaster options thatcan be set that drastically affectperformance. Below is a list of the mostcommonly used and how they affectperformance:max_connections: This option sets themaximum number of database backendsto have at any one time. Use this featureto ensure that you do not launch so manybackends that you begin swapping to diskand kill the performance of all thechildren. Depending on your application it

may be better to deny the connectionentirely rather than degrade theperformance of all of the other children.shared_buffers: Editing this option is thesimplest way to improve theperformance of your database server.The default is pretty low for mostmodern hardware. General wisdom saysthat this should be set to roughly 25% ofavailable RAM on the system. Like mostof the options I will outline here you willsimply need to try them at differentlevels (both up and down) and see howwell it works on your particular system.Most people find that setting it largerthan a third starts to degradeperformance.effective_cache_size: This value tellsPostgreSQL's optimizer how muchmemory PostgreSQL has available forcaching data and helps in determingwhether or not it uses an index. A largevalue increases the likelihood of using anindex. This should be set to the amount

13How To

PG Mag #00

14of memory allocated to shared_buffersplus the amount of OS cache available.Often this is more than 50% of the totalsystem memory.work_mem: This option is used to controlthe amount of memory usage in sortoperations and hash tables. While you mayneed to increase the amount of memory ifyou do a ton of sorting in your application,care needs to be taken. This isn't a systemwide parameter, but a per operation one.So if a complex query has several sortoperations in it, it will use multiplework_mem units of memory. Not tomention that multiple backends could bedoing this at once. This query can oftenlead your database server to swap if thevalue is too large. This option waspreviously called sort_mem in olderversions of PostgreSQL.max_fsm_pages: This option helps tocontrol the free space map. Whensomething is deleted from a table it isn'tremoved from the disk immediately, it issimply marked as "free" in the free spacemap. The space can then be reused forany new INSERTs that you do on the table.If your setup has a high rate of DELETEsand INSERTs it may be necessary toincrease this value to avoid table bloat.Note that max_fsm_pages is removed as ofPostgreSQL 8.4 and later.fsync: This option determines if all yourWAL pages are fsync'ed to disk before atransaction is committed. Having this on issafer, but can reduce write performance.If fsync is not enabled there is the chanceof unrecoverable data corruption. Turnthis off at your own risk.commit_delay and commit_siblings: Theseoptions are used in concert to helpimprove performance by writing outmultiple transactions that are committingat once. If there are commit_siblings

number of backends active at the instantyour transaction is committing then theserver waits commit_delay microsecondsto try and commit multiple transactions atonce.random_page_cost: This option controlsthe way PostgreSQL views non‐sequentialdisk reads. A higher value makes it morelikely that a sequential scan will be usedover an index scan indicating that yourserver has very fast disks.Note : Many of these options consumeshared memory and it will probably benecessary to increase the amount ofshared memory allowed on your system toget the most out of these options.

Hardware IssuesObviously the type and quality of thehardware you use for your database serverdrastically impacts the performance ofyour database. Here are a few tips to usewhen purchasing hardware for yourdatabase server (in order of importance):RAM: The more RAM you have the moredisk cache you will have. This greatlyimpacts performance, considering memoryI/O is thousands of times faster than diskI/O.Disk types: Obviously fast SAS disks areyour best option; however, high end SATAdrives are also very good. With SATA eachdisk is substantially cheaper and with that

How To Check out the wikiThe official wiki of the PostgreSQLproject (http:/wiki.postgresql.org) has avery nice article titled "Tuning YourPostgreSQL Server". You'll find there mostanswers to your performance questions.http://pgmag.org/0014

15

PG Mag #00

you can afford more spindles than withSAS on the same budget.Disk configuration: The best configurationis RAID 1+0 with as many disks as possibleand with your transaction log (pg_xlog)on a separate disk (or stripe) all by itself.RAID 5 is not a very good option fordatabases unless you have more than 6disks in your volume. With newer versionsof PostgreSQL you can also use thetablespaces option to put differenttables, databases, and indexes ondifferent disks to help optimizeperformance. Such as putting your oftenused tables on a fast SAS disk and theless used ones slower IDE or SATA drives.CPUs: The more CPUs the better,however if your database does not usemany complex functions your money isbest spent on more RAM or a better disksubsystem.In general the more RAM and diskspindles you have in your system thebetter it will perform. This is becausewith the extra RAM you will access yourdisks less. And the extra spindles helpspread the reads and writes over multipledisks to increase throughput and toreduce drive head congestion.Another good idea is to separate yourapplication code and your databaseserver onto different hardware. Not onlydoes this provide more hardwarededicated to the database server, but theoperating system's disk cache will containmore PostgreSQL data and not othervarious application or system data thisway.For example, if you have one web serverand one database server you can use across‐over cable on a separate ethernetinterface to handle just the web serverto database network traffic to ensure you

reduce any possible bottlenecks there. Youcan also obviously create an entirelydifferent physical network for databasetraffic if you have multiple servers thataccess the same database server.

Application IssuesThese issues typically affect both statefuland stateless applications in the samefashion. One good technique is to useserver side prepared queries for anyqueries you execute often. This reducesthe overall query time by caching thequery plan for later use.It should be noted however if you preparea query in advance using placeholdervalues (such as 'column_name = ?') then theplanner will not always be able to choosethe best plan. For example, if your queryhas a placeholder for the boolean column'active' and you have a partial index onfalse values the planner won't use itbecause it cannot be sure the value passedin on execution will be true or false.You can also obviously utilize storedprocedures here to reduce the transmit,parse, and plan portions of the typicalquery life cycle. It is best to profile yourapplication and find commonly usedqueries and data manipulations and putthem into a stored procedure.

About the authorFrank Wiles is an open sourceconsultant specializing in scaling andperformance . He founded RevolutionSystems (revsys.com) to help businessestake full advantage of all the benefits of the OpenSource Software revolution.

About the articleThe original presentation is availablehere : http://pgmag.org/0015

How To

PG Mag #00

16

FilesystemsOlder Linux distributions default to usingthe ext3 filesystem, which is reliable butnot the fastest around. The newer ext4 isfaster but still a bit immature; make sureyou load test heavily before deploying it. Agood middle ground is XFS, which is verystable at this point while also being quitefast. All these filesystems benefit from usingthe "noatime" mounting parameter. That

16 How To

Tuning Linuxfor

PostgreSQL

Linux is one of the most popularserver operating systems forPostgreSQL. One reason for that isthat it generally gives goodperformance. But there are a fewchoices and optimizations to takecare of if you want to get the bestperformance it's capable of.

eliminates updating information aboutwhen files were last accessed, somethingthat's normally not important fordatabases.

Read‐aheadPostgreSQL relies on the operatingsystem to do some amount of read‐ahead, which is essential to good readperformance on sequential scans of largetables. Linux has an excellent read‐ahead implementation, but it's set to alow number of blocks (256) to grab bydefault. 4096 blocks is a good startingpoint to use for that setting instead, andread‐ahead is set on block devices likethis:

/sbin/blockdev ­­setra 4096 /dev/sda

17

PG Mag #00

About the authorGreg Smith is the principal consultantof 2ndQuadrant US and the author of"PostgreSQL 9.0 High Performance":http://www.2ndquadrant.com/booksHis blog at http://blog.2ndquadrant.com regularlycovers Linux and PostgreSQL tuning strategies.

How To

Reliable & Fast WritesGetting the best performance fromPostgreSQL can require selecting theright hardware for that job. Andchoosing the wrong hardware orsetting it up incorrectly can put yourdata at risk. You can find articles onhardware selection and operatingsystem options needed for both goodperformance and reliable databasewrites at http://pgmag.org/0017For example, if you have a system with abattery‐backed write controller, thatgreatly improves write performance. ButXFS and ext4 filesystems will need the"nobarrier" mount option to work wellwith it.

Write cachingOn the write side, PostgreSQL expects theoperating system to buffer normal writes,only forcing them out to disk when thebackground checkpoints complete. Thesize of this write cache was much largerthan is normally preferred on Linuxkernels before 2.6.22, which includes thestill popular RedHat Enterprise 5. You canget the better new defaults on olderkernels like this:

echo 10 > /proc/sys/vm/dirty_ratioecho 5 > /proc/sys/vm/dirty_background_ratio

These settings allow 10% of RAM to beused as a write cache, preemptivelystepping up the rate it's written once itreaches 5%. These numbers may still betoo high on systems with many gigabytesof memory. Starting in Linux kernel2.6.29, these values can be set evensmaller using the new"dirty_background_bytes" and"dirty_bytes" settings. They replace thepercentage based values when you usethem. Note that some bulk operations in

PostgreSQL, particularly VACUUM, mayslow significantly if you reduce Linux'swrite cache by too much.

I/O SchedulingParticularly when running a workload witha heavy mix of read and write operations,Linux's kernel I/O scheduler can impactperformance when the system gets verybusy. It's hard to see a useful differencehere if you just do simple performancetesting, and the options alreadymentioned normally have a biggerimpact. But on a complicated databaseworkload, the scheduler can make asignificant difference too.The default scheduler, Completely FairQueuing (CFQ), is a good starting pointbut not optimal in every case. Theprimary alternative to consider is thedeadline scheduler, which can give betterresponse times with a read/write mixwhere response time latency isimportant.If you have a disk array with someintelligence in it, such as a good RAIDcontroller or external SAN storage, I/Oscheduling may just add overheadwithout any benefit. In those cases, usingthe NOOP scheduler may give bestperformance. It does a minimal amountof work, assuming the disks or theircontroller will make good decisionsinstead.

PG Mag #00

18

The CNAF (see p.19) moved its legacy BullInterel‐RFM2 database to PostgreSQL. Thatdecision was a reaffirmation of CNAF’scommitment to move towards open sourcesolutions. As a result of this, every monthPostgreSQL is now involved in the paymentof €3 billion in benefits to CNAF claimants.The project, which is fully up and running,is an integral part of the program totransform the information systems of thishighly sensitive organization.To improve the quality of its servicesagainst a backdrop of cost constraints,CNAF has put the emphasis on the searchfor better performance and more powerfulsystems. To achieve this, it decided tomodernize two of its strategic and rapidlyevolving applications, CRISTAL and SDP,which together account for 20 million linesof application software code and arecentral to its core business of calculating

people’s entitlements and paying benefits.The choice of a new database that meetsCNAF’s criteria in terms of performanceand openness was an obvious step, withPostgreSQL proving to be the best option.As Marc Pavie, Deputy Director of InformationSystems at CNAF, explained: “The choice ofPostgreSQL reiterates our commitment to opensource. With Bull as our prime contractor, wemigrated without any difficulty. WithPostgreSQL we are now benefiting from newfunctionality which is contributing to theoptimization of our information system. Butmost importantly the solution offers levels ofrobustness and performance largely in linewith our production imperatives andchallenges at a national level relating to thequality of our services.”The migration project started in October2008, following a feasibility study carried

Happy Users

PostgreSQL at CNAF :1 billion SQL queries per dayIn 2010 the CNAF, one of the biggest parts of the French SocialSecurity System, moved its core business applications to PostgreSQL.This use case is quite a success story, as it shows how PostgreSQL isperfectly fitted for big administrations, high workloads and largescale databases.

19

PG Mag #00

Happy Users 19Use Case

out by Bull and a prototype built incollaboration with CNAF’s teams. Anextensive testing phase established a soundbasis for the implementation, which tookplace over the nine months to April 2010.

« With PostgreSQL we arenow benefiting from newfunctionality which iscontributing to theoptimization of ourinformation system »In total, it took just 18 months to migratethe 168 databases involved, representing atotal of 4 Terabytes of data. Since then,almost a billion SQL queries are being runevery day. Regular administrative taskshave been automated and systemsupervision is carried out using an opensoftware solution, Nagios. In addition, thearchitecture that has been implementedfeatures dual high‐availability, with aremote business recovery site and locallybased service continuity facilities.One year after the implementation wascompleted, the PostgreSQL database isclearly proving its robustness and levels of

performance. As a result, the reduction inbatch processing times and transactionresponse times has led to improvements inthe quality of the service provided.Jean‐Pierre Barbéris, CEO of Bull France,commented: “We are very proud to havesupported CNAF in this project, which isunprecedented in Europe. The resultingbenefits bear witness not only to theextraordinary power and robustness ofOpen Source tools, but also to their abilityto support organizations’ most innovativeprojects while at the same time helpingthem to achieve greater room for maneuverfinancially.”CNAF is now considering someimprovements to take advantage ofPostgreSQL features such as migrating toPostgreSQL 9.x in order to reach evenhigher availability for the data servers.

About the authorDamien Clochard is one of the directorsof Dalibo. He is also webmaster of thepostgresql.fr website and involved inseveral PostgreSQL advocacy projects in France.

CNAF ?The Caisse Nationale d’Allocations Familiales (CNAF)is the family branch of the French social securitysystem, paying financial aid to families and providingfurther local, collective social action primarilythrough technical assistance and grants to local sociallife (town halls, nurseries, etc.). Every year, the CNAFprovides some €69 billion of benefits to 11 millionclaimants and overall 30 million persons concerned.Its information systems hold highly sensitive data,such as the French government’s minimum incomeguarantee for low‐income households.

PG Mag #00

20

This means that installing and changing themis now much simpler.For example, adding the ltree contrib moduleto a database is now as simple as:

$ select '1.1'::ltree;ERROR: type "ltree" does not existLINE 1: select '1.1'::ltree;

^$ create extension ltree;CREATE EXTENSION$ select '1.1'::ltree;ltree­­­­­­­1.1(1 row)

Removing it is also trivial:$ drop EXTENSION ltree;DROP EXTENSION

But the great stuff is that you can easily loadthe extension to a different schema, and afterloading you can easily alter it.

Waiting for 9.1

On 8th of February, Tom Lane committedthis patch:Core support for "extensions", which arepackages of SQL objects.

This patch adds the server infrastructure tosupport extensions. There is still onesignificant loose end, namely how to make itplay nice with pg_upgrade, so I am not yetcommitting the changes that would make all thecontrib modules depend on this feature.

In passing, fix a disturbingly large amount ofbreakage in AlterObjectNamespace() andcallers.

Dimitri Fontaine, reviewed by AnssiKääriäinen, Itagaki Takahiro, Tom Lane, andnumerous others

Extensions are a way to group variousdatabase objects together to allow easymanipulation. For starters – all contribmodules became extensions.

Extensionscoming inPostgreSQL 9.1The forthcoming version ofPostgreSQL will bring us a shiploadof exciting new features. Amongthem Extensions are a keyimprovment that will make it mucheasier for software vendors tosupply additional functionality forPostgreSQL.

21

PG Mag #00

Let’s see how it works:$ create extension ltree;CREATE EXTENSION$ create table test (z ltree);CREATE TABLE

OK. So I have a table with a column of ltreedatatype. But now I've decided I don’t wantltree‐related functions “polluting” thepublic schema, and I want to move it to thedesignated schema just for it:$ create schema ltree;CREATE SCHEMA$ alter extension ltree set schema ltree;ALTER EXTENSION

The table got modified too:$ \d testTable "public.test"Column | Type | Modifiers­­­­­­­­+­­­­­­­­­­­­­+­­­­­­­­­­­z | ltree.ltree | About the author

Hubert "Depesz" Lubaczewski is aDatabase Architect at OmniTI. His blog(http://www.depesz.com/) is dedicatedto the new features of the forthcoming version ofPostgreSQL.

While this might not seem like a big deal,in fact it is.Having all objects grouped together meansthat you can upgrade them, anddump/restore is easier. And instead ofdumping all functions/tables/typesdefinitions, the dump contains only:

# pg_dump | grep ltree­­ Name: ltree; Type: SCHEMA; Schema: ­;Owner: depeszCREATE SCHEMA ltree;ALTER SCHEMA ltree OWNER TO depesz;­­ Name: ltree; Type: EXTENSION; Schema: ­;Owner:CREATE EXTENSION ltree WITH SCHEMA ltree;­­ Name: EXTENSION ltree; Type: COMMENT;Schema: ­; Owner:COMMENT ON EXTENSION ltree IS 'data type forhierarchical tree­like structures';z ltree.ltree

The benefit of this will be appreciated byanyone who ever tried to load a dumpmade on some PostgreSQL server toanother, when there were loadedextensions (modules), and paths changed.Or the new version of a contrib moduledidn’t provide the same functions (somenew, some old disappeared).Writing extensions looks trivial, so I assumethat we will see migration of pgFoundryprojects to extensions approach soon.

About the articleThe original article is available at :http://postgr.es/p/‐6

Waiting for 9.1Postgres 9.1 Beta1PostgreSQL beta versions are feature‐frozen snapshots of the forthcoming9.1 major release. They are notmeant for production use. They areintended to preview and testupcoming features and to provide thepossibility for early feedback. If youare a developer or a DBA, testingPostgreSQL betas is one of the bestthings you can do to help get therelease out faster, with more featuresand less bugs.http://www.postgresql.org/developer/beta

PG Mag #00

22

One Database to Rule Them All

Everywhere I turn, I'm seeing articlesabout how relational databases are deadand all data will be stored in these flexibleeventually consistent data stores.Recently, at PG East, there was a MongoDBtrack where I had the opportunity to talkto many people either using NoSQL orconsidering using NoSQL and their reasonsfor using it were extremely varied.Some people had very legitimate reasonsfor using a NoSQL option over a traditionalrelational database like PostgreSQL. Thereare just some inheritant limitations inrelational theory that will never competewith a schema‐less document store when itcomes to performance. Inserting a parentchild relationship such as a new user of aweb application who has 2 addresses and 3phone numbers is much faster in MongoDB,than in PostgreSQL. In PostgreSQL, wewould split the user into a row in the usertable, 2 rows in the address table and 3rows in the phone table and let's justignore those pesky foreign key constraintsslowing things down, but in MongoDB, wejust insert a single BSON documentcontaining all of the information. Forapplications with extremely highthroughput rates that fit a NoSQL model, itis hard to argue with the performanceadvantages of this non relational method.Many people, however, did not fit intothese extreme usage scenarios and used

NoSQL as an easy to use persistent datastore. For many of these users,PostgreSQL may actually be a betterchoice, but unfortunately SQL isbecoming more of a foreign language toweb developers.Most applications do need to relateinformation across tables and that iswhere PostgreSQL outperforms the NoSQLoptions. To display in an application a listof all of the sales last month from usersin New York, a NoSQL database likeMongoDB would need to first find all ofthe users from New York and then usethat list as the input into the next queryto find the sales.A developer using PostgreSQL would justsimply join the sales table with the user

Opinion

The big new topic in databasestoday is the use of NoSQLimplementations such as MongoDB,Cassandra and Couchbase.

23

PG Mag #00

Opinionand address tables in a single query.PostgreSQL would then handle theoptimization of the query such as dealingwith the case of having 100,000 users inNew York, but only 10 total sales.PostgreSQL would know, based on thestatistics of the tables, that it is muchmore efficient to pull all of the sales anduse that as the criteria to pull the finalresult set. In the MongoDB case, thedeveloper needs to know which order ismost efficient, and can frequently bewrong.The manner in which applicationarchitecture is evolving implies that weare long past the days where relationaldatabases were a one‐size‐fits‐allsolution, but relational databases arenowhere near dead.The key to the continuing dominance ofrelational databases and expansion ofPostgreSQL use is how relational databasesinteract and coexist with new data stores.The improvements in the SQL/MEDfunctionality in 9.1 are a great first step.With a MongoDB foreign data wrapper, itwill allow MongoDB documents to appearas PostgreSQL rows allowing developers tojoin data in NoSQL data with relationaldata. Imagine being able to run a singlequery that will join data in your MongoDBinstance with your financial informationsitting in an Oracle database and insertingthat result into your Teradata DataWarehouse. PostgreSQL with SQL/MED isbuilding toward that and is putting itselfinto the position to be the one databaseto rule them all and bind them.

What is NoSQL ?NoSQL is an umbrella term for a looselydefined class of non‐relational datastores that break with a long history ofrelational databases and ACIDguarantees. Data stores that fall underthis term may not require fixed tableschemas, and usually avoid joinoperations. The term was firstpopularised in early 2009.NoSQL databases are intended toperform at internet scale and reject therelational model in favor of other (key‐value, document, graph) models. Theyoften achieve performance by having farfewer features than SQL databases.Rather than being a "jack of all trades,master of none", NoSQL databases tendto focus on a subset of use cases.

Get Ready forSQL/MED !SQL/MED (Management of ExternalData) is an extension to the SQLstandard that defines foreign‐datawrappers and datalink types to allowa DBMS to integrate date storedoutside the database.The implementation of thisspecification has begun in PostgreSQL8.4 and will be available in theforthcoming 9.1 version with the veryfirst foreign‐data wrapper : file_fdw,which can be used to access data filesin the server's filesystem.http://wiki.postgresql.org/wiki/SQL/MED

About the authorJim Mlodgenski is the founder ofCirrus Technologies, Inc. a consultingand product developmentorganization, dedicated to providingscalable data architectures.

PG Mag #00

24

SQL Query Analysis with pgFouine

About the authorMatt Tescher is a Software Engineer atElastra, in San Francisco, USA

Here are the basic steps to set up and usepgFouine:1. If you don’t already have it, you willneed to install php (via yum, manually,etc.):

yum install php

2. Get the latest version of pgFouine:wget http://.../pgfouine­1.2.tar.gztar ­zxvf pgfouine­1.2.tar.gz

3. In postgresql.conf, you will have tochange the default log line prefix. For thisexample, the log destination is set tostderr rather than syslog :log_destination = 'stderr'logging_collector = truelog_line_prefix = '%t [%p]: [%l­1] 'log_min_duration_statement = 0

Setting log_min_duration_statement to 0will log all SQL statements.4. Reload the postgres configuration:

sudo ­u postgres psqlselect pg_reload_conf();

5. You might also force the pg log to rollover (this will allow you to generate areport off of a fresh log file):select pg_rotate_logfile();

6. Now go do some things in your application(i.e. perform a sequence of activities that auser of your application might do), tocapture some of the SQL statements thatget run by the application.7. To generate a report, run pgfouine.phpon the command line, passing it the latestpostgres log to use:

./pgfouine.php ­logtype stderr \­file postgresql.log \> report.html

The resulting report will look somethinglike:

About the articleThe original article is available at :http://pgmag.org/0024

Tips & Tricks

pgFouine is a really nice Postgres loganalyzer that I have been using. Likepqa it can generate html reports froma Postgres log file which will displayslow running queries and most runqueries, among other statistics.