1. PostgreSQL 9.3.3 Documentation The PostgreSQL Global Development Group

2. PostgreSQL 9.3.3 Documentation by The PostgreSQL Global Development Group Copyright 1996-2014 The PostgreSQL Global Development Group Legal Notice PostgreSQL is Copyright 1996-2014 by the PostgreSQL Global Development Group. Postgres95 is Copyright 1994-5 by the Regents of the University of California. Permission to use, copy, modify, and distribute this software and its documentation for any purpose, without fee, and without a written agreement is hereby granted, provided that the above copyright notice and this paragraph and the following two paragraphs appear in all copies. IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PRO- VIDED HEREUNDER IS ON AN AS-IS BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. 3. Table of Contents Preface ................................................................................................................................................lx 1. What is PostgreSQL?.............................................................................................................lx 2. A Brief History of PostgreSQL.............................................................................................lxi 2.1. The Berkeley POSTGRES Project ...........................................................................lxi 2.2. Postgres95.................................................................................................................lxi 2.3. PostgreSQL..............................................................................................................lxii 3. Conventions..........................................................................................................................lxii 4. Further Information.............................................................................................................lxiii 5. Bug Reporting Guidelines...................................................................................................lxiii 5.1. Identifying Bugs.....................................................................................................lxiv 5.2. What to Report .......................................................................................................lxiv 5.3. Where to Report Bugs ............................................................................................lxvi I. Tutorial.............................................................................................................................................1 1. Getting Started ........................................................................................................................1 1.1. Installation..................................................................................................................1 1.2. Architectural Fundamentals........................................................................................1 1.3. Creating a Database....................................................................................................2 1.4. Accessing a Database .................................................................................................3 2. The SQL Language .................................................................................................................5 2.1. Introduction ................................................................................................................5 2.2. Concepts .....................................................................................................................5 2.3. Creating a New Table .................................................................................................5 2.4. Populating a Table With Rows ...................................................................................6 2.5. Querying a Table ........................................................................................................7 2.6. Joins Between Tables..................................................................................................9 2.7. Aggregate Functions.................................................................................................11 2.8. Updates.....................................................................................................................12 2.9. Deletions...................................................................................................................13 3. Advanced Features................................................................................................................14 3.1. Introduction ..............................................................................................................14 3.2. Views ........................................................................................................................14 3.3. Foreign Keys.............................................................................................................14 3.4. Transactions..............................................................................................................15 3.5. Window Functions....................................................................................................17 3.6. Inheritance ................................................................................................................20 3.7. Conclusion................................................................................................................21 II. The SQL Language......................................................................................................................22 4. SQL Syntax...........................................................................................................................24 4.1. Lexical Structure.......................................................................................................24 4.1.1. Identiers and Key Words............................................................................24 4.1.2. Constants......................................................................................................26 4.1.2.1. String Constants ..............................................................................26 4.1.2.2. String Constants with C-style Escapes............................................26 4.1.2.3. String Constants with Unicode Escapes..........................................28 4.1.2.4. Dollar-quoted String Constants.......................................................28 4.1.2.5. Bit-string Constants.........................................................................29 4.1.2.6. Numeric Constants ..........................................................................29 4.1.2.7. Constants of Other Types ................................................................30 iii 4. 4.1.3. Operators......................................................................................................31 4.1.4. Special Characters........................................................................................31 4.1.5. Comments ....................................................................................................32 4.1.6. Operator Precedence....................................................................................32 4.2. Value Expressions.....................................................................................................33 4.2.1. Column References......................................................................................34 4.2.2. Positional Parameters...................................................................................34 4.2.3. Subscripts.....................................................................................................35 4.2.4. Field Selection .............................................................................................35 4.2.5. Operator Invocations....................................................................................36 4.2.6. Function Calls..............................................................................................36 4.2.7. Aggregate Expressions.................................................................................36 4.2.8. Window Function Calls................................................................................38 4.2.9. Type Casts....................................................................................................39 4.2.10. Collation Expressions ................................................................................40 4.2.11. Scalar Subqueries.......................................................................................41 4.2.12. Array Constructors.....................................................................................41 4.2.13. Row Constructors.......................................................................................43 4.2.14. Expression Evaluation Rules .....................................................................44 4.3. Calling Functions......................................................................................................45 4.3.1. Using Positional Notation............................................................................45 4.3.2. Using Named Notation ................................................................................46 4.3.3. Using Mixed Notation..................................................................................46 5. Data Denition......................................................................................................................48 5.1. Table Basics..............................................................................................................48 5.2. Default Values ..........................................................................................................49 5.3. Constraints................................................................................................................50 5.3.1. Check Constraints........................................................................................50 5.3.2. Not-Null Constraints....................................................................................52 5.3.3. Unique Constraints.......................................................................................53 5.3.4. Primary Keys................................................................................................53 5.3.5. Foreign Keys................................................................................................54 5.3.6. Exclusion Constraints ..................................................................................57 5.4. System Columns.......................................................................................................57 5.5. Modifying Tables......................................................................................................59 5.5.1. Adding a Column.........................................................................................59 5.5.2. Removing a Column ....................................................................................59 5.5.3. Adding a Constraint.....................................................................................60 5.5.4. Removing a Constraint ................................................................................60 5.5.5. Changing a Columns Default Value............................................................60 5.5.6. Changing a Columns Data Type .................................................................61 5.5.7. Renaming a Column ....................................................................................61 5.5.8. Renaming a Table ........................................................................................61 5.6. Privileges ..................................................................................................................61 5.7. Schemas....................................................................................................................62 5.7.1. Creating a Schema .......................................................................................63 5.7.2. The Public Schema ......................................................................................64 5.7.3. The Schema Search Path..............................................................................64 5.7.4. Schemas and Privileges................................................................................65 5.7.5. The System Catalog Schema .......................................................................65 5.7.6. Usage Patterns..............................................................................................66 5.7.7. Portability.....................................................................................................66 iv 5. 5.8. Inheritance ................................................................................................................67 5.8.1. Caveats.........................................................................................................69 5.9. Partitioning ...............................................................................................................70 5.9.1. Overview......................................................................................................70 5.9.2. Implementing Partitioning ...........................................................................71 5.9.3. Managing Partitions.....................................................................................74 5.9.4. Partitioning and Constraint Exclusion .........................................................74 5.9.5. Alternative Partitioning Methods.................................................................76 5.9.6. Caveats.........................................................................................................76 5.10. Foreign Data ...........................................................................................................77 5.11. Other Database Objects ..........................................................................................78 5.12. Dependency Tracking.............................................................................................78 6. Data Manipulation.................................................................................................................80 6.1. Inserting Data ...........................................................................................................80 6.2. Updating Data...........................................................................................................81 6.3. Deleting Data............................................................................................................82 7. Queries ..................................................................................................................................83 7.1. Overview ..................................................................................................................83 7.2. Table Expressions.....................................................................................................83 7.2.1. The FROM Clause..........................................................................................84 7.2.1.1. Joined Tables ...................................................................................84 7.2.1.2. Table and Column Aliases...............................................................87 7.2.1.3. Subqueries .......................................................................................88 7.2.1.4. Table Functions ...............................................................................89 7.2.1.5. LATERAL Subqueries.......................................................................89 7.2.2. The WHERE Clause........................................................................................91 7.2.3. The GROUP BY and HAVING Clauses...........................................................91 7.2.4. Window Function Processing ......................................................................94 7.3. Select Lists................................................................................................................94 7.3.1. Select-List Items ..........................................................................................94 7.3.2. Column Labels.............................................................................................95 7.3.3. DISTINCT ....................................................................................................95 7.4. Combining Queries...................................................................................................96 7.5. Sorting Rows ............................................................................................................96 7.6. LIMIT and OFFSET...................................................................................................97 7.7. VALUES Lists ............................................................................................................98 7.8. WITH Queries (Common Table Expressions) ...........................................................99 7.8.1. SELECT in WITH...........................................................................................99 7.8.2. Data-Modifying Statements in WITH .........................................................102 8. Data Types...........................................................................................................................105 8.1. Numeric Types........................................................................................................106 8.1.1. Integer Types..............................................................................................107 8.1.2. Arbitrary Precision Numbers.....................................................................107 8.1.3. Floating-Point Types..................................................................................109 8.1.4. Serial Types................................................................................................110 8.2. Monetary Types......................................................................................................111 8.3. Character Types......................................................................................................111 8.4. Binary Data Types ..................................................................................................113 8.4.1. bytea Hex Format.....................................................................................114 8.4.2. bytea Escape Format................................................................................114 8.5. Date/Time Types.....................................................................................................116 8.5.1. Date/Time Input.........................................................................................117 v 6. 8.5.1.1. Dates..............................................................................................118 8.5.1.2. Times.............................................................................................118 8.5.1.3. Time Stamps..................................................................................119 8.5.1.4. Special Values ...............................................................................120 8.5.2. Date/Time Output ......................................................................................121 8.5.3. Time Zones ................................................................................................122 8.5.4. Interval Input..............................................................................................123 8.5.5. Interval Output...........................................................................................125 8.6. Boolean Type..........................................................................................................126 8.7. Enumerated Types ..................................................................................................127 8.7.1. Declaration of Enumerated Types..............................................................127 8.7.2. Ordering.....................................................................................................127 8.7.3. Type Safety ................................................................................................128 8.7.4. Implementation Details..............................................................................128 8.8. Geometric Types.....................................................................................................129 8.8.1. Points .........................................................................................................129 8.8.2. Line Segments............................................................................................129 8.8.3. Boxes..........................................................................................................130 8.8.4. Paths...........................................................................................................130 8.8.5. Polygons.....................................................................................................130 8.8.6. Circles ........................................................................................................131 8.9. Network Address Types..........................................................................................131 8.9.1. inet...........................................................................................................131 8.9.2. cidr...........................................................................................................132 8.9.3. inet vs. cidr............................................................................................132 8.9.4. macaddr ....................................................................................................132 8.10. Bit String Types....................................................................................................133 8.11. Text Search Types.................................................................................................134 8.11.1. tsvector ................................................................................................134 8.11.2. tsquery ..................................................................................................135 8.12. UUID Type ...........................................................................................................136 8.13. XML Type ............................................................................................................137 8.13.1. Creating XML Values ..............................................................................137 8.13.2. Encoding Handling ..................................................................................138 8.13.3. Accessing XML Values............................................................................139 8.14. JSON Type............................................................................................................139 8.15. Arrays ...................................................................................................................139 8.15.1. Declaration of Array Types......................................................................139 8.15.2. Array Value Input.....................................................................................140 8.15.3. Accessing Arrays .....................................................................................142 8.15.4. Modifying Arrays.....................................................................................143 8.15.5. Searching in Arrays..................................................................................146 8.15.6. Array Input and Output Syntax................................................................146 8.16. Composite Types ..................................................................................................148 8.16.1. Declaration of Composite Types..............................................................148 8.16.2. Composite Value Input.............................................................................149 8.16.3. Accessing Composite Types ....................................................................150 8.16.4. Modifying Composite Types....................................................................150 8.16.5. Composite Type Input and Output Syntax...............................................151 8.17. Range Types .........................................................................................................152 8.17.1. Built-in Range Types ...............................................................................152 8.17.2. Examples..................................................................................................152 vi 7. 8.17.3. Inclusive and Exclusive Bounds ..............................................................153 8.17.4. Innite (Unbounded) Ranges...................................................................153 8.17.5. Range Input/Output..................................................................................153 8.17.6. Constructing Ranges................................................................................154 8.17.7. Discrete Range Types ..............................................................................155 8.17.8. Dening New Range Types .....................................................................155 8.17.9. Indexing ...................................................................................................156 8.17.10. Constraints on Ranges............................................................................156 8.18. Object Identier Types .........................................................................................157 8.19. Pseudo-Types........................................................................................................159 9. Functions and Operators .....................................................................................................161 9.1. Logical Operators ...................................................................................................161 9.2. Comparison Operators............................................................................................161 9.3. Mathematical Functions and Operators..................................................................163 9.4. String Functions and Operators..............................................................................166 9.4.1. format.......................................................................................................180 9.5. Binary String Functions and Operators ..................................................................182 9.6. Bit String Functions and Operators........................................................................184 9.7. Pattern Matching ....................................................................................................185 9.7.1. LIKE...........................................................................................................185 9.7.2. SIMILAR TO Regular Expressions............................................................186 9.7.3. POSIX Regular Expressions......................................................................187 9.7.3.1. Regular Expression Details ...........................................................190 9.7.3.2. Bracket Expressions ......................................................................192 9.7.3.3. Regular Expression Escapes..........................................................193 9.7.3.4. Regular Expression Metasyntax....................................................196 9.7.3.5. Regular Expression Matching Rules.............................................197 9.7.3.6. Limits and Compatibility ..............................................................198 9.7.3.7. Basic Regular Expressions............................................................199 9.8. Data Type Formatting Functions ............................................................................199 9.9. Date/Time Functions and Operators.......................................................................206 9.9.1. EXTRACT, date_part...............................................................................210 9.9.2. date_trunc..............................................................................................213 9.9.3. AT TIME ZONE..........................................................................................214 9.9.4. Current Date/Time .....................................................................................215 9.9.5. Delaying Execution....................................................................................217 9.10. Enum Support Functions......................................................................................217 9.11. Geometric Functions and Operators.....................................................................218 9.12. Network Address Functions and Operators..........................................................222 9.13. Text Search Functions and Operators...................................................................224 9.14. XML Functions ....................................................................................................228 9.14.1. Producing XML Content..........................................................................228 9.14.1.1. xmlcomment ...............................................................................228 9.14.1.2. xmlconcat .................................................................................229 9.14.1.3. xmlelement ...............................................................................229 9.14.1.4. xmlforest .................................................................................231 9.14.1.5. xmlpi ..........................................................................................231 9.14.1.6. xmlroot......................................................................................232 9.14.1.7. xmlagg........................................................................................232 9.14.2. XML Predicates .......................................................................................233 9.14.2.1. IS DOCUMENT .............................................................................233 9.14.2.2. XMLEXISTS .................................................................................233 vii 8. 9.14.2.3. xml_is_well_formed..............................................................233 9.14.3. Processing XML ......................................................................................234 9.14.4. Mapping Tables to XML..........................................................................235 9.15. JSON Functions and Operators ............................................................................239 9.16. Sequence Manipulation Functions .......................................................................242 9.17. Conditional Expressions.......................................................................................245 9.17.1. CASE.........................................................................................................245 9.17.2. COALESCE ................................................................................................246 9.17.3. NULLIF.....................................................................................................247 9.17.4. GREATEST and LEAST..............................................................................247 9.18. Array Functions and Operators ............................................................................247 9.19. Range Functions and Operators............................................................................250 9.20. Aggregate Functions.............................................................................................252 9.21. Window Functions................................................................................................255 9.22. Subquery Expressions ..........................................................................................257 9.22.1. EXISTS.....................................................................................................258 9.22.2. IN .............................................................................................................258 9.22.3. NOT IN.....................................................................................................259 9.22.4. ANY/SOME .................................................................................................259 9.22.5. ALL...........................................................................................................260 9.22.6. Row-wise Comparison.............................................................................260 9.23. Row and Array Comparisons ...............................................................................260 9.23.1. IN .............................................................................................................261 9.23.2. NOT IN.....................................................................................................261 9.23.3. ANY/SOME (array) .....................................................................................261 9.23.4. ALL (array) ...............................................................................................262 9.23.5. Row-wise Comparison.............................................................................262 9.24. Set Returning Functions .......................................................................................263 9.25. System Information Functions .............................................................................266 9.26. System Administration Functions ........................................................................276 9.26.1. Conguration Settings Functions.............................................................276 9.26.2. Server Signaling Functions......................................................................277 9.26.3. Backup Control Functions .......................................................................278 9.26.4. Recovery Control Functions ....................................................................280 9.26.5. Snapshot Synchronization Functions.......................................................282 9.26.6. Database Object Management Functions.................................................282 9.26.7. Generic File Access Functions.................................................................284 9.26.8. Advisory Lock Functions.........................................................................285 9.27. Trigger Functions .................................................................................................287 9.28. Event Trigger Functions .......................................................................................288 10. Type Conversion................................................................................................................290 10.1. Overview ..............................................................................................................290 10.2. Operators ..............................................................................................................291 10.3. Functions ..............................................................................................................294 10.4. Value Storage........................................................................................................297 10.5. UNION, CASE, and Related Constructs..................................................................298 11. Indexes ..............................................................................................................................300 11.1. Introduction ..........................................................................................................300 11.2. Index Types...........................................................................................................301 11.3. Multicolumn Indexes............................................................................................303 11.4. Indexes and ORDER BY.........................................................................................304 11.5. Combining Multiple Indexes................................................................................304 viii 9. 11.6. Unique Indexes.....................................................................................................305 11.7. Indexes on Expressions ........................................................................................306 11.8. Partial Indexes ......................................................................................................306 11.9. Operator Classes and Operator Families ..............................................................309 11.10. Indexes and Collations........................................................................................310 11.11. Examining Index Usage......................................................................................310 12. Full Text Search ................................................................................................................312 12.1. Introduction ..........................................................................................................312 12.1.1. What Is a Document?...............................................................................313 12.1.2. Basic Text Matching ................................................................................313 12.1.3. Congurations..........................................................................................314 12.2. Tables and Indexes................................................................................................315 12.2.1. Searching a Table.....................................................................................315 12.2.2. Creating Indexes ......................................................................................316 12.3. Controlling Text Search........................................................................................317 12.3.1. Parsing Documents ..................................................................................317 12.3.2. Parsing Queries ........................................................................................318 12.3.3. Ranking Search Results ...........................................................................320 12.3.4. Highlighting Results ................................................................................322 12.4. Additional Features ..............................................................................................323 12.4.1. Manipulating Documents.........................................................................323 12.4.2. Manipulating Queries...............................................................................324 12.4.2.1. Query Rewriting..........................................................................325 12.4.3. Triggers for Automatic Updates ..............................................................326 12.4.4. Gathering Document Statistics ................................................................327 12.5. Parsers...................................................................................................................328 12.6. Dictionaries...........................................................................................................330 12.6.1. Stop Words...............................................................................................331 12.6.2. Simple Dictionary....................................................................................331 12.6.3. Synonym Dictionary ................................................................................333 12.6.4. Thesaurus Dictionary...............................................................................334 12.6.4.1. Thesaurus Conguration .............................................................335 12.6.4.2. Thesaurus Example .....................................................................336 12.6.5. Ispell Dictionary.......................................................................................337 12.6.6. Snowball Dictionary ................................................................................338 12.7. Conguration Example.........................................................................................338 12.8. Testing and Debugging Text Search.....................................................................340 12.8.1. Conguration Testing...............................................................................340 12.8.2. Parser Testing...........................................................................................342 12.8.3. Dictionary Testing....................................................................................343 12.9. GiST and GIN Index Types..................................................................................344 12.10. psql Support........................................................................................................345 12.11. Limitations..........................................................................................................347 12.12. Migration from Pre-8.3 Text Search...................................................................348 13. Concurrency Control.........................................................................................................349 13.1. Introduction ..........................................................................................................349 13.2. Transaction Isolation ............................................................................................349 13.2.1. Read Committed Isolation Level .............................................................350 13.2.2. Repeatable Read Isolation Level..............................................................351 13.2.3. Serializable Isolation Level......................................................................352 13.3. Explicit Locking...................................................................................................355 13.3.1. Table-level Locks.....................................................................................355 ix 10. 13.3.2. Row-level Locks ......................................................................................357 13.3.3. Deadlocks.................................................................................................358 13.3.4. Advisory Locks........................................................................................359 13.4. Data Consistency Checks at the Application Level..............................................360 13.4.1. Enforcing Consistency With Serializable Transactions...........................360 13.4.2. Enforcing Consistency With Explicit Blocking Locks............................361 13.5. Locking and Indexes.............................................................................................361 14. Performance Tips ..............................................................................................................363 14.1. Using EXPLAIN ....................................................................................................363 14.1.1. EXPLAIN Basics.......................................................................................363 14.1.2. EXPLAIN ANALYZE .................................................................................369 14.1.3. Caveats.....................................................................................................372 14.2. Statistics Used by the Planner ..............................................................................373 14.3. Controlling the Planner with Explicit JOIN Clauses............................................374 14.4. Populating a Database ..........................................................................................376 14.4.1. Disable Autocommit................................................................................376 14.4.2. Use COPY..................................................................................................376 14.4.3. Remove Indexes.......................................................................................377 14.4.4. Remove Foreign Key Constraints ............................................................377 14.4.5. Increase maintenance_work_mem........................................................377 14.4.6. Increase checkpoint_segments ..........................................................377 14.4.7. Disable WAL Archival and Streaming Replication .................................378 14.4.8. Run ANALYZE Afterwards........................................................................378 14.4.9. Some Notes About pg_dump...................................................................378 14.5. Non-Durable Settings ...........................................................................................379 III. Server Administration .............................................................................................................381 15. Installation from Source Code ..........................................................................................383 15.1. Short Version ........................................................................................................383 15.2. Requirements........................................................................................................383 15.3. Getting The Source...............................................................................................385 15.4. Installation Procedure...........................................................................................385 15.5. Post-Installation Setup..........................................................................................395 15.5.1. Shared Libraries.......................................................................................395 15.5.2. Environment Variables.............................................................................396 15.6. Supported Platforms.............................................................................................396 15.7. Platform-specic Notes ........................................................................................397 15.7.1. AIX ..........................................................................................................397 15.7.1.1. GCC Issues..................................................................................398 15.7.1.2. Unix-Domain Sockets Broken.....................................................398 15.7.1.3. Internet Address Issues................................................................398 15.7.1.4. Memory Management .................................................................399 References and Resources ................................................................400 15.7.2. Cygwin.....................................................................................................400 15.7.3. HP-UX .....................................................................................................401 15.7.4. IRIX .........................................................................................................402 15.7.5. MinGW/Native Windows ........................................................................402 15.7.5.1. Collecting Crash Dumps on Windows ........................................403 15.7.6. SCO OpenServer and SCO UnixWare.....................................................403 15.7.6.1. Skunkware...................................................................................403 15.7.6.2. GNU Make ..................................................................................403 15.7.6.3. Readline.......................................................................................403 x 11. 15.7.6.4. Using the UDK on OpenServer...................................................404 15.7.6.5. Reading the PostgreSQL Man Pages...........................................404 15.7.6.6. C99 Issues with the 7.1.1b Feature Supplement .........................404 15.7.6.7. Threading on UnixWare ..............................................................404 15.7.7. Solaris ......................................................................................................404 15.7.7.1. Required Tools ............................................................................404 15.7.7.2. Problems with OpenSSL .............................................................405 15.7.7.3. congure Complains About a Failed Test Program ....................405 15.7.7.4. 64-bit Build Sometimes Crashes.................................................405 15.7.7.5. Compiling for Optimal Performance...........................................406 15.7.7.6. Using DTrace for Tracing PostgreSQL.......................................406 16. Installation from Source Code on Windows .....................................................................407 16.1. Building with Visual C++ or the Microsoft Windows SDK.................................407 16.1.1. Requirements ...........................................................................................408 16.1.2. Special Considerations for 64-bit Windows ............................................409 16.1.3. Building ...................................................................................................410 16.1.4. Cleaning and Installing ............................................................................410 16.1.5. Running the Regression Tests..................................................................411 16.1.6. Building the Documentation....................................................................411 16.2. Building libpq with Visual C++ or Borland C++.................................................411 16.2.1. Generated Files ........................................................................................412 17. Server Setup and Operation ..............................................................................................413 17.1. The PostgreSQL User Account ............................................................................413 17.2. Creating a Database Cluster .................................................................................413 17.2.1. Network File Systems..............................................................................414 17.3. Starting the Database Server.................................................................................414 17.3.1. Server Start-up Failures ...........................................................................416 17.3.2. Client Connection Problems ....................................................................417 17.4. Managing Kernel Resources.................................................................................417 17.4.1. Shared Memory and Semaphores ............................................................417 17.4.2. Resource Limits .......................................................................................423 17.4.3. Linux Memory Overcommit....................................................................424 17.5. Shutting Down the Server.....................................................................................425 17.6. Upgrading a PostgreSQL Cluster .........................................................................426 17.6.1. Upgrading Data via pg_dump..................................................................427 17.6.2. Non-Dump Upgrade Methods..................................................................428 17.7. Preventing Server Spoong..................................................................................428 17.8. Encryption Options...............................................................................................429 17.9. Secure TCP/IP Connections with SSL .................................................................430 17.9.1. Using Client Certicates..........................................................................431 17.9.2. SSL Server File Usage.............................................................................431 17.9.3. Creating a Self-signed Certicate............................................................431 17.10. Secure TCP/IP Connections with SSH Tunnels.................................................432 17.11. Registering Event Log on Windows...................................................................433 18. Server Conguration.........................................................................................................434 18.1. Setting Parameters................................................................................................434 18.1.1. Parameter Names and Values...................................................................434 18.1.2. Setting Parameters via the Conguration File .........................................434 18.1.3. Other Ways to Set Parameters..................................................................434 18.1.4. Examining Parameter Settings.................................................................435 18.1.5. Conguration File Includes......................................................................435 18.2. File Locations.......................................................................................................437 xi 12. 18.3. Connections and Authentication...........................................................................438 18.3.1. Connection Settings .................................................................................438 18.3.2. Security and Authentication.....................................................................440 18.4. Resource Consumption.........................................................................................442 18.4.1. Memory....................................................................................................442 18.4.2. Disk..........................................................................................................443 18.4.3. Kernel Resource Usage............................................................................444 18.4.4. Cost-based Vacuum Delay.......................................................................444 18.4.5. Background Writer...................................................................................445 18.4.6. Asynchronous Behavior...........................................................................446 18.5. Write Ahead Log ..................................................................................................447 18.5.1. Settings.....................................................................................................447 18.5.2. Checkpoints..............................................................................................450 18.5.3. Archiving .................................................................................................450 18.6. Replication............................................................................................................451 18.6.1. Sending Server(s).....................................................................................451 18.6.2. Master Server...........................................................................................452 18.6.3. Standby Servers .......................................................................................453 18.7. Query Planning.....................................................................................................454 18.7.1. Planner Method Conguration.................................................................454 18.7.2. Planner Cost Constants ............................................................................455 18.7.3. Genetic Query Optimizer.........................................................................457 18.7.4. Other Planner Options..............................................................................458 18.8. Error Reporting and Logging ...............................................................................459 18.8.1. Where To Log ..........................................................................................459 18.8.2. When To Log ...........................................................................................462 18.8.3. What To Log ............................................................................................463 18.8.4. Using CSV-Format Log Output ...............................................................466 18.9. Run-time Statistics................................................................................................468 18.9.1. Query and Index Statistics Collector .......................................................468 18.9.2. Statistics Monitoring................................................................................469 18.10. Automatic Vacuuming........................................................................................469 18.11. Client Connection Defaults ................................................................................471 18.11.1. Statement Behavior................................................................................471 18.11.2. Locale and Formatting ...........................................................................474 18.11.3. Other Defaults........................................................................................476 18.12. Lock Management..............................................................................................477 18.13. Version and Platform Compatibility...................................................................478 18.13.1. Previous PostgreSQL Versions ..............................................................478 18.13.2. Platform and Client Compatibility.........................................................480 18.14. Error Handling....................................................................................................480 18.15. Preset Options.....................................................................................................480 18.16. Customized Options ...........................................................................................482 18.17. Developer Options..............................................................................................482 18.18. Short Options......................................................................................................485 19. Client Authentication ........................................................................................................486 19.1. The pg_hba.conf File........................................................................................486 19.2. User Name Maps ..................................................................................................492 19.3. Authentication Methods .......................................................................................494 19.3.1. Trust Authentication ................................................................................494 19.3.2. Password Authentication .........................................................................494 19.3.3. GSSAPI Authentication...........................................................................494 xii 13. 19.3.4. SSPI Authentication.................................................................................495 19.3.5. Kerberos Authentication ..........................................................................495 19.3.6. Ident Authentication.................................................................................497 19.3.7. Peer Authentication..................................................................................498 19.3.8. LDAP Authentication ..............................................................................498 19.3.9. RADIUS Authentication..........................................................................500 19.3.10. Certicate Authentication......................................................................501 19.3.11. PAM Authentication ..............................................................................501 19.4. Authentication Problems ......................................................................................502 20. Database Roles..................................................................................................................503 20.1. Database Roles .....................................................................................................503 20.2. Role Attributes......................................................................................................504 20.3. Role Membership .................................................................................................505 20.4. Function and Trigger Security..............................................................................506 21. Managing Databases .........................................................................................................508 21.1. Overview ..............................................................................................................508 21.2. Creating a Database..............................................................................................508 21.3. Template Databases..............................................................................................509 21.4. Database Conguration ........................................................................................510 21.5. Destroying a Database..........................................................................................511 21.6. Tablespaces...........................................................................................................511 22. Localization.......................................................................................................................514 22.1. Locale Support......................................................................................................514 22.1.1. Overview..................................................................................................514 22.1.2. Behavior...................................................................................................515 22.1.3. Problems ..................................................................................................516 22.2. Collation Support..................................................................................................516 22.2.1. Concepts...................................................................................................516 22.2.2. Managing Collations................................................................................518 22.3. Character Set Support...........................................................................................519 22.3.1. Supported Character Sets.........................................................................519 22.3.2. Setting the Character Set..........................................................................522 22.3.3. Automatic Character Set Conversion Between Server and Client...........523 22.3.4. Further Reading .......................................................................................525 23. Routine Database Maintenance Tasks...............................................................................526 23.1. Routine Vacuuming ..............................................................................................526 23.1.1. Vacuuming Basics....................................................................................526 23.1.2. Recovering Disk Space............................................................................527 23.1.3. Updating Planner Statistics......................................................................528 23.1.4. Updating The Visibility Map ...................................................................529 23.1.5. Preventing Transaction ID Wraparound Failures.....................................529 23.1.5.1. Multixacts and Wraparound........................................................531 23.1.6. The Autovacuum Daemon .......................................................................532 23.2. Routine Reindexing..............................................................................................533 23.3. Log File Maintenance...........................................................................................534 24. Backup and Restore ..........................................................................................................536 24.1. SQL Dump............................................................................................................536 24.1.1. Restoring the Dump.................................................................................537 24.1.2. Using pg_dumpall....................................................................................537 24.1.3. Handling Large Databases .......................................................................538 24.2. File System Level Backup....................................................................................539 24.3. Continuous Archiving and Point-in-Time Recovery (PITR)................................540 xiii 14. 24.3.1. Setting Up WAL Archiving......................................................................541 24.3.2. Making a Base Backup ............................................................................543 24.3.3. Making a Base Backup Using the Low Level API ..................................543 24.3.4. Recovering Using a Continuous Archive Backup ...................................545 24.3.5. Timelines..................................................................................................547 24.3.6. Tips and Examples...................................................................................548 24.3.6.1. Standalone Hot Backups .............................................................548 24.3.6.2. Compressed Archive Logs ..........................................................548 24.3.6.3. archive_command Scripts ........................................................548 24.3.7. Caveats.....................................................................................................549 25. High Availability, Load Balancing, and Replication.........................................................551 25.1. Comparison of Different Solutions.......................................................................551 25.2. Log-Shipping Standby Servers.............................................................................554 25.2.1. Planning ...................................................................................................555 25.2.2. Standby Server Operation........................................................................555 25.2.3. Preparing the Master for Standby Servers ...............................................556 25.2.4. Setting Up a Standby Server....................................................................556 25.2.5. Streaming Replication..............................................................................557 25.2.5.1. Authentication .............................................................................558 25.2.5.2. Monitoring...................................................................................558 25.2.6. Cascading Replication .............................................................................559 25.2.7. Synchronous Replication .........................................................................559 25.2.7.1. Basic Conguration.....................................................................559 25.2.7.2. Planning for Performance............................................................560 25.2.7.3. Planning for High Availability ....................................................561 25.3. Failover.................................................................................................................561 25.4. Alternative Method for Log Shipping ..................................................................562 25.4.1. Implementation ........................................................................................563 25.4.2. Record-based Log Shipping.....................................................................563 25.5. Hot Standby..........................................................................................................564 25.5.1. Users Overview.......................................................................................564 25.5.2. Handling Query Conicts ........................................................................566 25.5.3. Administrators Overview........................................................................568 25.5.4. Hot Standby Parameter Reference...........................................................570 25.5.5. Caveats.....................................................................................................570 26. Recovery Conguration ....................................................................................................572 26.1. Archive Recovery Settings ...................................................................................572 26.2. Recovery Target Settings......................................................................................573 26.3. Standby Server Settings........................................................................................573 27. Monitoring Database Activity...........................................................................................575 27.1. Standard Unix Tools.............................................................................................575 27.2. The Statistics Collector.........................................................................................576 27.2.1. Statistics Collection Conguration ..........................................................576 27.2.2. Viewing Collected Statistics ....................................................................576 27.2.3. Statistics Functions ..................................................................................589 27.3. Viewing Locks......................................................................................................591 27.4. Dynamic Tracing ..................................................................................................591 27.4.1. Compiling for Dynamic Tracing..............................................................592 27.4.2. Built-in Probes .........................................................................................592 27.4.3. Using Probes............................................................................................600 27.4.4. Dening New Probes ...............................................................................601 28. Monitoring Disk Usage.....................................................................................................603 xiv 15. 28.1. Determining Disk Usage ......................................................................................603 28.2. Disk Full Failure...................................................................................................604 29. Reliability and the Write-Ahead Log................................................................................605 29.1. Reliability .............................................................................................................605 29.2. Write-Ahead Logging (WAL) ..............................................................................607 29.3. Asynchronous Commit.........................................................................................607 29.4. WAL Conguration ..............................................................................................609 29.5. WAL Internals ......................................................................................................611 30. Regression Tests................................................................................................................613 30.1. Running the Tests .................................................................................................613 30.1.1. Running the Tests Against a Temporary Installation...............................613 30.1.2. Running the Tests Against an Existing Installation .................................614 30.1.3. Additional Test Suites..............................................................................614 30.1.4. Locale and Encoding................................................................................615 30.1.5. Extra Tests................................................................................................615 30.1.6. Testing Hot Standby.................................................................................615 30.2. Test Evaluation .....................................................................................................616 30.2.1. Error Message Differences.......................................................................616 30.2.2. Locale Differences...................................................................................617 30.2.3. Date and Time Differences ......................................................................617 30.2.4. Floating-Point Differences.......................................................................617 30.2.5. Row Ordering Differences.......................................................................618 30.2.6. Insufcient Stack Depth...........................................................................618 30.2.7. The random Test...................................................................................618 30.2.8. Conguration Parameters.........................................................................618 30.3. Variant Comparison Files .....................................................................................619 30.4. Test Coverage Examination..................................................................................619 IV. Client Interfaces .......................................................................................................................621 31. libpq - C Library ...............................................................................................................623 31.1. Database Connection Control Functions..............................................................623 31.1.1. Connection Strings...................................................................................629 31.1.1.1. Keyword/Value Connection Strings ............................................629 31.1.1.2. Connection URIs.........................................................................629 31.1.2. Parameter Key Words ..............................................................................630 31.2. Connection Status Functions................................................................................634 31.3. Command Execution Functions ...........................................................................638 31.3.1. Main Functions ........................................................................................638 31.3.2. Retrieving Query Result Information ......................................................645 31.3.3. Retrieving Other Result Information .......................................................648 31.3.4. Escaping Strings for Inclusion in SQL Commands.................................649 31.4. Asynchronous Command Processing...................................................................652 31.5. Retrieving Query Results Row-By-Row ..............................................................656 31.6. Canceling Queries in Progress..............................................................................657 31.7. The Fast-Path Interface.........................................................................................658 31.8. Asynchronous Notication...................................................................................659 31.9. Functions Associated with the COPY Command ..................................................659 31.9.1. Functions for Sending COPY Data............................................................660 31.9.2. Functions for Receiving COPY Data.........................................................661 31.9.3. Obsolete Functions for COPY ...................................................................662 31.10. Control Functions...............................................................................................664 31.11. Miscellaneous Functions ....................................................................................665 xv 16. 31.12. Notice Processing ...............................................................................................667 31.13. Event System......................................................................................................668 31.13.1. Event Types............................................................................................668 31.13.2. Event Callback Procedure......................................................................670 31.13.3. Event Support Functions........................................................................671 31.13.4. Event Example.......................................................................................672 31.14. Environment Variables .......................................................................................674 31.15. The Password File ..............................................................................................676 31.16. The Connection Service File ..............................................................................676 31.17. LDAP Lookup of Connection Parameters..........................................................677 31.18. SSL Support........................................................................................................678 31.18.1. Client Verication of Server Certicates ...............................................678 31.18.2. Client Certicates...................................................................................679 31.18.3. Protection Provided in Different Modes ................................................679 31.18.4. SSL Client File Usage............................................................................681 31.18.5. SSL Library Initialization ......................................................................681 31.19. Behavior in Threaded Programs.........................................................................682 31.20. Building libpq Programs.....................................................................................683 31.21. Example Programs..............................................................................................684 32. Large Objects ....................................................................................................................693 32.1. Introduction ..........................................................................................................693 32.2. Implementation Features ......................................................................................693 32.3. Client Interfaces....................................................................................................693 32.3.1. Creating a Large Object...........................................................................694 32.3.2. Importing a Large Object.........................................................................694 32.3.3. Exporting a Large Object.........................................................................695 32.3.4. Opening an Existing Large Object...........................................................695 32.3.5. Writing Data to a Large Object................................................................695 32.3.6. Reading Data from a Large Object ..........................................................696 32.3.7. Seeking in a Large Object........................................................................696 32.3.8. Obtaining the Seek Position of a Large Object........................................696 32.3.9. Truncating a Large Object .......................................................................697 32.3.10. Closing a Large Object Descriptor ........................................................697 32.3.11. Removing a Large Object ......................................................................697 32.4. Server-side Functions ...........................................................................................698 32.5. Example Program .................................................................................................698 33. ECPG - Embedded SQL in C............................................................................................704 33.1. The Concept..........................................................................................................704 33.2. Managing Database Connections .........................................................................704 33.2.1. Connecting to the Database Server..........................................................704 33.2.2. Choosing a Connection............................................................................705 33.2.3. Closing a Connection...............................................................................707 33.3. Running SQL Commands.....................................................................................707 33.3.1. Executing SQL Statements ......................................................................707 33.3.2. Using Cursors...........................................................................................708 33.3.3. Managing Transactions............................................................................708 33.3.4. Prepared Statements.................................................................................709 33.4. Using Host Variables ............................................................................................710 33.4.1. Overview..................................................................................................710 33.4.2. Declare Sections.......................................................................................710 33.4.3. Retrieving Query Results.........................................................................711 33.4.4. Type Mapping ..........................................................................................712 xvi 17. 33.4.4.1. Handling Character Strings .........................................................713 33.4.4.2. Accessing Special Data Types.....................................................713 33.4.4.2.1. timestamp, date...............................................................714 33.4.4.2.2. interval ............................................................................714 33.4.4.2.3. numeric, decimal.............................................................715 33.4.4.3. Host Variables with Nonprimitive Types ....................................716 33.4.4.3.1. Arrays .............................................................................716 33.4.4.3.2. Structures........................................................................717 33.4.4.3.3. Typedefs..........................................................................718 33.4.4.3.4. Pointers ...........................................................................719 33.4.5. Handling Nonprimitive SQL Data Types.................................................719 33.4.5.1. Arrays..........................................................................................719 33.4.5.2. Composite Types .........................................................................721 33.4.5.3. User-dened Base Types.............................................................722 33.4.6. Indicators..................................................................................................723 33.5. Dynamic SQL.......................................................................................................724 33.5.1. Executing Statements without a Result Set .............................................724 33.5.2. Executing a Statement with Input Parameters .........................................724 33.5.3. Executing a Statement with a Result Set .................................................725 33.6. pgtypes Library.....................................................................................................726 33.6.1. The numeric Type ....................................................................................726 33.6.2. The date Type...........................................................................................729 33.6.3. The timestamp Type.................................................................................732 33.6.4. The interval Type .....................................................................................736 33.6.5. The decimal Type.....................................................................................736 33.6.6. errno Values of pgtypeslib .......................................................................737 33.6.7. Special Constants of pgtypeslib...............................................................738 33.7. Using Descriptor Areas ........................................................................................738 33.7.1. Named SQL Descriptor Areas .................................................................738 33.7.2. SQLDA Descriptor Areas ........................................................................740 33.7.2.1. SQLDA Data Structure................................................................741 33.7.2.1.1. sqlda_t Structure.............................................................741 33.7.2.1.2. sqlvar_t Structure............................................................742 33.7.2.1.3. struct sqlname Structure .................................................743 33.7.2.2. Retrieving a Result Set Using an SQLDA ..................................743 33.7.2.3. Passing Query Parameters Using an SQLDA..............................745 33.7.2.4. A Sample Application Using SQLDA ........................................746 33.8. Error Handling......................................................................................................751 33.8.1. Setting Callbacks .....................................................................................751 33.8.2. sqlca .........................................................................................................753 33.8.3. SQLSTATE vs. SQLCODE...........................................................................755 33.9. Preprocessor Directives ........................................................................................758 33.9.1. Including Files .........................................................................................758 33.9.2. The dene and undef Directives ..............................................................759 33.9.3. ifdef, ifndef, else, elif, and endif Directives.............................................759 33.10. Processing Embedded SQL Programs................................................................760 33.11. Library Functions ...............................................................................................761 33.12. Large Objects......................................................................................................762 33.13. C++ Applications ...............................................................................................763 33.13.1. Scope for Host Variables........................................................................764 33.13.2. C++ Application Development with External C Mod