Your partner in System i Education

Embedding SQL in RPG Programs

Susan Gantnersusan.gantner@partner400.comwww.Partner400.comwww.SystemiDeveloper.com

Many RPG programmers have used interactive SQL as a tool to quickly browse data or to create test data scenarios, but have stopped short of embedding it into their RPG programs. Come to this session to learn why, when and how you may want to use SQL in conjunction with RPG. We will cover the basics of embedding SQL into RPG, including using SQL cursors.

The authors, Susan Gantner and Jon Paris, are co-founders of Partner400, a firm specializing in customized education and mentoring services for AS/400 and iSeries developers. After many years with IBM, including several years at the Rochester and Toronto laboratories, Jon and Susan are now devoted to educating developers on techniques and technologies to extend and modernize their applications and development environments. This is done via on-site custom classes for individual companies as well as conferences and user group events.

Together with her partner, Jon Paris, Susan authors regular technical articles for the IBM publication, IBM Systems Magazine, i5 edition (formerly iSeries Magazine and eServer Magazine, iSeries edition), and the companion electronic newsletter, i5 EXTRA (formerly iSeries Extra). You may view articles in current and past issues and/or subscribe to the free newsletter or the magazine at: http://www.ibmsystemsmag.com.

Susan and Jon are also partners in SystemiDeveloper, a company that hosts the RPG Summit and DB2 Summit conferences. See SystemiDeveloper.com for more details.

This presentation may contain small code examples that are furnished as simple examples to provide an illustration. These examples have not been thoroughly tested under all conditions. We therefore, cannot guarantee or imply reliability, serviceability, or function of these programs.

All code examples contained herein are provided to you "as is". THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE EXPRESSLY DISCLAIMED.

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 1-2 .

AgendaQuick review: What is SQL?

Syntax and rules for embedding SQL into programs

Using SQL cursors for processing individual rows in a set

Error detection and handling

Tips

Dynamic SQL (appendix only)

Data Manipulation Language (DML)SELECT - retrieves data; one row or multipleUPDATE - updates one row or multipleDELETE - deletes one row or multipleINSERT - adds one row or multiple

Data Definition Language (DDL)CREATE - creates schemas, tables, views, etc.DROP - deletes schemas, tables, views, etc.ALTER - change the format or attributes of a table

Data Control Language (DCL)GRANT & REVOKE - manages authorities to DB objects

Programming StatementsDESCRIBE, PREPARE - used for dynamic embedded SQLLogic control statements for SQL procedures, triggers and functions (SPL)

Types of SQL Statements

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 3-4 .

Interactive SQL - Use STRSQL commandTargeted primarily to programmers

Embedded SQL - Put into High Level Language (HLL)Such as RPG or COBOL

Query Manager - Report FormatterTargeted primarily to end usersAn alternative to Query/400

iSeries NavigatorSQL wizards, productivity and testing toolsGraphical alternative to Interactive SQL

SQL Stored Procedures and SQL TriggersEnforce business rules using SQL-based logic

Using SQL on System i

Quick Review of SQL SELECT Syntax

Enter SQL Statements Type SQL statement, press Enter.

> SELECT * FROM users WHERE clue > 0________ _0 rows returned._________________________ __________________________________________ __________________________________________ __________________________________________ __________________________________________ __________________________________________ __________________________________________ Bottom F14=Delete line F15=Split line F16=Select library F17=Select files F18=Select fields F24=More keys

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 5-6 .

RPG Interface - SourceRetrieve column/field values into program variables

One-to-one correspondence between SELECT and INTO listsSELECT....INTO expects only a SINGLE row/recordMultiple rows require the use of cursor operations

Note: /Free form allows embedded SQL only V5R4 and later

* No F spec needed !

D EmpNbr S 5 0 D Name S 25 D Job S 1

C/EXEC SQL C+ SELECT NAME, POS C+ INTO :Name, :Job C+ FROM EMPL C+ WHERE NBR = :EmpNbr C/END-EXEC /Free EXEC SQL SELECT NAME, POS INTO :Name, :Job FROM EMPL WHERE NBR = :EmpNbr;

All SQL statements must be coded on a C spec

SQL statements begin with /EXEC SQL in positions 7-15with the slash in position 7 (except in /Free form V5R4+)

... and end with /END-EXEC in positions 7-15 (except in /Free)

SQL statements may appear on the same line as /EXEC SQL/END-EXEC must be on a separate line

Between beginning and ending delimiters, all SQL statements must have + in position 7 (except in /Free form V5R4+)

Note: SQL statements CANNOT be coded in the /FREE parts of your C specs until V5R4

The SQL statement itself is free-form within the limitations listed here

Comments may be included at end of line preceded with --

Rules: Embedding SQL in RPG Code

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 7-8 .

Program variables referenced in the SQL statement are preceded with a colon (e.g., :EmpNbr)

Must not have names beginning with SQ, SQL, RDI, or DSNNames must not be > 64 characters longIf local variables have same names as other locals or globals, all definitions of that name must be identical

In V5R4, they need only be of the same type (e.g., numeric, character, date) but will still issue Severity 11 warning

Set GENLVL parameter to be 11 or higher to have them accepted

SQL statements cannot be included via a /COPY statementHowever, you can use the SQL INCLUDE statement instead

More Rules: Embedding SQL in RPG

C/Exec SQLC+ Include SrcMbrNameC/End-Exec

SELECT INTO SQL Statement is used to retrieve a single record from the database

If your statement returns more than 1 row, you must use another form of SELECT we will discuss later

D CustID S 7P 0D CustName S 30AD CustSales S 7P 0

C/EXEC SQL C+ SELECT CUSTID,CUSTNAM, CUSTSALESC+ INTO :CUSTID,:CUSTNAME,:CUSTSALES C+ FROM CUSTOMER C+ WHERE CUSTID = :InputCustC/END-EXEC

SELECT INTO

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 9-10 .

Updates and DeletesSQL updates and deletes can be embedded as well

Set-at-a-time updates do not require cursor operationsThis statement will update ALL records that meet the Where criteria

C/EXEC SQL C+ UPDATE empl C+ SET sal = sal + (sal * .10) C+ WHERE pos = 5 C/END-EXEC

Host structures are groups of variablesData structures in RPG

Structures can be used in SQL statementsReplaces list of variables

DS subfields must contain the correct number, type and sequence for SELECT columns

Using Structures in SQL

D CustInfo DS D CustID 7P 0 D CustName 30A D CustSales 7P 0

D InputCust S 7P 0

C/EXEC SQL C+ SELECT CUSTID, CUSTNAM, CUSTSALES C+ INTO :CustInfo C+ FROM CUSTOMER C+ WHERE CUSTID = :InputCust C/END-EXEC

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 11-12 .

External Data StructuresThe simple way to define a complete record format to match a SELECT * request

Declare an externally described Data StructureSubfields will be brought in to match all the fields in the record format

D CustRec E DS ExtName(Customer)

D InputCust S 7P 0

C/EXEC SQL C+ SELECT * C+ INTO :CustRec C+ FROM CUSTOMER C+ WHERE CUSTID = :InputCust C/END-EXEC

SQL Result SetsGroup of rows that are derived as a result of an SQL query.

The answer to your SQL question

Made up of rows and columns

SELECT nbr, nameFROM emplWHERE pos = 5

Nbr Name Pos Sex Sal10 AMY 2 F 120035 JOE 5 M 100030 JON 7 M 150020 DON 5 M 115025 ANN 8 F 1550

Nbr Name35 JOE20 DON

Result Set

SQL StatementProduces

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 13-14 .

Steps to access multiple rows:1. Declare cursor2. Open cursor3. Fetch a row (record)4. Process row (UPDATE, INSERT, etc)5. IF last row: go to Step 6,

ELSE go to Step 36. Close cursor

Selecting & Processing Multiple Rows

DECLARE CURSOR statementSimilar in function to HLL file declarations (F-specs or FD's)

No processing actually takes place - just definition

Host variables may be included in the statement

Created using an embedded SELECT commandmost SELECT clauses may be used - ORDER BY, GROUP BY, etc

Must be declared before being referenced"Before" meaning physically present in the source member earlier

If updating fields/columns, specify "For Update Of ..." clauseList the column(s) that may be updated using this cursor

C/EXEC SQL C+ DECLARE empcsr CURSOR FOR C+ SELECT nbr, nam, sal C+ FROM emp C+ WHERE dpt = :dept C+ FOR UPDATE OF sal C/END-EXEC

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 15-16 .

Actually executes the SQL Select statement

Builds the access path if necessary

Successful Open places the file cursor before the first row of the result set

Cursor must be closed before it can be re-opened

OPEN statement

C/EXEC SQL

C+ OPEN empcsr

C/END-EXEC

FETCH statement:

C/EXEC SQL

C+ FETCH NEXT FROM empcsr C+ INTO :number, :name, :salary

C/END-EXEC

Bring rows into the program One at a time for processingFields placed into host variables

one for one with fields in SELECT

Fetch Next is the default behaviorIf a cursor is declared as "scrollable", Fetch has other options

e.g., Prior, First, Last

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 17-18 .

Update or delete the current row of an updatable cursorCan only be done after successful Fetch operationAdd a "Where Current of" clause to the Update and Delete statements

Positioned Update and Delete Stmts

C/EXEC SQL C+ DECLARE empcsr CURSOR FOR C+ SELECT nbr, nam, sal C+ FROM emp ORDER BY nbr C+ FOR UPDATE OF sal C/END-EXEC

C/EXEC SQL OPEN empcsr C/END-EXEC

C/EXEC SQL C+ FETCH NEXT FROM empcsr C+ INTO :number, :name, :salary C/END-EXEC

C/EXEC SQL C+ UPDATE emp C+ SET sal = sal + :raise C+ WHERE CURRENT OF empcsr C/END-EXEC

Close the cursorCursor must be opened in order to be closed

DB2/400 may close cursors for other reasons also:job endactivation group endsprogram endsmodules endscommit or rollback without a 'with hold' clauseerror handling......

Close Statement

C/EXEC SQL

C+ CLOSE empcsr

C/END-EXEC

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 19-20 .

User Source Member

PrecompileModified Source Member

Processed SQL Stmts

Compile

Access Plans

Program

(temporary)

SQL Precompiler for Embedded SQL

Warning for Leading Edge RPGersThe SQL precompiler does not fully understand or support all the latest RPG language features

V5R4 allows embedded SQL in /Free calcs and relaxed local variable naming rulesExamples of things not supported as of V5R3 (not a complete list)

SQL embedded in /Free RPG calcs> 1 level of qualification for DS subfieldsProper recognition of local variables in subprocedures

Examples of things not supported UNTIL V5R3 (not a complete list)DS arrays for blocked fetches or insertsOne level of simple DS subfield qualification (not for DS array)LIKEDS and LIKEREC support (2nd parameter still not supported)

See the iSeries Information Center for details on limitations of RPG IV support

Topic is "Embedded SQL Programming" under "Database/Programming" topic

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 21-22 .

Create RPG/400 SQL applications using CRTSQLRPGCreates a *PGM object

Create RPG IV SQL applications using CRTSQLRPGICreates either a *PGM, *MODULE, or *SRVPGM object

Creating Programs with SQL Stmts

Create SQL RPG Program (CRTSQLRPG) Program . . . . . . . ________ Name Library . . . . . . *CURLIB Name, *CURLIB Source file . . . . . QRPGSRC Name, QRPGSRC Library . . . . . . *LIBL Name, *LIBL, *CURLIB Source member . . . . *PGM Name, *PGM Commitment control . . *CHG *CHG, *ALL, *CS, *NONE... Relational database . *LOCAL Text 'description' . . *SRCMBRTXT Precompiler options . *NOSRC__ *SRC, *SOURCE, *NOSRC... + for more values _______ Target release . . . *CURRENT *CURRENT, *PRV, V4R4M0, ... INCLUDE file . . . . *SRCFILE Name, *SRCFILE Library . . . . . *LIBL Name, *LIBL, *CURLIB

. . . .

To source file . . . . QSQLTEMP Name Library . . . . . . QTEMP Name, *LIBL, *CURLIB

Compiling RPG/400 SQL Programs

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 23-24 .

Create SQL ILE RPG Object (CRTSQLRPGI)

Object . . . . . . . _________ Name Library . . . . . *CURLIB Name, *CURLIB Source file . . . . QRPGLESRC Name, QRPGLESRC Library . . . . . *LIBL Name, *LIBL, *CURLIB Source member . . . *OBJ Name, *OBJ Commitment control . *CHG *CHG, *ALL, *CS, *NONE... Relational database *LOCAL Compile type . . . . *PGM *PGM, *SRVPGM, *MODULE Listing output . . . *NONE *NONE, *PRINT Text 'description' . *SRCMBRTXT Precompiler options. *XREF *XREF, *NOXREF, *GEN... + for more values ________ Target release . . . *CURRENT *CURRENT, *PRV, V4R4M0, ... INCLUDE file . . . . *SRCFILE Name, *SRCFILE Library . . . . . *LIBL Name, *LIBL, *CURLIB . . . . . Close SQL cursor . . *ENDACTGRP *ENDMOD, *ENDACTGRP . . . . . Debugging view . . . *NONE *SOURCE, *NONE To source file . . . QSQLTEMP Name Library . . . . . QTEMP Name, *LIBL, *CURLIB

Compiling RPG IV SQL Programs

Note that default on compiles is Commit *CHG

This means:If files aren’t journaled, programs will abnormally terminate whenever you attempt to use SQL on those filesIf your files are journaled, you will have uncommitted transactions unless you remember to explicitly commit themIf your shop does not normally use commitment control, consider using Commit *NONE on your compile commands

SQL and Commitment Control

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 25-26 .

When using ILE source view debuggerILE SQL program compiles automatically generate 3 different views with DBGVIEW(*SOURCE):

Use F15 to switch between:SQL Root Source viewSQL Output view (output from SQL precompiler)Listing view

Debugging Embedded SQL

V5R3+ Compile EnhancementWhat about compile options that are valid on CRTRPGPGM or CRTBNDRPG or CRTRPGMOD that are NOT present on the corresponding SQL compile statements?

With SQLRPGLE members, the best way to handle this is by embedding the compile options on the H spec

e.g., H DFTACTGRP(*No) ACTGRP(MYAG) BNDDIR('MYBNDDIR')This option has been available for many releases

New parameter added in V5R3:Works for other source types (e.g., SQLRPG, SQLCBL)

Also works for SQLRPGLE types if recommended H spec option is not usedCompiler Options (COMPILEOPT)

A 5,000 character parameter field allowing parameters to be passed on to the language compiler after the SQL precompile stepParameters that are included on the SQL compile already cannot be specified here because it will result in duplicate parameters being specifiedUse compile keywords with the values

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 27-28 .

Status always returned in the codeboth successful and unsuccessful statements

Programmer must check return codes within program

SQL Communications Area (SQLCA)contains feedback informationmust be included in all SQL programsRPG includes SQLCA automaticallyCOBOL programs must have specific include, as shown below:

Error Detection and Handling

/EXEC SQL INCLUDE SQLCA /END-EXEC

SQL Communications Area (SQLCA)SQLCAID Char(8) Structure identifying literal: "SQLCA"SQLCABC Integer Length of SQLCASQLCode Integer Return codeSQLErrML SmallInt Length of SQLErrMCSQLErrMC Char(70) Message Replacement textSQLErrP Char(8) Product ID literal: "QSQ" for DB2/400SQLErrD Array of Integers SQLErrD(1) - treated as Char(4); last 4 characters of

CPF or other escape message SQLErrD(2) - treated as Char(4); last 4 characters of

CPF or other diagnostic message SQLErrD(3) - for Fetch, Insert, Update or Delete,

number of rows retrieved or updated SQLErrD(4) - for Prepare, relative number indicating

resources required for executionSQLErrD(5) - for multiple-row Fetch, contains 100 if

last available row is fetched; for Delete, number of rows affected by referential constraints; for Connect or Set Connection, contains t-1 if unconnected, 0 if local and 1 if connection is remote

SQLErrD(6) - when SQLCode is 0, contains SQL completion message id

Error Detection and Handling

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 29-30 .

SQL Communications Area (SQLCA) continued

SQLWarn Char(11) Set of 11 warning indicators; each is blank, W, or NSQLWarn0 Char(1) Blank if all other SQLWARNx warning indicators are blank

W if any warning indicator contains W or NSQLWarn1 Char(1) W if a string column was truncated when assigned to host variableSQLWarn2 Char(1) W if null values were eliminated from a functionSQLWarn3 Char(1) W if number of columns is larger than number of host variablesSQLWarn4 Char(1) W if prepared Update or Delete statement has no a Where clauseSQLWarn5 Char(1) ReservedSQLWarn6 Char(1) W if date arithmetic results in end-of-month adjustmentSQLWarn7 Char(1) ReservedSQLWarn8 Char(1) W if result of character conversion contains the substitution characterSQLWarn9 Char(1) ReservedSQLWarnA Char(1) ReservedSQLState Char(5) Return code; "00000" if no error or warning

Error Detection and Handling

SQLCODE (SQLCOD in RPG) contains return code = 0 Successful statement execution > 0 Successful, with warning condition < 0 Unsuccessful - statement failed

SQLCODE value indicates exact error or conditione.g.. 100 = Row not found (or end of file)e.g.. -552 = Not authorized to object

SQLCODE values have corresponding messagese.g.. SQL0100 = Row not founde.g.. SQL0552 = Not authorized to &1.

Many SQL pgmrs also check SQLWN0A few warning conditions do not return > 0 SQLCODE valuee.g., If SQLCOD > 0 or SQLWN0 <> *Blank

SQLCODE Error Handling

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 31-32 .

Most SQLCODE values map to message IDs in QSQLMSG message file

MSGID is SQxxxxx where xxxxx is derived with the following rules:1. Calculate the absolute value of the SQLCODE value.2. Zero-fill the result with leading zeros to construct a 5-digit number.3. If the first digit in this number is 0, replace it with the letter 'L'.For example, SQLCODE value of -151 = message ID SQL0151SQLCODE value of -30090 = message ID SQ30090.

Use DSPMSGD to quickly find meanings of SQLCODE values

SQLCODE and Messages

Condition SQLCOD orSQLCODE

SQLSTT or SQLSTATE Class(1st 2 positions)

Successful 0 '00'

Warning >0 '01'

End of DataRow not found 100 '02'

Error <0 >='03'

SQLCODE and SQLSTATE

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 33-34 .

SQLCODE values vary by platformBy contrast, SQLSTATE values are standard across platformsHowever, SQLCODE is most widely used, especially for individual statements

SQLCODE is a deprecated function in current SQL standard"Deprecated" means function replaced with a newer methodUse of deprecated functions is typically discouraged

DB2 UDB for iSeries plans to continue to support SQLCODE for the forseeable future

SQLCODE vs. SQLSTATE

C/EXEC SQL C+ SELECT name INTO :nam C+ WHERE emp = :number C/END-EXEC

C Select C When SQLCod < 0 C ExSR SQLError C When SQLCod = 100 C ExSR NoMoreRows C When ( SQLCod > 0 Or SQLWn0 <> *Blank ) C ExSR SQLWarning C Other C ExSR ProcessRow C EndSL

Error Checking With SQLCOD

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 35-36 .

C/EXEC SQL C+ SELECT name INTO :nam C+ WHERE emp = :number C/END-EXEC

C Select C When %Subst(SQLStt:1:2) >= '03' C ExSR SQLError C When %Subst(SQLStt:1:2) = '02' C ExSR NoMoreRows C When %Subst(SQLStt:1:2) = '01' C ExSR SQLWarning C Other C ExSR ProcessRow C EndSL

Error Checking With SQLSTT

Use SQL source member typese.g., SQLRPG, SQLRPGLE, SQLCBL, SQLCBLLEPrompting won't work without SQL member type

You can prompt SQL statements in SEUYou MUST key both /EXEC SQL and /END-EXEC statements firstEnter the C+ for the SQL statementThen you can prompt (F4) for statements in betweenSame prompter as interactive SQL

To help diagnose run-time or performance problemsLook in your job log after running the SQL programMessages there often help diagnose problemsPut your job in debug mode before running SQL program, then look in your job logAdditional messages are put in job log when in debug mode which can be helpful in diagnosing SQL performance problems

Embedded SQL Tips

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 37-38 .

Static SQLSQL Statement structure known at compile timeBetter Performance

Dynamic SQLSQL Statements built during program executionGreater Flexibility

A few charts in the appendix of this handout illustrate the basics of using Dynamic SQL

Static vs. Dynamic SQL

Practical, effective solution for applications requiring complex data retrieval

Very flexible, functional application development toolembedded in a HLL programor entered interactively

Portability to other relational databasesEasy report writing with programmable flexibility

Similar skills across many relational databases

Summary

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 39-40 .

A different way to use SQL

SQL statements are not predefined in programDynamically created on the fly as part of program logic

SQL Precompiler cannot fully process dynamically created SQL statements

PREPARE statement is used in program logic to compile dynamically created SQL statements at run time

Simple dynamic SQL statement process:Build SQL statement in a character variablePREPARE the SQL statementEXECUTE the SQL statement

What is Dynamic SQL?

Where to use Dynamic SQLReport programs with user run time selection

FilesFieldsRecord selection criteriaSortingSQL built in functions

Whenever the exact syntax of an SQL statement cannot be determined beforehand

Dynamic SQL can be resource intensiveA dynamic SQL statement has to be parsed (interpreted) and executed at run timeNegative performance impactUse dynamic SQL only when necessary

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 41-42 .

Parameter Markers in Dynamic SQLDynamic SQL statements cannot contain host variables

e.g., :CUSTNO

Parameter markers are placed in embedded SQL statementsIndicated by ?Used to dynamically insert host variable data for predicate values and/or column assignmentsMultiple markers and values can be assignedValues are assigned to markers when the statement is executed

Example on next chart

C Eval SQLStmtFld = 'Delete From Customer Where -C CUSTNO = ?'

Dynamic SQL - Example

C If DeleteCorpC Eval Condition = 'Corp = ?' C Else C Eval Condition = 'CustNo = ?'C EndIf

C Eval SQLStmtFld = 'Delete From Customer Where 'C + Condition

C/EXEC SQL

C+ PREPARE DynSQLStmtC+ FROM :SQLStmtFld

C/END-EXECC If (SQLCod = 0) And (SQLWn0 = *Blank)C/EXEC SQL

C+ EXECUTE DynSQLStmt C+ Using :Cust

C/END-EXECC EndIf

©Copyright Partner400, 2003 - 2007. SQL in RPG - Page 43-44 .