IBM Database Conversion Workbench 3 -...

IBM Analytics

IBM Database Conversion Workbench 3.5

IBM PureData System for Analytics to IBM dashDB

Conversion Guide

Version: 3.5

Last Updated: June 12th, 2015

IBM Analytics

IBM Database Conversion Workbench © Copyright IBM Corp. 2015. All rights reserved Page 2 of 22

Table of Contents

1. Introduction .................................................................................................................................................................... 3

2. Prerequisites................................................................................................................................................................... 4

3. Overview of the Conversion Process .......................................................................................................................... 5

4. Set Up Your DCW Environment .................................................................................................................................... 6

4.1. Create a new DCW Conversion Project ................................................................................................................. 6

5. The IBM PureData System for Analytics to IBM dashDB Task Launcher ................................................................ 8

6. DDL Extraction ............................................................................................................................................................... 9

6.1. Extract DDL from a PureData System for Analytics Database .............................................................................. 9

6.2. Import the DDL into the DCW Project .................................................................................................................... 9

7. Compatibility Evaluation ............................................................................................................................................. 10

7.1. Format of the Compatibility Evaluation Report ..................................................................................................... 10

7.2. Generate the Compatibility Evaluation Report ..................................................................................................... 10

7.2.1. Prerequisites .......................................................................................................................................... 10 7.2.2. Evaluate Compatibility .......................................................................................................................... 10 7.2.3. Support for large DDL files ................................................................................................................... 11 7.2.4. Submitting your Compatibility Evaluation Report to IBM ................................................................. 11

7.3. Using the Compatibility Evaluation Report ........................................................................................................... 12

7.3.1. Prerequisites .......................................................................................................................................... 12 7.3.2. View through your web browser .......................................................................................................... 12 7.3.3. Overall Summary ................................................................................................................................... 12 7.3.4. DDL Code Evaluation ............................................................................................................................ 13 7.3.5. Unrecognized lines of code .................................................................................................................. 14 7.3.6. Support for BLU Acceleration .............................................................................................................. 14

8. Code Conversion ......................................................................................................................................................... 15

8.1. Auto-convert Oracle syntax to dashDB compatible syntax .................................................................................. 15

8.1.1. Pre-requisites ........................................................................................................................................ 15 8.1.2. Run the auto-conversion ...................................................................................................................... 15

8.2. Data Type Conversion .......................................................................................................................................... 15

8.3. Reviewing the converted code ............................................................................................................................. 15

9. Deploy Objects to the dashDB Database .................................................................................................................. 18

9.1. Create a Connection to the dashDB database ..................................................................................................... 18

9.2. Executing the converted DDL on dashDB from DCW .......................................................................................... 19

9.2.1. Pre-requisites ........................................................................................................................................ 19 9.2.2. Execute the DDL statements ................................................................................................................ 19

9.3. Execute the Converted DDL from the dashDB Console ...................................................................................... 20

Appendix 1 - Data-Type Conversion Table ...................................................................................................................... 21

1. Introduction The IBM® Database Conversion Workbench (DCW) integrates many of the tools used for converting from different database vendors to IBM database products, while providing an easy to use framework to take you through all the steps of the conversion process. DCW is provided as a plug-in to IBM Data Studio, a no-charge graphical tool based on an Eclipse® platform. IBM Data Studio helps database developers and administrators manage, administer, and develop heterogeneous database environments for increased productivity and collaboration. For more information, visit: http://www.ibm.com/software/data/optim/data-studio/ As an integrated workbench, DCW provides the following benefits to the user:

- Common graphical interface with uniform look and feel for all phases of the conversion - A consolidated process with well-defined steps based on best practices - Faster enablement for client and partner teams in conversion methodology - Wizards that guide users through all steps of the conversion

DCW offers the following step-by-step functions to facilitate the process.

1. DCW Task Launcher - An integrated help guide, which provides step by step instructions through the conversion process.

2. Compatibility Evaluation – Provides a report of the estimated compatibility ratio of PureData System for Analytics SQL and NZPLSQL* statements with dashDB, outlining the major issues of the conversion, code that was auto-converted, as well as code that must be fixed manually.

3. Code Conversion - Auto-converts SQL statements to dashDB compatible syntax. This streamlines a time consuming process.

* IMPORTANT NOTE: Support for analyzing NZPLSQL statements will be added to DCW in a future release.

http://www.ibm.com/software/data/optim/data-studio/

IBM Analytics


2. Prerequisites The IBM Database Conversion Workbench is a plug-in for IBM Data Studio. IBM Data Studio must first be installed on your machine before installing DCW. For the DCW 3.5 plug-in, the following requirements must be met:

IBM Data Studio 4.1.1 full client o Available for download at: http://www.ibm.com/developerworks/downloads/im/data/

For installation requirements and instructions, download the IBM DCW Plug-In Installation Guide: http://www.ibm.com/developerworks/data/ibmdcw.

http://www.ibm.com/developerworks/downloads/im/data/

http://www.ibm.com/developerworks/data/ibmdcw

IBM Analytics


3. Overview of the Conversion Process

A standard conversion consists of the following steps:

1. Extract DDL from source database 2. Generate Compatibility Evaluation report from extracted DDL

a. Estimate efforts based on incompatible syntax occurrences in the report 3. Automatically convert extracted DDL to syntax of the target database 4. Manually fix any additional incompatible syntax based on evaluation report 5. Create target database 6. Execute converted DDL to recreate database schema(s) 7. Extract data from source database 8. Deploy data into target database

The Database Conversion Workbench provides an integrated environment with many functions to help take the user through the conversion steps described above. It is highly recommended for novice users to follow the process from beginning to end. While DCW supports the use of the various conversion functions separately, this approach is beyond the scope of this guide and recommended only for more advanced users.

IBM Analytics


4. Set Up Your DCW Environment

4.1. Create a new DCW Conversion Project

In order to access the functions of the Database Conversion Workbench, you need to create a new DCW project.

i. From Data Studio, navigate to menu File > New > Project… then select the DCW Project wizard.

ii. Enter the Project Name, select “PureData System for Analytics” as the Source Database and

“dashDB” as the Target Database.

iii. Click Finish. You might be asked to switch to the Database Conversion perspective. Click Yes to confirm the switch, and note that the following views are available:

Project Explorer view – lists all projects and relevant files.

Data Source Explorer view – lists all existing database connection profiles.

Administration Explorer view – lists all existing database connection profiles and provides access to database administration functions.

DCW PureData System for Analytics to IBM dashDB Task Launcher – launches the various wizards and help topics.

IBM Analytics


IBM Analytics


5. The IBM PureData System for Analytics to IBM dashDB Task Launcher

The Database Conversion Workbench comes with a built in “Task Launcher”, which acts as an assistant to launch the various steps of the conversion process.

The Task Launcher is available after a DCW project has been created, and can be accessed at any time through the

‘Help’ menu. Included with the task launcher are the DCW Help Topics which offer directions on how to access the

various functions through your Project Explorer view, as well as helpful hints and step-by-step guidelines to follow for your conversion project. These help topics are automatically displayed when a process step is selected in the task launcher.

IBM Analytics


6. DDL Extraction

6.1. Extract DDL from a PureData System for Analytics Database

You can use the nzdumpschema command or a third-party application to extract the DDL and save it to a file. After the DDL file is available, import it into your DCW project. The procedure below describes how to extract the

DDL using the nzdumpschema command.

Note: Refer to the PureData System for Analytics Knowledge Center for more information about the

nzdumpschema command.

i. From the command line in a PureData System for Analytics system, execute the command below.

Note that the nzdumpschema command is located under under /nz/kit/bin/adm

nzdumpschema -R <db_name> <output_file>

where <db_name> is the database name and <output_file> is the name of the file into which the

DDL is written.

ii. Copy the <output_file> to the machine where the Database Conversion Workbench is installed.

6.2. Import the DDL into the DCW Project

After you have extracted the DDL from your source database, import the file into your DCW project.

i. In the Project Explorer view, right-click on your DCW Project and navigate to Database

Conversion > Import a DDL file. The Import wizard opens. ii. Specify the directory of the DDL file and select the DDL file you want to import. Select the folder (DCW

project) into which you want to import the DDL file. Click Finish.

iii. The DDL file is imported into your DCW Project, and automatically converted to a .sql extension.

IBM Analytics


7. Compatibility Evaluation With a simple click of a button, the IBM Database Conversion Workbench can analyze PureData System for Analytics DDL statements and create an assessment report which outlines the compatibility of the statements in use with the syntax of dashDB. The compatibility evaluation report includes an executive summary and a detailed list of DDL incompatibilities, as well as some suggested workarounds to fix the incompatible code. These issues can be addressed after the code conversion has been run in section 8.

7.1. Format of the Compatibility Evaluation Report

By default, the report is saved in an encrypted format in a file with a .xmle extension. Section 7.3 explains how to decrypt and visualize the report.

7.2. Generate the Compatibility Evaluation Report

The Compatibility Evaluation Report is generated by analyzing the DDL file of the source PureData System for Analytics database.

7.2.1. Prerequisites

Ensure that you have a DDL file imported into your DCW project as outlined in section 6.

7.2.2. Evaluate Compatibility

i. Right-click on the .sql file containing the DDL. Navigate to Database Conversion > Evaluate

Compatibility. The Compatibility Evaluation wizard opens.

Hint: You can run the compatibility evaluation on multiple DDL files at once. This generates a separate report for each DDL file.

Note: DCW automatically splits DDL files larger than 5 MB into separate files for processing. This process generates a single merged report. See section 7.2.3 Support for large DDL files for more information.

Note: DCW has a 100 MB size restriction on the input DDL file. See section 7.2.3 Support for large DDL files for more information.

IBM Analytics


ii. Select the Source SQL dialect as PureData System for Analytics and the Target SQL dialect as

dashDB then click Finish.

iii. The Compatibility Evaluation Report (.xmle file) is saved to your project folder.

7.2.3. Support for large DDL files

For DDL files that are larger than 5 MB, DCW pre-processes the file by splitting the large DDL file into smaller more manageable components.

DCW splits the large file into separate files, each of which is about 5 MB in size. The split files are created in a

folder named <original_DDL_filename>_parts, which can be located within your DCW Project.

To reduce the chance of error, DCW parses and processes each file separately and sequentially.

Note: The progress of each file is displayed by DCW. Part 1 of 9 indicates that DCW split the original DDL file into 9 component files and is currently processing the 1st file.

Although processed separately, DCW generates a single and merged Compatibility Evaluation Report for the original large DDL file. At the end of the process, you receive a result message, which details any component files that were missed due

to issues. You can rerun the Compatibility Evaluation on the missed files, which can be found in your DCW Project.

7.2.4. Submitting your Compatibility Evaluation Report to IBM

The Compatibility Evaluation generates an encrypted .xmle report as <file_name>-

<timestamp>_report.xmle. This encrypted report must be sent to IBM, and the decrypted report is emailed

back to you.

IBM Analytics


NOTE: The .xmle file does not contain any source code. It only contains the list of incompatibilities found and the number of occurrences in the original source code.

Using your desired email client, submit the .xmle file to [email protected]

NOTE: Emails sent by using gmail or hotmail accounts are not accepted.

An email response is sent back containing the HTML version of the Compatibility Evaluation Report, which can be viewed through your web browser.

7.3. Using the Compatibility Evaluation Report

The Evaluation Report is a key part of the conversion process, and outlines the various compatibility issues in the syntax between PureData System for Analytics and dashDB. This section outlines how to use the Evaluation Report to correctly address any issues that are reported, after the code conversion has been run.

7.3.1. Prerequisites

Ensure that you have the HTML version of the Evaluation Report. See section 7.2 Generate the Compatibility Evaluation Report for more details.

7.3.2. View through your web browser You can open the Evaluation Report using your preferred browser by double clicking on the file. You can also import the Evaluation Report into Data Studio, and use the native browser in Data Studio.

i. Right-click on your DCW Project and select Import. The Import wizard opens.

ii. Expand the General folder and select File System. Click Next.

iii. Select Browse, and navigate to the directory in which the HTML Evaluation Report is located. After

the folder is open, select the HTML Evaluation Report. Click Finish.

iv. The report is now in your DCW Project.

v. In your DCW Project, right-click on the HTML Evaluation Report and select Open With > Web

Browser. The Evaluation Report opens.

7.3.3. Overall Summary This section of the report shows the compatibility percentage of all NZPLSQL* and DDL statements that can run natively on a dashDB database.

NZPLSQL – Statistics by Object Type shows a breakdown and count of objects that were detected, as well as the number of object types that require attention.

mailto:[email protected]

IBM Analytics


DDL – Statistics by Statement Type shows the number of DDL statements that were detected, as well as the number of statements that require attention.

* IMPORTANT NOTE: Support for analyzing NZPLSQL statements will be added to DCW in a future release.

Total Number is the total count of statements and objects that was detected.

The Number that Require Attention is the count of statements/objects that are not immediately transferrable to dashDB and require attention.

Note: Statements and objects that can be auto-converted by DCW are not included in the count of “Number that

Require Attention”.

7.3.4. DDL Code Evaluation This section outlines the DDL features that are not supported on dashDB. These statements must be altered or removed before execution on dashDB. This can be done after the code conversion has been run in section 8.

You can expand or collapse each incompatible feature by clicking on the statement. For each issue identified, the

report provides a list of all occurrences with the corresponding line number of the statement in the original DDL

code snippet, the portion of the syntax that contains the error, and a recommended solution.

IBM Analytics


In addition to the features that are not supported, the report provides a DDL Features – Informational section. These are statements that were automatically converted by DCW to the most appropriate dashDB compatible syntax, so no further actions are required. The list is provided to help consultants identify specific cases where the conversion applied might not be ideal.

7.3.5. Unrecognized lines of code

The very last section of the report contains lines of code that the Compatibility Evaluation was not able to recognize. These statements might or might not be compatible with dashDB. You can check the code by referring

to the line number that is listed.

7.3.6. Support for BLU Acceleration

Since dashDB takes advantage of BLU Acceleration technology, the Compatibility Evaluation report contains two extra sections that list the BLU specific issues. The two BLU specific sections are:

BLU Incompatibilities: Lists features that are not supported by column-organized tables. Look at the proposed solution in the report to learn how to fix it.

BLU Stats: Contains the number of tables (CREATE TABLE statements) in the DDL file that can be created as column-organized tables.

Note, in the Overall Summary section, the compatibility ratio does NOT account for BLU specific issues found.

IBM Analytics


8. Code Conversion

The IBM Database Conversion Workbench offers a powerful tool that can auto-convert well known Oracle syntax to dashDB compatible syntax.

8.1. Auto-convert Oracle syntax to dashDB compatible syntax

8.1.1. Pre-requisites

Ensure that you have a DDL file of the source database as outlined in section 6.

8.1.2. Run the auto-conversion

i. Right-click on the DDL file of your source Oracle database in your DCW Project. Navigate to

Database Conversion > Convert Code… The Code Conversion wizard opens.

ii. Select the Source SQL dialect and Target SQL dialect. Click Finish.

iii. The converted DDL file is located in your DCW Project folder, named <originalfilename>-

<timestamp>_converted.sql.

Note: The code conversion automatically changes all statement terminators to ‘@’

8.2. Data Type Conversion

As part of the automatic code conversion, PureData System for Analytics data types are converted to the most suitable data type in dashDB. Refer to Appendix 1 - Data-Type Conversion Table for a listing of all data type conversions.

8.3. Reviewing the converted code

You can review the converted code in Data Studio by double clicking on the SQL file.

IBM Analytics


DCW does one of the following five things:

i. Remove code fragment – DCW comments out fragments of code that are not required on dashDB. These are tagged in the converted code with “Code Fragment”.

ii. Auto-convert code – DCW automatically makes updates to code to make it compatible with dashDB while maintaining the semantics of the original code. These statements are not tagged.

iii. Requires Attention – Code is marked as “requires attention” and is converted to dashDB compatible syntax so that the statement can compile and execute successfully. Although due to differences in behavior between dashDB and PureData System for Analytics, it might raise issues during runtime in very specific scenarios. For instance, DCW converts any NUMBER(N) where N is greater than 31 (max precision in dashDB) to NUMBER(31) which works for a great majority of scenarios.

iv. Evaluation Issue – DCW highlights statements that were identified in the Evaluation Report as being a potential issue. These are tagged in the converted code as “DCW Evaluation Issue”.

Whenever possible, DCW modifies DDL statements in order to make them compatible with column-

organized tables. These cases are listed in the Compatibility Evaluation report as Informational only.

At this point, you must manually edit the converted code to be fully compatible with dashDB.

1. Review code that requires attention. a. You can search the string “requires attention” by using CTRL+F.

b. In the following example, the data type of col2 was truncated to VARBINARY(32672) which is the dashDB limit. In most cases, this is not a problem. However, DCW has identified it as requiring attention so that the user is aware that this change has been made, and might become an issue if this columns stores text values longer than the permitted limit.

c. You can search the remaining converted code for the next statement that “requires

attention” and review it appropriately.

2. Review code with Evaluation Issue a. You can search the string “DCW Evaluation Issue” by using CTRL+F. b. In the following example, the ALTER TABLE … OWNER TO statement must be rewritten as

a TRANSFER OWNERSHIP statement, as outlined in the Compatibility Evaluation Report.

IBM Analytics


c. You can search the remaining converted code for other statements with “DCW Evaluation

Issue” and review them appropriately.

IBM Analytics


9. Deploy Objects to the dashDB Database This step involves executing the converted DDL statements against the target dashDB database.

9.1. Create a Connection to the dashDB database

i. From the dashDB Console, retrieve the connection information from Connect > Connection Settings,

on either the Connection without SSL page or the Connection with SSL page:

Host name

Port number

Database name

User ID

Password

ii. In the Database Conversion Workbench, go to the Administration Explorer tab and create a new connection.

iii. Select DB2 for Linux, UNIX and Windows as the database manager, and fill out the connection

information in the Properties section. Click Finish.

IBM Analytics


9.2. Executing the converted DDL on dashDB from DCW

9.2.1. Pre-requisites

Ensure that you have extracted the DDL statements from the source database.

Ensure that the Convert DDL function has been executed on the original DDL file.

If applicable, ensure that you reviewed and fixed any incompatibilities highlighted in the Compatibility Evaluation Report.

Ensure that the target database has been created and it is set up properly.

Ensure you have created a database connection as described on section 9.1.

9.2.2. Execute the DDL statements

i. Using the Project Explorer view, navigate to your DCW project.

ii. Double click on any SQL file to open the SQL Editor.

1. Use the “Click here to connect to a database” link at the top of the editor and select the target

database. 2. Switch to the “Validation” tab at the bottom of the SQL Editor and change the statement

terminator to ‘@’. 3. Go back to the “Editor” tab and click on the green play button on the top right corner to run the

DDL.

4. The results and status are displayed below the SQL Editor in the SQL Results tab.

iii. Alternatively, you can use the DCW Run SQL Files… command to execute multiple files at once.

IBM Analytics


1. Select the SQL file(s) that you want to run. You can also select the folder to run all of the SQL files in a folder.

2. Right-click and navigate to Database Conversion > Run SQL Files…

3. Set your Statement delimiter to ‘@’. Click Finish.

4. Select the Connection profile of your target database. Click Finish. 5. The scripts execute.

9.3. Execute the Converted DDL from the dashDB Console

You also have the option to execute the converted DDL directly from the dashDB

IBM Analytics


Appendix 1 - Data-Type Conversion Table The following table summarizes the PureData System for Analytics data types that are supported on dashDB, as well as the data types that are automatically converted by DCW. Data types that are not supported must be manually converted to a dashDB supported type before data movement can be done.

PureData System for Analytics Data Type IBM dashDB Data Type

BYTEINT, INT1 Converted to SMALLINT

BIGINT, INT8 BIGINT

BOOLEAN, BOOL Converted to CHAR(1)

CHAR(n), 0 <= n <= 254 CHAR(n)

CHAR(n), 255 <= n <= 32672 Converted to VARCHAR(n)

CHAR(n), n > 32672 Converted to VARCHAR(32762)

DATE DATE

DECIMAL, NUMERIC Converted to NUMERIC(18)

DECIMAL(p, s), NUMERIC(p, s), 1 <= p <= 31 NUMERIC(p, s)

DECIMAL(p, s), NUMERIC(p, s), p > 31 Converted to NUMERIC(31)

DOUBLE PRECISION DOUBLE PRECISION

FLOAT(p), 1<= p <= 6 REAL

FLOAT(p), 7<= p <= 15 DOUBLE PRECISION

INTEGER, INT4 INTEGER

INTERVAL Not supported

NCHAR(n), 1 <= n <= 63 NCHAR(n)

NCHAR(n), 64 <= n <= 8168 Converted to NVARCHAR(n)

NCHAR(n), n > 8168 Converted to NVARCHAR(8168)

NVARCHAR(n), 1 <= n <= 8168 NVARCHAR(n)

NVARCHAR(n), n > 8168 Converted to NVARCHAR(8168)

REAL REAL

SMALLINT (INT2) SMALLINT

ST_GEOMETRIC Not supported

TIME TIME

TIME WITH TIME ZONE, TIMETZ Not supported

TIMESTAMP TIMESTAMP

VARBINARY(n), 1 <= n <= 32762 VARBINARY(n)

VARBINARY(n), n > 32762 Converted to VARBINARY(32762)

VARCHAR(n), 1 <= n <= 32672 VARCHAR(n)

VARCHAR(n), n > 32672 Converted to VARCHAR(32672)

© Copyright IBM Corporation 2015 All Rights Reserved. IBM Canada 8200 Warden Avenue Markham, ON L6G 1C7 Canada IBM, the IBM logo, ibm.com and Tivoli are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at ibm.com/legal/copytrade.shtml Other company, product and service names may be trademarks or service marks of others. References in this publication to IBM products and services do not imply that IBM intends to make them available in all countries in which IBM operates. No part of this document may be reproduced or transmitted in any form without written permission from IBM Corporation. Product data has been reviewed for accuracy as of the date of initial publication. Product data is subject to change without notice. Any statements regarding IBM’s future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. THE INFORMATION PROVIDED IN THIS DOCUMENT IS DISTRIBUTED “AS IS” WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IBM EXPRESSLY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT. IBM products are warranted according to the terms and conditions of the agreements (e.g. IBM Customer Agreement, Statement of Limited Warranty, International Program License Agreement, etc.) under which they are provided.

http://ibm.com/legal/copytrade.shtml

Date post:	29-Apr-2018
Category:	Documents
Upload:	buixuyen
View:	239 times
Download:	0 times

IBM Database Conversion Workbench 3 -...

Documents