www.enselsoftware.com

Nucleus, Sybase IQ and Netezza for Oracle users

A brief guide of Nuclues, Sybase IQ and Netezza database for Oracle users.

Other database concept for those coming from Oracle backgroundNucleus is currently known as “Sand Analytic Server”

Category Oracle Nucleus Sybase IQ NetezzaInterfaceUsability & application

OLTP & DSS DSSOnly extensive read operation

DSSOnly extensive read operation

DSSOnly extensive read operation

SQL Plus sqlplus user/password@dbname

command terminates by ;

nisqlm connection_name dbname user/password

isql –U user_name –P password –S server

command terminates by “go” on a separate line

nzsql dbname user password

command terminates by ; or \g

GUI based admin tool

Oracle Enterprise Manager

Sand Enterprise Manager

Sybase Central Java edition

Import/export imp / exp ndlmLoading data from file

sqlldrcan only load data

ndlmcan both load and unload data to/from database

nzload

Server engine Oracle instance and background processes

Nserv, Octopus

Configuration tool Net Config Assistant nconfig Data config file Control files,

tnsnames.ora, listener.ora

nucleus.ini under /usr/nuclues

$SYBASE/interfaces

Page 1 of 18

www.enselsoftware.com

Creating a database

Use create database command in sqlplus

nconfig dbname

Starting database instance

Startup command in sqlplus

nserv connection_name instance_name

Concurrency manager?

Octopus

Starting database sqlplus /nologSQL>conn sys/pwd@db as sysdbaSQL>startup

nserv connection_name instance_name

Must be run from box where database server installed locally

start_asiq demodb.db

Stopping database SQL>shutdown Use stop_asiq or dbstop command

Database logical objects

Schema, user, table, view, PL/SQL procedure/function, package and much more

Schema, user, table, view only

Types of indices B Tree and Bitmap mainly

Both but Bitmap type is used extensively

Referential integrity

Primary/foreign keys/Not null values

Primary/foreign keys/ Not null values

Primary/foreign keys/ Not null values

Logging of operation

Redo log files (purely for recovery purpose)

Change files (often form part of data files)

Data consistency After commit/rollback

After applying changes

Querying past data Flashback query Time travel

Page 2 of 18

www.enselsoftware.com

Usually for a short period in the past

Even for a long period on the past

Backup & Restore Relevant files can be backed up OS level (physical) or using RMAN (logical)

Backup is carried at OS level only (physical).

Maintenance Complicated to maintain – usually dedicated DBA is required

Easier to manage. Dedicated DBA not essential.

Client connection Add in tnsnames.ora Create connection name in DSN using ODBC

InternalsPhysical component files

Data files, control files, Redo log files

Data files, log files

Memory processes DBWn, SMON, PMON, LGWR, CKPT etc.

Way data is stored Data is stored in data files. Redo log changes are frequently saved to data files.

Updates in Redo logs are treated as data files! Such data is merged with main data files only when user wants. Database has to be dismounted to apply redo log changes.

Data is stored in *.iq file

Logical data division

Tablespace -< data files

Dbspace -< data files (dbname.iq file)

Metadata storage System & Sysaux Catalog store,

Page 3 of 18

www.enselsoftware.com

tablespaces named as dbname.db for first physical file

Storing of records on disk blocks

A row of data is stored at one place in the disk.

Data is stored column wise (Domain concept). Unique column values are indexed in separate table and index is stored instead of data.

Transportation of database

Thru transportable table spaces but bit complex process.

Usually copying all relevant files and configuring similar database in another system is easier.

Specific examplesUnix environmental variables

$ORACLE_HOME $SYBASE

Selecting first (or random) 100 records

Select * from table where rownum < 101

Select * from schema.table with sample of 100

Select top 100 * from schema.table

Select * from schema.table limit 100

See database tables

Select * from dba_tables (or dba_views)

Select * from sys.systableShows tables & views

Seeing what SQL are running

Select * from sa_conn_activity()

View procedure code

Select * from sys.sysprocedure

Create a copy of a Create table Select * into

Page 4 of 18

www.enselsoftware.com

table new_table as select * from old_table where 1=2

new_table from old_table where 1=2

Note: if #new_table is used, it is created as temporary table which only exists in the same session

Saving data to database

commit commit [saved in session only]

commit work [saved permanently]

shutdown immediate keep change [saved permanently with entry point]

commit

Running an SQL file within SQL prompt

In SQL PlusSQL>@filename.sql

In Nisqlm SQL> .run filename.sql

To rename a table RENAME TABLE current_name TO new_name

ALTER TABLE schema.old_name RENAME new_name

ALTER TABLE current_name RENAME TO new_name

NOTE: An Oracle vs Sybase comparison is available here.http://www.rocket99.com/sybase/syb_vs_ora.html

Nucleus examples (similarities in line with Oracle)

Logging on to a database using nisqlm

Page 5 of 18

www.enselsoftware.com

nisqlm gwgr00 ltgroup0 dba/chuffer

Unloading (i.e. extracting data) a table (not all columns) using ndlm

ndlm gwgr00 ltgroup0 dba/chuffer sample_ndlm_config.ndl

The ndl file is similar to Oracle SQL Loader’s control file.

Sample ndl file for above command –

export @/usr/users/cp_dev/sbasak/nuc_extract.txt

{

NULL ""

RECORD{CAMPAIGN_ID *|CAMPAIGN_NAME *\n}select campaign_id, campaign_name from cas.campaign_history_lookup with sample of 10;}

If you are unloading all fields you can use “select *” also.

Sample ndl file for loading data into database

import @/MHDS2/dev/ltsbcp/cas/data/mfs/partitions/mfs_4way_0/load/GI_REP_FORECAST_R2L.dat@/MHDS2/dev/ltsbcp/cas/data/mfs/partitions/mfs_4way_1/load/GI_REP_FORECAST_R2L.dat@/MHDS2/dev/ltsbcp/cas/data/mfs/partitions/mfs_4way_2/load/GI_REP_FORECAST_R2L.dat@/MHDS2/dev/ltsbcp/cas/data/mfs/partitions/mfs_4way_3/load/GI_REP_FORECAST_R2L.dat

{

NULL ""

Page 6 of 18

www.enselsoftware.com

RECORD {

product *\tmedia *\tdate_sequence *\tcum_perc_total_sales *\tsum_sale_ind_keys *\tsum_sale_ind *\tcum_sales_total *\n

}CAS.GI_REP_FORECAST{

product productmedia mediadate_sequence date_sequencecum_perc_total_sales cum_perc_total_salessum_sale_ind sum_sale_indcum_sales_total cum_sales_total

}}

The above file can be called by ndlm in following fashion.

ndlm -u gwgr99_dev ltgroup_dev dba/ gen_GI_REP_FORECAST.ndl

A more efficient command example is (using parallel read/write)

nohup ndlm -f 32768 -d -b -u -w -k 4 -j 4 -e /tmp/campaign_history_adhoc_load2.err ltb99ut ltbcas_ute DBA/BLUESKY /tmp/campaign_history_adhoc_load2.ndl > /tmp/adhoc_load.log &

Describe a table (like desc table_name in SQL plus)

-- DESCRIBE A TABLESELECT

C.COLNO,C.NAME,D.TYPE,C.LENGTH

Page 7 of 18

www.enselsoftware.com

,C.DOMAINID,DN.NAME

FROMSYSTEM.TABLES T

JOIN SYSTEM.COLUMNS C ON T.TABLEID = C.TABLEIDJOIN SYSTEM.DATATYPES D ON C.DATATYPE = D.DATATYPEJOIN SYSTEM.DOMAINS DN ON C.DOMAINID = DN.DOMAINIDWHERE

T.TABLEID = (SELECT TABLEID FROM SYSTEM.TABLES WHERE NAME = UPPER('CAMPAIGN_HISTORY_LOOKUP'))AND T.SCHEMAID = (SELECT SCHEMAID FROM SYSTEM.SCHEMAS WHERE NAME = UPPER('CAS'))ORDER BY 1

;

Running an SQL command from OS command prompt

nisqlm connection_name db_name uid/pwd < filename.sql

nisqlm ccms99 ltbcms_dev dba/nucl3us < /usr/users/cp_dev/sbasak/test.sql

To exit from nisqlm, type .exit at the SQL> prompt.

How to find the dependent objects in Nucleus?

Usually most dependent objects on a table are views.You can find that information from SYSTEM.VIEWTEXT table in Nucleus.

Nserv and Octopus

In Nucleus, it is important to understand the difference between nserv and Octopus.

Nserv is the database engine. There can be only one active nserv connection to any database at any time. If you try to log in to database with another nserv session, the previous connection will be automatically disconnected. With nserv connection, you have unrestricted access to database.

Page 8 of 18

www.enselsoftware.com

With Octopus connection, you can log in to database and modify data but your changes will not be visible to other users (either nserv or another Octopus connection). With octopus, you can never make any permanent changes to the database. The advantage of using octopus is that, several users can use their own what-if scenarios with their own data – without actually affecting other users (as they won’t see this data). As a result, COMMIT WORK statement will work only on nserv connection and not on octopus connection.

Sybase IQ examples (similarities in line with Oracle)

Viewing a table structure

select t.table_id ,t.table_name ,t.table_type ,c.column_id ,c.column_name ,c.column_type ,c.pkey ,c.nulls ,c.width ,d.domain_name ,t.view_deffrom sys.systable tjoin sys.syscolumn c on t.table_id = c.table_idjoin sys.sysdomain d on c.domain_id = d.domain_idwhere t.table_name = upper('experian_omnibase')and user_name(creator) = 'CAS'

This query is similar to querying DBA_TABLES in Oracle.

To see what SQLs are currently running on database, run this command

Sp_iqcontext

Page 9 of 18

www.enselsoftware.com

To see database status (free space etc.) use

sp_iqstatus

ISQL commands sample

Log in to database as (either from database server of from client box – if available)isql -S lbgw_qa -U DBA -P SQL

Quit from isql by typing quit.

Sybase IQ architecture

Page 10 of 18

www.enselsoftware.com

IQ Store = Oracle’s logical tablespace which contains physical data file(s). In Sybase IQ, the data are actually indexes.Catalog store = Sybase IQ tables are logically defined here (i.e. table metadata) somewhat similar to Oracle’s System/Sysaux tablespace. It contains system tables and all stored procedures. Only one catalog store per database.

Page 11 of 18

www.enselsoftware.com

IQ Temporary store = Oracle’s temporary tablespace IQ Message log = Like Oracle’s alert log fileIQ Transaction log = Like Oracle’s Undo tablespace. Used for rollback/roll forward changes.

Sybase IQ Extract file (unload data from database)

SET TEMPORARY OPTION TIMESTAMP_FORMAT='YYYYMMDDHHNNSS'set temporary option temp_extract_column_delimiter='|'set temporary option temp_extract_name1 ='/tmp/TABLE_EXTRACT.dat'set temporary option temp_extract_name2 =''set temporary option temp_extract_null_as_empty='ON'set temporary option temp_extract_binary='OFF'goselect field1, field2, field3 from my_schema.mytable where x=4go

select * can also be used if all fields are required.

Sybase IQ load file (load data from database – like SQL Loader configuration file)

LOAD TABLE schema_name.table_name (field1, field2, field3 )FROM ‘/tmp/TABLE_EXTRACT.dat’DELIMITED BY '|'ESCAPES OFFLOG DELIMITED BY '|'IGNORE CONSTRAINT ALL 0MESSAGE LOG '/MHDS1/dev/sky/olive/data/admin/log/TABLE_LOAD.msg'ROW LOG '/MHDS1/dev/sky/cmd/data/admin/log/TABLE_LOAD.rej'ONLY LOG ALLNOTIFY 1000000GOCOMMITGO

The above command should load data to database.

Page 12 of 18

www.enselsoftware.com

Note: Unlike most loaders where each field in the file is separated by a field delimiter except the last field which just has the row delimiter, Sybase IQ loader expects the last field to also have the field delimiter and then followed by the row delimiter. The Unix “tr –d” command can be used to remove newline characters from load files.

Sybase Stored procedure example

This sample stored procedure will create a procedure which will count records of all tables under a given schema (user). This command can be run at isql prompt.

drop procedure get_table_summary;gocreate procedure cas.get_table_summary(in p_table_owner varchar(50) default 'gi_dev')begin declare tname varchar(100); declare row_count integer; declare x integer; declare err_notfound exception for sqlstate value '02000'; -- get all tables under current user declare c_table_details dynamic scroll cursor for select table_name from sys.systable where creator = user_id(p_table_owner); --check if the table exists select count(*) into x from systable where table_name = 'table_summary'; if x = 0 then -- create table create table cas.table_summary( table_name varchar(100) null, no_rows numeric(10) null, ) else -- table already exists message 'table already exists' to console; DELETE FROM CAS.TABLE_SUMMARY; COMMIT WORK; end if; open c_table_details with hold; -- Open the CURSOR for reading. CalculationLoop: loop -- Calculate summary for each table

Page 13 of 18

www.enselsoftware.com

fetch next c_table_details into tname; if sqlstate = err_notfound then leave CalculationLoop -- without it will be an infinite loop! end if; --select count(*) into row_count from tname; execute immediate 'select count(*) into row_count from ' || tname; insert into table_summary values( tname,row_count) ; commit work end loop CalculationLoop; close c_table_detailsend

To execute the above stored procedure from isql prompt, useexec get_table_summary(‘lb_prod’);go

If execution is successful, it will show this => (return status = 0)

To see the source code of the procedure/function, execute this

select * from sysprocedure where proc_name = 'get_table_summary'

To create index in Sybase, use this command

CREATE HG INDEX index_name ON table_name(column_name)

In Sybase, if you get “Database page size error” when trying to issue “select * from tablename”, execute this before “select * from tablename” command.

set temporary option FORCE_NO_SCROLL_CURSORS = 'ON'

It works in isql as well as 3rd party tools like WinSql.

Other Sybase caveats

Page 14 of 18

www.enselsoftware.com

The IQ doesn’t have queues for requests of changing its objects, like other RDBMSes (SQL Server, Oracle, ASE, ASA, MYSQL, DB2, etc) do. Therefore, every update/delete/insert/alter/create index to a table/view/procedure locks the object and the second transaction that’s trying to make a change, getting an error message. That’s why the IQ is not the right database to make transactions. The cache mechanism needs a database that gets many transactions from the application and can handle many update/delete/insert transactions. Therefore it uses ASA (Adaptive Server Anywhere) as a transactional database to get the changes from the application and the cache mechanism copy the new records to the IQ.

Netezza Examples

Show time – dbname(user)=> select now();

Describe a table

In nzsql prompt, type \d TABLENAME

Loading data to a Netezza table from an external text file (similar to SQL Loader)

For example, /tmp/nztest.txt file contains

100200300

Log on to nzsql (eg. nzsql lbcccms_dev cms_dev L10yds01 – it may be necessary to log on from same box where Netezza is installed or do an rsh/ssh if possible)

-- create external tableCREATE EXTERNAL TABLE ext_nztest SAMEAS nztest USING (dataobject ('/tmp/nztest.txt') compress false format 'text');-- load from external table (i.e. data file)

Page 15 of 18

www.enselsoftware.com

insert into nztest select * from ext_nztest;

If input data is tab delimited, then the tab DELIMITER ' ' character should be inserted from Unix prompt by pressing tab key(copy-paste from Windows won’t work)!

create external table X_AML_OFFER_LEVEL sameas AML_OFFER_LEVELusing ( dataobject ('/tmp/AML_OFFER_LEVEL_rollover.dat') dateStyle 'YMD' dateDelim '-' DELIMITER ' ' nullValue '' ctrlChars true ignoreZero true truncString true);insert into AML_OFFER_LEVEL select * from x_AML_OFFER_LEVEL;drop table X_AML_OFFER_LEVEL;

It can be called using this command

/nz/kit/bin/nzsql -d lbcccms_dev -f /tmp/CB10932_nz_rollover_load.sql -u cms_dev -pw L10yds01 -o /tmp/CB10932_nz_rollover_load.log

Netezza system tables

To see all usersSelect * from _v_user

To see relationship/columns of a table

select t.tablename ,t.objtype ,t.owner ,c.attname ,c.format_type

Page 16 of 18

www.enselsoftware.com

from _v_table tjoin _v_relation_column c on t.objid = c.objidwhere t.tablename = lower('gi_uss')

Sybase stored procedures

create procedure get_tables_structure(@p_user varchar(100))begin

declare tname varchar(100); declare row_count integer; declare val1 integer; declare val2 integer; declare err_notfound exception for sqlstate value '02000'; declare c_table_details dynamic scroll cursor for select t.table_name from sys.systable t where user_name(creator) = @p_user;

execute immediate 'select count(*) into val2 from sys.systable where table_name = ''summary_table'' and user_name(creator) = ''' || @p_user || '''' ; if val2 = 1 then -- table exists, so truncate execute immediate 'truncate table ' || @p_user || '.summary_table;'; else -- no table, so create execute immediate '' || 'CREATE TABLE summary_table (' || ' "table_name" char(128) NOT NULL DEFAULT NULL,' || ' "table_type" char(10) NOT NULL DEFAULT NULL,' || ' "column_id" unsigned int NOT NULL DEFAULT NULL,' || ' "column_name" char(128) NOT NULL DEFAULT NULL,' || ' "column_type" char(1) NOT NULL DEFAULT NULL,' || ' "pkey" char(1) NOT NULL DEFAULT NULL,' || ' "nulls" char(1) NOT NULL DEFAULT NULL,' || ' "width" smallint NOT NULL DEFAULT NULL,' || ' "domain_name" char(128) NOT NULL DEFAULT NULL)'; end if;

Page 17 of 18

www.enselsoftware.com

commit work;

open c_table_details with hold; CalculationLoop: loop fetch next c_table_details into tname; if sqlstate = err_notfound then leave CalculationLoop -- without it will be an infinite loop! end if; execute immediate 'select t.table_name ,t.table_type ,c.column_id ,c.column_name ,c.column_type ,c.pkey ,c.nulls ,c.width ,d.domain_name into ' || @p_user || '.temp_table1 from sys.systable t join sys.syscolumn c on t.table_id = c.table_id join sys.sysdomain d on c.domain_id = d.domain_id where t.table_name = ''' || tname || '''' ; execute immediate 'insert into ' || @p_user || '.summary_table select * from ' || @p_user || '.temp_table1'; execute immediate 'drop table ' || @p_user || '.temp_table1'; end loop CalculationLoop; commit work; close c_table_details

endgo--------------------------------------------------------------------------------exec get_tables_structure('lp_train_data');go

Page 18 of 18