Date post: | 27-Dec-2015 |
Category: |
Documents |
Upload: | daisy-norton |
View: | 244 times |
Download: | 4 times |
Internal and Confidential
COGNOS Decision Stream
Cognos CoE
April 19, 2023
Agenda
• Getting Started
• Components of DecisionStream
• Definitions
• DecisionStream Catalog
• Database Connections
• Fact Builds
• Dimension Builds
• JobStreams
• SQLTXT Designer
• SQL Term
• Command Line Interface
April 19, 2023
Modules
9. Definitions
1. Getting Started
2. Create a Catalog
5. Dimension Builds
6. Job streams
7. SQL Text Designer
8. Command Line Interface
3. Fact builds
4. Data Delivery Modules
April 19, 2023
Module 1
Getting Started
April 19, 2023
• The primary purpose of DecisionStream is to create integrated data marts by:– extracting operational data from multiple sources– merging and transforming the data to facilitate reporting and analysis– delivering transformed data into coordinated data marts
What is DecisionStream?
April 19, 2023
The Purpose of DecisionStream
Transformations
OLTP
OLTP
OLTPOLTP
OLTP
-x/y
+
Data
DataData
Integrated
Data Marts
April 19, 2023
Decision Stream Architecture
Design Client
API Scripts
PlatformsWindowsNT,95,98,
2000Desktop
DecisionStream Process Metadata
Operational Sources
DB2OracleSQL Server
InformixODBCSybaseText
MetaData Targets
ImpromptuTransformerArchitect
Microsoft SSASEssbase
Server Engine
NT or UNIXServerHPUXAIX
SolarisIBM AIX 4.2
COMPAQ TRU64 DE
CIS
ION
ST
RE
AM
DE
CIS
ION
ST
RE
AMCustom
DBMS
Flat FilesFiles
ERP
April 19, 2023
3
How DecisionStream Creates Data Marts
DimensionDelivery
5b.
4
Sales1
DimensionalFramework
Product Time LocationDim Data
Sources
2
DataSource
DataSource
Dat
aStr
eam
Dat
aStr
eam
Dat
aStr
eam
Dat
aStr
eam
MetaDataDelivery
5c.
5a.
FactDelivery
April 19, 2023
DecisionStream Interface
April 19, 2023
Interface Objects
• The core elements in a DecisionStream catalog are:
– Builds
– DataStreams
– Transformation Models
– Delivery Modules
April 19, 2023
Interface Objects (Cont’d)
• The core elements in a DecisionStream catalog are:
– JobStreams
– Dimensions
• Hierarchies and Lookups
• Templates
– Connections
– Functions
April 19, 2023
DecisionStream Tools
April 19, 2023
Module 2
Create a Catalog
April 19, 2023
What is a Catalog?
A catalog is a container for DecisionStream metadata.
April 19, 2023
• A DecisionStream Catalog provides a central repository for the information that defines how DecisionStream extracts, transforms, and delivers data.
• You can store a DecisionStream Catalog in either a dedicated database or a database that you use for other purposes, such as a target data mart.
• A Catalog consists of a set of data tables that can reside in any database for which DecisionStream has create permissions. You can create only one Catalog per database.
DecisionStream Catalog
April 19, 2023
Diagram illustrates
• How DecisionStream extracts and transforms data from the data sources and how it delivers data to the target marts
• The flow of data from the data sources to the target data marts
DecisionStream Catalog
April 19, 2023
Shared Items in a Catalog
• Library items can be used throughout the catalog in a variety of different builds and JobStreams.
• A catalog library consists of:
– Dimensions
– Connections
– Functions
April 19, 2023
Create a Catalog
You can create a Catalog in the DS_New sample database.
Steps
1. From the File menu, click New Catalog. If you are prompted to disconnect from the current Catalog, click OK.The New Catalog dialog box appears.
2. In the Name box, type MyCatalog and then click Next.3. In the left pane, click ODBC.4. In the Data Source Name box, click DS_New.5. Click Finish.DecisionStream creates the Catalog tables and opens the new Catalog.6. From the File menu, click Save Catalog.
DecisionStream Catalog
April 19, 2023
Backup and Restoration of a Catalog
• Backup files are useful for recovery and emergency situations.
• Write the catalog tables to an ASCII text file for later recovery.
April 19, 2023
Creating an ODBC Connection
Steps to define connections to ODBC databases set up for the Great Outdoors Company.
Steps
1. Open the MyCatalog catalog that you created in Chapter 2.2. In the Library, right-click Connections, then click Insert Connection.3. In the Alias box, type GO_Sales4. Click the Connection Details tab.5. In the left pane, click ODBC.6. In the Data Source Name box, click GOScer3.7. Click OK to close the Connection Properties dialog box.
Database Connections
April 19, 2023
Creating a connection to SQLTXT Database
Steps
1. In the Library, click the Connections folder.
2. From the Insert menu, click Library, and then click Connection.
3. In the Alias box, type DS_Sources
4. Click the Connection Details tab.
5. In the left pane, click SQLTXT.
6. In the Definition File box, click Browse, then select the DS_Sources.def file in the Cognos samples directory.
8. Click OK to close the Connection Properties dialog box.
Database Connections
April 19, 2023
Creating a connection to Oracle Database
Steps
1. In the Library, click the Connections folder.
2. From the Insert menu, click Library, and then click Connection.
3. In the Alias box, type DS_OracleSrc
4. Click the Connection Details tab.
5. In the left pane, click Oracle SQL*Net.
6. Enter the values for User Name, Password and SQL*Net Service.
8. Click OK to close the Connection Properties dialog box.
Database Connections
April 19, 2023
Module 3
Fact Builds
April 19, 2023
A DecisionStream fact build specifies how to extract, transform, and deliver data. Data may come from multiple data sources,
Each of which has its own naming and storage conventions.
A fact build can• acquire data from one or more data sources• merge and clean the source data• aggregate the data with reference to the dimensional framework• deliver the fact and dimension data to one or more data marts• deliver metadata to OLAP systems and Cognos BI applications
Fact Builds
April 19, 2023
Deliver
10100101…
The Fact Build Process
TransactionData
Sources
Structure Data
Sources
Acquire Star Join Merge
Transform Aggregate
PartitionFilter
Dimensional Framework
April 19, 2023
Fact Build Data Elements
Dimension
Measure
Attribute
Derivation
A dimension key or data that is used to qualify a measure.
Quantitative data or values that can be aggregated. These are usually Key Performance Indicators (KPIs).
Additional information that is not a dimension element or measure but that may be of interest.
A calculation or transformation.
April 19, 2023
Fact Build Wizard
The Fact Build Wizard guides you step-by-step through the process of creating a new fact build.
April 19, 2023
Types of Fact Builds
Fact build types provide default settings for the delivery of data marts.
April 19, 2023
The Fact Build Wizard: Add a Data Source A data source for the fact build is chosen from the available connections. This source contains the transactional data to be transformed.
April 19, 2023
The Fact Build Wizard: Create the SQL Data Source
Tables and columns to be read from
SQL statement created
April 19, 2023
The Fact Build Wizard: Map to Fact Build Elements
Elements of the Transformation Model
Columns returned from source data by SQL
April 19, 2023
The Fact Build Wizard: Modify Attributes
The elements of the Transformation Model are created automatically by the Fact Build Wizard but may need to be modified.
Elements of the Transformation Model
Click here to modify the types of elements
April 19, 2023
The Fact Build Wizard: Link to the Dimensional Framework
Dimension elements are related to reference structures.
April 19, 2023
The Fact Build Wizard: Fact Data Delivery
April 19, 2023
Fact Build Visualization
Dimensions associated with the build
Fact and Dimension tables
Metadata delivery to Impromptu, PowerPlay,and Architect
Transactional data sources
April 19, 2023
DataStream Mapping Visualization
You can visualize the mapping of the SQL columns to the build elements by clicking the DataStream tab.
April 19, 2023
Transformation Model Visualization
Use the Transformation Model tab to view the linkage of the dimension elements to the Dimensional Framework.
Mapped Reference Structures
April 19, 2023
Fact Delivery Visualization
By clicking the Fact Delivery tab, you can view the mapping of the elements to the target fact table.
April 19, 2023
Delivery Modules
• The three delivery module types are:
– fact
– dimension
– metadata
April 19, 2023
Steps to Execute the Fact Build
1. Right-click the BIMart factbuild, and then click Execute.The Execute Build dialog boxappears.
2. In the Execution mode box,click Normal, and then click OK.DecisionStream executes theFact build in a commandwindow, writing progressmessages to the screen andexecution log.
3. When the fact build executionIs complete, press Enter to closeThe command window.
Fact Builds
April 19, 2023
Run a Fact Build
April 19, 2023
Create a Fact Build Manually
Add a build1
Add and define the build elements
3
Define the source2
SQL columns Build elements
Map the elements to the columns4
April 19, 2023
Add a Dimension Element to the Fact Build
Insert
You have the option of adding the dimension element to deliveries after the fact table has been created
Associate and Set
Select to roll up to the higher level(s)
Select to include all records to avoid rejected data
Select to use the surrogate, if one is available, as a foreign key
April 19, 2023
Add a Measure to a Build
Insert and name it
Specify an aggregation method
Specify a merge behavior
April 19, 2023
What is a Derivation?
• Derivation is an element that DecisionStream calculates rather than obtains directly from the data source.
• It derives from any build element.
• It is an output-only element.
ProductRevenue = Quantity * UnitPrice
Quantity * UnitPrice
April 19, 2023
When to Use a Derivation?
• Use a derivation when:
– a calculation needs to be stored in the data mart to be shared by users
– the calculation is very complex and open to misinterpretation
– certain key performance indicators (KPIs) must be derived
– transformation is required
April 19, 2023
Operators for Derivations
Logical: =, !=
>=, <=
>, <, <>
IS
AND, OR
[NOT] LIKE
[NOT] BETWEEN
[NOT] IN
NOT ( ……….. )
Mathematical:
– ( )
– +
– -
– *
– /
April 19, 2023
General Functions for Derivations
Text
Mathematical
Logical
Type Conversion
Function Type Examples
Char(), Concat(), Trim(), Length(),...
Abs(), Floor(), Trunc(), Round(), Mod()
If(), IfNull()
ToChar(), ToDate(), ToInteger()
Date AddToDate(), SysDate(), DaysBetween()
Control
Member Member(), Level(), IsAncestor()
RowsUpdated(), Exit(), SQL(), LogMsg(),..
April 19, 2023
Control Statements
StatementsAssignment <variable> := <expression>;
Return return <expression>;
if…then…else…
if<expression>then<block>else<block>
while…do…
while <expression>do <block>
case… case <expression> ofbegin<value1>: <block>default: <block>end
<block> begin<statement>;end
Syntax
April 19, 2023
How to Define a Derivation
Test
InsertCalculate
April 19, 2023
When to Calculate a Derivation
By default , a derivation is calculated after aggregation.
You can choose to calculate the derivation before aggregation.
MEASURE MEASURE DERIVATION AGGREGATION
Revenue (Quantity*UnitCost)
Quantity * UnitCost AVGRevenue
AverageRevenue
Select the Calculate at Source box if you want to calculate first and then aggregate.
Select an aggregation method
April 19, 2023
Module 4
Data Delivery Modules
April 19, 2023
Delivery Process to a Single Data Mart
MetadataDelivery
FactDelivery
DimensionDelivery
Fact Build Delivery Modules Delivery Targets
Data Mart
April 19, 2023
Fact Delivery Modules
There are two groups of fact delivery modules.
Table delivery modules•Bulk Copy (BCP)
•DB2 LOAD
•Informix LOAD
•ORACLE SQL *Loader
•Red Brick Loader (TMU)
•Relational Table
Nontable delivery modules•Text File
•DataStream
•Essbase Direct Feed
•Essbase File Feed
April 19, 2023
Fact Delivery: Relational Table Delivery Module
• APPEND
• TRUNCATE
• REPLACE
• UPDATE/INSERT
• UPDATE
Refresh Type:
Commit Interval:
Target database Target table
Indexing options
Elements subscription for delivery
Key option1..
100 commit …200 commit……1000
April 19, 2023
Fact Delivery: Text File Delivery Module
Output Filenamefull directory path and file name or just file name.
Line/Field DelimiterNL(new line) TAB, COMMA, SPACE, NONE(no delimiter),or a character string.
Elements subscription for delivery
April 19, 2023
Dimension Delivery in a Fact Build
• In general you should not deliver dimensional data in a fact build. However, there are four exceptions to this rule:
– a demo or proof of concept
– data mart delivery from a warehouse
– dimensional data available only in fact data
– metadata delivery
April 19, 2023
Dimension Delivery Module
Add Relational Table
Add Dimension Table
Map the data set columnsto the template attributes
Create or select a template for delivery
Type or select a name for dimension table
TemplateAttributes
April 19, 2023
Metadata Delivery in a Fact Build
Provide a name and define the module properties for each metadata delivery
April 19, 2023
Metadata Delivery Modules
Specify a source and a dimension delivery for each dimension that you want to deliver
Dimension deliveries
To deliver BI metadata, you must declare dimension delivery modules for each dimension, but you do not need to deliver the dimension data.
Disable dimension deliveries if you do not want dimension data to be delivered through the fact build
Select a fact table and columns from the table that you want to deliver
April 19, 2023
Partitioned Delivery
Using DecisionStream, you can set vertical partitioning, horizontal partitioning, or both.
Horizontal PartitioningVertical Partitioning
Horizontal + Vertical Partitioning
April 19, 2023
Vertical Partitioning
Vertical partitioning is performed in a fact delivery by subscribing to the elements to be delivered.
Columns included in Fact Delivery
Columns excludedfrom Fact Delivery
April 19, 2023
Module 5
Dimension Builds
April 19, 2023
What is a Dimension Build?
A dimension build delivers data that describes a single dimension to the data mart. This may take the form of one or several tables.
Levels of the hierarchy
Reference structure to be delivered
Dimension Build
Dimension table
created Target Data Mart
April 19, 2023
A dimension build can deliver dimension data for many fact builds. Fact builds can also be used to deliver dimension data,but this may mean several fact builds delivering the same data,resulting in a duplication of effort. Dimension builds deliveronly dimension data.
The main use for dimension builds is to deliver conformed dimensions that form the dimensional reference for coordinated Data marts.
Each dimension build acquires data from a reference structure that you specify and delivers this, as dimension data, to thetarget data mart.
Dimension Builds
April 19, 2023
Set Up Reference Structures
For each reference dimension that you have set up, you must set up a hierarchy and the required templates. Each hierarchy models a view of the associated business dimension.
DecisionStream supports hierarchies based on
• the columns of one table (star schema)
• multiple tables (snowflake schema)
• the rows of one table (parent-child schema)
• parent-child structures without named levels (auto-level hierarchies)
Dimension Builds
April 19, 2023
Creating the Product Hierarchy
You create the Product hierarchy fromrelated tables in the GO_Sales database.The hierarchy follows a one-to-manyrelationship between tables.
A product line can have many producttypes, but a product type can haveOnly one product line. Each producttype can have many products, but aproduct can have only one producttype.
Dimension Builds
April 19, 2023
Steps to Select the Hierarchy Structure
1. From the Tools menu, clickHierarchy Wizard.
2. Click Create the hierarchy frommultiple tables (Snowflake Schema),and then click Next.
Steps to Define the Hierarchy
1. In the Enter the name of thehierarchy box, type Product
2. In the Select the referencedimension to use for this hierarchybox, click Product, and then clickNext.
Dimension Builds
April 19, 2023
Steps to Define the All Level
DecisionStream provides the option of creating astatic All level at the top of the hierarchy. • Click Next to accept the defaults and create the Alllevel.
Steps to Define the ProductLine Level
1. Click the ALLProduct level of the Producthierarchy, and then click Add. The Level Details window appears.2. In the Name box, type ProductLine3. In the Source database box, click GO_Sales.4. In the Source table box, click Browse for table, andthen double-click the ProductLine table.5. Move each table column from the Availableattributes pane to the Chosen Attributes pane.6. For the ProductLineCode attribute, select the Idcheck box. 7. For the ProductLine attribute, select the Captioncheck box and Click OK.
Dimension Builds
April 19, 2023
Steps to Define the ProductType Level1. Click the ProductLine level of the Product hierarchy, and then click Add.2. In the Name box, type ProductType and In the Source database box, click GO_Sales.4. In the Source table box, click Browse for table, and then double-click the ProductType table.5. Move each table column from the Available attributes pane to the Chosen attributes pane.6. For the ProductTypeCode attribute, select the Id check box. 7. For the ProductType attribute, select the Caption check box.8. For the ProductLineCode attribute, select the Parent check box. When you execute a build,DecisionStream creates links between a level and its parent by matching the Parent attribute of thecurrent level with the ID attribute of the level above. Then Click OK.Steps to Define the Product Level1. Click the ProductType level of the Product hierarchy, and then click Add. 2. In the Name box, type Product and In the Source database box, click GO_Sales.4. In the Source table box, click Browse for table, and then double-click the Product table.5. Move the ProductNumber, ProductName, and ProductTypeCode table columns from theAvailable attributes pane to the Chosen attributes pane.6. Select the Id check box for ProductNumber and Caption check box for ProductName
attribute.7. For the ProductTypeCode attribute, select the Parent check box and Click OK.
Dimension Builds
April 19, 2023
Dimension Builds
Steps to View the Product Hierarchy
Visualization
• In the Tree pane, click the Product
hierarchy.
The Visualization pane should look Like this:
The next task is to create the Product
dimension build and use this to
Deliver The Product hierarchy.
April 19, 2023
The Dimension Build Wizard
The Dimension Build Wizard guides you step-by-step through the process of creating a new dimension build.
April 19, 2023
Steps to Create a Dimension Build
1. From the Tools menu, click Dimension BuildWizard. The Dimension Build wizard appears.2. In the Name box, type Product and then click Next. 3. In the Schema Type box, click Star Schema.5. In the Dimension to be delivered box, click Product.6. In the Reference item to be delivered box, clickProduct (H).7. In the Deliver into connection box, click DS_Marts,and then click Next.8. Click Next to accept default naming conventions.9. Click Next to accept the default build features.10. Click Next to accept the default build properties.When using conformed dimensions in a data mart, itis a good idea to use surrogate keys rather thanbusiness keys to reference the dimensions. 11. Select the Add surrogate keys to thedimension tables check box, and then click Next.12. Click Finish to close the Dimension Build wizard.
Dimension Builds
April 19, 2023
Steps to View the Dimension Build
• In the Tree pane, click the Product dimension build.
The Visualization pane should look like this:
Note that DecisionStream has created a template,
D_Product, which is used the deliver the primary
reference structure.
Steps to Deliver the Dimension Build
1. Right-click the Product dimension build, and
then click Execute.
The Execute dialog box appears.
2. In the Execution mode box, click Normal.
3. Click OK.
DecisionStream executes the dimension build in a
command window.
4. When the dimension build execution is complete,
press Enter to close the commandwindow.
Dimension Builds
April 19, 2023
Dimension Build Schemas
• One dimension table:
– Star schema
– Parent-child schema
• Several dimension tables:
– Snowflake schema
– Optimal star schema
– Optimal snowflake schema
April 19, 2023
Star Schema
Date• Order Date• Week• Month• Year
Order Fact• Customer Cd• Sales Rep Cd• Product Cd• Order Date• Order_Qty• Order_Line_Value
Product Name
Product• Product Cd•• Product Type Cd• Product Type Desc• Product Line Cd• Product Line Name
Sales Staff• Sales Rep Cd• Sales Rep Name• Sales Area Cd• Sales Area Desc
Customer• Customer Cd• Customer Name• Customer Type Cd• Customer Type Desc
CustomerType
Customer
SalesArea
SalesRep
OrderHeader
OrderLine
ProductLine
ProductType
Product
April 19, 2023
Snowflake Schema
Order Fact• Customer Cd• Sales Rep Cd• Product Cd• Order Date• Order_Qty• Order_Line_Value
Customer Type• Customer Type Cd• Customer Type Desc
Customer• Customer Cd• Customer Name• Customer Type Cd
Sales Rep• Sales Rep Cd• Sales Rep Name• Sales Area Cd
Sales Area• Sales Area Cd• Sales Area Desc
Product• Product Cd• Product Name• Product Type Cd
Product Line• Product Line Cd• Product Line
Name
Date• Order Date• Week• Month• Year
A time-based dimension is typically not snowflaked
Product Type• Product Type Cd• Product Type Desc• Product_Line Cd
April 19, 2023
Parent-Child Schema
•
••••
Customer••Customer Name• Customer Parent
Order Fact•••••Order Qty•Order Line Value Product
••Product Name•Product ParentOrganization
••Sales Rep Name•Sales Rep Parent
Not a parent-child relationship
Date• Order Date• Week• Month• Year
Order Fact• Customer Cd• Sales Rep Cd• Product Cd• Order Date• Order_Qty• Order_Line_Value
Product• Product Cd• Product Name• Product Parent Cd
Sales Staff• Sales Rep Cd• Sales Rep Name• Sales Rep Parent Cd
Customer• Customer Cd• Customer Name• Customer Parent Cd
April 19, 2023
The main tasks required to define and use a Conformed dimension are
. Create a primary reference structure.
. Deliver the primary reference structure using a dimension build.
. Create a secondary reference structure using the dimension data delivered for the primary reference structure.. Associate the secondary reference structure with a fact build.
Dimension Builds
April 19, 2023
Module 6
Job Streams
April 19, 2023
Job Control and the Data Warehouse
The basic processes of the data warehouse require automation for the warehouse to function effectively.
Raw Data
Automation of:• Data Extraction• Data Transformation• Data Loading• Exception/Error Handling• Logging/Notification
Target Data Mart
April 19, 2023
What Is a JobStream?
• A JobStream is a DecisionStream component used to coordinate data mart management tasks.
• The JobStream Visualization pane helps to understand the JobStream workflow.
April 19, 2023
Characteristics of JobStreams
• A JobStream is associated to a single catalog.
• Nodes are processed in series or in parallel.
• A JobStream is aware only of objects in its catalog but can call system commands (for example, databuild.exe) that can reference other catalogs.
Nodes within a single JobStream
Catalog-specific JobStreams contained in separate folder
April 19, 2023
Add a JobStream to a Catalog
JobStreams are added to a catalog and are specific to that catalog.
Enter a name and other properties for the JobStream
April 19, 2023
Add Variables to a JobStream
Define variables to use when creating procedure, condition, and SQL nodes.
The COUNTER JobStream variable is used in the Counter Test node
April 19, 2023
Main Components of a JobStream
a fact or dimension build to be executed
tests conditions for branching to other nodes
functions, variables, or actions to be performed
SQL statement(s) to be executed
a container for one or more nodes that can include nested jobs
Build
Condition
Procedure
SQL
Job
April 19, 2023
Build Node
Add a fact or dimension build node to a JobStream to automate the extract-transform-load process.
Fact build that uses conformed dimensions
Conformed dimension builds
April 19, 2023
SQL Node
Use an SQL node to include SQL statement(s) to be executed during the job control process.
SQL statements to create indexes on a fact table previously loaded by the JobStream
Database referenced by the SQL
April 19, 2023
Procedure Node
Use a Procedure Node to coordinate processing around a build.
The Execution Log procedure writes a message to the log file using the LogMsg function
April 19, 2023
Result Variable
• All job nodes (except condition nodes) set the Boolean variable RESULT to indicate success or failure.
• Condition nodes check RESULT by default.
• A node may deliver its result to another variable if specified.
April 19, 2023
Condition Node
Use a condition node to test a condition before executing another JobStream component .
If the condition specified in Counter Test is False, then End Automation Report executes
April 19, 2023
Job Node
Use a job node to break large jobs into a series of smaller jobs.
If the FactBuild job node is reached, a separate process will run that contains its own logic
April 19, 2023
Link JobStream Nodes
Each node in a JobStream process must be linked to at least one other node.
The Drop Indexes SQL node will be completed BEFORE the SalesFact build node that it is linked to
April 19, 2023
Execute Nodes Sequentially or In Parallel
These nodes are run in parallel These nodes are run sequentially
April 19, 2023
Action on Failure
You can specify what DecisionStream does when a node fails.
CONTINUE: Continue processing the next node
TERMINATE: Terminate the current flow--continue with the next flow
ABORT: Stop ALL processing immediately
April 19, 2023
Run a JobStream
April 19, 2023
Module 7
SQL Text Designer
April 19, 2023
You use the SQLTXT Designer to configure theSQLTXT definition files used withinDecisionStream to access text files as thoughthey are in an SQL database.
SQLTXT Definition FilesA SQLTXT definition file (.def) defines a virtualdatabase of text file tables. It associates eachTable with a text file in the database directory. Italso specifies the structure of the tables.
Steps to create a SQLTXT definition file
1. In the SQLTXT Designer, from the File menu, click New.2. From the File menu, click Save As.3. In the File name box, type a name for the file.Note: The definition file must have the extension .def.4. Click Save.
SQLTXT Designer
April 19, 2023
Steps Import Delimited Records with DelimitedColumns
1. Open the appropriate SQLTXT database.2. From the Edit menu, click Import Table Data.The Import SQLTXT Table Data File dialog box appears.3. Select the text file to import, and click Open.4. Click Delimited records.5. In the Record delimiter box, click the delimiterused to separate individual records in the file.6. In the Number of header rows box, enter thenumber of header rows in the file. The SQLTXTDesigner uses these header rows to name thecolumns.7. In the Decimal symbol box, type a singlecharacter that is used for the decimal point innumeric values.
SQLTXT Designer
April 19, 2023
8. In the Character set box, enter the encodingused in the text file.9. In the Digit grouping symbol box, type thecharacter that is used for the thousandsseparator in numeric values. 10. In the Preview rows box, enter the number ofrows that you want to view in the wizard.11. Click Next.12. Select Delimited columns.13. In the Delimiter type box, click the delimiterused to separate the columns in the table.14. In the Quote character box, click the quotationcharacter used to enclose individual columnswithin a record.15. Click Next.
SQLTXT Designer
April 19, 2023
DecisionStream automatically names andattempts to specify the data type of each column.You can override these selections.
16. To rename a column, click the column nameand type a new name.17. To change a data type, double-click theappropriate column, and in the Data type dialogbox, click a different data type. If you selectNUMBER, you must also specify the number ofdigits expected after the decimal point in theScale box. If you select DATE, you must alsospecify the format of the date in the Format box.18. Click OK to close the Data type dialog box.19. Click Process All Records to ensure that allrecord lengths and data types are correct.20. Click Finish. The wizard adds the table to thedatabase, and the SQLTXT Designer appears.
SQLTXT Designer
April 19, 2023
Steps to define a table manually in SQLTEXT Designer:
1. Click the Database icon. 2. From the Edit menu, click Insert Table. An empty
table appears in the tree.3. Click the new table and, from the Edit menu, click
Rename.4. Type the new name for the table.5. In the Header lines text box, type the number of
header rows present in the file.6. In the Trailer lines text box, type the number of
trailer lines in the file. SQLTXT ignores these rows when it accesses data
in the file.7. In the Character set box, enter the encoding used
in the text file. You can click an encoding type from the list, or type
the code page into the box.
SQLTXT Designer
April 19, 2023
8. In the Action on row error box, click the action to apply when faulty data is read.
9. Select or clear the Remove trailing whitespace from fields check box as appropriate.
10. Specify whether columns in the input text file are of fixed length or delimited .
11. In the Record delimiter box, click the delimiter used to separate individual records.
12. If you selected Delimited records, in the Column delimiter box, specify the delimiter used to separate the columns in the table.
13. If you selected Delimited records, click the quotation character used to enclose individual columns within a record.
14. If you selected Delimited records, select the Ignore extra columns check box to ignore any excess fields.
SQLTXT Designer
April 19, 2023
Adding Columns to the table manually:
1. Click the table to which you want to add a column.
2. From the Edit menu, click Insert Column.A new column appears beneath the selected table.
3. In the Data type box, click the data type for the column, and specify appropriate length for that column.
4. If the column can accept null values, select the Nullable check box.
5. If the column forms part of the primary key of the table, select the Key check box.
SQLTXT Designer
April 19, 2023
Creating an SQL Query (SELECT Statement)
1. From the Tools menu, click SQLTerm.2. In the box at the top of the SQLTerm window,click the database with which you want to work.3. Use the Cognos SQL check box to specifywhether you want to use the native SQL orCognos SQL for the database you are accessingwhen you construct the SQL statement.Select or clear the check box as appropriate.
Tip: Using Cognos SQL makes it easier to port your project between database platforms.
Note: The default for the Cognos SQL check boxis determined by whether you selected theCognos SQL check box in the ConnectionProperties dialog box.
SQL Term
April 19, 2023
4. In the Database Objects pane, expand the tree,and then click the table or column for whichyou want to add a SELECT statement.5. To generate the SQL statement use one of thefollowing methods:• Right-click, and then click either Add TableSelect Statement, or Add Column Select Statement.• Click the Source SQL tab. Hold down Ctrl anddrag the table or column you selected inthe Database Objects pane to the Source SQL tab.DecisionStream clears any existing query thatmay be on the Source SQL tab and generates theSQL SELECT statement for the selected table orcolumn.6. You are now ready to execute the SQLstatement.
SQL Term
April 19, 2023
Module 8
Command Line interface
April 19, 2023
Executing a Fact build from a DecisionStream Catalog
DATABUILD -c<catdb_driver> <catdb_items> <databuild_name>[<options>]
Where
<catdb_driver> The database driver that DecisionStream must use to connect to the
catalog. <catdb_items> The connection string that DecisionStream must pass to the
database driver to connect to the catalog. <databuild_name> The name of the build that
you want to execute. When you invoke DATABUILD with this syntax, it executes the named
build that resides in the named catalog.
Example
DATABUILD -cODBC "DSN=SalesCat;UID=Sales;PWD=ds123" US_Sales
This command invokes DATABUILD to execute the build named US_Sales. To retrieve the
Build definition, DATABUILD connects to the ODBC data source named 'SalesCat' as user
Sales', with password 'ds123'.
Command Line Interface
April 19, 2023
To Execute a Dimension Build from a Catalog
DIMBUILD -c<catdb_driver> <catdb_items> <dimbuild> [<options>]Where<catdb_driver> The database driver that DecisionStream must use to connect to thecatalog.<catdb_items> The connection string that DecisionStream must pass to the databasedriver to connect to the catalog. For example, to connect as user 'DBA', with password
'PASS',to a catalog that resides in an ODBC data source named “Sample”, you would specify:"DSN=Sample;UID=DBA;PWD=PASS"<dimbuild> The name of the dimension build that you want to execute.Example
DIMBUILD -cODBC "DSN=SalesCat;UID=Sales;PWD=ds123" PeriodThe command above invokes DIMBUILD to execute the dimension build named Period. Toretrieve the dimension build definition, DIMBUILD connects to the ODBC data source
named'SalesCat' as user 'Sales', with password 'ds123'.
Command Line Interface
April 19, 2023
RUNDSJOB
This command executes a JobStream from a DecisionStream catalog.
Syntax
RUNDSJOB <catdb_driver> <catdb_items> <jobstream> [<options>]<options> ::= [-e<runenv>] [-j<subjob>] [-A<dbalias_file>] [-C][-L<logfile>] [-P] [-O] [-R] {-V<name>[=<value>]}Where<catdb_driver> The database driver that DecisionStream must use toconnect to the catalog that contains the JobStream.<catdb_items> The connection string that DecisionStream must passto the database driver to connect to the catalog. For example, to connect as user 'DBA',
withPassword 'PASS', to a catalog that resides in an ODBC data source named “Sample”, youwould specify:"DSN=Sample;UID=DBA;PWD=PASS"<jobstream> The name of the JobStream that DecisionStreammust execute.-e Denotes the run environment for the JobStream.
Command Line Interface
April 19, 2023
RUNDSJOB – Continue …<runenv> The run environment that DecisionStream must apply to identify the correctdatabase alias.-j Denotes the subjob of the JobStream that DecisionStream should execute.<subjob> JobStreams can nest jobs. This option enables you to execute a job that is
nestedwithin the JobStream.-A Denotes the text file that contains the database alias definitions that DecisionStreamshould use for the JobStream.<dbalias_file> The name and path of the text file that contains database aliasdefinitions that DecisionStream must use for the build.-C Executes the JobStream in check-only mode. That is, DecisionStream executes theJobStream but does not deliver any data.-L Causes DecisionStream to log JobStream progress to the specified file.<logfile> The name and path of a text file to which DecisionStream should log build progress.-P Causes DecisionStream to pause at the end of JobStream execution and to prompt for
akeystroke before terminating.
Command Line Interface
April 19, 2023
RUNDSJOB – Continue …
-O Causes DecisionStream to execute any builds in the JobStream in object creation mode. That is, DecisionStream executes the builds and creates any required fact and dimension tables and metadata structures, but does not deliver data to them.
-R Causes DecisionStream to restart the JobStream at the point of a previous interruption the JobStream restarts at the earliest node that did not successfully complete.
-V Denotes that the parameter is a named variable and value pair.
<name> The name of a variable that DecisionStream must use during item substitution.
<value> The value that the corresponding named variable takes.
Example
RUNDSJOB ODBC "DSN=Sales;UID=admin" example_job -jbuild_fiscal -R
This example invokes RUNDSJOB to restart the subjob named 'build_fiscal' that is nested in
The job named 'example_job'. The DecisionStream catalog resides in the ODBC data sourceNamed 'Sales', to which DecisionStream connects as user 'admin' with no password.
Command Line Interface
April 19, 2023
CATEXPThis command exports a catalog item to a DecisionStream file-based specification.SyntaxCATEXP <catdb_driver> <catdb_items> <export_file> <comp_type>
[<comp_name>] [-a] {-V<name>[=<value>]} <comp_type> ::= A|B|C|D|F|H|J|L|S|T
Where<catdb_driver> The database driver that DecisionStream must use to connect to thecatalog.<catdb_items> The connection string that DecisionStream must pass to the databasedriver to connect to the catalog. For example, to connect as user 'DBA', with password
'PASS',to a catalog that resides in an ODBC data source named 'Sample', you would specify:"DSN=Sample;UID=DBA;PWD=PASS“ <export_file> The path and name of the fileto which you want to export the catalog component. <comp_type> The type of component that you want to export.
Command Line Interface
April 19, 2023
CATEXP – Continue …
-a Causes DecisionStream to append the exported component to the named file.
<name> The name of a variable.
<value> The value that the associated variable takes.
<comp_type> The type of component that you want to export. The comp_type
values have the following meanings:
Command Line Interface
April 19, 2023
CATIMPThis command imports a text-based component definition into a DecisionStream
catalog.
Syntax
CATIMP <catdb_driver> <catdb_items> <import_filename> <filetype> [<comp_name> [<caption>[<notes>]]] [-d<dbalias>][-o][-I] {-V<name>[=<value>]} <filetype> ::= B|C|D|F|J|R|SWhere<catdb_driver> The database driver that DecisionStream mustuse to connect to the catalog.<catdb_items> The connection string that DecisionStream must pass to the databasedriver to connect to the catalog. For example, to connect as user 'DBA', with password
'PASS',to a catalog that resides in an ODBC data source named 'Sample', you would specify:"DSN=Sample;UID=DBA;PWD=PASS“ <import_filename> The path and name of the text file that defines the component.<filetype> The type of component that you want to export.
Command Line Interface
April 19, 2023
CATIMP – Continue…
<comp_name> The name of the individual component that you want to import.
This is optional for reference structures, aliases, and functions, but mandatory
for other component types.
<caption> An optional caption, or business name, that DecisionStream should
use for the item for display or presentation purposes.
<notes> Optional notes that appear within DecisionStream Designer.
Command Line Interface
April 19, 2023
CATIMP – Continue…
Example
CATIMP ODBC "DSN=SalesCat" SalesBld.bld B US_Sales "US Sales -o
This command invokes CATIMP to import the build US_Sales from the text file
Sales.bld, into the catalog that resides within the ODBC data source named
'SalesCat', overwriting any build with the same name.
Within the catalog, CATIMP gives the build US_Sales the caption 'US Sales'.
CATIMP connects to the named data source as the default user, with no password.
Command Line Interface
April 19, 2023
CATIMP – Continue…
Order of Import
Because DecisionStream validates objects that reside in a catalog, you must import text
Definitions in dependency order. For example, do not import a build until the catalog contains
the collections and all the aliases and hierarchies that it uses. Normally, you should import
text definitions in this order:
1. Aliases
2. User-defined functions
3. Dimension table templates
4. Reference structures (hierarchies and lookups)
5. Collections
6. Builds and dimension builds
7. JobStreams
Command Line Interface
April 19, 2023
Definitions
• Build
This is a DecisionStream Unit of work that goes through the steps of Acquisition, Transformation, and Delivery. There are two types of build, fact builds and dimension builds.
• Collection
When DecisionStream is driven from a text file specification, the transformations are split into a build and a collection. The build deals with the delivery of data and the collection deals with the sources of data and the transformation process.
• Component
A fact build, dimension build, collection, hierarchy, lookup jobstream, user-defined function, template or connection in a catalog.
April 19, 2023
Module 9
Definations
April 19, 2023
Definitions
• Condition Node
A condition node provides a branching mechanism between nodes for conditional execution. Each condition node can have many nodes linking to it, but only two output links (True, and False).
• Connection
All the information required to make a connection to a database. The actual information required varies according to DBMS and connection method.
• Data Mart
A database that holds data for analysis. It usually implies that the data and its organization is of limited scope or is targeted at a particular group of users and business subjects, for example orders.
• Data Source
Somewhere from which the DecisionStream can acquire data.
April 19, 2023
Definitions
• DataStreamA DecisionStream object that gathers together all the data sources from which the build or reference structure acquired data. Data streams merge these sources to present a consider whole to other DecisionStream objects.
• DesignerDecisionStream Designer is the Windows software that you use to produce build specifications and Dimensional Frameworks.
• DimensionA Key Component of a business by which you want to analyze your data. For example, the products that your company sells and the retail outlets in which these products are sold.
• Dimension BuildA Decision Stream build process that delivers only dimension data.
April 19, 2023
• Dimension TemplateA dimension template lists the columns that may exists in a table of dimension data, together with the behaviour of each column.
• Domain TypeIt is the domain that is used to deliver dimension data at the end of a build. The two types are Dynamic ( where the domain is build up from data sources as rows are processed) and Reference (where the reference object associated with the dimension is taken as the domain)
• FactbuildA DecisionStream build that delivers fact data. May optionally deliver dimension data, and metadata, allowing a private datamart to be built from within a single build.
• Foster ParentAn artificially introduced member that acts as a parent for members that either have no defined parent or whose defined parent cannot be found at the next hierarchy level.
Definitions
April 19, 2023
• JobStreamA DecisionDtream object that combines other DecisionStream object to form a logical process.
• Job NodeA job node is actually a JobStream within a JobStream. This allows larger jobs to be broken down into a series of smaller jobs. When DecisionStream encounters a job node it processes all the steps within the sub job, and only when these steps are complete, does it move on to the next node in the sequence.
• Leaf NodeIn a hierarchy, a member that has no children.
• Lookup
A simple reference object that consists of members that are not arranged into hierarchical levels.
Definitions
April 19, 2023
• MappingThe relationship between columns (from an SQL query) and the DecisionStream objects to which they relate. DecisionStream has mappings that relate data sources to transformation model elements and structure data columns to level attributes.
• Message FrequencyThe rate at which DecisionStream generates messages as it executes a build. Input message frequency refers to how many data rows DecisionStream acquires between messages. Output message frequency refers to how many data rows DecisionStream delivers between messages.
• MetadataData that describe the delivered data. Typically, metadata defines the data structure and the domain of each dimension. It can also provide aliases that reporting tools can use to provide meaningful descriptions.
Definitions
April 19, 2023
• Never OutputAn indication on a transformation model element that DecisionStream must not make the element available for delivery.
• Object Creation ModeA build execution mode in which fact tables, dimension tables, and metadata are created but no fact or dimension data is delivered.
• Output FilterA filter that can be specified in a delivery module to restrict delivery of rows to the target system. The output filter takes any expression that evaluates to TRUE or FALSE.
• Output LevelA level of hierarchy at which DecisionStream will output data. Output data may be generated by aggregation.
Definitions
April 19, 2023
• PivotTo move a dimension from on axis to another. For example, consider a relational table that has a single column (called product) that holds all product values. Assume that this has one column fro each business measure (units, revenue, etc.) The product column could be pivoted to produce one column for each product and the business measures pivoted to produce one row for each measure.
• Procedure NodeA procedure node can contain one or more DecisionStream functions or statements. This node can be used to coordinate processing around builds for such activities as checking for input files, sending mail messages and alerts, and generating custom logging and auditing messages.
• Ragged HierarchyA hierarchy that has leaf nodes at more than one level. Also called an unbalanced hierarchy.
Definitions
April 19, 2023
• Reference ExplorerA DecisionStream object that gathers together all reference structures that relate to a particular business dimension.
• Reject FileA text file in which DecisionStream stores records that the engine rejects. In these files, each rejected record has the reason for rejection as its first field.
• Sample RateWhen reading from a data source, DecisionStream can apply a sample rate so that it does not process every row. For example, a sample rate of 10 results in DecisionStream processing every tenth row and ignoring the intermediate rows.
• Slowly Changing DimensionA business dimension in which one or more non-key values may change without a corresponding change in the key values.
Definitions
April 19, 2023
• SQL NodeUse an SQL node to enter a single SQL statement that DecisionStream performs when the SQL node is reached in the JobStream flow. DecisionStream automatically commits the successful SQL statements, but roll back if the statement fails.
• SQLTermA terminal monitor application that provides a common interface to the various relational database products that DecisionStream supports.
• SQLTXT DesignerIt is a TOOL which enables text files to be described in a way that subsequently allows SQLTerm to access the data that the text files hold.
• TransformationThe re-expression of a stream of data in some different format. Typically the merging and aggregation of source data to provide delivery elements.
Definitions
April 19, 2023
• Star SchemaA star schema represents the dimension in a single table. Each level of the delivered hierarchy is represented by one or more columns of this table. In order to use the Dimension Build wizard to create a dimension build that supports slowly changing dimensions, you must use this schema type.
• Surrogate KeySurrogate keys are numeric keys that can replace business keys in the context of a data mart. Whereas the natural ID may be text-based, the surrogate key is always an integer.
• Tree PaneThe left pane of the Designer window where DecisionStream arranges the objects of the catalog in a tree-like hierarchical fashion.
• Transformation ModelCollection of dimension, measure, attribute and derivation elements in a fact build.
Definitions
April 19, 2023
Thank YouThank You