© 2013 IBM Corporation

®

Tivoli Common Reporting Best Practices

Dan Krissell, IBM TCR Architect

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareOverview

2

v3v2 Applicable to TCR 2.x Applicable to TCR 3.1

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareInstalling TCR v3.1

3

Unofficial installation walk through– LINK

Use the JazzSM custom installation mode

Only install the components you need

v3

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwarePrereq scanner

Don’t continue the installation until the prereq scanner passes

Common failed prereqs include:– Missing Linux system RPMs – Open file handlers set too low

Downloadable off the web (first link):http://pic.dhe.ibm.com/infocenter/tivihelp/v3r1/index.jsp?topic=

%2Fcom.ibm.tivoli.tcr.doc_211%2Frtcr_soft_and_hard_reqs.html

Part of the product:<JazzSMImage>/PrereqScanner/TCR.sh

4

v3v2

v2

v3

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareDon’t use Derby

TCR v2 uses Derby out of the box to house reporting artifacts– Fine for demos and POCs– Performance bottleneck, not enterprise– Not supported in production

Moving off Derby:– Normal case (Derby > 1GB):

http://pic.dhe.ibm.com/infocenter/tivihelp/v3r1/topic/com.ibm.tivoli.tcr.doc_211/ttcr_reconf_db.html

– Large Derby:https://www.ibm.com/developerworks/mydeveloperworks/files/form/anonym

ous/api/library/9641dcf4-c5b8-413c-8ae8-9c461dd84a09/document/0cdff487-dee6-4070-941c-f40ed8d0afd1/media/Exporting-Importing%20Derby%20Content%20Store_v01.pdf

5

v2

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareWAS Java Heap setting

Max heap is set to 1024 MB out of the box

Better value is 1843 MB (32 bit, v2.x) and 2-4 GB (64 bit v2.1.1 & 3.1)

http://www-01.ibm.com/support/docview.wss?uid=swg21585678

http://pic.dhe.ibm.com/infocenter/tivihelp/v3r1/topic/com.ibm.psc.doc_1.1.0/admin/psc_t_admin_jvm_memory.html

Also, there is an Administration Services task to update the heap from the web UI

6

v3v2

v2

v3

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareUDA temp files

Created during report generation for local processing (non-database processing)

Located at <cognos>/temp/UDA*

Can get very large, monitor the directory.

Should get cleaned up by Cognos. But sometimes files are abandoned, for example due to abnormal ending. OK to delete them when the server is stopped.

If they become excessive it maybe worth narrowing down which report(s) are active then the files become hug and investigate those reports for unneeded local processing

7

v3v2

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareSort Buffer

Queries not processed entirely in the database may require local processing that involves sorting

This is done in memory, with overflows into temporary disk storage. Defaults to a small value of 4MB per query (process serves multiple

queries) This is the same 32 bit native process (BIB\usTKServerMain) is the

reporting engine, so you can’t just bump it way up. Increase the value slowly, while watching the BIBusTKServerMain

memory usage

tcr_cogconfig.sh Or, Windows Start menu

8

v3v2

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareProcessors

By default, two report engine processes run, when required– Each engine can run multiple queries

If reports are backed up in the queue waiting to be run and IF your reporting server has CPU and memory to spare, you can increate the number of reporting engines– For checking the queue, see a chart later in this deck

– A rough formula, assuming you have memory for each new 32-bit reporting engine (~ 2GB), is: # of CPU’s X 2 = Max number of reporting servers

9

v3v2

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareProcessors, continued

http://www.ibm.com/developerworks/data/library/cognos/page61.html

10

v3v2

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareLoad Balancing TCR Servers

Easier to configure in v3 Be sure TCRs are at the same version

Consists of first moving off Derby, then pointing several TCRs to a common database

Doc shows separating frontend/backend, this is not required for load balancing

http://pic.dhe.ibm.com/infocenter/tivihelp/v3r1/topic/com.ibm.tivoli.tcr.doc_211/ttcr_configuring_2engines.html

When installing the 2nd TCR instance, point to the existing database instance one is already using

http://pic.dhe.ibm.com/infocenter/tivihelp/v3r1/topic/com.ibm.psc.doc_1.1.0/tcr_original/ttcr_configuring_2engines.html

11

v3v2

v2

v3

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareCurrent System Metrics

Reporting high level statistics since the server has been up can be viewed in the administrative UI

Is the queue backed up now? How backed up did it get?

12

v3v2

• Need to look at audit reports for report level details, later chart

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareAudit reports

Cognos sample audit reports are on the TCR image1. Direct logging to a database2. Increase logging level3. Install sample audit report package

– All reports except one just read the logging database. But one report, ‘Report Usage’ is Java-based. Requires updating/compiling

History by report/package, history by user, failed reports, etc.

http://pic.dhe.ibm.com/infocenter/cbi/v10r1m0/index.jsp?topic=%2Fcom.ibm.swg.im.cognos.ug_cra.10.1.0.doc%2Fug_cra_id4425SampleAuditReports.html

13

v3v2

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareFile system for saved reports

In certain scenarios, you may want to write scheduled reports to the file system. 2 Methods: A. Allow a 'save to file system' prompt

– Only save selected reports to the file system• User chosen file name• Overwrite or not• Append date or not

 

B. Automatically save all reports to the FS when they are saved in TCR– Saves report with auto appended name to avoid overwrite– XML is also written with details on report parameters, running user, creation time, etc– Optionally, call user provided script - Useful for automation  

Note: Regardless of which method, it’s up the administrator to manage the files once they are written to the file system

14

v3v2

Recommended

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareFile system for saved reports, con’t

A: Allow a 'save to file system' prompt1. Set the root directory:

– http://publib.boulder.ibm.com/infocenter/c8bi/v8r4m0/index.jsp?topic=/com.ibm.swg.im.cognos.inst_cr_winux.8.4.1.doc/inst_cr_winux_id17308inst_SaveReportOutputToFileSystem.html

– Follow the section "Save Report Output Outside IBM Cognos 8" to set the archive root directory 

– Note, If on Windows, Cognos Configuration is in the start menu. For other platforms, launch tipv2Components\TCRComponent\cognos\bin\tcr_cogconfig.sh

 2. After saving and closing Cognos Configuration, restart TCR.

 3. Configure the subdirectory level:

– http://publib.boulder.ibm.com/infocenter/c8bi/v8r4m0/index.jsp?topic=/com.ibm.swg.im.cognos.ug_cra.8.4.1.doc/ug_cra_i_LocalFileOutputLocation.html

– Now when you run a report, there is an option to save to the file system with options to over write file names, append timestamps, etc.

15

v3v2

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareFile system for saved reports, con’t

B: Automatically save all reports to the FS when they are saved in TCR1: Allow to save to the file system:

– http://publib.boulder.ibm.com/infocenter/c8bi/v8r4m0/index.jsp?topic=/com.ibm.swg.im.cognos.inst_cr_winux.8.4.1.doc/inst_cr_winux_id17308inst_SaveReportOutputToFileSystem.html

– Follow the section "Save Report Output Inside IBM Cognos 8" to allow saving to the file system

– Note, If on Windows, Cognos Configuration is in the start menu. For other platforms, launch tipv2Components\TCRComponent\cognos\bin\tcr_cogconfig.sh

2. After saving and closing Cognos Configuration, restart TCR. 

3. Configure the output location and optionally automation script:– http://publib.boulder.ibm.com/infocenter/c8bi/v8r4m0/topic/com.ibm.swg.im.cognos.ug_cra.

8.4.1.doc/ug_cra_i_System-WideReportOutputLocation.html#System-WideReportOutputLocation

– Now when you save any report in TCR, is it also saved to the file system. If you provided a CM.Outputscript, it is invoked when reports are saved.

16

v3v2

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareFile System Archival

To enhance system performance, extend scalability, and reduce costly database space

NOT meant to be read directly by user, cryptic file names

http://www-01.ibm.com/support/docview.wss?uid=swg21620975&aid=1

http://pic.dhe.ibm.com/infocenter/cbi/v10r2m0/index.jsp?topic=%2Fcom.ibm.swg.ba.cognos.c8bizos_ig.10.2.0.doc%2Fc_zig_ecm_intro.html

17

v3

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareMax report execution time

Global setting

Useful if you know you should not have any reports running over X amount of time

Many other tuning parameters to look through here, too

18

v3v2

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareWhat SQL is run against my database?

Look at any query in Report Studio

19

v3v2

• Compare the Native SQL with the Cognos SQL• Where clauses in the Cognos SQL and NOT in the Native SQL

are being processed locally, on the reporting server

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareWhat SQL is run against my database?

Sometimes it is useful to see the actual SQL being executed against the database. For this, turn on native query logging:– http://pic.dhe.ibm.com/infocenter/cbi/v10r1m0/index.jsp?topic=

%2Fcom.ibm.swg.im.cognos.ug_cra.10.1.0.doc%2Fug_cra_id4321SettheLoggingLevel.html

20

v3v2

• May need to restart TCR• SQL is logged to <cognos>/logs/cogserver.log

• Note: DQM has it’s own logging, a later slide

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareWhat’s happening on my server right now

See upcoming, past, and present activities View details on any activity Filter by user, date, status, priority, owner, type, scope

by package Administer schedules Even cancel running reports

21

v3v2

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareDynamic Query Mode

Dynamic Query Mode (DQM) is a new Java-based reporting engine– Decrease time to value with JDBC support (DB2, Oracle, MSSQL,

Netezza) DB2 driver is preinstalled

– Optimized SQL generation of complex queries– 64-bit, takes advantage of system resources

Existing reports from TCR 2 (Cognos 8) continue to use native connections

Framework Manager project or publish must explicitly be changed to use

DQM, single checkbox.

Reports need to be regression tested before assuming they can just be

switched to DQM

New data model projects should use DQM, it is the strategic direction22

v3

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareDynamic Query Mode

The package dictates which data source type will be used

More information What you need to know about data sources in TCR v3:https://www.ibm.com/developerworks/mydeveloperworks/files/form/anonymous/api/library/9641dcf4-c5b8-

413c-8ae8-9c461dd84a09/document/ed346f9d-1910-4150-a3e9-4cbdb47a5ffa/media/Data_Sources_in_TCR_3.1.pdf

DQM Cookbook: http://www.ibm.com/developerworks/data/library/cognos/infrastructure/cognos_specific/page529.html

23

v3

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareWhen not to use Active Reports

Active Reports contain all the data and charts required for the interactive

report

Default governor of 5000 rows of data

Warning will prevent report running with larger result set

Can be changed in the data model

Html only, no pdf or Excel

24

v3

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareBIRT

BIRT reporting engine still exists to serve older reports

Same old v2.2.2, to guarantee existing reports still run

NOT suggested to use for any new usage

25

v3v2

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareUpgrade Report Specifcations

After importing reports created in TCR 2 (Cognos 8), some time pre-production, you should upgrade the report spec in the admin UI

Otherwise, EVERY TIME the report is run, the report spec XML is upgraded … resulting in a small performance loss every run.

26

v3

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareSteps to Building a Rich and Compelling Reports

Take a stepped approach:1. Plan the report2. Build the content

a. Start by putting relevant data into a list or crosstab– This way you can be sure you are getting the data you expect before moving on.

Save a ton of debug.

b. Add filters and formattingc. Then add visualization

3. Size, style and lay out the content4. Define the interactive behavior (for Active Reports)5. Test the report6. Deploy the report

27

v3v2

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareLayout Component Reference Objects

Use Layout Component Reference Objects

– Makes for quick and simple changes later– Reports reference the single copy. Later updates only

require you to change the single source copy

28

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwarePrereq Reports

Use the prereq reports that more products with dynamic schemas are shipping – Checks the schema and tables exists instead of debugging it later– Integrated help links for failed

29

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwarePerformance Enhancements on Queries

Advanced topic

https://www.ibm.com/developerworks/community/wikis/home?lang=en#/wiki/IBM%20SmartCloud%20Monitoring/page/Reporting%20-%20Performance%20enhancements%20%26%20Best%20practices

For example:

Avoid shared dimension usage when data is coming from the same table by

exposing identifiers in the tables

Examine SQL to see if proper indexing is applied for popular reports

30

v3v2

IBM Software Group | Lotus software

IBM Software Group | Tivoli softwareOther Resources

TCR devWorks – report catalog, videos, articles, msg board https://www.ibm.com/developerworks/community/groups/service/html/communityview?

communityUuid=9caf63c9-15a1-4a03-96b3-8fc700f3a364

Field guide to TCR https://www.ibm.com/developerworks/mydeveloperworks/files/form/anonymous/api/library/9641dcf

4-c5b8-413c-8ae8-9c461dd84a09/document/4950a286-dcf8-4022-8ade-e2d18ca35ca2/media/A%20Field%20Guide%20to%20Tivoli%20Common%20Reporting%20v01r02.pdf

Free 3 hour web-based TCR 2 fundamentals class• http://www-304.ibm.com/jct03001c/services/learning/ites.wss/us/en?

pageType=course_description&includeNotScheduled=y&courseCode=TOS33

Cognos proven practices – Large collection, wide range of advanced topics http://www.ibm.com/developerworks/data/library/cognos/cognosprovenpractices.html

31

IBM Software Group | Lotus software

IBM Software Group | Tivoli software

Backup

32

IBM Software Group | Lotus software

IBM Software Group | Tivoli software

Cognos Active Report Common Techniques

– Leveraging the Basics to Create Rich Report Objects

– Visual Techniques

– Leveraging HTML Items

– Extending Base Functionality

IBM Software Group | Lotus software

IBM Software Group | Tivoli software1. Change Chart Type and View Details

TECHNIQUE Provide multiple views (charts, maps, or tables) of the same dataEXAMPLE

IMPLEMENTATION Use a static deck or a static tab control to provide multiple cards (or

tabs) with the same data in renderings34

IBM Software Group | Lotus software

IBM Software Group | Tivoli software2. Side-by-Side Comparison Chooser

TECHNIQUE A side-by-side display of the same data for two different itemsEXAMPLE

IMPLEMENTATION Mirrored controls and decks leveraging a layout component reference of the data

content in the second deck Use a separate variable for each item (ex: 2 city variables are used in the

example above)35

IBM Software Group | Lotus software

IBM Software Group | Tivoli software3. Drilling Down to Details

TECHNIQUE Upon selecting data, drill-down to more detailsEXAMPLE Drilling down to the details for a particular product line

IMPLEMENTATION Use decks and data decks organized in a horizontal table to define a

drill path36

IBM Software Group | Lotus software

IBM Software Group | Tivoli software4. Toggled List Column Display

TECHNIQUE User control over which columns are visible on a listEXAMPLE Checkboxes that control which measures are displayed in a list

IMPLEMENTATION Use a static check box control to drive the visibility of associated

columns37

IBM Software Group | Lotus software

IBM Software Group | Tivoli software5. Disabling Until a Condition is Met

TECHNIQUE Keep a part of a report disabled until other conditions are met

through user interactionEXAMPLE Showing a default card and disabling another control until a value is

chosen from another control

IMPLEMENTATION Create a default card to be shown when no value is set and/or setup

a control enabled expression requiring the variable to not be empty

38

IBM Software Group | Lotus software

IBM Software Group | Tivoli software6. Cascading Prompts

TECHNIQUE Choices available in the next control depend on the value selected in the

previous controlEXAMPLE Cascading prompting through a hierarchy of data

IMPLEMENTATION Disable next control(s) until variable set by previous control has a value Filter next control(s) by the value of the previous control’s variable

39

IBM Software Group | Lotus software

IBM Software Group | Tivoli software7. Cascading Animated Prompts

TECHNIQUE Show a second prompt control only after a selection from the first has

been madeEXAMPLE Progressive disclosure of a cascading prompt control

IMPLEMENTATION Put the secondary control(s) in an animated deck and show the card

only when the previous prompt has been answered

40

IBM Software Group | Lotus software

IBM Software Group | Tivoli software8. Hidden Report Regions

TECHNIQUE Create an area in the report that can be displayed or hiddenEXAMPLE Ideal for making additional content or controls available on demand

IMPLEMENTATION Put a two-card static deck into a collapsing 2 column (or row) table

41

IBM Software Group | Lotus software

IBM Software Group | Tivoli software9. “Top Metrics” Area

TECHNIQUE A visual technique to highlight “key metrics” in a textual wayEXAMPLE Highlight the most important measures and metrics

IMPLEMENTATION Using a table in a data deck, add measures to each column cell in

the first row of the table, and labels for the corresponding measures in the second row

42

IBM Software Group | Lotus software

IBM Software Group | Tivoli software10. Clickable Exploding Pie Slices

TECHNIQUE Clicking on a pie slice explodes out that piece of the pieEXAMPLE

IMPLEMENTATION Create a pie chart in a data deck and use an expression on the chart

to identify the pie slice to explode43

IBM Software Group | Lotus software

IBM Software Group | Tivoli software11. Custom Content Buttons

TECHNIQUE Create custom looking items that support interactive behaviorEXAMPLE A custom look vertical button bar containing a micro chart

IMPLEMENTATION Use a list for vertical or a crosstab for horizontal orientation content Augment or override the default contents of the cells

44

IBM Software Group | Lotus software

IBM Software Group | Tivoli software12. Toggled Conditional Styling

TECHNIQUE Allow user to toggle on and off conditional styles on contentEXAMPLE Button bar to toggle conditional styling in a crosstab

IMPLEMENTATION Create a static deck with duplicated contents on each card Apply a different conditional style on each card’s content

45

IBM Software Group | Lotus software

IBM Software Group | Tivoli software13. Sliding to Visualize Changes

TECHNIQUE Sliding through time on a slider control to gain insight on the dataEXAMPLE Sliding through years to see heat map data on map change over time

IMPLEMENTATION Use a live updating slider to slide through content in a data deck

46

IBM Software Group | Lotus software

IBM Software Group | Tivoli software14. Scrolling Marquee

TECHNIQUE Scroll data (to be passively consumed) across the screenEXAMPLE A scrolling marquee of related measures to the report’s primary

content

IMPLEMENTATION Wrap data in a repeater element between two HTML items

containing opening and closing marquee tags (<marquee> and </marquee>)

47

IBM Software Group | Lotus software

IBM Software Group | Tivoli software15. Embedded Videos

TECHNIQUE Include a link to a video form an external source (ex: from YouTube

or a corporate internal or external web site) EXAMPLE

IMPLEMENTATION Put well formed HTML video code into an HTML item in the report Note: video will only be available when user is connected

48

IBM Software Group | Lotus software

IBM Software Group | Tivoli software16. Absolutely Positioned Content

TECHNIQUE Absolutely position content on top of other content in a reportEXAMPLE

IMPLEMENTATION Wrap content in HTML Items that positions the content absolutely

• Before content:<div style=“position: absolute; left: ?px; top: ?px; width: ?px; height: ?px;”>

• After content: </div>

49

IBM Software Group | Lotus software

IBM Software Group | Tivoli software17. Tree Control

TECHNIQUE A tree control to more easily navigate and find data in a hierarchical

structure

EXAMPLE A hierarchical tree of Product line >

Product type > Product name

IMPLEMENTATION Use decks inside of lists to build up

a structure based on the data hierarchy

50