Date post: | 07-Nov-2014 |
Category: |
Technology |
Upload: | ibm-danmark |
View: | 1,553 times |
Download: | 3 times |
CSC REFERENCE CASEIBM NETEZZA
John V FabienkeChief Architect, Nordic Architecture Practice
I. CSC Enterprise Business Intelligence capability overview 3
II. Zürich Netezza case study 7
III. CSC Nordics Netezza PoC capability 22
Agenda
Capability Overview
CSC Proprietary and Confidential 04/08/2023 05:21 AM 4
CSC Enterprise Business Intelligence Framework
CSC Proprietary and Confidential 04/08/2023 05:21 AM 5
CSC Enterprise Performance and Information Management Practice (EPIM)
EPIM Practice (40+ SMEs)
BI Capability Data Management Capability
GOVERNANCE
Strategy
Planning & Knowledge Management
Architecture & Disruptive Technologies
BI SOLUTION and SERVICETower of Competency
DATA SOLUTION and SERVICES Tower of Competency
BI Solution Analytics Solutions
BI Services Analytics Services
Data Solutions Integration Solutions
Data Services Integration Services
EMEA Enterprise Information Capability Group (120+ SMEs)
Offshore Capability (800+ SMEs)
BI Capability Data Management
BI Capability Data Management
BI Services Analytics Services
BI Services Analytics Services
Data Services Integration Services
Data Services Integration Services
GLOBALPRACTICE
GLOBALOFFERINGS
GLOBALDELIVERY
CSC Proprietary and Confidential 04/08/2023 05:21 AM 6
CSC’s Global Business Intelligence Alliances
Zürich Case StudyIBM Netezza solution
CSC Proprietary and Confidential 04/08/2023 05:21 AM 8
Netezza Overview
Business problem to solve
• Client’s current mainframe and DB2 based infrastructure that their financial close processing, data warehouse processing, regulatory reporting and Business Intelligence reporting runs on was over 4 years old
• Client was at full capacity on multiple fronts and the DB2 environment required a significant investment to upgrade
• With the introduction of the U2Z program data into client’s month-end close the system was no longer able to meet SLAs for the month-end close process
• Even after 9 months of tuning the DB2 environment SLAs were still unable to be met
CSC Proprietary and Confidential 04/08/2023 05:21 AM 9
Netezza Overview
What scenarios were considered?
• Netezza was suggested as a simple, high-performance solution• Teradata was included due to their being the long-term industry leader• IBM’s traditional offering was considered as well• Oracle was dismissed due to TCO, lack of existing footprint and skill sets
CSC Proprietary and Confidential 04/08/2023 05:21 AM 10
Netezza Overview
PoC performance results
• Netezza and Teradata reporting consistently in the 10x – 200x performance improvement range across the board
• Analytic run queries ran 150 to 18,000 times faster than the current DB2 platform on both Netezza and Teradata
• Compared to DB2 performance differences between Teradata and Netezza were negligible
CSC Proprietary and Confidential 04/08/2023 05:21 AM 11
Netezza Overview
Reference calls
• Teradata is usually used as an Enterprise Data Warehouse “workhorse”
• Netezza is usually used as a reporting engine
• Cost was the major factor in choosing Netezza over Teradata
‒ A few references had purchased both platforms for contract negotiations
• Both platforms returned over 95% of user queries within 60 seconds
CSC Proprietary and Confidential 04/08/2023 05:21 AM 12
Netezza Overview
Netezza was chosen
• Teradata’s version 13 was not compatible with the client’s version of Business Objects and Informatica
• Teradata ran out of time to provide data points used to measure the month-end processing performance
• The Netezza platform was a less complex solution than Teradata
‒ Better fit for client’s out-sourced model
CSC Proprietary and Confidential 04/08/2023 05:21 AM 13
High-performance databaseengine streaming joins,aggregations, sorts, etc.
SQL CompilerQuery PlanOptimizeAdmin
Slice of User DataSwap and Mirror partitionsHigh speed data streaming
Netezza Overview
SMP Hosts
Snippet Blades™ (S-Blades™)
Disk Enclosures
CSC Proprietary and Confidential 04/08/2023 05:21 AM 14
EDW Architecture Overview
DMS
CESAR
CIID
COS
RADIUS/PALM
ZNAW
DICE
ZDW
CORE WORKBENCH
Application Architecture: EDW
ZEA C Class
Work Bench/EDB
BIW Farmers
Mainframe
EDW on DB2
INPUT OUTPUT
Informatica
Netezza
nzload
EDW BO
ZDW
CSC Proprietary and Confidential 04/08/2023 05:21 AM 15
Netezza OverviewPoC Query Timings
Business Objects Reports & SAS queries DB2 Timings (sec) NZ Timing ROWS
Comm Analysis Template – Dec 2 ‘08 - 23 113.516
Comm Analysis Template – Dec ’08 - AON 227 60 56.134
HPY AY Loss Ratio Analysis 1.380 51 190.688
Policy Effective year Template 2.460 14 888.324
Comm Analysis Template – Dec 2 ‘08 7.166 41 341.234
Comm Analysis Template – Dec 2 ‘08 - Marsh 2.150 15 44.308
Direct Written Premium - 2006 300 118 1.322.590
Construction Expiration List – Date Promt 2.024 22 29.510
Domestic Linked to Customer with Foreign 2009 1.800 17 20.597
International Report – Final – Dec Results 2.100 36 29.755
NSBs Large Account Renewals 660 40 3.567
Comm Analysis Template – Dec 2 ‘08 - AJG 7.826 15 33.263
Comm Analysis Template – Dec 2 ‘08 - Wells Fargo 6.854 13 25.290
HPA Comm Analysis 5.971 16 4.182
Comm Analysis Template – Dec 2 ‘08 - HUB 2.840 14 15.508
Producer Code to Dist ID - 17 29.780
Query 1 38.317 43 1.480.812
Query 2 54.707 5 4.169
CSC Proprietary and Confidential 04/08/2023 05:21 AM 16
Netezza Overview
DB2 and Netezza landscape
• DB2 has not gone away, a few applications still live on DB2
• DB2 environment was moved to InfoSphere Change Data Capture (CDC)
• Informatica environment is completely on DB2
• Working on shrinking the DB2 environment
• Netezza replication to Reston can be accomplished via Netezza utilities
‒ NZ_MIGRATE
‒ Data is sent compressed
‒ 2 TB per hour throughput
CSC Proprietary and Confidential 04/08/2023 05:21 AM 17
Netezza Overview
How have things improved with Netezza?
• Month-end processing is down from 9 days to 3 days
‒ Within 6 months client anticipate to have this down to 2 days
• ADS processing does not stop for month-end (no catch-up time)
• Queries can run on the Netezza platform during month-end processing
• SAP feed down from day 3 to day 1 of Month-end processing
• Netezza load and unload speed of 2 TB per hour
• Developer mindset change from sequential to set processing
• Business Object query time has dropped significantly
‒ Average BO query on DB2 took over 3,100 seconds
‒ Average BO query on Netezza now runs in 6.5 seconds
CSC Proprietary and Confidential 04/08/2023 05:21 AM 18
Netezza Overview
How have things improved with Netezza?
• On November 18, 2011 16.172 queries ran on the TF12
‒ Largest number of queries run on a single day
‒ Average query time was 0.83 seconds
‒ Longest query time was 47 minutes (Informatica)
‒ Peek of 8,243 queries completed per hour
‒ Host CPU utilization never went above 10%
‒ SPU (blade) utilization never went above 17%
• EDW conversion PoC
‒ 4.1 million updates ran in 18 seconds
‒ Executed on TF6
‒ Informatica & post session task
CSC Proprietary and Confidential 04/08/2023 05:21 AM 19
Netezza Overview
Bumps along the way
• Netezza and Linux upgrades have caused issues
• Netezza off-hours support is poor
• Technical Account Manager (TAM) knowledge is lacking
• Integration error debugging knowledge is lacking
• 48 concurrent query limit on each Netezza appliance have caused issues
• Service provider requirements not accommodated for
‒ Features lacking for guaranteeing SLA’s
‒ Limited multi-tenancy features
• Netezza documentation is lacking or sometimes incorrect
Light at the end of the tunnel
• IBM will bring stability to these issues
• Client is member of the Netezza Advisory Board to escalate issues
CSC Proprietary and Confidential 04/08/2023 05:21 AM 20
Netezza Overview
What’s Next?
• More projects being developed on Netezza
• Plan Migration of ZEA to Netezza
• Look at other systems to migrate (EDW + CDW)
• Netezza upgraded in 2012
‒ 2x performance
‒ 5x or better compression
‒ 4TB/hour load and unload speeds
‒ Concurrent Query limit raised to 100
‒ Smart Caching
‒ Query Mix: large percentage of short, tactical queries
‒ Granular Zone Maps (potential 24x reduction in reads)
CSC Proprietary and Confidential 04/08/2023 05:21 AM 21
Netezza Overview
What’s Next?
• Informatica Pushdown Optimization (PDO)
‒ Informatica mapping converted to SQL
‒ SQL is submitted to Netezza
‒ All data stays on Netezza
‒ All processing happens within Netezza
• PDO example – Creation of Combined Mart
‒ Before: DB2 only method – 18 to 20 hours
‒ After: Netezza conversion – 6 to 8 hours
‒ Future: Netezza with PDO – expected 1 to 2 hours
Netezza PoC capabilityCSC Nordics
CSC Proprietary and Confidential 04/08/2023 05:21 AM 23
Cognos + TargitNetezza
Scandihealth POC Setup
Mainframe + DB2
PushNear Real Time Reports
CDC server
Selected TablesXfer Changes
CSC Proprietary and Confidential 04/08/2023 05:21 AM 24
Business case
Optimize hospital beds
• Calculate availability of hospital beds more frequently in order to optimize utilization
• Utilization of beds is an important KPI for a hospital
• Before
– At best the customers would get calculations once a day
• After
– Calculations performed every 15 minutes
CSC Proprietary and Confidential 04/08/2023 05:21 AM 25
Findings
All business requirements met
• Data is updated in near real-time• Queries can now be very frequent• No incremental MIPS cost for queries and reports• The load CDC imposed on mainframe MIPS was insignificant
Thank You Q&A