QA Challenges in Data Integration Projects &

iCEDQ SoftwareData Warehouse | ETL | Data Migration | Data Lake | MDM

ETL Testing & Monitoring Platform

The QA Challenge

Lack of Automation

Lack of Agility

Clutter

No Test Driven Development

Lack of Regression Testing

Data Warehouse | ETL | Data Migration | Data Lake | MDM

www.icedq.com

vs. iCEDQ has in-Memory Rules Engine

Tests ETL processes by Validating the output data

Validation Rule

Reconciliation Rule

Tests ETL Transformations by Reconciling Source Data Vs. Target Data

ETL

Too many tools – Excel, SQL, Scripting

Not repeatable and consistent Can’t test Millions of rows Forced to do sampling Time wasted in coding rather

than testing Disorganized Time & resource consuming

A

iCEDQ Shift

www.icedq.com

BWaterfall SDLC vs. Agile

iCEDQ Shift

www.icedq.com

CClutter vs. Organized & Documented

START

Business Requirements

Test Results

Workflow

QA Dashboard

Audit (Testing) Requirements

Audit Rules & Scripts

Test Cases

iCEDQRepository

iCEDQ Shift

www.icedq.com

Conventional Waterfall SDLC vs.

Testing is too late in the project. 8th Step! QA team sitting idle for developers to finish Last minute rush

Step 1. Get Requirements

Step 2. Design the

TablesStep 3 Create Data Mapping

Step 4. Develop ETL

Code

Step 5. Get Test

RequirementsStep 6. Write

Test CasesStep 7. Write Physical Test Step 8. Test Step 9.

Deploy

Step 1. Get Requirements

Step 2. Design the

Tables

Step 3 Create Data

Mapping

Step 4. Develop ETL

CodeStep 5.

TestStep 6. Deploy 7 8 9

RefactorReduce

timeline by 33%

Step 5. Get Audit Requirements

Step 6. Write Test Cases

Step 7. Create

Physical Rules

iCEDQ enables Test First Approach Test Driven Development - TDD Lift n’ Shift the QA to the beginning of project

iCEDQ -TDD

DiCEDQ Shift

www.icedq.com

Qualit

y ?

Start

QA QC

Regression Testing & Release Management

Conventional data integration testing lacks… Tools for regression

testing Sign-Off & release

management Continues Build

Management Quality Control

iCEDQiCEDQ Shift

Business Requirements

Audit Requirements

Test Cases

Regression Testing Deployment

Release Management

Quality Control & Production Monitoring

Source Sys.

Biz. Users Management

E

www.icedq.com

Why iCEDQ?

Automation Test high volume of data Regression Testing Need Agile and/or Test Driven Development Lower Project Risks & Cost Improve Productivity Compliance Security Organized & Collaborative Reporting, Dashboard & Notification Transparency & Feedback Shrink Project Timeline

Data Warehouse | ETL | Data Migration | Data Lake | MDM

www.icedq.com

Development & QA Testing

Customer Finds It Post Release

Leads to Litigation, Fines, Reputation Loss

1x

100x

10,000xSave $$$$$$

Early is Better!

Cos

t of D

elay

$

Time www.icedq.com

iCEDQ SERVER

iCEDQ Architecture

Audit Rules Engine

iALM iGOVERNiQAiAUDIT Scheduler

RepositorySQL Server |Oracle

Configuration Files

Parameter Files Log Files

Data Exceptions

Files

GUI Web Service CLI/Shell RPC Report

JDBC

Text File

Cobol*

Other Files

Mainframe*

Data Adapters

Oracle | SQL Server | Redshift | DB2 | My SQL |

Netezza |…

Hadoop HIVE | HP Vertica

XML | EDI | JSON

CSV | Fixed Width | Excel

Cobol Files

ISAM | KISM | KSDS | QSAM |VSAM

BIG DATA

Redshift, HADOOP, Hive, S3, HP VERTICA, Cassandra, MONGODB, HDFS…www.icedq.com

Who uses iCEDQ?

Stock Exchange Banks Insurance Manufacturing Healthcare E-Commerce Manufacturing …

www.iCEDQ.com

game changer:nouna newly introduced element or factor that changes an existing situation or activity in a significant way

ETL Testing & Monitoring Platform

- Merriam Webster

www.iCEDQ.com