1

November 10-13, 2015

Kevin D. Smithemail: kdsmith@forsythe.commobile: +1 813-340-0335

A Forsythe Division

Kevin D. Smith

Oracle Open World Review

Golden Gate / ODI & Big Data

2

Thank you for joining us today

33

DAY’S AGENDA

10:15 - 10:30 Welcome / Intro / Meta7 / OOW (Kevin Imm)

10:30 - 11:30 Systems and Storage Announcements (Chris Brillante)

• M7/T7 Chip and Systems

• Engineered Systems

• Storage

11:30 - 12:15 Oracle Cloud Offerings (Eric Frank)

12:15 - 12:45 Lunch and Discussion

12:45 - 1:30 GoldenGate / ODI / Big Data Tools (Kevin Smith)

1:30 - 2:00 Database and Red Stack Security (Kevin Smith)

44

SPEAKER PROFILE – KEVIN SMITH

Kevin Smith - Master Consultant, Meta7

Over twenty years of business & technical experience in Information Technology encompassing Service Oriented Architecture (SOA), Application & System Integration for large implementations across multiple platforms & operating systems

Expertise in Oracle, DB/2, Sybase, & MS SQL

Multiple platform support: HP, SunOS, AIX, and Linux Unix; Windows; MVS, OS/390, z/OS

Oracle GoldenGate 10g to 12c

Certified GoldenGate Implementation Specialist GoldenGate

Co-Founder of International GoldenGate Oracle Users Group – www.iggoug.org

55

META7 is a division of Forsythe Technologies

Platinum Oracle partner

Deeply skilled, uniquely experienced

Focused on the Oracle Red Stack

Flexible, creative and accommodating

Reliable on-time and on-budget delivery

State of the Art Technology Evaluation Center

Secure hosting and Managed Services

Flexible financial support

META7

66

Consulting and Managed Services

12c DB Upgrade & Migrations

SPARC Refreshes

IBM Power -> Oracle Systems Migrations

Oracle-on-Oracle Storage

DB Security

Engineered Systems Deployments

Oracle Cloud Strategy

META7 - AREAS OF FOCUS

7Oracle Open World 2015

NoETL Engine100% Native Data

Transformation

DataIntegrator

Big DataPreparation

GoldenGate

DataQuality

Data ServiceIntegrator

MetadataManagement

Non-invasive CDC, Realtime streaming data

delivery

Profile, Cleanse, Match, and

Remediate Data

Prepare, Secure, Enrich and Publish Unstructured Data

Catalog, Trace and View Models across the Enterprise

Federate Data Across DBs, Services and Applications

88

Part of a Family of Oracle Data Integration Technologies

• Comprehensive Integration

• ELT/ETL for Bulk Data

• Service Bus

• Process Orchestration

• Human Workflow

• Data Grid

• Business Data / Metadata

• Statistical Analysis

• Time Series Reporting

• Integrated Data Quality

• Metadata Harvest

• Metadata Catalog

• Data Lineage

• Interactive Search

• Interactive Browse

• Impact Analysis

• Heterogeneous E-LT & ETL

• High-speed Transformations

• OLAP Data Loading

• Data Warehouse Loading

• Real Time Data Replication

• Changed Data Capture

• DBMS High Availability

• Disaster Tolerance

• Data Service Modeling

• XQuery Data Federation

• Data Security/Redaction

• XA Compliance

• Cleansing & Parsing

• De-duplication

• High Performance

• Integrated w/ODI

99

Oracle Database Replication Technologies

Data Guard

Physical

Logical

Usable only when the source and the target are identical

Operating System and Database Version

Streams

While still supported and used, Streams had a number of issues (which is why Oracle acquired GoldenGate) and is no longer Oracle's strategic replication product, it's on a continue and converge path and will continue to be supported but not extended to support multi-tenant container databases

Change Data Capture (CDC)

A subset of Streams capabilities: Support will continue but capabilities will not be enhanced for multi-tenant container databases

Materialized View, Database Links, 3rd Party Products

1010

What Makes An Ideal Replication Solution?

Replication apply time configurable from near-zero to a user defined pre-set delay

Scalable to handle high transaction volumes

Minimal footprint and impact to existing systems

Modular architecture supporting multiple data sources and targets

Fault tolerant

Maintains transactional integrity

Reliable

Secure

Stable

1111

What Is GoldenGate?

A software tool that creates a tunnel between two, or more physically separate systems such that they can behave as a single logical system

Middleware that provides a data communications channel between a source from which transactions will be read and a target to which the transactions will be written

GoldenGate connections canbe homogeneous, for exampleOracle Database to OracleDatabase, or heterogeneous,for example Teradata to Oracle Database

Supports data transformation

Can replicate DML and DDL

Unlike Oracle Service Bus ... understands transactions

1212

Logical Replication Architectures

UnidirectionalReporting Instance

Bi-directionalActive:Active Failover

Peer-to-PeerLoad Balancing, High Availability

BroadcastData Distribution

ConsolidationData Warehouse/Mart/Store

CascadingScaling, Database Tiering

13

Business Cases For Oracle GoldenGate

1414

GoldenGate Business Cases (1:3)

Load Distribution

The current server infrastructure is not sufficient to handle the demands placed upon it; for example it would save the organization both time and money if analytic queries were run against a separate database performance optimized for those queries

Upgrading

The organization wishes to upgrade from one version of a product to a different version of the same product; for example 11.2.0.3 to 12.1.0.2 with near-zero downtime

Homogeneous Migration

The organization wishes to migrate a database from one hardware platform to another; for example Oracle Database on Solaris to Oracle Database on Linux

Heterogeneous Migration

The organization wishes to migrate from one database product to another; for example from stand-alone SQL Server to Oracle RAC on an ODA

1515

GoldenGate Business Cases (2:3)

Data Distribution

The organization wishes to make data available at different physical locations without dependency upon the internet; for example the corporate head quarters are in London England but the manufacturing facilities are in Latin America

Data Consolidation

The organization has multiple locations where data is collected and wishes to have a single source that combines them all for management reporting and analysis, for example there are 8,200 retail stores in North America, each with its own on-site database and the corporate headquarters needs to be able to run a single query across data from one region's stores or all stores in all regions

Data Access Sharing

The organization wishes to have two separate locations with equal access to data with the ability to perform DML and DDL, for example the organization considers each office a DR site for the other

1616

GoldenGate Business Cases (3:3)

Security

The organization stores both public and confidential information some of which it does not wish to make vulnerable by making it visible through-out the organization; for example data collection from an e-commerce site that processes both orders and credit cards

Auditing

The organization wants a database of suspicious transactions for fraud investigators where the source might be multiple databases, from multiple vendors; for example the organization has both Oracle and DB2 databases and wants a single repository for internal auditors

Data Transformation

The organization wants to load data from OLTP (On-Line Transaction Processing) systems into one or more data warehouses, Operational Data Stores (ODS) or Decision Support Systems (DSS); for example the data warehouse collects data from only specific tables and columns and must load them into cubes

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Open World 2015 17

Announcing: Oracle GoldenGate New Capabilities

Zero Downtime Upgrades & Data Migration

High Availability& Disaster Recovery

Application andReport Offloading

Big Data, Data Warehouseand Data Marts

Message Bus, SOA/JMS, Coherence

Global Data SynchronizationAcross the Enterprise

Real-time Analytics& Data Streaming

New Capabilities GG Studio for easy design

Metadata in Trail File

Delivery to Kafka

Enhanced Monitoring w/Lag

Performance Improvements

MySQL 5.7. SQL Server SE, ALO

Key Benefits Reliability most proven CDC on

the planet – don’t lose data!

Performance when low latency matters most – GG is the best

Breadth of coverage on all your targets, Oracle, Non-Oracle, Big Data and NoSQL

Cap

ture

Trai

l

Ro

ute

Del

iver

Pu

mp

GoldenGate

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

Streaming Big Data

Batch Big Data

NoSQL

Cap

ture

Trai

l

Ro

ute

Del

iver

Pu

mp

Oracle Open World 2015 18

NEW: Oracle GoldenGate for Big Data

GoldenGate

GoldenGate for Big Data Same trusted architecture used

by 1000’s of customers

Data delivery to Big Data

Lambda/Kappa streaming data

Key Benefits Less invasive on Source DBs

when compared to Batch Processing such as Sqoop or ETL processes

Simple ingestion for 1:1 data architecture for populating “raw data” zones

Real-time data delivery for streaming analytics/apps

Reliable, proven at scale

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

Database as a Service

Hadoop as a Service

3rd Party Cloud

Cap

ture

Trai

l

Ro

ute

Del

iver

Pu

mp

Oracle Open World 2015 19

NEW: Oracle GoldenGate Cloud Service

GoldenGate Cloud Service Available in the Public Cloud via

Subscription or Hourly basis

Key Benefits Oracle Database Cloud Service

delivery via GoldenGate

Exadata Cloud Service delivery via GoldenGate

Big Data Cloud Service delivery to Hadoop and NoSQL

3rd Party Cloud More Choices run as BYOL on

other Clouds for delivery to any supported Database

On Premise

DatabaseCloud Service

ExadataCloud Service

Big DataCloud Service

GGCS

GGCS

GoldenGate

20

GoldenGate Supported Systems

2121

Supported Sources

HP SQL/MP (Tandem)

HP SQL/MX

IBM DB2 z/OS

IBM DB2 (LUW)

IBM DB2 for i (AS/400)

IBM Informix

Java Messaging Queues (JMS)

Microsoft SQL Server

Oracle Database

Oracle MySQL

Oracle TimesTen

Sybase ASE

Teradata

2222

Supported Targets

Base24

EMC Greenplum

GG For BigData: Hadoop File System (HDFS, Hive, HBase, MongoDB, Kafka)

HP Enscribe

HP SQL/MP (Tandem)

HP SQL/MX

IBM DB2 (LUW)

IBM DB2Remote

IBM DB2 System i (AS/400)

IBM DB2 System z (mainframe)

IBM Informix

IBM Netezza

Java Message Queue (JMS)

Microsoft SQL Server

Oracle Database

Oracle MySQL

Oracle TimesTen

PostgreSQL

Sybase ASE

Teradata

Flat Files

XML Files

JSON File

AVRO File (across the BigDatastack)

2323

Supported Operating Systems

HPUX IA64

HP NonStop Itanium

IBM AIX

IBM iSeries

IBM z/Linux

IBM z/OS

Linux x86-64

Oracle Solaris Sparc

Oracle Solaris x86-64

Windows x86-64Copyright © Puget Sound Oracle Users Group

2424

Complete Logical Flow with Managers

ReplicatPump Trail File

TargetSource

Extract Trail File

Network

Network

Manager Manager

The Pump process communicates with the Target's Manager and requests a Collector process to transfer the trail file

2525

Logical Flow: Bi-directional

ReplicatExtract Trail File Pump Trail File

TargetSource

Target Source

Replicat Trail File Pump Trail File Extract

ManagerManager

26

Troubleshooting and Debugging

2727

Troubleshooting Considerations

The most common problems are network stability issues

The second most common problems are caused by DBAs and Developers making schema changes that create collisions

For example inserting rows into the target that will later cause a failure when an identical key is replicated from the source

The third most common issues relate to adding and altering tables creating supplemental logging failures

Help your GoldenGate admin identify processing -intensive tables so they can be replicated using separate processes

There are special considerations for ASM and RAC dependent upon GoldenGate and Database version ... do your research

Bidirectional replication is hard ... not because of the database and not because of GoldenGate but because of application designs not capable of supporting it

28

Basic GoldenGate Security

2929

Credential Store

Deploy GoldenGate version 12.1 with a Credential Store to hold login useridand passwords to protect the database

[oracle@gg00a gghome_1]$ cd dircrd

[oracle@gg00a dircrd]$ ggsci

Oracle GoldenGate Command Interpreter for Oracle

Version 12.1.2.1.0 OGGCORE_12.1.2.1.0_PLATFORMS_140727.2135.1_FBO

Linux, x64, 64bit (optimized), Oracle 11g on Aug 7 2014 09:14:25

Operating system character set identified as UTF-8.

Copyright (C) 1995, 2014, Oracle and/or its affiliates. All rights reserved.

GGSCI (gg00a) 1> ADD CREDENTIALSTORE

Credential store created in ./dircrd/.

GGSCI (gg00a) 1> exit

[oracle@gg00a dircrd]$ ls -l

total 4

-rw-r----- 1 oracle oinstall 324 May 9 12:06 cwallet.sso

GGSCI (gg00a) 1> ALTER CREDENTIALSTORE ADD USER ggadmin PASSWORD ora123 ALIAS ggadm

Credential store in ./dircrd/ altered.

GGSCI (gg00a) 1> dblogin useridalias ggadm

Successfully logged into database.

3030

OOW Highlights: Oracle Data Integrator

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Open World 2015 31

Announcing: Oracle Data Integrator New Capabilities

On Any DB/DWOn Exadata On Big Data

0

50

100

150

200

250

300

350

42XFASTER

ETL

INFA

/ IB

M

Best ETL for any Exadata App

Run ODI on the Exadata Host

ODI Optimized for SmartScan

Proven w/1000’s of customers

Native SQL for 3rd Party DB’s

Faster ETL and lower TCO

Oracle Data Integrator

First to market with Spark

Reuse maps across languages

Executable runs anywhere

New Capabilities Lifecycle Management updates

to core versioning systems

Simplify setup and deployment of Big Data transformations

Performance improvements across all E-LT platforms

Key Benefits Open Source friendly – run E-LT

on Apache Spark/Hadoop

Best Performance – take processing to the data

No Hardware required – run on the DB host or on Apache big data cluster

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Open World 2015 32

Oracle Data Integrator Advanced Big Data Edition

SQOOP

OGG

Oracle Data Integrator

SQOOP+ NativeLoaders

API/File

Flume/Kafka

OLH/OSCH

Big Data SQL

Data Integrator for Big Data Batch data ingestion with

Sqoop or native loaders

Generate data transformations in Hive, Pig or Spark

Extract data into external DBs, Files or Cloud

Key Benefits NoETL Engine designed from

scratch for E-LT execution

Non-invasive does not require any install on cluster

Decoupled mappings allow for portable runtime execution, saving development time, and simplifying debugging

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Open World 2015 33

Oracle Data Integrator for Cloud

Data Integrator for the Cloud Deploy ODI anywhere and

execute ETL on the data

Key Benefits Integrated with OPC (Oracle

Public Cloud)

Flexible Deployment architecture that can run on-premise, Java Cloud Service or Big Data Cloud Service

Broad Connectvity for sourcing data from any Apps or DBs

3rd Party Cloud

Flexible run ODI with BYOL on other clouds

On Premise

ODI

SaaS Apps

Oracle Data Integrator

ODI/EDQ

JavaCloud Service

ODI

Big DataCloud Service

DatabaseCloud Service

ExadataCloud Service

StorageCloud Service

Business IntelligenceCloud Service

3434

META7 - AREAS OF FOCUS

3535

OOW Highlights: Oracle Metadata Management

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Open World 2015 36

Announcing: Oracle Metadata Management 12.2.1

1. Business Glossary – simple tools to catalog, link and collaborate on business terms

2. Metadata Management – horizontal and semantic data lineage for all big data sources

Business/Metadata Catalog Report to Source Lineage

Impact Analysis

Audit, Versioning & Diff Reports

Social/Collaboration Features

Annotations and Tagging

Automatic Stitching

Comprehensive Harvesting 3rd Party BI Metadata

3rd Party ETL Metadata

3rd Party DB Metadata

3rd Party Modeling Tools

Hadoop & NoSQL Metadata

Metadata Standards

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Open World 2015 37

Open Source, Standards and Third Party Integrations

Adaptive Altova Apache HCatalog Apache Hive/HQL Borland CA ERwin Cloudera Impala COBOL Copybook DataStax Embarcadero EMC ProActivity GentleWare Google BigQuery Grandite Hadapt Hive Hortonworks Hive IBM Cognos IBM DB2 IBM DataStage IBM Discovery IBM Federation Server IBM Lotus Notes IBM Netezza IBM Rational Rose IBM Rational Architect Informatica Metadata Manager

Informatica PowerCenter CoSORT ISO SQL Standard (DDL) MapR Hadoop Hive MicroFocus Microsoft Access Microsoft Office Excel Microsoft Visio Microsoft SQL Server Microsoft SSIS Microsoft Visual Studio Microstrategy Magic Draw OMG CWM Standard OMG UML Standard Oracle BI Answers Oracle BI Enterprise Edition Oracle BI Server Oracle DAC Oracle Data Integrator Oracle Data Modeler Oracle Database Oracle Designer Oracle Hyperion Applications Oracle Hyperion Essbase Oracle Warehouse Builder

Pivotal Greenplum PostgreSQL QlikView SAP BO Crystal Reports SAP BO Designer SAP BO Desktop Intelligence SAP BO Repository SAP BO Data Integrator SAP BO Data Steward SAP Master Data Management SAP Sybase PowerDesigner SAP Sybase ASE Database SAS Data Integration Studio SAS BI Server SAS Information Map SAS Metadata Management SAS OLAP Server Select Sparx Architect Syncsort Tableau Talend Teradata Tigris Visible W3C DTD & XSD Schema

Metadata Harvesting (Glossary, Lineage & Impact Analysis)Key Standards

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |38

OOW Highlights: Oracle Data Quality

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Open World 2015 39

Announcing: Oracle Enterprise Data Quality 12.2.1

New Capabilities Enhanced High Availabilty

WebLogic Clustering

DB Failover Support

Improved Matching Flexibility

Scoring for Matching

Easier Definitions

Hadoop/Hive Certification

Audit Framework Support

REST Web Services

Key Benefits Simpler tools to very efficiently

create match rules

Accuracy of scoring models and easier access to parameters

Oracle Enterprise Data Quality

Fusion DQ (EDQ)

EDQ 12c

Profile Audit Transform Match Parse

WebLogic 12c

DaaS DQ (EDQ)

EDQ

40

Thoughts and Conclusions

4141

Conclusion (1:4)

In my opinion, GoldenGate is the best homogeneous and heterogeneous replication tool on the market

Very small cpu, memory, and storage footprint

Capable of solid security lockdown

Robust capabilities for subsetting and transformation

Substantial tools for troubleshooting and debugging

Substantial capabilities for performance tuning

Easy to patch

Easy to upgrade

Stable

4242

Conclusion (2:4)

Oracle Data Integrator

Provides a new declarative design approach to defining data transformation and integration processes, resulting in faster and simpler development and maintenance.

Based on a unique E-LT architecture (Extract - Load Transform), Oracle Data Integrator not only guarantees the highest level of performance possible for the execution of data transformation and validation processes but is also the most cost-effective solution available today.

Oracle Data Integrator provides a unified infrastructure to streamline data and application integration projects.

4343

Conclusion (3:4)

Oracle & Big Data

Platform for Big Data

Faster time to actionable insights

Big Data Preparation Cloud Service

Data visualization tools

Big Data Discovery Cloud Service

Common Customer approach to Big Data

Internal use to derive more value from their data assets

Add on services to end-users to foster greater customer affinity

Stand-alone clients offerings on a paid basis that allow monetization of Big Data assets

4444

Conclusion (4:4)

Metadata Management

Metadata Harvest

Metadata Catalog

Data Lineage

Interactive Search

Interactive Browse

Impact Analysis

Data Quality

Cleansing & Parsing

De-duplication

High Performance

Integrated w/ODI

Hadoop/Hive Certification

4545

How can we help you be successful?

In my opinion GoldenGate is the best homogeneous and heterogeneous replication tool on the market

Very small cpu, memory, and storage footprint

Capable of solid security lockdown

Robust capabilities for subsetting and transformation

Substantial tools for troubleshooting and debugging

Substantial capabilities for performance tuning

Easy to patch

Easy to upgrade

Stable

If you have any questions about

how Meta7 can help or on what

Meta7 provides, we would like to

talk with you after the

presentations.

46

Thank you for your time

Questions ?Address questions directly to: kdsmith@forsythe.com

Next EventTuesday, November 17, 2015:

Update on Oracle GoldenGate for DBA’s

12:00 ET / 11:00 CT

All Technical Briefingshttp://meta7.forsythe.com/Resources

contact: Kevin D. Smithemail: kdsmith@forsythe.commobile: +1 813-340-0335skype: kevindsmith-Tampa

47

www.meta7solutions.com

Kevin D. Smith 813-340-0335

KDSMITH@Forsythe.com