Big Iron to Big Data: Do more with the data you’ve got
Kevin Markwick
Regional Sales Director
Leading the next technology [r]evolution:
Big Iron to Big Data
Big Iron to Big Data: a fast-growing market segment composed of solutions that optimize traditional data systems and deliver mission-critical data from these systems to next-generation analytic environments.
Syncsort Confidential and Proprietary - do not copy or distribute 2
Trusted Industry Leadership
Syncsort Confidential and Proprietary - do not copy or distribute 3
500+Experienced & Talented
Data Professionals
>6,000Customers
196850 Years of Market Leadership
& Award-Winning Customer Support
84of Fortune 100 are Customers
3xRevenue GrowthIn Last 12 Months
The global leader in Big Iron to Big Data
Macro Trends Require Big Iron to Big Data Strategies
Digital business is driving significant growth in workloads managed by legacy data systems
▪ IBM i and Mainframes run the core transactional applications of most enterprises
▪ Growing touchpoints on mobile & online increasing transaction volumes and workload unpredictability
▪ Critical focus on data infrastructure optimization & availability initiatives to meet demand and manage costs
Innovation is enabling analysis of larger, more diverse data
▪ Enterprises making major investments in new Big Data repositories for greater insights
▪ Mobile & online data sources must beanalyzed quickly & efficiently
▪ Next-gen analytic environments must contain valuable app & machine data from legacy systems that is liberated, integrated & trusted
4
Big Iron to Big Data
Syncsort Confidential and Proprietary - do not copy or distribute
Big Iron to Big Data Market Size and Forecast
Syncsort Confidential and Proprietary - do not copy or distribute
$20.4 billionlarge, stable
Data Infrastructure Optimization &
Availability software market
$8 billion~20% annual
growth
Big Datasoftware market
$1.2 billion~25% annual growth
Big Iron to Big Data software market
Big Iron to Big Data
Big IronBig Data
source: Estimates based on 2016 Wikibon study
5
And These Strategies Must Drive Value Across Multiple Domains
▪ Reduce computing costs on legacy data systems including mainframes & IBM i Power Systems
▪ Improve availability, reliability and integrity
▪ Meet growing security and compliance requirements
Data Liberation, Integration & Integrity
▪ Unlock mainframe and IBM i data for machine learning and advanced analytics
▪ Access, transform, integrate & deliver data to analytic environments
▪ Ensure data quality, lineage, security ▪ Enable data consumption on premise
and in the cloud
6
Data Infrastructure Optimization & Availability
Big Iron to Big Data
Syncsort Confidential and Proprietary - do not copy or distribute
Customer Use Cases & Strategic Partnerships
Syncsort Confidential and Proprietary - do not copy or distribute
Data Infrastructure Optimization
• Mainframe Optimization
• Cross-Platform Capacity Management
• EDW Optimization
• Application Modernization
Data Availability
• High Availability & Disaster Recovery
• Mission-Critical Migration
• Cross-Platform Data Sharing
• IBM i Data Security & Audit
Big Iron to Big DataA fast-growing market segment composed of solutions that optimize traditional data systems and
deliver mission-critical data from these systems to next-generation analytic environments.
7
Customer Use Cases & Strategic Partnerships
Syncsort Confidential and Proprietary - do not copy or distribute
Data Integration
• Mainframe Access & Integration for Machine Data
• Mainframe Access & Integration for App Data
• High-performance ETL
Data Quality
• Data Governance
• Customer 360
• Big Data Quality & Integration
• Data Enrichment & Validation
Big Iron to Big DataA fast-growing market segment composed of solutions that optimize traditional data systems and
deliver mission-critical data from these systems to next-generation analytic environments.
8
Customer Use Cases & Strategic Partnerships
Syncsort Confidential and Proprietary - do not copy or distribute
Data Infrastructure Optimization
• Mainframe Optimization
• Application Modernization
• EDW Optimization
• Cross-Platform Capacity Management
Data Availability
• High Availability & Disaster Recovery
• Mission-Critical Migration
• Cross-Platform Data Sharing
• IBM i Data Security & Audit
• Mainframe Access & Integration for Machine Data
• Mainframe Access & Integration for App Data
• High-performance ETL
Data Integration
Data Quality
• Data Governance
• Customer 360
• Big Data Quality & Integration
• Data Enrichment & Validation
Big Iron to Big DataA fast-growing market segment composed of solutions that optimize traditional data systems and
deliver mission-critical data from these systems to next-generation analytic environments.
9
Differentiated Product Portfolio & Technical Expertise
Syncsort Confidential and Proprietary - do not copy or distribute
Data Infrastructure Optimization
Data Availability
Data Integration
Data Quality
Market-leadingdata quality capability
Best-in-class resource utilization and performance, on premise
or in the cloud
#1 in high availability forIBM i and AIX Power Systems
Industry-leading mainframe data access and highest
performing ETL
• Trillium Software System
• Trillium Quality for Big Data
• Trillium Precise
• Trillium Cloud
• Trillium Global Locator
• DL/2
• Zen Suite
• MFX® for z/OS
• ZPSaver Suite
• EZ-DB2
• EZ-IDMS• DMX & DMX-h
• DMX AppMod
• athene®
• athene SaaS®
• MIMIX Availability & DR
• MIMIX Move
• MIMIX Share
• iTera Availability
• Enforcive IBM i Security
• Ironstream®
• Ironstream® Transaction Tracing
• DMX & DMX-h
• DMX Change Data Capture
Big Iron to Big DataA fast-growing market segment composed of solutions that optimize traditional data systems and
deliver mission-critical data from these systems to next-generation analytic environments.
10
USE CASE CHALLENGES DESCRIPTION & BUSINESS VALUE CUSTOMERS
Mainframe Optimization
• Contain hardware and software costs
• Improve performance• Meet/exceed SLAs to
support the business
• Best-in-class solutions for optimizing mainframe performance and resource utilization
• Dramatically reduce the costs by reducing billable CPU time, containing overall software costs and delaying hardware upgrades
• Meet SLAs/batch windows to better serve the business
Cross-Platform Capacity Management
• Capacity, performance incidents impact business
• Difficult to get single view across enterprise
• Lack of time/resources
• Single solution to proactively manage capacity across your enterprise IT services and infrastructure
• Avoid service disruptions• Predict and minimize IT infrastructure costs• Free up staff, maximize accuracy
EDW Optimization
• EDW costs are escalating, performance suffering due to ELT/ETL processing
• Moving data and processing to data lake is complex, takes a long time
• Solutions to shift data and ELT workloads to the data lake quickly and efficiently
• Lower EDW costs and improve performance for Business Intelligence • Increase productivity and get faster time to value by automating,
optimizing and future-proofing development without requiring special skills
Application Modernization
• Minimize risk • Match performance of
application on new system• Lack of skills/resources• Time to value
• Enables migration of applications off the mainframe, with minimum or no modifications, to run on other platforms with the same or faster performance
• Seamless integration with leading re-hosting solutions• No manual coding or mainframe expertise required
Delivering Business Value: Data Infrastructure Optimization
Syncsort Confidential and Proprietary - do not copy or distribute 11
USE CASE CHALLENGES DESCRIPTION & BUSINESS VALUE CUSTOMERS
High Availability & Disaster Recovery
• Minimize risk and costs of downtime
• Meet SLA, compliance requirements
• Resource constraints
• Proven IBM i solution virtually eliminates downtime and data loss for businesses of all sizes, from SMB to enterprise
• Reduce risk and costs of planned administrative downtime, component failures, software defects, operator error, malicious users, and local/regional disaster
Mission-Critical Migration
• Complexity, downtime and risk of migrations
• SLAs, compliance
• Near-zero downtime workload and system migration• Mitigate risk• Ease strain on internal IT resources
Cross-Platform Data Sharing
• Data spread across systems leads to stale, incomplete information
• Traditional approaches of data movement are inefficient, costly
• Real-time data sharing between all leading databases, operating systems, platforms (physical, virtual, cloud)
• Improve decision-making, business productivity with accurate, up-to-date information
• Easy to use – automated, no programming required• Cost effective
IBM i Data Security & Audit
• IBM i has no comprehensive security management
• Risk to your business• Compliance, governance• Lack of IBM i visibility/skills
• Most comprehensive, easy-to-use security & compliance solution for IBM i
• Get complete oversight/control of IBM i security without special skills• Protect critical data from ever-changing threats – external and internal• Satisfy auditors and meet the strictest compliance regulations
Delivering Business Value: Data Availability
Syncsort Confidential and Proprietary - do not copy or distribute 12
USE CASE CHALLENGES DESCRIPTION & BUSINESS VALUE CUSTOMERS
Mainframe Access & Integration for Machine Data
• Mainframe data left out of enterprise-wide views
• Data is diverse, complex• Skills, expertise shortages
• Collect, transform and forward mainframe operational and security data into analytics platforms like Splunk Enterprise
• Include mainframe with your other systems for 360-degree view of your enterprise
• Easy to use - no mainframe expertise required
Mainframe Access & Integration for Application Data
• Critical enterprise data trapped in Mainframe silo –missing from data lake
• Mainframe is expensive• Working with Mainframe
and Big Data requires hard-to-find skills
• Unparalleled technology and expertise for loading Mainframe data into the data lake and integrating it with MapReduce or Spark
• Deliver powerful new insights by combining mainframe data with other data sources
• Save on Mainframe storage and processing costs• Achieve faster time to value• Use skills you already have - no mainframe expertise required
High Performance ETL
• Manual coding is resource-intensive, slow, inflexible and difficult to maintain
• Data Integration software requires tuning for performance/efficiency
• ETL jobs must be rewritten for new execution frameworks
• Fast, efficient, simple no-code ETL solution that runs on any platform – including MapReduce and Spark – on premise or in the cloud
• Get business insights faster with up to 10x lower elapsed processing times, linear scalability
• Shift staff to higher-value initiatives• Future-proof your environment• Save money by optimizing your system resources
Delivering Business Value: Data Integration
Syncsort Confidential and Proprietary - do not copy or distribute 13
USE CASE CHALLENGES DESCRIPTION & BUSINESS VALUE CUSTOMERS
Data Governance
• Regulatory compliance, mitigate risk
• Data volume/complexity• Lack of visibility,
accessibility, trust in the data
• Data profiling integrated with leading data governance and data lineage solutions to identify/assess data across your organization and determine compliance with business rules – on premise or in the cloud
• Identify data issues fast so you can fix them and stay compliant• Have confidence in your data• Empower users with collaborative, web-based tools
Customer 360
• Grow customer base, share of wallet, loyalty
• Increasing volume, complexity and sources of customer data
• Data profiling, quality and enrichment solutions deliver a common, unified, trusted view of your customer data, across all systems and sources – on premise or in the cloud
• Maximize the value, efficiency and integrity of every customer interaction, across the entire customer lifecycle
Big Data Quality & Integration
• Maximize the value/ROI of the data lake
• Ensure the data lake is trusted source of data
• Highly-scalable, enterprise-level data quality that supports Big Data frameworks like MapReduce and Spark to process the largest and most complex data volumes in the data lake – on premise or in the cloud
• Trusted the insights from your most critical data assets• Achieve fast time to value with flexible deployment and ease of use
Data Enrichment & Validation
• Need complete, accurate data for effective customer engagement
• Customer data constantly changes
• Real-time data enrichment, validation and verification of third-party data sources, including email, landline/mobile phone, postal address and internet connectivity
• Engage with your customers and prospects more effectively• Mitigate risk by validating customer data
Delivering Business Value: Data Quality
Leading U.K.
Bank
Syncsort Confidential and Proprietary - do not copy or distribute 14
DATA AVAILABILITY….AND MORE
Security Journal (QAUDJRN)
QHIST Log
File Journals
Message Queue
NEW: Operational Data
Exit Point Traffic*
Encryption*
•View DataView Data*
SQL Audit*
IBM iData Stream
Cross-Platform
Audit
Syslog /SIEM
IFS FilesSPLU
NK
Output TypesData Export
Market Landscape and Key Concepts: IBM i Data
*Requires use of our security products
Having The Big Picture allows organizations analyze these data sets to proactively and reactively
mitigate different risks. Typical reasons to perform data analysis include; compliance with security
policies and/or regulations, oversight of IT operations, and system troubleshooting.
Key Differentiators: Robust Filtering, Batch or Real-Time, Performance Optimized
Market Landscape and Key Concepts: Splunk
High-Level Summary of Splunk • First machine data analysis software
• Market leader
• The “Google for log files” enterprise tool
• Can be expensive –particularly with growing data volumes
• Original target for our Ironstream® products
• More mature partnership
Market Landscape and Key Concepts: SIEM
• SIM: Log management, analytics, compliance reporting
• SEM: Real-time monitoring & incident management
• Advanced Threat Detection
• Basic security monitoring
• Investigation and incident response
Security Information & Event Management
Market Landscape and Key Concepts: SIEM & Big Data
Data Type Use Case Main Players Data Size
API for Other Data
Sources
OS logs can be
correlated against
application activity
Splunk, Elastic
Stack, Sumo Logic
Even Bigger Data
Operations Data Security data &
operations data
converge
Splunk ITSI, Elastic
Stack, Sumo Logic,
Logpoint, QRadar,
LogRhythm
Big Data
SIEM Data Security threat
detection, monitoring
& response
QRadar, Splunk ES,
LogRhythm,
ArcSight, AlienVault
Little Data
SPLUNK
Market Landscape and Key Concepts:Stream Your Big Iron Data into The Industry-Leading Platform For Machine Data
Machine Data: Any Location, Type, Volume
Online
ServicesWeb
Services
ServersSecurity
GPS
Location
StorageDesktops
Networks
Packaged
Applications
Custom
AppsMessaging
TelecomsOnline
Shopping
Cart
Web
Clickstreams
Databases
Energy
Meters
Call Detail
Records
Smartphones
and Devices
RFID
On-
Premises
Private
Cloud
Public
Cloud
Platform Support (Apps / API / SDKs)
Enterprise Scalability
Universal Indexing
Answer Any Question
DeveloperPlatform
Report &analyze
Custom dashboards
Monitor & alert
Ad hoc search
Mainframe
We make Critical IBM i and Mainframe Machine Data USEABLE in Splunk with Ironstream®
IBM i
What Can You Do with Security Analytics Using IBM i Data?
Detect Data Movements – Inbound/Outbound FTP
Dataset access operations– Determine potential security threats based on
unauthorized access attempts– Ensure only authorized users are accessing critical
datasetsPrivileged/non-privileged User Activity Monitoring
– Unusual behavior pattern – off hours connections– High number of invalid logon attempts
Attack Detection– Intrusion, Scans, Floods
Authentication Anomalies– Entered the building at 08:30 but logged on from
another country at 09:00Maintain Compliance
– SOX, HIPAA, PCI, GDPR, Etc.
Key Use Cases – Problems we Solve
Enterprise Security and Compliance/SIEM
– Detect and prevent security threats
– Ensure compliance
– Ensure audits pass
IT Operations Analytics/ITOA/Service Intelligence
– Bigger picture of what's happening in the environment
– Make better decisions to take control of the IT infrastructure
– Ensure SLAs Met
What Can You Do with Security Analytics Using IBM i Data?
Detect Data Movements – Inbound/Outbound FTP
Dataset access operations– Determine potential security threats based on
unauthorized access attempts– Ensure only authorized users are accessing critical
datasetsPrivileged/non-privileged User Activity Monitoring
– Unusual behavior pattern – off hours connections– High number of invalid logon attempts
Attack Detection– Intrusion, Scans, Floods
Authentication Anomalies– Entered the building at 08:30 but logged on from
another country at 09:00Maintain Compliance
– SOX, HIPAA, PCI, GDPR, Etc.
Hadoop | The New Centerpiece of the Modern Data Architecture
What do customers want to use their Hadoop clusters for?
1. ETL
2. Analytics*
3. Data Blending
4. Active Archive
5. EDW / Mainframe Optimization
What do customers get out of it? – Business Benefits
Increase Productivity
Reduce Costs
Next-Gen Analytics
Increase Revenue and Growth
Archive Data
Increase Agility
Get More for EDW/ Mainframe Investment
Retain Data for Compliance
Free Mainframe Resources
Implementation Challenges
1. Data Quality: Assessing and improving quality of data as it enters and/or in the data lake.
2. Skills/Staff: Need to learn a new set of skills, Hadoop programmers are difficult to find and/or expensive.
3. Data Governance: Including data lake in governance initiatives and meeting regulatory compliance.
4. Rapid Change: Frameworks and tools evolve fast, and it’s difficult to keep up with the latest tech.
5. Fresh Data (CDC): Difficult to keep data lake up-to-date with changes made on other platforms.
6. Mainframe: Difficult to move mainframe data in and out of Hadoop/Spark.
7. Data Movement: Difficult to move data in and out of Hadoop/Spark.
0
10
20
30
40
50
% of People Who Consider this a Top Challenge (Rated 1 or 2)
Big Data Challenges
Data Quality Skills Governance Rapid Change
CDC Mainframe Data Movement Cost
Key Concepts: Security and Compliance in IBM i Environments
The IBM i is not inherently a secure system, it is however, extremely securable
Being in compliance doesn’t automatically mean the system is secure
Segregation of duties
– Auditors tend not to approve of tools developed in-house or modules that are part of an ERP
Key Concepts: Security and Compliance in IBM i Environments
New/open-source protocols create access headaches
– Legacy, proprietary protocols and experienced administrators now cohabitate with new/open-source protocols and young techies. All of the access points need to be controlled.
The IBM i is now more exposed to the worldwide hacker community (they finally discovered that it’s a very valuable target)
– First session dedicated to IBM i was presented at Def Con in 2015
Most customers have a security program in placeand claim to be confident in it
Customers are much more involved in, and knowledgeable about, security than in the recent past, which should send a clear message to security software vendors: “We aren’t neophytes so don’t waste our time explaining things we already know, get directly to the point about how you can solve my problems”
From Vision’s 2017 Security Survey - IBM Power
My organization’s IT systems are less secure
My organization’s IT systems are about as secure as they were last
year
My organization’s IT systems are more secure
Don’t know
0% 10% 20% 30% 40% 50% 60%
How secure are your organization’s IT systems today compared to a year
ago?
Yes
No
Don't know
0% 10% 20% 30% 40% 50% 60% 70% 80%
Do you have a formal security program in place for IT systems?
Very confident
Somewhat confident
Neither confident norunconfident
Somewhat unconfident
Very unconfident
0% 10% 20% 30% 40% 50%
How confident are you in the effectiveness of your
organization’s security program?
Compliance is a primary driver of IT security
• It is rare to find a company today that doesn’t need to comply with a regulation or external audit requirement
• Countries in secondary markets have their own regulations (Angola, Lebanon, Colombia, etc…) or are in the process to bringing them online (China)
Compliance needs
Industry best practices
Results of security audit
Risk-based framework
Cost-value analysis
Experience from past attacks
Board recommendations
Don’t know
Other (please specify)
0% 10% 20% 30% 40% 50% 60% 70% 80%
What resources and information were used to define your organization’s security program?
From Vision’s 2017 Security Survey - IBM Power
From Vision’s Risk Management Survey – IBM i, Dec. 2016 – respondents are Vision customers
What IBM i security software is already implemented?
45% of respondents have not implemented security software and 12% don’t know if they have (yellow highlight)
40% have implemented a Help Systems security product (in red)
4.5% have implemented Enforciveand 1.7% Cilasoft
Numbers are likely biased toward North America so would likely be different in other geographies
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%
Help Systems Powertech tools
Help Systems Bytware tools
Townsend Software
Enforcive Enterprise Security
Help Systems SkyView tools
NetIQ Secure Configuration…
Help Systems CCSS tools
Linoma Crypto Complete
Help Systems Safestone tools
Cilasoft Suite of products
IBM Power SC
Raz-Lee iSecurity suite of products
Cosyn Software
Secure Sphere Imperva
I don't know
Other (please specify)
We have not deployed security…
Security Solutions
Finding Opportunities with the Security Risk Assessment Tool
What it Is:
– Free, self-service assessment tool that enables IBM i administrators and security officers to identify security vulnerabilities by conducting real-time, fully controlled penetration attempts on their IBM i environment.
– Customer receives an Executive Summary of the results, along with specific recommendations for remediation.
– Customer registers for access on our website, downloads the tool, installs the software. Assessment runs as a native job on the target computer.
It´s all about Data replication
Todays market
Storage Replication
• Easy to use
• Pushed by IBM/partners
Software Replication
• Near ´Zero´downtime
• Continous Data Protection
• Optimum in RPO and RTO
Mimix for FlashCopy
• Hybrid solution
• Different Data with indivdual
RPO/RTO requirements
The Game-changer
The Idea• Opens the market of storage replication that was mostly inaccessible
• Win net new logos
• Provides a new opportunity to return to customers we lost to SAN replication
The Channel• Opens a whole new channel of partners focused on SAN storage
• Vision direct customers with storage replication in partnership with sophisticated partner
IBM FlashCopy Challenges
Switched offIPL phaseApplication downQUIESCEFORCE WRITE DATAJust FlashCopy
CertainCompromise
Uncertain
File 3
File1
File2
File 3
File 1File 2
tFlash-suspendtr2
t r3
t r1
Rollback Flash-none suspend
RCV0001
RCV0001 RCV0002
RCV0001
t1
RCV0001 RCV0002
t2 t3 t4
Tue 06:00am
Tue12:00pm
Tue 06:00pm
FlashCopy Consistency
FlashCopy Journal Chain FlashCopy Timeline
FlashCopy Commit & Rollback
Flash1
Flash2
Flash3
LIVE