© 2009 IBM Corporation
Integrated Data Management
Data Security & Privacy for iSeries
Dean CompherBig Data Portfolio Technical Sales Specialist
www.db2Dean.com
facebook.com/db2Dean
@db2Dean
© 2009 IBM Corporation
Integrated Data Management
Perimeter Defenses No Longer Sufficient
2
“A fortress mentality will not work in cyber. We cannot retreat behind a Maginot Line of firewalls.”
Outsourcing
Web-Facing Apps
Legacy App Integration/SOA
Employee Self-Service, Partners & Suppliers
Insiders (DBAs, developers, outsourcers, etc.)
Stolen Credentials (Zeus, etc.)
- William J. Lynn III, U.S. Deputy Defense Secretary
© 2009 IBM Corporation
Integrated Data Management
Addressing the Full Lifecycle of Database Security & Compliance
© 2009 IBM Corporation
Integrated Data Management
Agenda
• Data Security – Guardium Database Activity Monitoring
• Alert on Access Policy Violations
• Audit and Report Activity
• Data Privacy – Otpim Test Data Management
• Mask Data Copied to Test
• Create Subsets
• Automate Test Data Refresh
• Improve Security with Better Testing
© 2009 IBM Corporation
Integrated Data Management
Guardium Database Activity Monitoring
© 2009 IBM Corporation
Integrated Data Management
Collector
Real-Time Database Monitoring with InfoSphere Guardium
• Non-invasive architecture
– Outside database
– Minimal performance impact (1-3%)
– No DBMS or application changes• Cross-DBMS solution• 100% visibility including local DBA
access
• Enforces separation of duties• Does not rely on DBMS-resident logs
that can easily be erased by attackers, rogue insiders
• Granular, real-time policies & auditing– Who, what, when, how• Automated compliance reporting, sign-
offs & escalations (SOX, PCI, NIST, etc.)
Host-based Probes (S-TAPs)
© 2009 IBM Corporation
Integrated Data Management
Scalable Multi-Tier Architecture
•Integration with LDAP, IAM, IBM Tivoli SIEM, IBM TSM, Remedy, …
iSeries
© 2009 IBM Corporation
Integrated Data Management
Extend real-time Data Activity Monitoring to also protect sensitive data in data warehouses, Big Data Environments and file shares
Integration with LDAP, IAM, SIEM, TSM, Remedy, …
NEW
Big Data Environments
DATA
InfoSphere BigInsight
s
© 2009 IBM Corporation
Integrated Data Management
• Providing complete and native data security solution for System I (DB2 6.1, 7.1)
• Monitors privileged user activity in real time
• Enables complete separation of duties • Helps satisfy auditor’s requirements and
ensure compliance
Protect sensitive data on your System i deployments, ensuring compliance to mandates like PCI easily and cost effectively
Protect sensitive data on your System i deployments, ensuring compliance to mandates like PCI easily and cost effectively
Extended data security platform coverage
S-TAP for System i
S-TAP for System i
System i S-TAP for System i
© 2009 IBM Corporation
Integrated Data Management
3 Types of Rules
SQL Query
Result Set
Database Server
Database
Exception (ie. Invalid table)
There are three types of rules:
1. An access rule applies to client requests
2. An extrusion rule evaluates data returned by the server
3. An exception rule evaluates exceptions returned by the server
1
2
3
© 2009 IBM Corporation
Integrated Data Management
Fine-Grained Policies with Real-Time Alerts
Application Server
10.10.9.244
Database Server
10.10.9.56
© 2009 IBM Corporation
Integrated Data Management
2. Extrusion Definition to Alert on Unauthorized Results Set
• Monitor 10.10.9.248
• SQL Server database
• Not user Bill
• Send Alert per match
© 2009 IBM Corporation
Integrated Data Management
•Should my customer service rep view 99 records in an hour?•Is this normal?Is this normal?
Monitoring Data Extrusion
© 2009 IBM Corporation
Integrated Data Management
3. Policy Exception Rule - Preventing Attacks
Rogue users know what they’re looking for, but...
SQL injection leads to SQL errorsSQL errors!
Guardium: 100% visibility with real-time alerts …
They don’t always know where to find it!
Brute force attacks result in failed failed loginslogins!
© 2009 IBM Corporation
Integrated Data Management
• Issue: App server uses generic service account to access DB -- which doesn’t identify WHO initiated transaction (connection pooling)
• Solution: Track access to application user associated with specific SQL commands
• Deterministic identification vs. time-based “best guess”
• Out-of-the-box support for all major enterprise apps (Oracle EBS, PeopleSoft, SAP, Siebel, Business Objects, Cognos, etc.)
• Plus custom apps (WebLogic, WebSphere, Oracle AS, etc.)
• No changes to applications
Identifying Fraud via Application-Layer Monitoring
•Application Server
•Database Server
•Joe •Marc
•AppUser
© 2009 IBM Corporation
Integrated Data Management
Workflow Automation
• Schedule & automate tasks
• Compliance reporting
• Automatically generate reports
• Distribute to oversight team
• Track electronic sign-offs
• Escalate when required
• Store process trail in secure repository
• Demonstrates oversight process for auditors
© 2009 IBM Corporation
Integrated Data Management
23
Accelerators• Software modules harnessing Guardium's extensive capabilities to
address the requirements of security mandates
• Customizable mandate-specific reports, policies, tools and workflows
• Greatly improve security and streamline audit preparation
• Increased operational efficiency through automation of compliance
• Simplified validation of broad ranges of requirements
Sarbanes-Oxley
PCIGLBAHIPAABase II
© 2009 IBM Corporation
Integrated Data Management
Protect data in real-time and ensure compliance in unstructured Hadoop big data environments
Introducing Hadoop Activity Monitoring Monitor and Audit Hadoop activity in real-time to support compliance requirements and protect data
• Real time activity monitoring of HDFS, MapReduce, Hive and HBASE data sources• Automated compliance controls• Fully integrated with InfoSphere Guardium solution for database activity monitoring • View Hadoop systems with other data sources
Big data brings big security challenges As big data environments ingest more data, organizations will face significant risks and threats to the repositories in which the data is kept
Big data environments help organizations: Process, analyze and derive maximum value from these new data formats as well as traditional structured formats in real-time
Make more informed decisions instantaneously and cost effectively•Turn 12 terabytes of Tweets into improved product sentiment analysis• Monitor 100’s of live video feeds from surveillance cameras to identify security threats
NEW
© 2009 IBM Corporation
Integrated Data Management
Expand system openness and integration with Universal Feed
Universal Feed opens InfoSphere Guardium system, enabling all capabilities to be applied to custom applications and niche data sources
• Open InfoSphere Guardium protocol (agent to Collector) integration to clients and 3rd party companies
Provides a means of supporting fragmented segments of the market: custom applications, niche databases, etc.
Data auditing model; not a SIEM
• Customer/partner responsible for developing interface to system to be integrated (e.g. S-TAP equivalent)
Open industry standard protocol used to simplify development
• Supports full capabilities, or subset of InfoSphere Guardium capabilities
Monitoring and protection Real-time Secure audit trail, compliance workflow automation, etc.
© 2009 IBM Corporation
Integrated Data Management
Universal Feed Overview
Guardium Appliance
Universal Feed Agent
CapturingEvents
Universal Feed Agent Agent developer for universal feed agent
Partner Customer 3rd Party
Guardium Collector Accepting connections from the Universal Feed
Agent
Sending Audit Data via Guardium messages
Sending Information
-- - ---- - - - - - - - ---- -- - - -
Process & StoreAudit messages
Send Alert
Responsible for capturing events with audit interest
Responsible for sending the audit data using Guardium defined messages
Processing and storing audit data
Alerting if Universal Feed Agent doesn’t send heart beat
Responsible for receiving and processing Guardium messages (policies, pings, etc)
Sending information to Universal Feed agent (policy, pings, etc)
-- - ---- - - - - - - - ---- -- - - -Receiving &
processing
Guardium Toolkit
Agent developer
© 2009 IBM Corporation
Integrated Data Management
InfoSphere Optim Data Privacy & Test Data Management
© 2009 IBM Corporation
Integrated Data Management
Source DataSource Data
InfoSphere Optim: Intelligent Move of Structured Data
CurrentCurrent
Production
ExtractExtractExtractExtract
RestoreRestoreRestoreRestoreRetrievedRetrieved
Production Archive
Contextual DataContextual Data
Reference DataReference Data
Intelligent Move of Structured Data is a process that captures contextual source data for the purpose of Archiving and Accessing historical data
Data PrivacyData Privacyfor Test Data for Test Data Data PrivacyData Privacyfor Test Data for Test Data
SQL access to SQL access to Archived DataArchived Data
or Development Test Data
or Populating Test Databases with privatized data
Universal Access to Archived DataUniversal Access to Archived Data
ODBC / JDBC XML Report WriterApplication IBM Mashup
© 2009 IBM Corporation
Integrated Data Management
Supporting Enterprise Environments
Organization environments are diverse, yet interrelated therefore what you use to manage the data MUST support across your environment
Data GrowthData PrivacyTest Data Management Application Retirement
Discovery
© 2009 IBM Corporation
Integrated Data Management
Our Unique Capability: The Complete Business Object
DBA viewReferentially-intact
subset of data
DBA viewReferentially-intact
subset of data
Business view “reference
snapshot” of business activity
Business view “reference
snapshot” of business activity
OracleOracle
DB2DB2
SybaseSybase
AdabasAdabas
Federated access to data and metadata
Federated access to data and metadata
Related LUW Files or
Documents
Related LUW Files or
Documents
© 2009 IBM Corporation
Integrated Data Management
31
Information Management
31
- Archive & Delete
- Archive Only
- Reference Only
F0411 F0413
F0414
AP Ledger AP Header
AP Details
F0010 F0006
BU MasterCompany Master
F0901
F0902
F4008
Tax AreaAccountMaster
F0018
Batch Control F0101
F0909
AccountBalances
F0911T
F0011
F0911
Account LedgerLedger Tag
Chart of A/C
Tax table
AB Master
F0012
F0025
AAI’s
LT Master
F0004UDC Types
F0005UDC
F00151Currency Exchange Rate Header
Currency RestatementRate
F1113
F0008Fiscal Date Pattern
F0015Currency Ex. Rate
F0013Currency Codes
F11151Currency Ex. Rate Calculation
F0014Payment Terms
F0401Supplier Master
Example: JD Edwards Accounts Payable Archiving
© 2009 IBM Corporation
Integrated Data Management
• DB Relationships are automatically derived from database RI rules
• Application Specified Relationships
• Can be defined individually to Optim
• Can be imported into Optim from DDL
• Can be automatically discovered by InfoSphere Discovery
• Shared by all Optim components
OPTIM
Relationships OptimDIRECTORY
Tables
Referential Integrity Rules
AccessDefinitions
DB Aliases
Maps
Stored in Database- Catalog- System Tables- Data Dictionary
A Word About Relationships...
© 2009 IBM Corporation
Integrated Data Management
Automate Discovery and Accelerate Information Understanding
• Significant Acceleration of Information Agenda projects
• Application/Data Consolidation, Migration & Retirement
• Data Growth Management
• Master Data Management and Data Warehousing
• Test Data Management
• Sensitive Data De-identification
• Why is this Different?
• Data-based discovery
• Automate discovery of business entities, cross-source business rules & transformation logic
• Evaluate multiple data sources simultaneously
• Identify & remediate cross-system rules and inconsistencies
© 2009 IBM Corporation
Integrated Data Management
InfoSphere Optim Deep Dive:Test Data Management
© 2009 IBM Corporation
Integrated Data Management
Drivers for Test Data Management Projects
• Quality• Bad data • Unidentified test cases• Test Automation approach (Rational Borland MI…) • Verification of test results
• Parallelism (Multiple Sandboxes)
• Tunnel effect • Multi project testing
• Storage
• Reduce storage • Include into a cost control project
• Data Privacy / Compliance
© 2009 IBM Corporation
Integrated Data Management
Unit Test
How Does Test Data Management Impact Storage Cost?
Production
Training
System Test
UAT
Integration
Before TDM
With TDM
Production 500GB 500GB
Training 500GB 25GB
Unit Test 500GB 25GB
System Test 500GB 500GB
UAT 500GB 25GB
Integration 500GB 25GB
Test Data 2.5TB 0.6TB
© 2009 IBM Corporation
Integrated Data Management
37
Relational Extract
Relational Extract
Relational Edit
Relational Compare
Relational Edit
Inspect and Add Datato Test Error Routines
Correct Errors inProduction Data
Compare Before/AfterData
Compare Before/AfterData
TEST
Go Production !!!
Create/Modify Application
Refresh Test Data
Optim Archive
Archive Old DataArchive Old Data
Subset and Privatize
Copy Production Data for Testing
InfoSphere Optim Test Data Management Solution
© 2009 IBM Corporation
Integrated Data Management
ExtractFile
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
-- ---- ---- ---- ------- ----CUST
-- ---- ---- ---- ------- ----ORD
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
DETL
TESTDB
-- ---- ---- ---- ------- ----CUST
-- ---- ---- ---- ------- ----ORD
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
DETL
QADB
LOAD
INSERT/UPDATE
LoadFiles
The Relational Extract Facility
Extract a relationally intact subset from production database(s)
• Extract data and/or object definitions• From multiple tables (files) that are related• From multiple tables (files) that are not related• From single tables (files)• All data or subset
• Define a new set of test tables
• Populate Target databases
• Refresh Target databases
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
-- -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ----
CUSTOMERS
ORDERS
DETAILS
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
-- -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ----
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
CUSTOMERS
ORDERS
DETAILS
-- ---- ---- ---- ------- ----CUST
-- ---- ---- ---- ------- ----ORD
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
DETL
NewDB
Create
-- ---- ---- ---- ------- ----CUST
-- ---- ---- ---- ------- ----ORD
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
DETL
New_DB
Create
Saves:Programmer/DBA timeDisk space utilizationTesting interference
© 2009 IBM Corporation
Integrated Data Management
Traditional vs. Relational Tools
One table/view at a time
No edit of related datafrom multiple tables
FIND CUSTOMERNOTE INFOEXIT TABLE
FIND ORDERSNOTE INFOEXIT TABLE
FIND DETAILSNOTE INFOEXIT TABLE CUSTOMERS
ORDERS
DETAILS
........................ ........................ ........................ ........................ ........................
Single Table Editors The Relational Editor
Simultaneous browse/edit of related data from multiple tables
Speeds time to create boundary test cases.
Simplifies edit process.
© 2009 IBM Corporation
Integrated Data Management
Optim’s Relational Compare Facility
• Single-table or multi-table compare
• Creates compare file and/or compare Report of results
• For application testing, QA, and to verify database contents
• Enhances productivity by finding unexpected changes in the data
SOURCE 1
SOURCE 2
COMPAREPROCESS
OptimCOMPARE
FILE
OptimCompareREPORT
........................ ........................ ........................ ........................ ........................
Interactive Browse
Verify Test Results
Saves QA Validation timeImproves Test Accuracy
© 2009 IBM Corporation
Integrated Data Management
Architecture: Test Data Management/Data Privacy
Optim Repository(OptimDir)
Optim Server
Extract files
QFED
Windows
Windows, Unix, Linux, zOs
Test system 1
Test system 2
Test System 3
-- ---- ---- ---- ------- ----CUSTOMER
-- ---- ---- ---- ------- ----EMPL
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
-- ---- ---- ---- ------- ----HR
-- ---- ---- ---- ------- ----EMPL
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
-- ---- ---- ---- ------- ----FINANCE/BUDGET
-- ---- ---- ---- ------- ----EMPL
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
Mas
k on
ext
ract
Mask on load
LoadFiles
Mask on insert
Test System 4
-- ---- ---- ---- ------- ----FINANCE/BUDGET
-- ---- ---- ---- ------- ----EMPL
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
Oracle 9HP UX
Sql Svr 2K ??Windows ??
OptimWorkstation
Application 2
Server Name• Server address or name •DB Alias• Connectivity via DB Client softwareWork Directory• Server File SystemStorage Profile• Storage and retention policy
Server Name• Server address or name •DB Alias• Connectivity via DB Client softwareWork Directory• Server File SystemStorage Profile• Storage and retention policy
DB2/i
© 2009 IBM Corporation
Integrated Data Management
Optim Data Privacy
© 2009 IBM Corporation
Integrated Data Management
Optim™ Data Privacy Solution
Production
Contextual,Application- Aware,
Persistent Data Masking
Contextual,Application- Aware,
Persistent Data Masking
EBS / Oracle
Custom / Sybase
Siebel / DB2
Test
EBS / Oracle
Custom / Sybase
Siebel / DB2
• Substitute confidential information with fictionalized data
• Deploy multiple masking algorithms
• Provide consistency across environments and iterations
• Enable off-shore testing
• Protect private data in non-production environments
© 2009 IBM Corporation
Integrated Data Management
Drivers for Privacy of non production data
• Regulatory & Compliance
• PCI
• HIPPA
• EU Safe Harbour
• ….
• Offshoring test
• Sub subcontracting test & dev.
• Good business practice
• Sensitive data
• Training environnements
© 2009 IBM Corporation
Integrated Data Management
ExtractFile
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
Transform / mask sensitive data
-- ---- ---- ---- ------- ----CUST
-- ---- ---- ---- ------- ----ORD
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
DETL
TESTDB
-- ---- ---- ---- ------- ----CUST
-- ---- ---- ---- ------- ----ORD
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
DETL
QADB
LOAD
INSERT/UPDATE
LoadFiles
Data Privacy in Application Testing
Extract a relationally intact subset from production database(s)
• Most Secure Approach
• Extract data only
• Convert during extract
•Extract file already contains masked data
•Can be shared with testers to reuse
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
-- -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ----
CUSTOMERS
ORDERS
DETAILS
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
-- -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ------ -- ------ -- --------- ----
-- ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ------ ---- ---- ---- ------- ----
CUSTOMERS
ORDERS
DETAILS
Only Users authorized to see Private data
© 2009 IBM Corporation
Integrated Data Management
Masking Functions
Column Map Map unlike column names
Transform/mask sensitive data
Datatype conversions
Column-level semantic date aging
Literals
Registers
Calculations
Default values
Substring
Exits
Currency conversion
Social Security (US ……)
Credit Card
Hash Lookup
Lookup
Random Lookup
NAME tables (US)
ADDRESS table (US)
Shuffle
String manipulation
…
…
…
© 2009 IBM Corporation
Integrated Data Management
Consistent Masking and Propagation across the Enterprise
Masked fields are consistent
Data is masked
SS#s
157342266
132009824
SS#s
157342266
132009824
DB2
SSN#s
134235489
323457245
SSN#s
134235489
323457245
Client Billing Application
© 2009 IBM Corporation
Integrated Data Management
Thank You