Post on 19-Jun-2015
description
transcript
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved
Transforming IT Infrastructure
Tim GormanTechnical Consultant
Oracle OpenWorld 2014
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved
Virtualized storage is fast becoming the new norm.
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved
Virtualized storage is fast becoming the new norm.
Nobody can justify provisioning non-production environments the way they
did up to now.
Private and confidential© 2014 Delphix. All Rights Reserved© 2014 Delphix. All Rights ReservedPrivate and confidential 4
Accelerating the tempo of application development
Fortune 1 Retailer #1 Social Network #1 Financial Services #1 Network Equipment #1 Cable
#1 Wholesale #1 Food Service Co. #1 Office Supplies #1 Apparel & Footwear #1 Chip Manufacturing
#1 Pharma #1 Auction Marketplace #1 Total Healthcare #1 Aerospace #1 Computer Access
#1 CPG #1 ETL Software #1 Availability Service #1 Mutual Life Ins. #1 Satellite TV
#1 State Gov #1 Cruise Line #1 Retirement Fund #1 IT Services #1 Game Software
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 5
• The problem• Attempting to find a solution• A solution• Delphix• Wrap-up / Q&A
Agenda
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 6
• PROD is what we care most about…
The problem
PROD
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 7
• PROD is what we care most about…– But behind every PROD…
• There is QATEST…
The problem
PROD QATEST
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 8
• PROD is what we care most about…– But behind every PROD…
• There is QATEST, DEV…
The problem
PROD QATEST DEV
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 9
• PROD is what we care most about…– But behind every PROD…
• There is QATEST, DEV…• …and other project-specific environments to support
The problem
PROD QATEST DEV
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 10
• Also, as PROD grows…
The problem
PROD PROD
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 11
DEVQATEST
• Also, as PROD grows…– Then so do the supporting non-PROD environments…
The problem
PROD
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 12
The problem
CEO
COO
CFO CMO CSO CIO
• And so inevitably the situation shifts from this…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 13
The problem
CEO
COO
CFO CMO CSO CIO
StorageAdministrator
• To this…
* Inspired by Connor McDonald (http://connormcdonald.wordpress.com/)
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 14
• Some try to mitigate the problem by copying subsets of PROD…
Attempting to solve the problem…
PROD QATEST DEV UAT
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 15
• Some try to mitigate the problem by copying subsets of PROD…
• …but it is difficult and time-consuming and expensive to create referentially-correct subsets of databases.
Attempting to solve the problem…
PROD QATEST DEV UAT
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 16
• Some try to mitigate the problem by copying subsets of PROD…
• …but it is difficult and time-consuming and expensive to create referentially-correct subsets of databases. Most give up…
Attempting to solve the problem…
PROD QATEST DEV UAT
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 17
• What inevitably happens is that cloning is performed less frequently…
• …meaning that schemas and data are older, dirtier due to previous testing, and too small to accurately represent reality
Attempting to solve the problem…
PROD QATEST DEV UAT
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 18
PROD
• Creating snapshots with mirror splits…– Mirror the volume one extra copy
Attempting to solve the problem…
PRODPROD
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 19
PROD
• Creating snapshots with mirror splits…– Mirror the volume one extra copy– Then cease writes to that extra copy at a specific time and
make it separately addressable as a separate volume
Attempting to solve the problem…
PRODPROD
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 20
PROD
• Creating snapshots with mirror splits…– Mirror the volume one extra copy– Then cease writes to that extra copy at a specific time and
make it separately addressable as a separate volume– To create another snapshot, “re-silver” the volume into the
mirror set
Attempting to solve the problem…
PRODPROD
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 21
PROD
• Creating snapshots with mirror splits…– Mirror the volume one extra copy– Then cease writes to that extra copy at a specific time and
make it separately addressable as a separate volume– To create another snapshot, “re-silver” the volume into the
mirror set– And it’s ready to split again
Attempting to solve the problem…
PRODPROD
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 22
PROD
• Creating snapshots with mirror splits…– Mirror the volume one extra copy– Then cease writes to that extra copy at a specific time and
make it separately addressable as a separate volume– To create another snapshot, “re-silver” the volume into the
mirror set– And it’s ready to split again
– While we’ve solved the problem of quickly creating new copies
Attempting to solve the problem…
PRODPROD
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 23
PROD
• Creating snapshots with mirror splits…– Mirror the volume one extra copy– Then cease writes to that extra copy at a specific time and
make it separately addressable as a separate volume– To create another snapshot, “re-silver” the volume into the
mirror set– And it’s ready to split again
– While we’ve solved the problem of quickly creating new copies• We haven’t solved the problem of using a full image
Attempting to solve the problem…
PRODPROD
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 24
PROD
• Creating snapshots with mirror splits…– Mirror the volume one extra copy– Then cease writes to that extra copy at a specific time and
make it separately addressable as a separate volume– To create another snapshot, “re-silver” the volume into the
mirror set– And it’s ready to split again
– While we’ve solved the problem of quickly creating new copies• We haven’t solved the problem of using a full image• And re-silvering is not a fast operation, so we need time to “reload”
Attempting to solve the problem…
PRODPROD
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 25
Attempting to solve the problem
CEO
COO
CFO CMO CSO CIO
StorageAdministrator
We’re still at square one…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 26
Attempting to solve the problem
CEO
COO
CFO CMO CSO CIO
StorageAdministrator
We’re still at square one…
…and it gets even worse
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 27
Attempting to solve the problem
CEO
COO
CFO CMO CSO CIO
StorageAdministrator
We’re still at square one…
…and it gets even worse
Because now we have to predictthe size of everything withperfect accuracy
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 28
Attempting to solve the problem
CEO
COO
CFO CMO CSO CIO
StorageAdministrator
We’re still at square one…
…and it gets even worse
Because now we have to predictthe size of everything withperfect accuracy
and then pre-allocatefor copies as well as volumes
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 29
Attempting to solve the problem
CEO
COO
CFO CMO CSO CIO
StorageAdministrator
We’re still at square one…
…and it gets even worse
Because now we have to predictthe size of everything withperfect accuracy
and then pre-allocatefor copies as well as volumes
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 30
So, in a perfect world what we’d like to see is something like this…
Attempting to solve the problem…
PROD
DEV for Tom
DEV for Dick
DEV for Sally
DEV for Harry
DEV for Becky
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 31
Attempting to solve the problem…
PROD
DEV
Tom Dick Sally
Harry
Becky
…in the real world we see this…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 32
• It’s not easy for developers or testers to share environments simultaneously
Attempting to solve the problem…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 33
• It’s not easy for developers or testers to share environments simultaneously– Changes made by each user conflict with one another
Attempting to solve the problem…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 34
• It’s not easy for developers or testers to share environments simultaneously– Changes made by each user conflict with one another
• So it becomes necessary to serialize access to the limited number of environments
Attempting to solve the problem…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 35
• It’s not easy for developers or testers to share environments simultaneously– Changes made by each user conflict with one another
• So it becomes necessary to serialize access to the limited number of environments– Each user queues at the resource, so their project takes more
time elapsed
Attempting to solve the problem…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 36
• It’s not easy for developers or testers to share environments simultaneously– Changes made by each user conflict with one another
• So it becomes necessary to serialize access to the limited number of environments– Each user queues at the resource, so their project takes more
time elapsed– Each user destructively changes the resource
Attempting to solve the problem…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 37
• It’s not easy for developers or testers to share environments simultaneously– Changes made by each user conflict with one another
• So it becomes necessary to serialize access to the limited number of environments– Each user queues at the resource, so their project takes more
time elapsed– Each user destructively changes the resource
• And now we need a way to reset or clean up the environment between each use
Attempting to solve the problem…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 38
• It’s not easy for developers or testers to share environments simultaneously– Changes made by each user conflict with one another
• So it becomes necessary to serialize access to the limited number of environments– Each user queues at the resource, so their project takes more
time elapsed– Each user destructively changes the resource
• And now we need a way to reset or clean up the environment between each use– Reset or clean up takes time, which causes further delays
between uses
Attempting to solve the problem…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 39
• The biggest problem with cloning environments and databases is all the duplication
A solution…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 40
• The biggest problem with cloning environments and databases is all the duplication– Most (if not all) of the blocks comprising the database or file-
system exactly the same
A solution…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 41
• The biggest problem with cloning environments and databases is all the duplication– Most (if not all) of the blocks comprising the database or file-
system exactly the same
• What if we store only one copy of each version of a block?
A solution…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 42
• The biggest problem with cloning environments and databases is all the duplication– Most (if not all) of the blocks comprising the database or file-
system exactly the same
• What if we store only one copy of each version of a block?– Copy-on-write (COW)
• Never modify a block, but insert a new version instead
A solution…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 43
• The biggest problem with cloning environments and databases is all the duplication– Most (if not all) of the blocks comprising the database or file-
system exactly the same
• What if we store only one copy of each version of a block?– Copy-on-write (COW)
• Never modify a block, but insert a new version instead
• Then let the storage figure out what we’re supposed to be seeing?
A solution…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 44
• The biggest problem with cloning environments and databases is all the duplication– Most (if not all) of the blocks comprising the database or file-
system exactly the same
• What if we store only one copy of each version of a block?– Copy-on-write (COW)
• Never modify a block, but insert a new version instead
• Then let the storage figure out what we’re supposed to be seeing?– The storage maintains the relationships and versions on
retrieval
A solution…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 45
• The biggest problem with cloning environments and databases is all the duplication– Most (if not all) of the blocks comprising the database or file-
system exactly the same
• What if we store only one copy of each version of a block?– Copy-on-write (COW)
• Never modify a block, but insert a new version instead
• Then let the storage figure out what we’re supposed to be seeing?– The storage maintains the relationships and versions on
retrieval
• And then make it portable so that ANY platform (Unix/Linux, Windows) can use it?
A solution…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 46
• The biggest problem with cloning environments and databases is all the duplication– Most (if not all) of the blocks comprising the database or file-
system exactly the same
• What if we store only one copy of each version of a block?– Copy-on-write (COW)
• Never modify a block, but insert a new version instead
• Then let the storage figure out what we’re supposed to be seeing?– The storage maintains the relationships and versions on
retrieval
• And then make it portable so that ANY platform (Unix/Linux, Windows) can use it?– NFS (network file system)
A solution…
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 47
• Cloning the old way…
A solution…
PROD DEV
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 48
• Cloning the smart way…
A solution…
PROD DEV
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 49
• Cloning the smart way…– As changes are made to the clone, only those changed blocks
are stored…
A solution…
PROD DEV
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 50
• Cloning the smart way…– As changes are made to the clone, only those changed blocks
are stored…
• Takes less storage, faster to copy• The cost of the clone has gone way down in terms of
storage and time
A solution…
PROD DEV
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 51
• Cloning the smart way…– As changes are made to the clone, only those changed blocks
are stored…
• Takes less storage, faster to copy• The cost of the clone has gone way down in terms of
storage and time
A solution…
PROD
CapEx
DEV
Private and confidential© 2014 Delphix. All Rights ReservedPrivate and confidential© 2014 Delphix. All Rights Reserved 52
• Cloning the smart way…– As changes are made to the clone, only those changed blocks
are stored…
• Takes less storage, faster to copy• The cost of the clone has gone way down in terms of
storage and time
A solution…
PROD
CapEx OpEx
DEV
Delphix Engine
• Software appliance installed as a virtual machineStorage LUNs are presented through the hypervisorVirtualized data is presented to target servers using NFS
Delphix Engine
• Software appliance installed as a virtual machineStorage LUNs are presented through the hypervisorVirtualized data is presented to target servers using NFS
• Target server(s) are installed with Oracle databaseVirtual databases are just Oracle databases on NFSDatabase files reside on the Delphix Engine
Delphix Engine
• Software appliance installed as a virtual machineStorage LUNs are presented through the hypervisorVirtualized data is presented to target servers using NFS
• Target server(s) are installed with Oracle databaseVirtual databases are just Oracle databases on NFSDatabase files reside on the Delphix Engine
• Source database(s) are copied into the Delphix Engine• Using standard RMAN functionality
• Not just a point-in-time image, but a continuous TimeFlow
© 2014 Delphix. All Rights ReservedPrivate and confidential 56
The Delphix Ecosystem: Source Environment
Source Environme
nt
Target Environments
SSH, JDBC
NFS
Any Storag
e
FC
The Source Environment is the host or cluster on which the instance(s) for the source database runs. Typically this is the production environment.
SSH, DSP
JDBC
© 2014 Delphix. All Rights ReservedPrivate and confidential 57
The Delphix Ecosystem: Source Database
Target Environments
SSH, JDBC
NFS
Any Storag
e
FC
The Source Database is the collection of Oracle files that make up an Oracle Database. Using RMAN APIs, a full level 0 backup is initially taken directly into Delphix. Level 1 incremental backups and archive/redo data will be used to keep the Delphix System up to date. RMAN operations are run via the Delphix SnapSync service. Log shipments are performed via the LogSync service.
SSH, DSP
JDBC
Source Environme
nt
© 2014 Delphix. All Rights ReservedPrivate and confidential 58
The Delphix Ecosystem: Delphix Server
Target Environments
SSH, JDBC
NFS
Any Storag
e
FC
The Delphix Server resides in a VMWare infrastructure and can use any storage as a backend. It hosts the Delphix management GUI and can be accessed via a web browser, the Delphix CLI, or RESTful API.
SSH, DSP
JDBC
Source Environme
nt
© 2014 Delphix. All Rights ReservedPrivate and confidential 59
The Delphix Ecosystem: dSource
Target Environments
SSH, JDBC
NFS
Any Storag
e
FC
The dSource is the Delphix Server’s compressed representation of a Source Database. Incremental change data is ingested into the dSource, building a window of change data over time called the Timeflow. This Timeflow data serves as the basis for all Virtual Database storage within Delphix.
SSH, DSP
JDBC
Source Environme
nt
© 2014 Delphix. All Rights ReservedPrivate and confidential 60
The Delphix Ecosystem: Target Environment
Target Environments
SSH, JDBC
NFS
Any Storag
e
FC
A Target Environment is a host or cluster that runs the Oracle instance(s) for Virtual Databases. This is typically the same database servers where development, QA, reporting, or other target databases run today.
SSH, DSP
JDBC
Source Environme
nt
© 2014 Delphix. All Rights ReservedPrivate and confidential 61
The Delphix Ecosystem: Virtual Database
Target Environments
SSH, JDBC
NFS
Any Storag
e
FC
A Virtual Database, or VDB, is a fully functional read/write database that runs on a Target Environment. Storage for the VDB is completely managed by the Delphix Server and provided to the Target Environment via NFS. All changes to the VDB are stored back on the Delphix Server.
SSH, DSP
JDBC
Source Environme
nt
Delphix Engine features and use cases
• Platforms DatabasesVMware, Amazon EC2 Oracle, SQLServer, PostgreSQL, Sysbase
• Application DataUnix/Linux and Windows file-systems
• Data MaskingIntegrated compliance and security
• Business intelligence and data warehousingAid ETL with production copies, or offload BI/DW workload
• Upgrades, migration, and modernizationUpgrade OS/database, migrate to Linux, migrate to new data center or cloud
• ReplicationHigh-availability and fault-tolerance for the Delphix Engine
Private and confidential© 2014 Delphix. All Rights Reserved© 2014 Delphix. All Rights Reserved
THANK YOUANY QUESTIONS?
Private and confidential
Em: Tim.Gorman@Delphix.comTw: @TimothyJGorman
Web: http://Delphix.com/
Make yourself at home at booth #821!