Testing Delphix: easy data virtualization

19Tips&techniques19
SOUG Newsletter 5/2014 – Sonderausgabe
Testing Delphix:
easy data virtualization
Franck Pachot, dbi services
File system technologies such as snapshots, clo-
ning, copy-on-write are arising to handle the ever
increasing size of databases and the multiplying
number of required copies (backups, preprod, test
environment). But there are new technologies to
learn beyond the basic storage system options. EMC,
NetAPP, Oracle ZFS Storage Appliance are hardware
appliance and requires specialized skills. But there
is a new breed of products that have arisen to tackle
both the technical obstacle as well as the specia-
lized skills obstacle. The new product is a software
appliance: it runs as a VMware virtual machine and
exposes the virtual storage through NFS.
The software is intimately integrated with the
database providing power and ease of use. At
Oracle Open World I tested it and discussed with
the oracle experts Kyle Hailey (Ace and Oaktable)
and Tim Gorman (Ace Director and Oaktable) that
are now working at Delphix.
Why data virtualization?
What is the size of your database? Let’s say you have a
small to medium size – 200GB. And you generate 20GB of
archivelog every day. Let’s go into production needs first.
You requires to keep 30 days of backup. With daily full back-
ups that’s more than 5TB of backupsets. If you do weekly full
backups and daily incrementals, you may reduce it to 2TB.
Ok 2TB for backups. Now you need environments for pre-
production (testing upgrades), UAT (testing releases), qualifi-
cation (testing performance). Your UAT users dream about
having their own database for each testers, so they don’t
overlap on data.
And now about developers. If you want them to release
efficient application you should give them an environment
with representative volume of data. And once again each de-
veloper team has its own set of data set. How many database
do you need to provide?
That requirements exists for years. But today the delivery
delays have decreased a lot. We must be agile. Developers
are committing to continuous integration. They have a bunch
of test cases to run on a fixed set of data. And they need to
rewind the database each time.
I forget one environment: bugfix. What if you find an ap-
plication bug on production? You have a patch but want to
test it. If you have Active Data Guard (i.e. Enterprise Edition
plus ADG option), you can use one of the physical standby to
do a snapshot standby. What if you were able to just click on
a GUI and provision a new copy instantaneously?
Well, it’s a bit more than a click. It is a click and slide:
Yes: you can choose the point in time snapshot that you
want to copy.

20 Tipstechniques
Source databases
So you have your existing databases and they can act as
source for cloning. You don’t have to do anything about them
except installing the Delphix Toolkit on their server.
Here is the screen to define the source server. Toolkit de-
ployment is automatic:
You just set the host connection and where to install the
Toolkit.
Then you add the databases.
How does it works? Delphix will use RMAN to get the
source database.
The first time you add the source database, it will do a full
backup through RMAN. And only the first time. Then only in-
cremental changes are collected (using incremental backups
and/or archived logs) to maintain the copy on the Delphix
server.
The Delphix server is a VMware virtual machine. You al-
locate storage as you want and Delphix maps its own file sys-
tem onto the storage. Delphix uses a copy-on-write filesys-
tem in order to be able to rewind to any state between the first
initialization (or the defined window) and the current time. In
addition to that the storage is compressed by Delphix. Then
the copy is 2 or 3 times smaller than the source database and
it is able to flashback in the past.
Clone databases
Then you have the base: a copy of your production which
you can view as of any timestamp.
And with a few clicks you can provision a clone immed
iately: data blocks are not copied until they are changed.
I have the ’#cloneattack’ environment on my laptop. Del-
phix is running and I can connect with any web browser to get
the full interface, The server running in VMware Workstation:
And the console, connected from my browser. Here I
show the source database properties. Delphix has a nice
GUI: you click on the upper right arrow to flip between sum-
mary and detail:

21Tipstechniques21
It will create automatically an instance for it (I need the
Delphix toolkit installed on the target server as well) and I can
change the default init.ora parameters, file location, listener
etc.
Ok, after a few screens Delphix is doing everything auto-
matically. I can control the running jobs: DB_PROVISION cre-
ates my instance and database and DB_SYNC manages the
data virtualization to show the data from the source point-in-
time I have chosen.
Ok, now I want to get a copy of it – as of 08:30 this morn-
ing:
I have then to define where to put it (storage is virtualized
but I need a server to run the instance) and a name for it:

22 Tipstechniques
How long does it take? That’s just a few minutes and the da-
tabase is opened in read write, ready to accept connections
with the connection string defined by the listener and data-
base configuration.
You can see it on the target server:
[oracle@linuxtarget ~]$ ps -edf | grep pmon
oracle 57363 1 0 02:19 ? 00:00:00 ora_pmon_VEmp519
oracle 60795 1 0 03:29 ? 00:00:00 ora_pmon_MyCopy
oracle 61180 61147 0 03:33 pts/0 00:00:00 grep pmon
and the listener is listening on it:
[oracle@linuxtarget ~]$ lsnrctl status
LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 30-SEP-2014 03:34:01
Copyright (c) 1991, 2009, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=linuxtarget.delphix.local)(PORT=1521)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 11.2.0.1.0 - Production
Start Date 29-SEP-2014 21:49:14
Uptime 0 days 5 hr. 44 min. 47 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/listener.ora
Listener Log File /u01/app/oracle/diag/tnslsnr/linuxtarget/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=linuxtarget)(PORT=1521)))
Services Summary...
Service “MyCopy” has 1 instance(s).
Instance “MyCopy, status READY, has 1 handler(s) for this service...
Service “VEmp_519” has 1 instance(s).
Instance “VEmp519, status READY, has 1 handler(s) for this service...
The command completed successfully
I like to see the structure of the database with RMAN ’re-
port schema’:
RMAN report schema;
using target database control file instead of recovery catalog
Report of database schema for database with db_unique_name MYCOPY
List of Permanent Datafiles
===========================
File Size(MB) Tablespace RB segs Datafile Name
---- -------- --------- ------- ------------------------
1 680 SYSTEM *** /mnt/provision/MyCopy/datafile/u02/app/orcl/data/u01/app/oracle/oradata/orcl/system01.dbf
2 530 SYSAUX *** /mnt/provision/MyCopy/datafile/u02/app/orcl/data/u01/app/oracle/oradata/orcl/sysaux01.dbf
3 60 UNDOTBS1 *** /mnt/provision/MyCopy/datafile/u02/app/orcl/data/u01/app/oracle/oradata/orcl/undotbs01.dbf
4 38 USERS *** /mnt/provision/MyCopy/datafile/u02/app/orcl/data/u01/app/oracle/oradata/orcl/users01.dbf
List of Temporary Files
=======================
File Size(MB) Tablespace Maxsize(MB) Tempfile Name
---- -------- ------------------- ----------- --------------------
1 20 TEMP 32767 /mnt/provision/MyCopy/temp/temp1.dbf

23Tipstechniques23
We see that the datafiles are on the NFS mountpoint, with
the name of the copy and the name of the source as well.
And finally I like to see all what happened on the back-
ground:
[oracle@linuxtarget trace]$ cat alert_MyCopy.log
Tue Sep 30 03:26:01 2014
Starting ORACLE instance (restrict)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
…
System parameters with non-default values:
processes = 150
event = 10052 trace name context forever
filesystemio_options = setall
_omf = DISABLED
control_files = /mnt/provision/MyCopy/datafile/u02/app/orcl/data/u01/app/oracle/oradata/orcl/control01.ctl
_controlfile_enqueue_timeout = 4500
db_block_size = 8192
compatible = 11.2.0.0.0
log_archive_dest_1 = location=/mnt/provision/MyCopy/source-archive MANDATORY
log_archive_format = arch_%t_%s_%r.log
db_create_file_dest = /mnt/provision/MyCopy/datafile
remote_login_passwordfile = EXCLUSIVE
audit_sys_operations = FALSE
job_queue_processes = 0
_system_trig_enabled = FALSE
audit_trail = NONE
db_name = orcl
db_unique_name = MyCopy
open_cursors = 300
_diag_hm_rc_enabled = FALSE
…
ALTER DATABASE MOUNT
…
alter database rename file
'/u02/app/orcl/data/u01/app/oracle/oradata/orcl/system01.dbf'
,'/u02/app/orcl/data/u01/app/oracle/oradata/orcl/sysaux01.dbf'
,'/u02/app/orcl/data/u01/app/oracle/oradata/orcl/undotbs01.dbf'
,'/u02/app/orcl/data/u01/app/oracle/oradata/orcl/users01.dbf'
to
'/mnt/provision/MyCopy/datafile/u02/app/orcl/data/u01/app/oracle/oradata/orcl/system01.dbf'
,'/mnt/provision/MyCopy/datafile/u02/app/orcl/data/u01/app/oracle/oradata/orcl/sysaux01.dbf'
,'/mnt/provision/MyCopy/datafile/u02/app/orcl/data/u01/app/oracle/oradata/orcl/undotbs01.dbf'
,'/mnt/provision/MyCopy/datafile/u02/app/orcl/data/u01/app/oracle/oradata/orcl/users01.dbf'
What I see here is that this has been designed by people
that really knows how oracle works in depth. It’s not just a
simple clone script that works only in the common case. It’s
real DBA stuff.
Then what do you have?
You have a database that takes minimal storage and is seen
as an exact point-in-time copy of the source database.
You can use it, rewind it to another point in time, you can
refresh it from source, you can clone it to another copy etc.

24 Tipstechniques
Use Cases
So what can you do with virtual databases?
■ You can give a database to each tester, each developer
and they have a full volume database without taking too
much storage
■ You can give a database for continuous integration that
can be rewind before each test (Delphix has an API to
control without the GUI)
■ You set up QA environments with real data. Choosing a
point-in-time let open a database in an ’end-of-month’
situation for example if you want to test a monthly job.
■ You can have synced databases for protection. If the
Delphix VM is set with high availability, then you can get
an exact copy of a production database that is not
available anymore.
■ You can quickly provision a database for bugfix, showing
an exact situation in the past where you had the bug
happening.
■ You can use it for backups: you reduce storage and
reduce RPO and RTO. Do you prefer RMAN backups as
it detects block corruption? Why not automating a daily
clone provisioning that you open and run a rman validate
database on it?
■ You can avoid lot of the storage you use for backup
retention. You can get yearly snapshots to keep 10 years,
monthly snapshots to keep 2 years and be able to get a
copy from any point-in time from the previous two
months.
■ Your BI ODS database can be refreshed with Delphix in
a few minutes. Remember: Delphix maintains its copy
of the production database (synchronizing with incre-
mentals and archivelogs) so you don’t impact production
performance even with high activity on the clone.
Those are the few Use Cases that are the most common
I see in many companies.
Conclusion
I’ve heard of Delphix for months, mainly because people
at Delphix are well known in the Oracle community. And Jon-
athan Lewis has presented that solution on several occa-
sions. His performance tuning activity often show the need to
have an exact copy of production at a specific state in order
to reproduce the issue.
But I like to learn about a product by testing it instead of
reading docs or looking at presentations. And the #cloneat-
tack organized at the Oracle Open World was a good occa-
sion for that. By playing with it just a few hours, I can see how
it addresses some common questions frequently raised by
my customers. Both in operation and development teams.
This article is only about what I’ve tested. There is a lot
more. Delphix is not only for Oracle. It does the same for
Microsoft SQL Server. PostgreSQL is also supported. ■
Contact
dbi services
Franck Pachot
E-Mail:
franck.pachot@dbi-services.com
Besuchen Sie jetzt die Oracle-Originalkurse bei Digicomp: www.digicomp.ch/oracle
Neue Kurse zu Oracle Database 12c!
0844 844 822, info@digicomp.ch, www.digicomp.ch
A N Z E I G E

Testing Delphix: easy data virtualization

Recommended

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Testing Delphix: easy data virtualization (20)

More from Franck Pachot (10)

Recently uploaded (20)

Testing Delphix: easy data virtualization