Active/Active Database Solutions with Log Based Replication in xDB 6.0
2 likes1,834 views
The document outlines the features and benefits of xdb 6.0, a database replication tool designed for active/active database solutions. It highlights efficient replication methods, such as log-based multi-master replication, easier publication table configuration, and improved conflict detection performance. xdb facilitates high availability and scalability, making it suitable for diverse data environments.
![© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 10
• When you have a lot of tables to choose
from when building your publication, the
wildcard selection dialog is a useful tool
• Here we see a few examples
− % as a wildcard character to identify all
tables of the _gb designation
− [character lists] to identify more patterns
with specific characters
• Detailed Help is available in the client
• See Documentation section 7.1 for
information on the wildcard selector UI.
Easier to Configure Publication Tables in Bulk
Use sophisticated pattern matching rules to identify tables for publication](https://image.slidesharecdn.com/xdb6-151216184648/85/Active-Active-Database-Solutions-with-Log-Based-Replication-in-xDB-6-0-10-320.jpg)
Active/Active Database Solutions with Log Based Replication in xDB 6.0
- 1. © Copyright EnterpriseDB Corporation, 2015. All rights reserved. 1 Active / Active Database Solutions with Log Based Replication in xDB 6.0
- 2. © Copyright EnterpriseDB Corporation, 2015. All rights reserved. 2 • xDB Overview − Features − Use Cases − Components of a Multi Master Replication System • Supporting Active / Active Database Solutions with xDB 6.0 − Faster and more efficient replication with log-based Multi Master replication for Postgres Plus and PostgreSQL − Easier to configure publication tables in bulk with pattern matching selection rules − Ensuring High Availability with integration of the 'Control Schema’ − Improved performance in conflict detection rules Agenda
- 3. © Copyright EnterpriseDB Corporation, 2015. All rights reserved. 3 xDB is a highly flexible database replication tool to address multiple problem sets What is xDB? Multi Master: Consolidated view from any node and multiple conflict detection / resolution rules • Write Availability & Scalability • Warm Standby • Localized Data Access Single Master: Heterogeneous (Oracle, SQL Server, PG, PPAS) & filtered data sources • Offload Reporting or BI Queries from the OLTP system • Migrate or test systems at low cost
- 4. © Copyright EnterpriseDB Corporation, 2015. All rights reserved. 4 Features • Distributed Publication / Subscription Architecture • Snapshot and continuous synchronization modes • Replicate one or more tables • Define and apply row filters • Flexible replication scheduler • Supports cascading replication • Replication History Viewer • Graphical Console and CLI Use Cases • Write Availability • Write Scalability • Warm Standby Servers • Localized Data Access • Offloading Reporting and Business Intelligence Queries • Testing Systems in Parallel • Migrating Data xDB Feature Highlights
- 5. © Copyright EnterpriseDB Corporation, 2015. All rights reserved. 5 Why xDB over other replication tools? • Rich graphical management console − complete control over your replication system − a history viewer − a replication scheduler • Command line operations suitable for scripting. • Multi-Master is an ideal solution for geographically dispersed data centers or co-located redundant data sets. It provides a critical component of a 24x7 near zero downtime maintenance operation. • Single Master Heterogeneous Sources provide integration and a path to reduce other license and maintenance costs
- 6. © Copyright EnterpriseDB Corporation, 2015. All rights reserved. 6 • Publication Server & Configuration Files: − Java application; configures publication database and master nodes for replication; performs replication. • Controller database (Can be any Master Node): − Primary provider of the replication system information to the publication server. − Must be first one running and accessible to publication server; can be set to any node. • Master Definition Node − First node added to the MMR system − Table definitions and optionally initial data to publish − Role can be reassigned • Master Node − Replicated & updatable data stores • More Info in Documentation Section 2.3.1.x – Physical Components of xDB system Architecture and Physical Components of xDB MMR xDB is a set of programs + datastores containing configuration information and metadata that form a replication system.
- 7. © Copyright EnterpriseDB Corporation, 2015. All rights reserved. 7 • xDB Overview − Features − Use Cases − Components of a Multi Master Replication System • Supporting Active / Active Database Solutions with xDB 6.0 − Faster and more efficient replication with log-based Multi Master replication for Postgres Plus and PostgreSQL − Easier to configure publication tables in bulk with pattern matching selection rules − Ensuring High Availability with integration of the 'Control Schema’ − Improved performance in conflict detection rules Agenda
- 8. © Copyright EnterpriseDB Corporation, 2015. All rights reserved. 8 • Changes to rows in the source tables are extracted from the Write-Ahead Log segments (WAL files). • xDB connects via walsender interface. Changes are streamed on continuous basis. Eliminates need to poll. • The extracted changes are converted to an in-memory data structure and applied to the target tables by means of SQL statements executed using JDBC. • See Documentation section 2.2.10 for information on the log-based method. Faster and more efficient replication with log-based MMR for PPAS & PG 9.4 Capture data changes without impacting the online transaction processing rate High-level architecture of how xDB communicates with database server to fetch logical changes.
- 9. © Copyright EnterpriseDB Corporation, 2015. All rights reserved. 9 • Changes to rows in the source tables are extracted from the Write-Ahead Log segments (WAL files). • xDB connects via walsender interface. Changes are streamed on continuous basis. Eliminates need to poll. • The extracted changes are converted to an in-memory data structure and applied to the target tables by means of SQL statements executed using JDBC. • See Documentation section 2.2.10 for information on the log-based method. Faster and more efficient replication with log-based MMR for PPAS & PG 9.4 Capture data changes without impacting the online transaction processing rate
- 10. © Copyright EnterpriseDB Corporation, 2015. All rights reserved. 10 • When you have a lot of tables to choose from when building your publication, the wildcard selection dialog is a useful tool • Here we see a few examples − % as a wildcard character to identify all tables of the _gb designation − [character lists] to identify more patterns with specific characters • Detailed Help is available in the client • See Documentation section 7.1 for information on the wildcard selector UI. Easier to Configure Publication Tables in Bulk Use sophisticated pattern matching rules to identify tables for publication
- 11. © Copyright EnterpriseDB Corporation, 2015. All rights reserved. 11 • In an xDB MMR systems, if any master goes offline, remaining nodes continue to synchronize amongst themselves. • When an offline master is brought back, pending transactions are synchronized. • Each node serves as a backup of other nodes* • As part of the Control Schema integration project, normal replication configuration (control tables, procedures/functions, sequences…) and history tables are also replicated amongst master nodes. • Thus, there is resiliency built into the system such that you could lose any database node and replication operations will continue • See Documentation 6.11 for more information *Keep care when using Table Filters, as some nodes may not have all data replicated onto it. Ensuring High Availability Control Schema Control Schema Control Schema
- 12. © Copyright EnterpriseDB Corporation, 2015. All rights reserved. 12 • There is however special significance of the Controller Database − Publication server accesses it for configuration information − So any changes made to system config using GUI or CLI is done on Controller DB first • For planned maintenance − You can identify a different node to be your Controller using the GUI Console or CLI • For unplanned maintenance − Edit the xDB Configuration file to contain connection information of another master node − Restart publication server. − You can now use the CLI or GUI tool to connect to, monitor and manage the replication system. • See Documentation 6.11 for more information Ensuring High Availability
- 13. © Copyright EnterpriseDB Corporation, 2015. All rights reserved. 13 • xDB 5.x conflict detection query is based on IN clause for doing the comparison of PK attribute(s). • i.e. SELECT s.* FROM _edb_replicator_pub.rrst_public_orders WHERE order_id IN (1,5,10,15,20,21); • Performance starts to degrade as table grows or if PK type is not numeric. • We replaced the existing query with a ‘common table expression’ optimize the query execution time. • i.e. WITH temp_table AS (SELECT… Improved Performance in Conflict Detection Over 100x improvement with large tables and non-numeric PK Internal Testing: Create table and seed 10 million rows of text data create table x (t text not null); insert into x select md5(n::text) from (select generate_series(1, 10000000)) w(n); Show timing with old and new algorithm timing on test=# select xdb60_style(); --Time: 3,629.105 ms (3 seconds) test=# select xdb51_style(); --Time: 2,198,287.369 ms (36 minutes!)
- 14. © Copyright EnterpriseDB Corporation, 2015. All rights reserved. 14 • Active / Active Databases with fast and efficient log based replication and high performance conflict detection algorithms • Pattern matching selection rules make it easier to configure publication tables in bulk • Simple High Availability is now possible with integration of the 'Control Schema’ Summary – xDB 6.0 Enables New Replication Solutions!
- 15. © Copyright EnterpriseDB Corporation, 2015. All rights reserved. 15