Presentation online application upgrade of oracle's bug db with edition-based redefinition
- 1. 1
- 2. 2 Oracle OpenWorld Latin America 2010 December 7–9, 2010
- 3. 3 Oracle OpenWorld Beijing 2010 December 14–16, 2010
- 4. <Insert Picture Here> Online Application Upgrade of Oracle's Bug DB with Edition-Based Redefinition Louise Morin, Principal Member of Technical Staff Bryn Llewellyn, Product Manager Database Server Technologies Division, Oracle HQ
- 5. 5 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle‟s products remains at the sole discretion of Oracle.
- 6. 6 <Insert Picture Here> Program Agenda • Introduction • Challenge of online application upgrade • Brief EBR overview • Bugdb application characteristics • Steps to EBR readiness • EBR exercise • Testing • Summary • References • Questions and Answers
- 7. 7 <Insert Picture Here> “Eat your own dog food.” “Drink your own champagne.” Larry CEO, Oracle
- 8. 8 <Insert Picture Here> Program Agenda • Introduction • Challenge of online application upgrade • Brief EBR overview • Bugdb application characteristics • Steps to EBR readiness • EBR exercise • Testing • Summary • References • Questions and Answers
- 9. 9 Online application upgrade the challenge • The database holds data! – Upgrades change the representation of some of the data – Most of it is untouched – All of it must survive! • There‟s code in the database • Synonyms, views, PL/SQL • You change this with create or replace • Online users of the old app must experience no changes • Must make changes “in secret” • Need old and new in concurrent use
- 10. 10 <Insert Picture Here> Program Agenda • Introduction • Challenge of online application upgrade • Brief EBR overview • Bugdb application characteristics • Steps to EBR readiness • EBR exercise • Testing • Summary • References • Questions and Answers
- 11. 11 Edition-based redefinition the solution • The edition – A database has one or many editions – Different editions can hold differently defined occurrences of the same object • The editioning view • Tables hold the union of data needed by old and new tables are not editionable • An editioning view projects a table differently into each edition views, synonyms, and PL/SQL objects are editionable • The crossedition trigger • A forward crossedition trigger synchronizes from old to new • A reverse crossedition trigger synchronizes from new to old
- 12. 12 In place / Conflicts prevented • You need only your production database – People have tried to buy “in secret” with a second database • Conflicts are detected and prevented before they happen • Concurrent users of the old and new apps might attempt conflicting changes • A two database approach cannot synchronize transactionally (too slow) – so need human conflict resolution • These benefits of EBR were stated a priori as MUST requirements
- 13. 13 The NE on E prohibition • There‟s no edition-extended syntax – One object refers to another, as ever, using Owner.Name – An editioned object can see only those other editioned objects that are in its own edition – An editioned object sees noneditioned objects ordinarily • Therefore, a noneditioned object cannot depend on an editioned object • There‟s no way for a noneditioned object to say in which edition it should look for the editioned object it wants to depend on
- 14. 14 The NE on E prohibition – cont • A table can have a column defined by a UDT – The Employees table is not editioned – The Phone_Numbers column depends on the Phone_Numbers_Type (nested table) – A user-defined type (collection or ADT) is a PL/SQL object – Sounds like a gotcha • Editions-enabling a user • An object whose type is noneditionable is never editioned • All objects whose type is editionable are editioned when their owner is editions-enabled • All objects whose type is editionable are not editioned when their owner is not editions-enabled
- 15. 15 Readying for EBR • Existing objects – Decide which objects should be editioned – Identify the NE on E prohibition breakers – “Move” these to owners who won‟t be editions-enabled – Migrate the data they define – Now you can editions-enable the users you wanted to • Cover existing tables with new editioning views • Rename tables to obscure names • Restore the former names with covering EVs • “Move” grants, triggers, VPD policies from table to EV • Congratulations! This was your last offline application upgrade
- 16. 16 Case study • The HR sample schema, that ships with Oracle Database, represents phone numbers in a single column: – Diana Lorentz 590.423.5567 – John Russell 011.44.1344.429268 • Users now need to ring phone numbers from any country in the world • So we want a uniform representation with two columns: Country Code; and Number Within Country
- 17. 17 Case study
- 18. 18 <Insert Picture Here> The edition-based redefinition exercise proper The app has already been EBR-readied
- 19. 19 Maintain_EmpsEmployees Pre_Upgrade Employees_ ID Ph. … Starting point. Pre-upgrade app in normal use. edition editioning view table PL/SQL package
- 20. 20 Post_Upgrade Pre_Upgrade Employees Maintain_Emps Start the edition-based redefinition exercise. Create the new edition as the child of the existing one. This is fast because initially all the editioned objects are just inherited. ID Ph. … Employees_ Maintain_EmpsEmployees
- 21. 21 Pre_Upgrade ID Ph. … Employees_ Cntry # Create the replacement columns in the underlying table. The editioning view shields the app from this change. Post_Upgrade Employees Maintain_Emps Maintain_EmpsEmployees
- 22. 22 Post_Upgrade Pre_Upgrade Maintain_EmpsEmployees ID Ph. … Employees_ Cntry # Change Employees to select the new columns. Change Show_Employess to implement the new behavior. Maintain_EmpsEmployees
- 23. 23 Post_Upgrade Pre_Upgrade Maintain_EmpsEmployees ID Ph. … Employees_ Cntry # Fwd Xed Create the forward crossedition trigger. Maintain_EmpsEmployees crossedition trigger
- 24. 24 Post_Upgrade Pre_Upgrade Maintain_EmpsEmployees Fwd Xed Rvrs Xed ID Ph. … Employees_ Cntry # Create the reverse crossedition trigger. Maintain_EmpsEmployees
- 25. 25 Rvrs Xed ID Ph. … Employees_ Cntry # Post_Upgrade Pre_Upgrade Maintain_EmpsEmployees Fwd Xed Apply the transform to the data for the new app to use Maintain_EmpsEmployees
- 26. 26 Post_Upgrade Pre_Upgrade Maintain_EmpsEmployees Fwd Xed Rvrs Xed ID Ph. … Employees_ Cntry # Hot rollover period. Maintain_EmpsEmployees
- 27. 27 Maintain_EmpsEmployees Post_Upgrade Pre_Upgrade Maintain_EmpsEmployees ID Ph. … Employees_ Cntry # The Pre_Upgrade edition is retired. The edition-based redefinition exercise is complete.
- 28. 28 Nota bene • Online application upgrade is a high availability subgoal • Traditionally, HA goals are met by features that the administrator can choose to use at the site of the deployed application – independently of the design of the application – without the knowledge of the application “vendor” • The features for online application upgrade are used by the application “vendor” – when preparing the application for EBR – when implementing an EBR exercise • Site administrators, of course, will need to understand the features
- 29. 29 <Insert Picture Here> Program Agenda • Introduction • Challenge of online application upgrade • Brief EBR overview • Bugdb application characteristics • Steps to EBR readiness • EBR exercise • Testing • Summary • References • Questions and Answers
- 30. 30 • Defects and enhancement requests tracking system • Mission critical • 24/7 High availability requirement – 4 nodes RAC linux-X64 RDBMS 11g Release 2,~ 500 gb , ias 10.1.2 – Small custom mod/plsql application – Uses SES (context index), VPD, AQ – avg ~ 400 concurrent users in development and support LOBs – OLTP Web interface, many critical interfaces and hundreds of sqlnet clients (sqlplus and db links access). Oracle Bugdb Application Characteristics
- 31. 31 EBR Project Goal • Run online application upgrades – without performance impact – without wrong results – following EBR readiness best practices • Proof of concept approach – Users selected based on high availability benefits – Edition notion introduction
- 32. 32 Oracle IT Organization – Get All Teams Involved • Architecture/security – process change, new technology, security review • Operation – refresh test env, maintain mid tiers, EMGC monitoring, perform backups … • Database administrators – db upgrade, patches, tuning... • Release mgt – collect change requirements, coordinate user acceptance testing w/ LOBs rep., approve application releases and patches • App development – code, test, and automate patching • Migration development – integrate acquisitions • Application user base – Line of Business representatives
- 33. 33 EBR Implementation Project Phases • Readying the application for EBR • Define the scope • Editions-enabling the users • EV layer Creation • EBR exercise – change packages and triggers • EBR exercise – include schema changes
- 34. 34 Oracle Change Control Management Process • Code in a dev db -> users test in a UAT db -> stage patch application -> production • Data migration in a clone db -> test -> Prod
- 35. 35 <Insert Picture Here> Program Agenda • Introduction • Challenge of online application upgrade • Brief EBR overview • Bugdb application characteristics • Steps to EBR readiness • EBR exercise • Testing • Summary • References • Questions and Answers
- 36. 36 Readying for EBR Database Administrators Tasks • Upgrade db to 11.2 • Set compatible => 11.2.0 • Increase SGA, tune memory parameters ahead • Capture sql tuning set to run RAT (Real Application Testing) performance comparison before and after each change. Run AWR diff reports • Grant developer role previously limited to dba role – Select_catalog_role, Select any dictionary – Create a ebr_admin package
- 37. 37 Readying for EBR Define the scope • Categorize non system schemas with objects into 2 buckets based on their HA req. and functionality • HA req. : online transactions user interfaces and queries • No HA benefits: AQ interfaces and interfaces using MVs • Check the conditions for a successful editions enabling • Change the application to resolve the NE on E prohibition breakers – UDT (user defined type) – Public synonyms
- 38. 38 Schema design before EBR readiness ID Prod … Bug_table1 Bug_index1 Materialized views Bugctx_index2 VPD policies Grants, roles, privs webbug_ apis MOS Public synonyms Views Remote objects (db link access) AQ interface Triggers UDT buginfo interface_ apis Constraints
- 39. 39 Readying for EBR Rules for Enable Editions to Succeed There's no non-editioned object depending on editioned object type of this user, or if FORCE is specified • To check for this condition : Identify object pairs dependencies in target schemas • Review pair of occurring objects discovered and move the rule breakers to a schema which will be not be edition enabled NE on E prohibition breakers will fail w/ ora-38820, or become invalid, if –force is used when you enable editions
- 40. 40 Readying for EBR Rules for Enable Editions to Succeed There‟s no evolved object type • To check for this condition : select type_name,version# from user_type_versions where version# > 1; • You need to reset the object type with command : alter type ... Reset; Rule breakers will get ora-38820 error "user has evolved object type“ or ORA-22374: cannot reset the version of a type with table dependents. In such case, you would have to drop the table, reset the type and reimport the table.
- 41. 41 Readying for EBR Development Team Tasks • Make application changes to remove the NE on E – For UDT, was used for AQ, move to another schema • Used dbms_aq apis to dequeue, stop the queue, and recreate in the non editioned schema – For public synonyms : • Drop the public synonyms for the 2 schemas • Create an on login trigger that set current_schema to main app schema and create a function to identify app users (excludes sys, system, dbsnmp users) • Create private synonyms between the 2 targeted schemas so packages will compile
- 42. 42 Readying For EBR Editions-Enabling the Users • Alter user BUG enable editions force; • Alter user PRODCOMP enable editions force; • Recompile • Procedure for developers (dbms_utility.validate) • Procedure for EBR admin (sys.utl_recomp.recomp_parallel; ) • Check for any new invalids • select * from dba_invalid_objects;
- 43. 43 Schema EBR ready ID Prod … Bug_table1# Bug_index1 Materialized views bugctx_index2 VPD policies Grants, roles, privs webbug_ apis MOS synonyms Views Remote objects (db link access) AQ interface On delete cascade triggers Orioncc.UDT buginfo interface_ apis Constraints Editioning Views BUG_Table1 Triggers CtxTriggers
- 44. 44 Readying for EBR EV (Editioning Views) Layer Creation • Rename tables to substr(table_name,1,29) || „#‟ • Create EVs with same table names and same column names in same order (using dbms_metadata api) • Drop grants on tables and recreate on the EVs • Recreate VPD policies on EVs • Generate new files for EVs definition for source control check-in • Change all table definitions in source control to match the new table name # on the db
- 45. 45 <Insert Picture Here> Program Agenda • Introduction • Challenge of online application upgrade • Brief EBR overview • Bugdb application characteristics • Steps to EBR readiness • EBR exercise • Testing • Summary • References • Questions and Answers
- 46. 46 EBR Exercise Application change SYS > CREATE EDITION MR12 AS CHILD OF ORA$BASE; SYS > GRANT USE ON EDITION MR12 TO BUG; BUG > EXEC DBMS_SESSION.SET_EDITION_DEFERRED('MR12'); BUG> @ InstallMR12code.sql SYS > exec sys.utl_recomp.recomp_parallel; SYS> select * from dba_invalid_objects; SYS> ALTER DATABASE DEFAULT EDITION = MR12;
- 47. 47 EBR Exercise Monitoring the hot rollover • Monitor session_edition_id on v$session SELECT SYS_CONTEXT('Userenv', 'Current_Edition_Name') FROM DUAL; select object_name, session_edition_id, type, inst_id, v.status, count(*) from gv$session v, dba_objects where session_edition_id=object_id group by object_name, session_edition_id,type, inst_id, v.status order by object_name, session_edition_id,type, inst_id, v.status
- 48. 48 EBR Exercise Monitoring the hot rollover – cont. • Observe background (MMON) and users 1.Web interface users 2.Sqlplus, db link access 3.Critical interfaces (connection pooling, AQ interface) 4.Other agents (ex. Emgc dbsnmp, CRS oraagent.bin) • Terminate remaining sessions using previous edition prior to retiring the edition
- 49. 49 <Insert Picture Here> Program Agenda • Introduction • Challenge of online application upgrade • Brief EBR overview • Bugdb application characteristics • Steps to EBR readiness • EBR exercise • Testing • Summary • References • Questions and Answers
- 50. 50 Testing • Functional testing – Special attention to triggers firing and VPD policies • User Interface testing – use 11.2.0.2 service SET ORA_EDITION=e1 sqlplus usr/p@11202 sql> SHOW EDITION • Concurrent edition usage test (during OAU) • Scheduler jobs • Sql loader – control file “into table”=<evname> • Cursors stats
- 51. 51 Performance Testing • Use Real Application Testing • Mass update the parsing_schema_name to match the schema defined in the on login trigger (BUG) – select * from dba_sqlset where name like 'EBR%'; • Get the ID ex. 25 – update wri$_sqlset_statements set parsing_schema_name='BUG' where sqlset_id=25; – update wri$_sqlset_plans p set parsing_schema_name='BUG' where stmt_id in (select id from wri$_sqlset_statements where sqlset_id=25);
- 52. 52 <Insert Picture Here> Program Agenda • Introduction • Challenge of online application upgrade • Brief EBR overview • Bugdb application characteristics • Steps to EBR readiness • EBR exercise • Testing • Summary • References • Questions and Answers
- 53. 53 Summary • Oracle Database 11g Release 2 brings a revolutionary new capability that supports online application upgrade – This means preserving most of the data, changing some of it, and changing code – all while it‟s in continuous use – It‟s called edition-based redefinition – It works in place (no need for a second db). Old and new representations of the same data are synchronized transactionally. No need for human conflict resolution.
- 54. 54 Summary • A mission critical Oracle Corp internal application has gone live with EBR – the application has been readied (11.2 platform; users are editions-enabled) – an EBR exercise has been conducted with zero downtime – users didn‟t even notice the hot rollover
- 55. 55 Summary • The mission critical internal application is BugDB – Half of all Oracle employees, world wide, are authorized to use it and might need to at a moment‟s notice – There are typically hundreds of concurrent end-user sessions – There are also very many mechanical clients
- 56. 56 • Read the edition-based redefinition chapter in the Oracle Database Advanced Application Developers‟ Guide, 11.2 • Download Bryn‟s OpenWorld 2010 session S318090 • Read Bryn‟s whitepaper: published on the High Availability subpage under the Database page on OTN • Listen to Bryn‟s 30-minute recorded presentation download.oracle.com/technology/products/database/files/Edition_Based_Redefinition_Overview_ReadMe.pdf download.oracle.com/technology/products/database/files/Edition_Based_Redefinition_Overview.zip • Internet search for edition-based redefinition References
- 58. 58 The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle‟s products remains at the sole discretion of Oracle.
- 59. 59
- 60. 60