Agile db testing_techniques
2 likes814 views
This document discusses agile database testing techniques, including why testing is important, how to implement automated unit testing of PL/SQL code using a framework like UtPLSQL, how to validate database schema installations, approaches for testing reports, implementing daily database builds, and logging and monitoring techniques. Automated unit testing helps improve code quality by making tests repeatable and increasing test coverage. Validating schema installs and reports helps catch errors. Daily database builds and logging/monitoring aid in debugging.
Agile db testing_techniques
- 1. Tarik R. Essawi VeriSign Inc. Agile Database Testing Techniques
- 2. Introduction • Real world best practices • Proven Techniques • Why Test ? • How to greatly improve – PL/SQL Quality – Database Schema Installations – Reports – Build Process – Debugging and Monitoring
- 3. Why Test ? • You will spend time testing your code. – You can manually test your code every time a change is made or a build is deployed • Manual testing is haphazard • Code coverage is sporadic at best – You can create an automated unit test once that can be run every time the code changes • Tests are repeatable • Code Coverage is repeatable
- 4. Finding Defects • Defects will be found, the only question is when: – Options 1)You can find the defects during development 2)Quality Assurance can find the defects 3)Customers can find the defects in production
- 5. What is Agility • A rapid change of velocity or direction in response to a stimulus • Many different forms and methods – Test Driven Development – Extreme Programming – Scrum • Foundation of Agility is Testing – Creates confidence to change direction
- 6. Traditional QA role • Responsible for finding defects • If defects are not found in QA they are found in production
- 7. A Different Approach • Guilty Until Proven Innocent • Developers must prove code is correct • QA role becomes: – Corroborate test results – Verify the Delivered Code meets Requirements – Focus on integration tests of corner cases
- 8. Side Effects • Number of builds reduced • Amount of rework reduced Week0 Week1 Week2 Week3 Week4 Week5 Week6 Week7 Week8 0 5 10 15 20 25 30 35 40 Time Man Hours Development Lifecycle Automated Testing Maual Testing
- 10. UtPlsql Installation • UtPlsql is a plsql unit testing framework. It is one of many, So if you are not using plsql other testing frameworks exist that accomplish the same goal. • Installing the package – Download zip file from • http://utplsqlsql.sourceforge.net • Unzip the file. – Create user • connect to the database as system • create user utplsql identified by utplsql default tablespace ?? temporary tablespace ??; • grant create session, create table, create procedure, create sequence, create view, create public synonym, drop public synonym to utplsql; • alter user utplsql quota unlimited on ??; • You can also just grant DBA privileges. – In the code directory where the file is unzipped, login to sqlplus as utplsql and run @ut_i_do install (you must pass in the parameter “install”). – Create public synonyms for all of the packages owned by utplsql.
- 11. Developing the Tests • A test package is created for each application package – For example, to test PKGORDERITEMS a package called UT_ PKGORDERITEMS is created • Each package contains a ut_setup, ut_teardown, and then individual procedures to test the code • The tests are self-contained • Tests are based on assertions
- 12. Assertions • Assertions are the heart of the testing framework • You will prove through the assertion that the results are correct • Below are the basic assertions that can satisfy 80% of test cases. – utassert.eqquery – compares multiple columns of two queries or result sets – utassert.eqqueryvalue – compares a single column returned from a query – utassert.eq –compares two values – utassert.throws – is used to test that exceptions are being thrown and caught properly • For the other 20% you can also – Compare tables, pipes, refcursors, and collections for equality – Check for null values, table counts and whether previous assertions passed or failed – You can also extend the assert procedure through inheritance and create your own
- 13. Running Tests • Compile the test package under the same schema as the calling package • Run the test by calling utplsql.test. This is one of many ways to execute the test SET SERVEROUTPUT ON SIZE UNLIMITED BEGIN utplsql.test(package_in=>'PKGORDERITEMS', recompile_in=>FALSE); DBMS_OUTPUT.put_line (utplsql2.runnum); END;
- 14. Running Tests Con’t • Review the output, which will say > SSSS U U CCC CCC EEEEEEE SSSS SSSS > S S U U C C C C E S S S S > S U U C C C C E S S > S U U C C E S S > SSSS U U C C EEEE SSSS SSSS > S U U C C E S S > S U U C C C C E S S > S S U U C C C C E S S S S > SSSS UUU CCC CCC EEEEEEE SSSS SSSS OR > FFFFFFF AA III L U U RRRRR EEEEEEE > F A A I L U U R R E > F A A I L U U R R E > F A A I L U U R R E > FFFF A A I L U U RRRRRR EEEE > F AAAAAAAA I L U U R R E > F A A I L U U R R E > F A A I L U U R R E > F A A III LLLLLLL UUU R R EEEEEEE .
- 16. Install Uninstall Procedures • Install and Uninstall scripts should not require any operator input • Script should be simple and straightforward spool install.lst PROMPT Calling define_var.sql... @@define_var.sql /********************************************* * Creating Orders **********************************************/ @cr_orders.sql /********************************************* * Creating OrderItems **********************************************/ @cr_order_items_table.sql spool off; !grep 'ORA-' *.lst !grep 'SP2-' *.lst exit
- 17. Validating Schema Installs • Schema installs are one-time tasks where quality is extremely critical • Errors during a schema install can cause an entire release to be rolled back or cause the maintenance window to be missed due to debugging • Validation scripts evolved based on the following scenarios: – Errors being written to files and never reviewed – Errors never being written to files at all – Scripts not executing, which means no error is generated – Scripts that exit part of the way through and never execute the remaining commands
- 18. Validating Schema Installs con’t • Verify that every object that was expected to be created actually exists. • Create an assertion package that will verify the actual versus the expected. • Generate an error if the assertion fails. • Items to check – Tables – Indexes – Constraints – Synonyms and Grants – Reference data was inserted or updated as expected – Anything created during the install…
- 19. Schema Install Validation Example • Below is an example of a database verification script. The assertion procedure was written in-house and raises an error if the first and second parameters are not equal. The verification script are tailored to each deployment. declare v_actual number; Begin -- Verify TABLE Tables select count(*) into v_actual from user_tables where table_name = ‘ORDER_ITEMS' and tablespace_name = 'DATA'; PkgValidation.assert(1, v_actual,' Verify Order Items'); -- Verify TABLE INDEXES select count(*) into v_actual from user_indexes where index_name in ('IDX_ORDER_ITEMS', 'PK_ORDER_ITEMS') and tablespace_name = 'INDEX'; PkgValidation.assert(2, v_actual,' Verify Index Creation'); -- Print Validation Summary dbms_output.put_line('======================================================='); dbms_output.put_line('**Assertions passed : '||PkgValidation.v_assertions_passed); dbms_output.put_line('**Assertions Failed : '||PkgValidation.v_assertions_failed); dbms_output.put_line('**Total Assertions : '||PkgValidation.v_total_assertions); dbms_output.put_line('======================================================='); end;
- 20. Report Testing • Report Testing can be challenging: – Reports contain millions of rows – A single report can consist of multiple pages of SQL Code • Automating Report Testing can be accomplished: – Load the report into the database using sqlloader or external tables – Once the report is accessible within the database the source data can be compared to the report data – Assertions can be used to compare each record in the report with the source data and vice versa – This initial testing will identify common errors with complex table joins and date ranges – Controlled Datasets will validate the end to end testing of the system from the customer channel through to the reports
- 21. Daily Builds
- 22. Daily Build Database • The development and test environments must be functionally equivalent to production • The hardware can differ but the logical structure of the database must be an exact replica • Use the DBMS_METADATA package
- 23. DBMS_METADATA • DBMS_METADATA.SET_TRANSFORM_PARAM( DBMS_METADATA.SESSION_TRANSFORM,'STORAGE',false); • DBMS_METADATA.SET_TRANSFORM_PARAM( DBMS_METADATA.SESSION_TRANSFORM,'SQLTERMINATOR',true); • DBMS_METADATA.SET_TRANSFORM_PARAM( DBMS_METADATA.SESSION_TRANSFORM,'SEGMENT_ATTRIBUTES',false); • DBMS_METADATA.SET_TRANSFORM_PARAM( DBMS_METADATA.SESSION_TRANSFORM,'TABLESPACE',false);
- 24. DBMS_METADATA con’t • Examples of how to reverse engineer the database • SELECT DBMS_METADATA.GET_DDL('TABLE',table_name) FROM USER_TABLES; • SELECT to_char(DBMS_METADATA.GET_DDL('INDEX',index_name)) FROM USER_INDEXES; • SELECT to_char(DBMS_METADATA.GET_DEPENDENT_DDL('OBJECT_GRANT' ,table_name)) FROM USER_TABLES; • SELECT to_char(DBMS_METADATA.GET_DEPENDENT_DDL('CONSTRAINT‘ ,table_name)) FROM USER_CONSTRAINTS where constraint_type in ('P','C'); • SELECT to_char(DBMS_METADATA.GET_DEPENDENT_DDL( 'REF_CONSTRAINT‘,table_name)) FROM USER_CONSTRAINTS where constraint_type = 'R';
- 25. Daily Builds Recipe: Daily Build Database API to Checkout Source Code Install Scripts Database Validation Scripts Self Contained Unit Tests Uninstall Scripts
- 26. Daily Builds con’t • At times uninstall scripts may not be complete enough to undo changes to the data dictionary • An alternative to uninstall scripts is using Flashback Database • Before running the install log the current scn – DBMS_FLASHBACK.GET_SYSTEM_CHANGE_NUMBER() • When complete with all tests flashback the database – FLASHBACK DATABASE TO SCN ?;
- 28. Logging and Monitoring CREATE TABLE EVENTLOG ( EVENTLOGID NUMBER(9) NOT NULL, SEVERITYID NUMBER(2) DEFAULT 1 NOT NULL, LOCATION VARCHAR2(100 BYTE) NOT NULL, ERRORMSG VARCHAR2(4000 BYTE), ADDITIONALINFO VARCHAR2(4000 BYTE), CREATEDDATE DATE, CREATEDBY NUMBER(9) ) • http://log4plsql.sourceforge.net/
- 29. Debug Flag • Example 1 – log info message if (pkgeventlog.getdebugflag) then pkgeventlog.createeventlog(1,'PkgOrderItems.AddOrderItems', 'info','Get transid',sysdate,1); end if; • Example 2 – log error message EXCEPTION WHEN OTHERS THEN vError:=substr(DBMS_UTILITY.FORMAT_ERROR_BACKTRACE ,1,100); pkgeventlog.createeventlog(4,'PkgOrderItems.AddOrderItems’, 'error',vError,sysdate,1); RAISE;
- 30. Summary • Everything can be tested • Developers are Guilty until Proven Innocent • Robust testing has great side effects such as: – Reduced Rework – Spend more time developing new code instead of fixing old code • Agility requires confidence • Confidence = Tests that prove the system is correct
- 31. Resources • forwardthinkingdb.com – Database Architecture Resources • agiledata.org – Agile Database Development
- 32. Become a Complete Oracle Technology and Database Professional • Join the IOUG online at www.ioug.org and get immediate access to: – Member Discounts and Special Offers – SELECT Journal – Library of Oracle Knowledge (LoOK) – Member Directory – Special Interest Groups – Discussion Forums: – Access to Local and Regional Users Groups: – 5 Minute Briefing: Oracle – Volunteer Opportunities
- 33. Stop by the IOUG Booth • Moscone Center West – 2nd Floor – Register for Raffles and Giveaways – Learn more about IOUG – Fill out a SIG interest card
- 35. Save the Date! May 3-7, 2009 Orange County Convention Center West Orlando, Florida