20121213 qa introduction smileryang
- 1. Quality Assurance in Software Industry Smiler Yang 2012/12/13
- 2. Outline • About me • Roles in a project • Why a software needs QA • QA tasks • Testing Methodology • Examples • Some stories in real world • Happy and Unhappy • Q&A
- 3. Roles in a project • PM (Product Manager) • JM (Project Manager) • DM (Developer Manager) • QM (QA Manager) • Architecture • RD • QA • UI Designer • Technical Writer
- 4. Why a software needs QA? • Did you test your program? – I think it’s “Yes” – But you might not consider everything… So, we need--- • Different people • Different knowledge • Different environment
- 5. QA tasks • Requirement Clarification • Bug Reporting • Test Plan • Bug Verification • Test Spec • Document Testing • Test Case • Test Automation • Environment Preparation • Beta training • Functional Testing • Customer on-site • System Testing • Exploratory Testing
- 7. Testing Methodology (1/5) • Black Box Testing – Perform the tests on the basis of functional properties and requirements of a software – Focus on the selected test data and the interpretation of test results – Needs experience… – Includes • Functional Test • Error-handling Test • Boundary Test • Performance Test • Stress Test • Usability Test • I18N Test (internationalization test) • …
- 8. Testing Methodology (2/5) • White Box Testing – Need to understand overall system architecture – Inspect code of function implementation – Focus on testing the paths of control flow and data flow • Go through all independent paths • Exercise logical decisions • Executed all loops with boundary condition checked
- 9. Testing Methodology (3/5) • Test Plan – Purpose of Test Plan • Sync Testing Direction with JM/DM/RD • To Give all testing directions/information to QA Engineer – Information required in a test plan • MRD/PRD/Functional Spec/High Level Design – Schedule of each development process, Feature List, System Requirement, Product Team Information • Configuration Compatibility Testing – hardware compatibility / software (OS/Application) compatibility • What is the user expecting to the product? – How does Customer use the product? – on what kind of the environment?
- 10. Testing Methodology (4/5) • Define the Scope of testing of Features and Environments – Doing and not doing – Decide the priority of Features and Environment • Define of Bug Severity – Severity: Critical/Serious/Non-Critical/Enhancement/Suggestion • Define Testing methods • Resource Planning – People – Environment • Preparation of the testing – Knowledge: QA knowledge, product knowledge, … – Environment: Software, Hardware,… • Process for Testing/Management – Bug Tracking System – Checking method of the progress – Bug Review Meeting –…
- 11. Testing Methodology (5/5) • Test Case – What is Test Case? • The set of input data, operation steps and expected result • Operation steps includes conditions, configurations, environment information, … – What is good Test Case? • The purpose of the testing is to improve the quality by finding the errors. • Test case should cover all requirements. • Test case should take the wrong input/output and wrong operation into consideration. • Test case should have clear description.
- 12. Examples
- 13. Example of Test Case Creation • Use Case: – Account Login in a web portal • Test cases?? – Hint: • Functional Test • Error-handling Test • Boundary Test • Performance Test • Stress Test • Usability Test • I18N Test
- 14. Example of Bug Reporting • Issue: User failed to login the web portal • How to describe a problem? • The information of a bug should include: – Title – Environment – Version/build number – OS/Platform – Priority – Severity – Defect Type – Frequency – Description – Steps – Expected results – …
- 15. Some stories in real world
- 16. QA and JM <Before the milestone day> • JM: Any issues of the candidate build? • QA: So far we found #358 is the problem for…, and #359 is the issue when user…., and….. • JM: Have RD found the root cause for these issues? • …. • … • Then “3M” may have a meeting to review the issues.
- 17. QA and RD < After QA submit a bug to RD> • RD: This problem didn’t appear in my machine. • QA: It did happen in my testing environment! • … • .. • .
- 18. QA and Customer <One day…> • Customer: The service is not available after I used the system for only 2 days. • QA: What did you do on the machine today? Do you think it’s normal yesterday? • Customer: I don’t know. I just moved the machine from my office to server room. • QA: Why did you move the machine? • … • ..
- 19. A day of a QA • Read emails • Clean environment and install new build. • Testing, bug verification, … • Find new bug and submit it to RD. • Discussion for the current issues. • Discussion for the new features. • Write test plan / test spec / test case • Clarify, reproduce issues and collect more debug log… • Review document, write automation, ….. • Reading, study group, …
- 20. Happy and Unhappy • Happy • Unhappy – Improve the quality of the – Too many tasks software – “Why didn’t you find the bug – Provide the solution that before the release?” customers really need – Schedule VS Quality – Learn new things – … – …
- 21. Q&A
- 22. Thank you~