How to Make Test Automation for Cloud-based System
- 1. How to Make Test Automation For Cloud-Based System by Nick Babich
- 2. Telephony Automation Team Alexey Georgy Alexandra Nikita Alexey Igor 2
- 3. Contents 1. RingCentral product and system specifics. 2. Automation: What-When-Where-How and Why 3. Questions and Answers 3
- 6. RingCentral: Cloud Business Communications 6
- 7. RingCentral Service Telephony is a cornerstone service for the RingCentral. It consist of many different features and each new release brings more to production. 7
- 10. What? Type of Automation 10
- 11. 11
- 12. Testing Types ● Functional Tests ● Acceptance Tests ● A/B Tests ● Unit Tests ● Stress Tests ● Security Tests Business Facing Technology Facing Support Programming CritiqueProduct 12
- 14. How to Define Scope for Automation ● Automate the most critical cases (P0 cases, especially one which are hard-to-execute manually). ● Automate the most frequently used scope (included in regular regression cycles). ● Automation for the most automatable scope (easy to automate and move to the execution). Make a distinction between “ephemeral” and “long-standing” cases 14
- 19. Testing Process Create new test scope Define a scope for automation & automate it Execute automated scope Who: Scrum team QA engineers. Task: Create test design for new features. Deliverable: Prioritized test scope, proposal for automation Who: Automation QA engineers. Task: Automate test cases Deliverable: Test scripts ready for execution Who: Scrum team QA or DEV. Task: Execute automated test cases. Deliverable: Execution results. Understanding product quality. 19
- 20. Test Automation Plan Development Team Step 1: Starting development new telephony feature. Development Team Step 2: Prioritized test design for new functionality Development Team Step 3: Automation for the most critical test cases Telephony Automation Team Step 4: Continue automation for 2-nd and 3-rd category Telephony Automation Team Step 5: Delivering automated cases to the customers 20
- 22. 22 Test Environment and Production
- 23. Test Environment Update ADB Codebase change or Manual request Start Automatic Deployment System SVN New Build 23
- 24. Automatic Deployment System 3 2 1 Deploy → Verify (Post Deployment Verification) → Execute Regular Scope 24
- 25. 25 Distributed Test Execution Scheme
- 27. Tools • Framework and automated tests cases (scripts). • Test Environment • Dashboard for test execution results and reports. • Bug tracking system 27
- 29. How It Works 29
- 30. Execution Scheme Test Environment Deployment Test Scope Execution Send Execution Results to Dashboard New Build Available 30
- 31. Framework and Automated Test Cases Business Layer Test Scripts Layer Application Part (exe) Command line application with a set of parameters (What we execute and Where) Set of modules required for test scripts (WebDriver, Rest API, Oracle ORA) Automated test cases (test scripts) Powered by 31
- 33. Test design and Test script 33
- 34. Test script report Powered by 34 • Start time (timestamp) • Group (e.g. feature) • Execution result (Passed / Failed / Skipped) • Screenshots and attachments (if test failed during execution ).
- 35. Execution Results for the Scope ● Archive for all test reports ● Tested build (Branch and Build number) ● Backend log data and system traces Powered by 35
- 36. Execution Report in Dashboard • Configuration properties • Backend build (from execution results) • Execution Statistics (passed / failed / blocked) 36 Powered by
- 38. Why? 38
- 39. 39 Metaphor Behind Test Automation
- 40. 40 Cost: Manual vs. Automated Testing
- 41. 41 Benefits of Automated Testing Global Continuous Integration Team 6 Development Team Regression Team Total number of test cases: 5500 Total number of Automated: 2100 On a daily basis On a weekly basis On a daily basis Automated
- 42. Test Automation Disadvantages 42 1. Test scripts base requires regular support and actualization. 2. Troubleshooting for environment deployment and execution results analysis. 3. It works only on long-term basis.
- 43. Important Moments ● Feedback from stakeholders ● Task Prioritization ● Popularization of automation 43
- 44. 44
Editor's Notes
- #29: Нужно использовать уже готовые и достаточно стабильные компоненты и библиотеки.