Incremental test automation for Retailers to save money
Download as PPTX, PDF1 like302 views
Incremental test automation helps retailers save money by optimizing their cost of quality. Implementing test automation incrementally based on factors like revenue traffic and budget allows retailers to focus testing on high priority areas. This leads to earlier detection of bugs, faster delivery of features, and reduced failure costs. A case study describes how a retail chain benefited from using a digital commerce quality automation framework with an incremental approach. They saw reductions in regression cycle times, test development efforts, and overall test suite size.
![©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND
How DCqaf Helped a Leading Retail Chain
ChallengesAbout the Retailer
An American retail chain specializing in
counterculture-related clothing and
accessories.
Has more than 600 stores and online portal
for selling apparel and branded merchandise.
Also has two co-brands with online-only
presence
Longer regression cycles due to absence of effective
automation
High execution time of existing test automation suite
No concrete Test automation process and no CI
implementation
In need of Parallel execution implementation for
Automation suite
Incremental Test Automation
Solution
Revisited existing regression suite to first eliminate test cases which are duplicate or has similar verification points
Rationalized the test suite and consolidated it to keep it lean(reduced it by 25%) while having the same test
coverage
Implemented DCqaf using Selenium & Ruby-Cucumber.
With in-built reusable and platform-specific(SFCC in this case) methods and back end verification scripts, Test script
Development started with minimal setup time.
With DCqaf’s in-built features, new co-brands were easily added to existing suite with around 60% reduction in
development time. This resulted in high ROI for new brands [ < 6 months]
Incrementally automated the new scripts for each sprint and revenue drawing view ports were prioritized for
development
Sauce Labs Cloud platform was setup while using Jenkins as CI Pipeline](https://image.slidesharecdn.com/incrementaltestautomation-190731071651/85/Incremental-test-automation-for-Retailers-to-save-money-20-320.jpg)
Incremental test automation for Retailers to save money
- 1. How Incremental Test Automation helps Retailers Save Money
- 2. ©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND Sr. Test Automation Consultant, Aspire Systems [email protected] Speaker Bhanu Prakash P
- 3. ©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND About Aspire Global technology services firm with core DNA of Software Engineering Specific areas of expertise around Software Engineering, Digital Services, Testing and Infrastructure & Application Support International headquarters in Singapore with presence across US, UK, Benelux, Middle East and India Recognized 9 consecutive times as “Best Place to Work for” by GPW Institute Vertical focus among Independent Software Vendors and Retail, Distribution & Consumer Products 2750+ employees; 170+ active customers CMMI Maturity Level 3, ISO 9001:2015 and ISO 27001 : 2013 certified A Great Place to Work - 9 Glorious Wins in a Row Quality Maturity Level Technology Awards Forrester Rated Aspire one of the Top Continuous Testing Service Providers (2017) UK India Middle East Singapore US Poland Netherlands Mexico
- 4. ©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND 1 Importance of Cost of Quality (CoQ) 2 How Test Automation Helps Optimize CoQ 3 The Need for Incremental Test Automation 4 Platforms that Facilitate Test Automation for Retailers 5 Recommended Tools and Methodologies 6 How a Retail Chain Benefited from Incremental Test Automation – Use Case Agenda
- 5. Importance of Cost of Quality
- 6. ©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND Importance of Quality on Cost Cost of Quality – A financial measure of Quality Performance in an organization Prevention Costs Appraisal Costs Internal Failure Costs External Failure Costs Cost of Good Quality Cost of Bad Quality Cost of Quality
- 7. ©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND Ever - Increasing Test Coverage A Decade ago.. A pure website-based commerce Now !! Customer experience is being delivered in multiple view ports on different OS platforms What do Retailers Need Now? A seamless shopping experience across all the viewports All features and changes to be tested and certified on majority of the above platforms
- 9. ©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND Test Automation for Optimizing CoQ Test Automation should be planned right from the design phase of the project and should fall under Preventive Costs Shift Left Approach will help in keeping a control on appraisal costs, as majority of the test activity costs lies in the Appraisal Delay in introducing Test Automation into STLC will lead to increased Internal and External Failure Costs Test Automation allows testers to concentrate on validating new features and changes to system there by finding critical bugs early This leads to an early Go-No Go decision which leads to faster delivery of the features
- 10. ©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND Test Automation – Do’s & Don’ts 100% Test Automation and High expectations on ROI Not planning for Test Automation Maintenance Automating Applications planned for Major Redesign or Sun-setting in the near future High Flaky or Unstable Functional Tests High Critical and Revenue Generating Flows Stable Functional Tests Labor Intensive Tests Unit/Integration Tests Build Verification Tests Factors to be considered while planning Test Automation Common Pitfalls to be avoided while designing Test Automation
- 11. The Need for Incremental Test Automation
- 12. ©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND Incremental Test Automation – Why ? Evolving Requirements Inherent to the Retail Industry, requirements keep changing based on shopping trends – This translates to more number of builds for QA team Test Across Devices Seamless shopping experience across multiple shopping platforms is a costly affair. Incremental and Selective Automation of revenue generation flows across different channels keeps the costs to minimum while reducing internal/external Failures Scalable Testing Cloud Platforms enhances Incremental Test Automation by testing across different viewports without any added costs Early to Market Quick to market leads to first mover’s advantage. So retailer should able to certify the changes faster. Incremental test automation helps in this and let QA team give confidence & faster sign-off Frequent Testing Utilizing CI/CD Pipeline helps in verifying the sanity of builds frequently through out the day helps developers detect conflicts in the code and deliver fixes well before QA team pickup the build
- 13. ©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND Incremental Test Automation – How ? 67% 11% 10% 10% 2% iOS Android Macintosh Windows Other OS 62% 24% 14% Safari Chrome Other browsers Device/Revenue Traffic Below charts show snapshot of traffic by OS and by browser, for a typical retailer. This should be prioritized based on which view ports should be automated first Budget allocation Versatile Automation tools are available in market – both Open Source & Licensed Based on budget planned and scope of applications being automated, appropriate Automation tools should be selected for implementation Open Source Licensed Incremental Test Automation should be driven by below factors
- 14. ©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND Incremental Test Automation – How ? Ease of Use Ease of use (level of programming ability, installation/setup, language support) will also be a driving factor if there are mix of desktop and web apps Business users wanting to use Compatibility with other tools/infrastructure Data driven Framework Keyword Driven Framework CI/CD Integration CI/CD Integration and Incremental Test Automation go hand-in-hand, as they derive benefits from each other. Robust and well planned CI/CD pipeline enhances Test automation benefits many fold by Helping DEV team to unit test their code for each code check-in and deliver a quality build to test QA team then uses it for verifying Regression flows and provide a faster sign-off.
- 15. Platforms that Facilitate Test Automation for Retailers
- 16. ©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND Cloud Device Platform Cloud-based Infrastructure solution consisting of pool of Virtual machines and Real Devices Easy and Remote access to all infrastructure for all teams distributed across globe Enables organizations to provide access only on need basis to modern and legacy devices so that an inventory of devices need not be purchased and maintained on- premises Enhances the capability of Incremental Automation by accelerating QA and there by ensures faster Time to Market Full support to most of the CI/CD Tools with plug-n-play capability with existing setup Intelligent Reports Dashboard along with screenshots and playback of execution Virtual (Emulated/Simulated) devices as well as Physical devices of various configurations – OS, Storage, RAM
- 17. ©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND Digital Commerce quality automation framework (DCqaf) E-Commerce UAT Release Automation Frame work Reusable Scripts Product Specific Scripts Test Data Config Files Object Repository Selenium Core Engine Version Control Continuous Integration Remote VMs or Cloud Device Platform HTML Reports / Extent Reports QA Team Report & Failure Analysis On Success
- 18. ©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND DCqaf – Key Advantages Technology Scalability Ease of Use Test Reports 3rd Party and Backend Integrations Reusability Tool Stack (eCom) - Selenium Webdriver – Java and Cucumber – Ruby Framework Tool Stack (mCom) – Appium Web driver with Java and Ruby Tool Stack (POS) - Test Complete with Jscript Flexible programming language compatibility Multi test data execution for a test case Low complexity framework with ease of scripting Automation testing support for 3rd party e-commerce Tools – Taxation, Fraud Management, Loyalty, Customer Interaction Touchpoints Integrated with TestRail and Jenkins Built-in Integration with Sauce Labs Regression Testing support for multiple browsers/versions Support for mobile devices - Web and Native App Multi-brand, Multi-site Support Test reports customization Easy to interpret Shared with stakeholders after execution 100+ Generic methods to support most of e- commerce Website components Product specific reusable scripts like for Demandware and Magento
- 19. Use Case
- 20. ©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND How DCqaf Helped a Leading Retail Chain ChallengesAbout the Retailer An American retail chain specializing in counterculture-related clothing and accessories. Has more than 600 stores and online portal for selling apparel and branded merchandise. Also has two co-brands with online-only presence Longer regression cycles due to absence of effective automation High execution time of existing test automation suite No concrete Test automation process and no CI implementation In need of Parallel execution implementation for Automation suite Incremental Test Automation Solution Revisited existing regression suite to first eliminate test cases which are duplicate or has similar verification points Rationalized the test suite and consolidated it to keep it lean(reduced it by 25%) while having the same test coverage Implemented DCqaf using Selenium & Ruby-Cucumber. With in-built reusable and platform-specific(SFCC in this case) methods and back end verification scripts, Test script Development started with minimal setup time. With DCqaf’s in-built features, new co-brands were easily added to existing suite with around 60% reduction in development time. This resulted in high ROI for new brands [ < 6 months] Incrementally automated the new scripts for each sprint and revenue drawing view ports were prioritized for development Sauce Labs Cloud platform was setup while using Jenkins as CI Pipeline
- 21. ©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND How DCqaf Helped a Leading Retail Chain Tools DCqaf with Selenium Web driver with Ruby + Cucumber Framework Cloud Platform: Sauce Labs Continuous Integration: Jenkins Results 0 10 20 30 40 50 60 Reduction in Regression suite size by Rationalization Reduction in development time for newly added co-brands Reduction in turn around time for Regression cycle with help of Jenkins CI & Sauce Labs25% 60% 50%