Continuous testing & devops with @petemar5hall
5 likes1,265 views
This document discusses testing software in high frequency delivery environments using continuous testing and DevOps practices. It outlines how continuous testing is not just about test automation, but also includes automated management of environments, application feedback through monitoring, and engaging in XP practices. DevOps helps by automating building, testing, and deployment to provide consistency and tools for teams. Characteristics of high frequency delivery environments include automating infrastructure, testing, and deployment to reduce errors and allow for smaller, more frequent deliveries. This allows for a single view of quality and faster restore times when issues arise.
Continuous testing & devops with @petemar5hall
- 1. Testing software in high frequency delivery environments. Continuous Testing & DevOps
- 2. - Lead consultant & trainer at Lean Software Services - 16+ years working in software development, based in London & Madrid Peter Marshall MAIL [email protected] TWITTER @petemar5hall LINKEDIN https://www.linkedin.com/in/petedmarshall
- 3. Training Coaching Consulting Madrid Barcelona London Copenhagen - Testing - DevOps - Continuous Delivery - Automation - Virtualisation
- 4. Gain insight into how continuous testing, DevOps, and organisational structure can alter the way we think about testing. Understand how DevOps practices contribute to the test effort. Objectives
- 5. Who is here today?
- 6. Creating environments deploying to environments testing environments looking at requirements pair programming/ testing creating test data writing automated checks running through regression scripts what else does a tester do? triage mitigating delays
- 7. Who is testing in an agile environment? What about doing continuous delivery? Who has a devops culture / team / role?
- 8. Sources: Applied Software Measurement: Global Analysis of Productivity and Quality by Capers Jones, & IBM The Future of Testing Most forms of testing find fewer than 30% of all bugs almost 40% of testers have seen an increase in the frequency of release Productivity and quality are directly coupled: projects with high quality have high productivity and vice versa
- 9. How can devops and continuous testing help?
- 10. Agile gave us..
- 12. the test and QA headache is compounded further by the need to release faster and more often Amazon is on record as making changes to production every 11.6 seconds on average in May of 2011. Facebook releases to production twice a day. Many Google services see releases multiple times a week, and almost everything in Google is developed on mainline. Etsy push unto 70 changes a day to its monolithic software!
- 13. the presence of a tester can sometimes introduce more risk than they mitigate..
- 16. Continuous Testing …not just about test automation! (ignore wikipedia) Actually includes, but not limited to: Automated testing Automated management of production and non production environments Automated application feedback through monitoring Automated evaluation of business objectives Evaluating product concepts Testing engaging in XP practices organisational change
- 17. The DevOps movement is built around a group of people who believe that the application of a combination of appropriate technology and attitude can revolutionize the world of software development and delivery. Stephen Nelson-Smith - 2010 its about: Automating building, testing, and deployment Bringing consistency to the way we use and create technical assets Providing teams with the tools required to achieve technical excellence DevOps
- 18. vs. more facts.. high performing organizations ship code 30 times faster (and complete these deployments 8,000 times faster), have 50% fewer failed deployments, and restore service 12 times faster than their peers. The longer they had been employing these practices, the better they performed. source: ThoughtWorks, 2014
- 19. automate the build Reduce general coding errors, create a single source of truth, ensure basic component integration
- 20. vs.script your configuration! say NO to hand crafted configuration files!
- 21. automate the infrastructure Remove configuration issues by having a repeatable mechanism for creating environments.
- 22. vs.automate the testing continuous testing does away with the concept of regression testing. ..an effective software delivery process requires no regression testing stages. It becomes integral to everything we do..
- 23. automate deployment say NO to sneaker net say NO manually to copying files say NO to manually pulling and pushing servers in and out of load balancers deploying software is one of the most mission critical aspects of what we do, let a machine do it!
- 24. monitor One of the greatest and overlooked tools in the arsenal of the tester is monitoring. Realtime access to application logs Realtime access to error logs Realtime access to infrastructure logs
- 25. “A good, easily accessible event & error monitoring system is essential for anyone releasing software. Build an early warning system into your environment!”
- 26. “Understanding your systems steady state allows you to react quickly to any deviation”
- 27. Build Agent & Devel The delivery pipeline Pre commit tests Code review PO/UX review Branch / Merge Build Unit tests Contract tests Component tests Coverage Static analysis Component performance End to end component tests NFR tests Exploratory Test performance testing Story sign off / Demos NFRs Analytics Monitoring Production tests canary or blue / green release CI Environments Developer Local Pre-Production Production Virtualised Full stack environments Commit Auto-Deploy Promote Release Build Pipeline: There can be multiple pipelines, usually per system, should execute in minutes Commit: The commit can include changes to the infrastructure, database, or application code. Outputs: Software, infrastructure and configuration Environment manifests Release notes
- 28. Create a single view of quality
- 30. Demo
- 31. Thank you! TWITTER @petemar5hall Agile Consultant & Technical Architect @planday Peter Marshall fiveMAIL [email protected] LINKEDIN https://www.linkedin.com/in/ petedmarshall