Use Docker to Enhance Your Testing
- 1. W5 DevOps & Testing 5/10/17 11:30 Use Docker to Enhance Your Testing Presented by: Glenn Buckholz Coveros Brought to you by: 350 Corporate Way, Suite 400, Orange Park, FL 32073 888-‐-‐-‐268-‐-‐-‐8770 ·∙·∙ 904-‐-‐-‐278-‐-‐-‐0524 -‐ [email protected] -‐ http://www.starwest.techwell.com/
- 2. Glenn Buckholz Glenn Buckholz leads CI and deployment automation efforts at Coveros. Using his more than fifteen years of industry experience, Glenn brings success to his customers. Beginning his career as a consultant implementing automated test frameworks, he introduced the concept of change management to many, many projects. After moving on from consulting, Glenn settled down at the Public Company Accounting Oversight Board as their full-‐time enterprise change manager. He eventually ventured back into the real world at Coveros, where he specializes in implementing agile practices, implementing CI, and engineering configuration management instead of simply documenting it.
- 3. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. Agility. Security. Delivered. 1 Use Docker to Enhance Your TesFng Glenn Buckholz Coveros, Inc.
- 4. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. 2 Agenda • Intro • Docker Overview • How Docker Changes TesFng • DevOps Approach • DemonstraFon • Using Docker to Talk to Developers • Speeding Up The Feedback Cycle • QuesFons
- 5. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. 3 IntroducFon • Docker is a popular form of containerizaFon • ContainerizaFon is a server virtualizaFon method in which the kernel of an operaFng system allows the existence of mulFple isolated user-‐space instances, instead of just one. • Other Container Technologies • Jails -‐ BSD • Chroot -‐ Filesystem Level • Xen paravirtualizaFon -‐ Processes Hypervisor • LXC or LXD -‐ Linux ContainerizaFon • VirtualizaFon vs ContainerizaFon • VirtualizaFon allows many OS images to use the same hardware. • ContainerizaFon allows many pieces of so[ware to use the same OS in an isolated fashion.
- 6. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. Docker Overview • Docker has 3 major components • Images • Filesystem of the container. • Containers • Running instance of an image. • Registries • Repository of images. • Docker Hub is a global public repository. • Local Registry on the docker host itself • Private Registry Image Container Private Registry
- 7. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. Docker Overview • Docker Host • Hardware and OS that is shared among containers. • Docker Service mulFplexes the OS • NIC’s • Devices • Storage • CPU limits • IO limits • Intra Host CommunicaFon • HA • Scaling • Load Balancing
- 8. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. Docker Overview • Git version control • docker commit and docker push • PromoFng containers • When the container is where/what you want docker tag imageid registryhost:port/tagname docker push registryhost:port/tagname • Make the environment change around the container. Keep the container constant. • Use mount points for configs • Use mount points for logs • Make the container network agnosFc • Make the thing that you put in producFon the thing you had in dev and test.
- 9. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. DevOps Approach • Dockerfile creaFon • Automate from known stock images • Quality Gates • Container “Unit Tests” • Not strictly unit tests but you are tesFng one high level piece of funcFonality. • Container “IntegraFon Tests” • Use docker swarm or some other orchestraFon tool to put together a temporary environment, run minimal tesFng. • Container “Nightly TesFng” • Run a full set of automated tests overnight. • Container “Manual TesFng” • A[er nightly tesFng release the containers to manual testers.
- 10. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. DevOps Approach Hourly (Automatic) Nightly Release Candidate Release QualityGate Unit,Integration QualityGate UItesting
- 11. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. DevOps Approach • Promote the container through some or all the different stages automaFcally. • Dev and test should have high levels of automaFon. • Each quality gate is automaFc and triggered by changes to the Dockerfile OR the underlying applicaFon. • Failures are preserved for some Fme (24-‐72 hours) Dev Test Stage Prod Failure
- 12. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. Demo • Tester Developer InteracFon Demo “I need a blue seal balancing an orange box on his nose.”
- 13. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. Speeding Up Feedback Cycles • Developers can have complete access to containers to look at logs • Failed containers can be saved to a private registry • Testers can demo the defect on demand. Old Way: New Way: Find Bug Fight About it Compromise Fix it Fix itFind Bug Demonstrate
- 14. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. Demo • Technical DemonstraFon
- 15. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. How Docker Changes TesFng • The operaFng system is domain of specialists due to the impact of one wrong change. • The host should sFll be maintained by specialists. • Containers put barriers between the so[ware stack and the OS. • Allow the testers direct access to the containers. • Containers can now be tagged with test arFfacts. • Test case failed. • Code branch. • Testers can capture the state of the failure. • This is more complete than a descripFon. • Developers can start the container in failed state and debug.
- 16. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. Why This Makers • CI/CD Cannot work in a meaningful way without a quick way to pass informaFon • Bug interpretaFon by the tester takes Fme • Version matching environments takes Fme • Developers understanding the issues takes Fme • Gelng the applicaFon in the right state takes Fme • Following a CI/CD doctrine can make you fail faster. • Docker can tell you why you failed faster.
- 17. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. Using Docker to Talk to Developers • Sample of an old style bug report • What a bug report could look like. Title: Something didn’t work Description: The thing didn’t work Release number: 1.2.2.1 Expected Behavior: The thing should work Steps to reproduce: 1.Do stuff 2.Do more stuff Expected Behavior: The thing should work and it failed when I tried X Image ID: SOME LONG GUID
- 18. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. Using Docker to Talk to Developers System Version System StateLogs Description of Event 6030ba23a646dad11c3d9fc7ce7b7adb2fbbc92ffee87e82b8667728a74cec2a
- 19. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. What do I, as a tester, need to know? • High level overview of how docker funcFons. • What a docker commit does. • The difference between an image and a container. • What is a private repository. • Don’t air your dirty code in public. • How to do or have someone automate a docker push to a private repo. • How this can change your defect reports. • More concise, rely on the container
- 20. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. Docker TesFng and Agile • How does this “fit” into agile • By reducing the tesFng feedback cycle meaningful manual tesFng can fit within sprints • Slight change of story/epic format • User Story • The test case • Container id • Sprint planning can demand certain features and branches be tested in a parFcular container for UAT. • Clearly sets deployment expectaFons • Can be part of the definiFon of done, must deploy and work on container X
- 21. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. AdopFon • Docker is new and I don’t have it. • This is an opportunity, you can make tesFng be part of the process. • Upfront implementaFon is cheap and you can guide process. • We have docker but they didn’t think of tesFng • Show with a private repo it can be a cheap add for great gain • We already do what you are suggesFon, no actually it's cooler • OK, Let me take you out for lunch.
- 22. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. Conclusion • As a tester I do not need to know about COW filesystems, orchestraFon, swarms, or any other advanced containerizaFon topic, just basic concepts. • If we implement docker I should be able, as a tester, to get command line access to the container to help get more informaFon for my defect reports. (So[ware Version, container version, log access) • Using an image of a broken container is an extremely effecFve addiFon to my defect tracking. • Containers can allow me, as a tester, to communicate more informaFon with developers effecFvely.
- 23. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 21 QuesFons?
- 24. © COPYRIGHT 2017 COVEROS, INC. ALL RIGHTS RESERVED. References • ContainerizaFon DefiniFon -‐ hkps://en.wikipedia.org/wiki/OperaFng-‐system-‐level_virtualizaFon