Don't hate, automate. lessons learned from implementing continuous delivery
0 likes1,846 views
Lab Zero Innovations, Inc. discusses their transition to continuous delivery (CD) and the lessons they've learned, highlighting the importance of automation in software development processes. They emphasize the significance of collaboration between developers and QA teams, the challenges of running tests efficiently, and the need for structured environments to support testing and deployment. The document provides insights into their build processes, tools used, and strategies for improving test performance and coordination.
Don't hate, automate. lessons learned from implementing continuous delivery
- 1. Don’t Hate, Automate. Lessons learned from implementing Continuous Delivery. Matt Wilson, CTO Lab Zero LAB ZERO INNOVATIONS, INC. PIER 9, THE EMBARCADERO SAN FRANCISCO, CA 94111 T:415.839.6861 F:415.839.6901 [email protected] HTTP://LABZERO.COM
- 2. Overview Who we are Our evolution toward Continuous D* What we’ve learned along the way Questions & Answers LAB ZERO INNOVATIONS, INC.
- 3. Who We Are Product development, design, and strategy team based in SF. Early-stage entrepreneurs and startup founders: earfl.com, meez.com, gofish.com, echosign. com, veodia.com. Held senior product design & dev roles at Apple, Google, Yahoo, Kodak, Accenture. Our clients are often startups and Fortune 500s. Founded in 2008 Many of our clients hire us just to build out their Continuous D* rigs. LAB ZERO INNOVATIONS, INC.
- 4. Our path to Continuous D* and what we’ve seen
- 5. In the beginning there was Just a build… We had code and compiled it. We even had some unit tests for some areas of the code. The official build ran nightly but we certainly weren’t using it for fast feedback. Then we started deploying with automation. “cap production deploy” Those were some fancy pants. Then we started running our tests on every check in. Then we needed to automate the validation of our app running in different environments Then we started automating the creation of our environments. Just can’t have enough. LAB ZERO INNOVATIONS, INC.
- 6. Necessity struck We have to start new projects very often. The teams we collaborate with want to see progress often. Projects started without a clear delivery process were often slower to deliver over time. At some point manually running our deployment scripts every time we wanted to share our work seemed like too much hassle. </first_world_problem> We need to know that the feature works in the QA environment. We figured out how to get Continuous D* running on our projects in less than a day. LAB ZERO INNOVATIONS, INC.
- 7. Our Webapp “Build” Rspec: Unit tests and generates code coverage reports. Jasmine + PhantomJS: Javascript unit testing. Cucumber: We write our ATDD suites in Cucumber and use Capybara to actually drive browsers. Brakeman: Static security analyzer to help us find obvious security vulnerabilities. Capistrano: Automated deployments of RoR, PHP, Java... applications. LAB ZERO INNOVATIONS, INC.
- 8. Our Mobile “Build” Xcodebuild, Ant+javac: iOS and Android command line tools. OCUnit, JUnit: iOS and Android unit testing frameworks. Cucumber: Using Calabash to actually drive the native devices and emulators. TestFlight: Deploy every passing build to TestFlight for easy distribution to team devices. LAB ZERO INNOVATIONS, INC.
- 9. The story starts at your laptop and ends in production. LAB ZERO INNOVATIONS, INC.
- 10. TDD the first “d” is for “driven”
- 11. Build Failed
- 12. “When the light is green, the trap is clean.” --Ghostbusters
- 13. It’s hard to keep the light green or meaningful if tests are written afterwards. Build your culture of automation awareness. There is a correlation between writing tests and running them. Those that write them tend to run them more often. Developers and QA both need to run tests before committing. If developers aren’t running the tests because they take too long, it’s worth the investment to speed them up. CD can fall apart quickly if the team doesn’t react early to failed builds. LAB ZERO INNOVATIONS, INC.
- 14. Speed will matter when you’re doing it right
- 15. How long does it take to deliver a feature to QA? LAB ZERO INNOVATIONS, INC.
- 16. Ouch, my tests! We want to run as many tests as we can but once we’ve invested in a fair amount of browser and emulator testing they get really slow. When they get slow we become reluctant to run them. Tests that don’t run, can’t find bugs. The longer you wait to run them all, the less likely they are to still pass. Tests that are broken and remain unfixed become dead weight. Releasing under-tested products often means we’re asking too much of our QA teams and usually won’t find bugs until they are in production. LAB ZERO INNOVATIONS, INC.
- 17. Usual suspects for slowness in the pipe Heavy use of fixtures and db cleaning (factory-girl and database-cleaner) Consider using mocks/stubs instead of db’s when possible. Android emulator Get a real device and plug it into your CI server. Assets compiled on each server at deploy time Pre-compile your static assets on the build server and deploy archives instead. LAB ZERO INNOVATIONS, INC.
- 18. Parallulala, parallaluh, parallelize them Selenium tests are just too slow to not do this Remote browser tests will need you to run multiple tests at once. Especially when you start considering running your tests in more than one browser/OS combo. To do this well you should lean on the PaaSs out there like Solano CI (beaucoup parallelization-fu), Sauce Labs (browsers and mobile emulators, can be used from Solano), Travis CI, etc... Unit tests will need it too If you’re doing TDD, you likely already have this problem. LAB ZERO INNOVATIONS, INC.
- 19. Continuous D* wants to use your test automation across many environments
- 22. You’ll need a few environments to pull this off. CI Server Run tests here first to find anything that should prevent us from deploying the app. Unit tests, static analysis, DevInteg First deployed spot for the app, the environment will be volatile and will serve little use other than for CI. May also serve as QA server if the numbers of commits are low. QA1, 2, 3.. You may need one of these for each product team. Staging Runs in production and is often used as a place to validate the app in production before going live. Production Live. LAB ZERO INNOVATIONS, INC.
- 23. We want to get the features out, but they have to be verified in QA and Staging before we go live. LAB ZERO INNOVATIONS, INC.
- 24. Co-own yer ATDD with QA
- 25. Force multiplier Share the same ATDD code Devs and QA should collaborate on their functional testing investment. Many organizations separate these groups activities which leads to redundant investments. Devs need to lead in the spirit of TDD Relying only on QA for functional automation causes flapping red->red->green->red->green builds. It works better when you work together Devs can do the heavy lifting on writing reusable steps and tests. QA can add more cases if they see fit. Devs should expect to train and support QA on the tools. Allow QA to audit With our QA hats on, let’s have an open conversation about what should be tested and what is missing. This is how we can focus our beams to make the light brighter. LAB ZERO INNOVATIONS, INC.
- 26. Running tests just one thing your CI server can do
- 27. Robots, ACL’s and users, what could go wrong? Empower folks with a magic button There are likely things that your team is always asking someone else to do for them. De-spof it by making a job that folks can click whenever needed, even from their phones. Zomg seriously. Database Refreshes Nearly every webapp project needs to be able to reset/refresh the dev/qa/staging databases easily. Production deploy jobs This is your parent’s liquor cabinet: make sure the kids don’t get into it, but make it easy to open when needed. LAB ZERO INNOVATIONS, INC.
- 28. Thank You! Questions? [email protected] twitter/aim/yim/skype: w1150n LAB ZERO INNOVATIONS, INC. PIER 9, THE EMBARCADERO SAN FRANCISCO, CA 94111 T:415.839.6861 F:415.839.6901 [email protected] WWW.LABZERO.COM