From Continuous Integration to Continuous Delivery and DevOps

Editor's Notes

• #5: Speech notes -------------------------------- Used for 20 years The only major upgrade I know of was a new color. industry trend of shortening products’ life cycles, moving from years and months to weeks and days. This phone was in use for 20 years in Italy. From the 60’. The only major upgrade I know of was a new color. My smartphone is 18 months old, I receive apps updates almost every day, I already updated firmware twice, and 2 new models has been presented since I bought it. The Industry is constantly shortening products’ life cycles. CD enables IT to lead this trend, release cycles are moving from years and months to weeks and days or even hours. Extra notes: Industry trend of shortening products’ life cycles, more and more teams and organizations release-cycles are moving from years and months to weeks and days.
• #6: Speech notes -------------------------------- Main question here is: is CD beneficial to your business? How? I’ll share answer I found in my experience. I encourage you to find your own answers for your specific context.
• #7: Speech notes -------------------------------- A vintage test card of Italy's public broadcasting company:RAI From the name, RAI, the initials of the 3 main benefits I observed:
• #8: Speech notes --------------------------------- from risky painful releases to predictable, safe and regular events => a release become a non-event - marketing events decoupled by tech releases, without risks of failure CD turns risky painful releases to production into predictable, safe and regular events or non events. from a technical point of view, they can still be marketing events but without risks and anxiety. Thanks to early and frequent releases, it is possible to see if the organization is building the right thing in the right way
• #9: - More Features implemented over time - No value created until delivery For all this time we don’t know if features are what user wanted, if they solve user’s problem, or if there is a tech problem we keep going without knowing it, until we release
• #10: Speech notes -------------------------------- - Every feature is released early and it create value - Problems are detected early, when it is easier and cheaper to fix them. - The consequences of a mistake are limited by the small scope and size of each incremental release For Anyone => preferred option Early adopters Requestor of the feature Internal users
• #11: Speech notes -------------------------------- Remaining 10% get almost never completed problems discovered only late during integration or hardening release to production followed by weeks of expensive bug-fixing and firefighting With CD instead real progress becomes - Visible - Tangible Have you ever be accountable for the outcome of a project? If so you probably experienced the ‘90% done’ syndrome, when the gant chart reach the 90% progress and then the remaining 10% get almost never completed. It usually happens when deadlines loom, all certainties about the real progress of a project disappear. Some other times significant problems are discovered only late into the integration or hardening phase. §Other times it is the release to production that becomes a lengthy and painful experience, followed by weeks of bug-fixing and firefighting. With CD instead the real progress of the release becomes Visible Tangible in every moment
• #12: Speech notes -------------------------------- For Anyone => preferred option Early adopters Requestor of the feature Internal users
• #14: Speech notes -------------------------------- Many companies I visited in EU and US told me about misalignment they experience between IT and the business. Sometime IT and the business were blaming each other.
• #15: Speech notes -------------------------------- Sometime even between different department inside IT there’s misalignment. Look at this picture
• #16: Speech notes -------------------------------- - reduces the distance between business – customers/market => a checkpoint at every release, many chances to steer - Support mutual understanding business – IT => software dev become tangible when released => closer collaboration around each “feature” implemented CD spurs closer communication, mutual understanding, synchronization and cooperation between business and IT, and between all roles and departments involved in the project. CD also reduces the distance between the business and the customers. Furthermore, CD closes the gap between market, business, and IT.
• #17: Speech notes -------------------------------- Life before Google maps, explorers like Cock or Magellano or Colombo Nowadays exploring new markets, new technologies, new domains and applications increase organization’s responsiveness: it reduces the time required to react to new circumstances. Before Google maps people could get lost and large parts of this planet were unknowns. There were explorers like Cock or Magellano or Colombo that planned the expeditions to cross the edge of the known and discover new places. Nowadays we are still - Exploring new markets - Experiment new technologies - Discovering new domains and applications CD support this exploration, make it easier and faster CD can increase organization’s responsiveness: it reduces the time required to react to new circumstances.
• #18: Speech notes -------------------------------- When I joined Ferrari F1 racing team to help advancing the adoption of Lean and Agile practices, we have been asked to go faster and safer. You don’t want a bug to compromise the result of a race or the championship. It seems a paradox. Actually CD helped here.
• #19: Speech notes -------------------------------- CD resolve the tension between ITops and Prod/SW Dev When IT Ops wins, When Prod Dev wins, Instead collaboration that increasing the throughput + stabilizing production systems, increasing innovation + predictability This is another apparent paradox. On the right: IT operations aims to keep the production systems up and running in a stable way. On the left: Product development aims to release new, live features to stay ahead of the competition. These two goals have traditionally been seen as conflicting, like in a zero-sum game. When Prod Dev wins, devs throws releases over the wall and IT Ops release them and pray and sometime start with firefighting. When IT Ops wins, organization implement governance standards as SOX, PCI DSS, ITIL, COBIL in a very bureaucratic way that slow down new releases and development CD instead has the paradoxical effect of increasing the throughput of new features and stabilizing production systems, increasing innovation and predictability together at the same time. Extra notes: Pressure to change ○  pressure to innovate (i.e. stay ahead of competitors); ○  technical novelty and variety in the technologies used for the project; ○  uncertainty (e.g. from the domain, the market, the users, the partners, the suppliers) Need for stability ○  interruption or disruption of services provided cause significant loss of money; ○  services provided are life-critical CD also helps with the compliance to standards such as SOX, PCI DSS, ITIL, and COBIT. And in many cases, problems with external policies are due to implementations of an interpretation of a policy, while there are other, lightweight ways to achieve the policy goals that actually produce better outcomes.
• #20: Let’s look at implementing cd, starting with prerequisites, and checking how much cd are we already doing
• #21: Speech notes -------------------------------- - Let’s start with prerequisites to CD! After a public workshop a CTO told me he recognized many of the problems I described during the day and that solutions proposed made lot of sense to him. He also told that the IT dep was not ready to adopt those advanced practices so he asked where they could start to prepare for CD?
• #22: Speech notes -------------------------------- 2 prereqs Each build on top of the other, Iterative dev and CI Jim Highsmith, one of the 2 authors of the Agile Manifesto that work in TW (the other is Fowler) suggest this progression: Start with iterative development Add CI Add CD Extra notes:
• #23: Speech notes -------------------------------- At every iteration a new feature is created, an existing feature is improved/evolved, ready to be released experimented first in the 1930 in the 1990 its use became common. In 2000 US Department of Defense states a clear preference for Iterative development. Nowadays Waterfall and its evolutions and extensions such as V-Model are considered obsolete. And just-do-it approaches inadequate for teams bigger then 2
• #24: Speech notes -------------------------------- - incremental vs evolutionary (better) - After evolutionary and iterations shorter than 1 week => flow-pull systems such as Kanban
• #25: - After CI : Deploy & Test automation - That Includes architecture improvements to support automatic configuration for multiple environments (test, staging, prod) Design to make code testable Architecture to Enable hot deploys Automate remediation plans
• #26: Mentioned 1st time 1991 (by Grady Booch) 1st article on CI in 2000, focus on devs integrating, not on tool Here definition from MF
• #27: Build and automated tests run and return a report in 5-15mins
• #28: Speech notes -------------------------------- Can skip
• #29: Merge in mainline are so frequent that branches are rare exceptions. Features not finished are hidden from the front-end (with feature toggles), and back-end code is tested for not breaking existing functionalities (latent code, branch by abstraction).
• #30: When devs works on different branches
• #31: Each branch accumulates overtime lot of potentially conflicting or interfering changes
• #32: - Only when devs start the merge war, conflicts and interferences are spot, - resolving merge conflicts is painful and time-consuming, when not impossible
• #33: With frequent merges on the mainline, conflicts are less likely, and very easy to fix with a quick chat Interferences are spot immediately what they are cheaper to fix
• #34: Speech notes -------------------------------- After the Whys we can talk about the whats. What is CD than ?
• #35: Speech notes -------------------------------- CD has 3 fundamental ingredients. People/social ingredient is the most underestimated and misunderstood ingredient.
• #36: - Def by Martin Fowler - Can you release in production at any time during the sprint/iteration ?
• #37: Def by Jez Humble, the guy how wrote the book. notice this business point of view
• #38: Speech notes -------------------------------- - Let’s do a quick self-assessment (a proper assessment take between 1 and 3, but this can give us a general idea of there your team/org stands)
• #39: Speech notes -------------------------------- 1 – from the first to the last iteration of your release, anytime during any iteration 2 – Let’s make an example, I’m pushing a feature that needs a new db table: db change script and update to the remediation plan
• #40: Speech notes -------------------------------- - typically in 5 to 15 minutes you get a report with the results - When you do manual QA and it takes days you cannot get this fast feedback
• #41: Speech notes -------------------------------- - Anyone can do it (QA, Bus stakeholder, PO/PM), it’s completely automatic, no need for manual steps - even with db changes, configuration changes, etc.
• #43: 1-2 weeks for QA => Less than 1 day for QA
• #44: they improved the productivity of developers by 2-3x, Like in the consumer printer market, it was incredibly competitive, where new and innovative offerings were showing up nearly every month. Their problem was that the firmware group was having tremendous difficulty keeping up with the demand for new, innovative features,
• #45: - Simplification reduced planning effort from 20% to 5%, one of the biggest saving Build cycle time: 1 week -> 3 hours (10-15 builds per day)
• #47: Green field product release
• #48: - Things easy to achieve by a small group of people, and everyone benefit from them after they are done - Basic remediation plan = only for backward compatible changes
• #49: - Changes that involve anyone (i.e. keep existing tests working when changing a feature, create remediation plan also for backward incompatible changes)
• #50: Speech notes -------------------------------- Do companies need to hire specifically for DevOps? Extra notes:
• #51: Speech notes -------------------------------- DevOps doesn't require particular roles; in fact, many believe that creating a separate role for DevOps misses the point that developers and operations people should share ownership of the software, end to end. DevOps extends collaboration to include infrastructure, release management, support, security, and other roles in IT operations. Extra notes:
• #52: Speech notes -------------------------------- This is the traditional divide between devs and IT Ops Different skillset, language, bosses
• #53: Speech notes -------------------------------- Solution: A IT Ops joining dev team since the beginning of a new request to add requirements related to deployment automation and support A dev joining IT Ops team to help automate the release and to help with support They rotate, build a common language and mutual understanding and so collaboration
• #54: Speech notes -------------------------------- This misses the point Moves devs and IT Ops even more far away
• #56: Speech notes -------------------------------- Companies often ask for advises on picking the right tool
• #57: Speech notes -------------------------------- This is an example of different types of tools and some tools I’ll give you soon a reference with a complete list of tool Extra notes:
• #58: Speech notes -------------------------------- Kief Morris, author of Infrastructure as code book
• #59: Speech notes -------------------------------- A successful product will grow – maybe tool will be changed Good separation of responsibility between tool and script/automation you create A successful product will grow over time and so will the related code-base. The implementation of CD will thus have to evolve too. In order to support this growth easily, a CD tool should implement the pipeline concept as a native first-class concept. It should be cross-platform compatible to support a large variety of technologies. And it should be able to distribute the build tasks across many computers, as in a build-grid, to support increased workload. It should support scripting to enable refactoring and support VCS. And finally it should follow a clear separation of concerns to enable a proper integration of different tools and to allow easy replacement of tools over time when needed. Extra notes:
• #60: Speech notes -------------------------------- Companies that are transitioning to Agile and Lean ask what’s the relation with CD and Agile and Lean, are they compatible? They support each other
• #64: Can skip
• #66: Speech notes -------------------------------- Let’s have an overview of the main principles and practices
• #67: Speech notes -------------------------------- All remaining principles derive from this main principle. The same for bug fixes, new features, new products, or emergencies This main principle incorporates the whole idea of CD: create a repeatable, reliable way to release software. 
• #68: Speech notes -------------------------------- Everybody Devs, IT Ops, Marketing, etc. is responsible for making sure the software constantly and increasingly generates business value. Have the sw fulfill the real needs of the users keeping the software functioning well in production Extra notes: All the people involved throughout the value stream of the product, from concept to cash, should share this common goal and be evaluated as a team against this common goal. This is essential to creating an environment where communication and collaboration patterns required to succeed are possible and likely to happen.
• #75: Speech notes -------------------------------- The key idea is to automate the recovery after a bad release into production. When a new version of an application is released into production but starts to malfunction or users report a showstopper bug, different techniques can help to remedy the problem. Challenge All the cases where there are 2 moving parts Data bases double challenge because the size of the data Decouple the deploy of the moving parts for example with Rollbacks and forward-compatible interim versions. Extra notes:
• #76: Can skip
• #77: Speech notes -------------------------------- Test/stage envs equal to prod Safely replace prod servers/infra Easy support for many nodes Automate the creation of test/prod environments Networking (DNS, etc) Web/mail servers Etc Infrastructure code. The goal of infrastructure automation and infrastructure as code is to have environments that are always in a well-known state and to avoid unnecessary variations that can invalidate the result of the tests or be a source of bugs in production that are very hard to reproduce. And it become quick and easy to add or replace a server and to create a test environment that is as similar as possible to the prod one. Extra notes: Infrastructure automation is intended to automatically create from scratch every test and production environment, from the operating system to the installation and configuration of the required software and services (networking, DNS, Web server, email server, etc.), and to automatically apply all the configurations expected for the environment.
• #83: Speech notes -------------------------------- Extra notes: