Terraform GitOps on Codefresh
4 likes2,615 views
The document discusses the implementation of GitOps practices using Terraform and Codefresh to simplify infrastructure management by employing pull requests for operations. It highlights the challenges associated with traditional DevOps and Terraform management, emphasizing the need for repeatability, predictability, and auditing in deployments. The content also includes best practices, a live demo overview, and resources for further assistance in adopting these methodologies.
![Example codefresh.yaml. (Continued)
plan:
title: Run `terraform plan`
stage: Plan
fail_fast: true
image: ${{build_image}}
working_directory: *cwd
environment:
- TF_COMMAND=plan
commands:
- set +e -xo pipefail
- terraform plan | tfmask | scenery | tee plan.txt
- export TF_EXIT_CODE=$?
- github-commenter < plan.txt
- '[ $TF_EXIT_CODE -ne 1 ]'
# define step called “init”
# give it a title
# associate it with a stage of the pipeline
# exit on errors
# docker image to use
# working directory (e.g. terraform code)
# environment variables
# (used for our github comment template)
# commands we should run in this step
# shell flags
# terraform plan, mask secrets, format it
# record exit code of terraform plan
# comment back to PR with plan output
# exit code of 0 or 2 is success; 1 is error
Steps can be Entirely Customized.
PLan Step](https://image.slidesharecdn.com/terraformgitopsoncodefresh-190412143000/85/Terraform-GitOps-on-Codefresh-33-320.jpg)
![Example codefresh.yaml. (Continued)
apply:
title: Run `terraform apply`
stage: Apply
fail_fast: true
image: ${{build_image}}
working_directory: *cwd
environment:
- TF_COMMAND=apply
commands:
- set +e -xo pipefail
- terraform apply | tfmask | tee apply.txt
- export TF_EXIT_CODE=$?
- github-commenter < apply.txt
- '[ $TF_EXIT_CODE -eq 0 ]'
# define step called “apply”
# give it a title
# associate it with a stage of the pipeline
# exit on errors
# docker image to use
# working directory (e.g. terraform code)
# environment variables
# (used for our github comment template)
# commands we should run in this step
# shell flags
# apply the terraform plan and mask output
# (run apply using previous plan)
# $PLANFILE ensures WYSIWYG
# Comment back on github with outcome
# Expect an exit code of zero
Apply Step](https://image.slidesharecdn.com/terraformgitopsoncodefresh-190412143000/85/Terraform-GitOps-on-Codefresh-34-320.jpg)
Terraform GitOps on Codefresh
- 1. Terraform GitOps How to do Operations by Pull Request <[email protected]> https://cloudposse.com/ @cloudposse +
- 2. What to Expect Feelings of OMG Aha! Moments... Totally Sweet Ops What is GitOps? (not rocket science) Why it’s awesome (and you’ll agree) How to get started… (our way) And... Live demo. . .. Q&A . ...
- 3. Who is this dude? Founder of a DevOps Professional Services Company We’ve pioneered Collaborative DevOps for Companies (cloudposse.com) SweetOps M e (Erik Osterman) ( 100% Open Source )
- 4. Infrastructure as a Service Everything as Code, SDNs Serverless & Lambdas Mesh Networking, Operators Container Management Platforms CI/CD Everywhere, ChatOps, GitOps DevOps Renaissance (kubernetes, ecs, mesos, swarm)
- 5. DevOps Complicated Manual Rollouts via the terminal Poor Audit Trails (huge risk) Not clear what’s been deployed Out of date documentation No one knows how to make changes The “Industry” (configuration drift)
- 6. Terraform more problems Deploying infrastructure is not like deploying a web app (no easy rollbacks) Terraform is more like a database migration tool Terraform does not automatically rollback on errors Terraform plans are a best guess of what’s to happen Terraform apply will regularly fail Terraform apply on merge risks destabilizing master
- 7. I test some changes at home... For Example….
- 8. “I ^ it worked... on my machine.” SWEAR .
- 9. Then comes… Launch Day Production
- 10. The Math is Simple A*B*C*D*E*F = impossible to manage A = # of tools pinned to versions B = # of dependencies pinned to versions C = # of AWS accounts D = # of project environments (per acct) E = # of number of developers F = # of customers (our case) Too many permutations to keep straight This is why we don’t run things “natively”
- 13. Goal: Make it Easy to Terraform Stuff. (e.g. enable anyone on team to easily spin up RDS Database with Terraform)
- 14. Let’s Practice GitOps. Use Git as a System of Record for the desired state of configuration Do Operations by Pull Request for Infrastructure as Code Then use Continuous Delivery to apply changes to infrastructure (basically it’s a CI/CD for DevOps) See output from terraform in GitHub comments (E.g. “Plan: 23 to add, 2 to change, 15 to destroy.”)
- 15. GitOps Objectives Repeatable - Apply changes the same way every time (even your entire stack all at once!) Predictable - Know what’s going to happen (e.g. before you merge) Auditable - See what was done (e.g. when things were applied. see if there were errors) Accessible - Anyone who can open a PR can contribute
- 17. Automate Anything (if it runs in a container)
- 18. How We Use Codefresh Terraform Cloud Formation Helm → K8S Helmfile Because we can run any command
- 19. But will it work with... Terragrunt? YES GITLAB? YES BITBUCKET? YES ANSIBLE? YES
- 20. About Codefresh Yet another CI/CD solution, only better. 1. Stick everything you want to automate into containers 2. String containers together in a pipeline, run them in parallel 3. Trigger pipelines on webhooks, comments, releases, etc. Slack Notifications Approval Steps GitHub Comments
- 24. Step One: Open Pull Request
- 25. Step Two: Review “Auto Plan”
- 26. Step Three: Seek Approval Code Review
- 27. Step Four: Deploy Changes
- 28. Step Five: Merge Pull Request
- 29. Sneak Peak
- 30. That was easy.
- 31. How to get started 1. Signup for Codefresh 2. Add codefresh.yaml to each terraform repo 3. Get back to work (sorry it’s that easy). Or ask us for help =)
- 32. Example /codefresh.yaml. init: title: Run `terraform init` stage: Init fail_fast: true image: ${{build_image}} working_directory: *cwd environment: - TF_COMMAND=init commands: - eval "$(chamber exec atlantis -- sh -c "export -p")" - eval "$(ssh-agent)" - echo "${ATLANTIS_SSH_PRIVATE_KEY}" | ssh-add - - terraform init # define step called “init” # give it a title # associate it with a stage of the pipeline # exit on errors # docker image to use # working directory (e.g. terraform code) # environment variables # (used for our github comment template) # commands we should run in this step # export environment from chamber to shell # start an SSH agent # load SSH key so we can pull private repos # run terraform init with s3 backend Steps can be Entirely Customized. Init Step
- 33. Example codefresh.yaml. (Continued) plan: title: Run `terraform plan` stage: Plan fail_fast: true image: ${{build_image}} working_directory: *cwd environment: - TF_COMMAND=plan commands: - set +e -xo pipefail - terraform plan | tfmask | scenery | tee plan.txt - export TF_EXIT_CODE=$? - github-commenter < plan.txt - '[ $TF_EXIT_CODE -ne 1 ]' # define step called “init” # give it a title # associate it with a stage of the pipeline # exit on errors # docker image to use # working directory (e.g. terraform code) # environment variables # (used for our github comment template) # commands we should run in this step # shell flags # terraform plan, mask secrets, format it # record exit code of terraform plan # comment back to PR with plan output # exit code of 0 or 2 is success; 1 is error Steps can be Entirely Customized. PLan Step
- 34. Example codefresh.yaml. (Continued) apply: title: Run `terraform apply` stage: Apply fail_fast: true image: ${{build_image}} working_directory: *cwd environment: - TF_COMMAND=apply commands: - set +e -xo pipefail - terraform apply | tfmask | tee apply.txt - export TF_EXIT_CODE=$? - github-commenter < apply.txt - '[ $TF_EXIT_CODE -eq 0 ]' # define step called “apply” # give it a title # associate it with a stage of the pipeline # exit on errors # docker image to use # working directory (e.g. terraform code) # environment variables # (used for our github comment template) # commands we should run in this step # shell flags # apply the terraform plan and mask output # (run apply using previous plan) # $PLANFILE ensures WYSIWYG # Comment back on github with outcome # Expect an exit code of zero Apply Step
- 35. Live Demo 1. Add User 2. Open PR 3. Run Plan 4. Seek Approval (or not) 5. Apply 6. Merge
- 36. Demo Time!
- 37. Our Best Practices Use Geodesic as our cloud automation shell Use IAM STS for short lived AWS credentials (not hardcoded credentials) Use GitHub CODEOWNERS Use .tfvars for non-secrets Use SSM Parameter Store + KMS for Secrets Use scenery for clean output; tfmask to sanitize output Atlantis “Best Practices”
- 38. Why do you care? Teamwork.
- 39. GitOps Stop living dangerously. Start using GitOps. https://github.com/runatlantis/atlantis ● Practice total transparency in operations ● Enable team collaboration ● Reduce access to environments → increase security ● Increase Productivity, Simplify Maintenance, Ensure Repeatability
- 40. Where can I ask questions? slack.sweetops.com Join our community!
- 41. Links Example Pipeline on GitHub cpco.io/codefresh-gitops github.com/cloudposse/tfmask github.com/cloudposse/geodesic github.com/cloudposse/github-commenter
- 42. Office Hours with Cloud Posse ● Expert Advice — Prescriptive solutions to your questions ● Reduced Time to Market — know your options & eliminate analysis paralysis ● Trusted Partner — who learns your stack and understands your problems ● Recorded Strategy Sessions — Weekly or Biweekly Cadence (30m-1hr) ● Easy Scheduling — via Calendly or recurring events ● Shared Slack Channel — for private communications (~12 hour SLA) What you get... Why you want it... $500/mo - 2 hours
- 43. Hire us. =) A Totally Sweet DevOps Professional Services Company 100+ Free Terraform Modules github.com/cloudposse Active Community sweetops.com/slack Awesome Documentation docs.cloudposse.com 415 5 3 5 86 15 hello@ cloudposse.com (free consultation)