Best Practices of Infrastructure as Code with Terraform
5 likes4,742 views
The document discusses best practices for using Terraform in infrastructure-as-code to provision dynamic infrastructure efficiently. It outlines challenges such as the need for coding knowledge, change tracking, and collaboration, alongside solutions like utilizing workspaces, modules, and version control. Additionally, it emphasizes the importance of a structured approach to manage code and environments to avoid issues like 'ball of mud' configurations and upgrade disruptions.
![475 def update
476 return update_api if api_request?
477
478 if authorized_action(@account, @current_user, :manage_account_settings)
479 respond_to do |format|
480
481 custom_help_links = params[:account].delete :custom_help_links
482 if custom_help_links
483 @account.settings[:custom_help_links] = custom_help_links.select{|k, h| h['state'] != 'delete
484 hash = index_with_hash[1]
485 hash.delete('state')
486 hash.assert_valid_keys ["text", "subtext", "url", "available_to"]
487 hash
488 end
489 end
490
491 params[:account][:turnitin_host] = validated_turnitin_host(params[:account][:turnitin_host])
492 enable_user_notes = params[:account].delete :enable_user_notes
493 allow_sis_import = params[:account].delete :allow_sis_import
494 params[:account].delete :default_user_storage_quota_mb unless @account.root_account? && !@accou
495 unless @account.grants_right? @current_user, :manage_storage_quotas
496 [:storage_quota, :default_storage_quota, :default_storage_quota_mb,
497 :default_user_storage_quota, :default_user_storage_quota_mb,
498 :default_group_storage_quota, :default_group_storage_quota_mb].each { |key| params[:account]
499 end
500 if params[:account][:services]
501 params[:account][:services].slice(*Account.services_exposed_to_ui_hash(nil, @current_user, @a
502 @account.set_service_availability(key, value == '1')
503 end
504 params[:account].delete :services
505 end
506 if @account.grants_right?(@current_user, :manage_site_settings)
507 # If the setting is present (update is called from 2 different settings forms, one for notifi
508 if params[:account][:settings] && params[:account][:settings][:outgoing_email_default_name_op
509 # If set to default, remove the custom name so it doesn't get saved
510 params[:account][:settings][:outgoing_email_default_name] = '' if params[:account][:setting
511 end
512
513 google_docs_domain = params[:account][:settings].try(:delete, :google_docs_domain)
514 if @account.feature_enabled?(:google_docs_domain_restriction) &&
515 @account.root_account? &&
516 !@account.site_admin?
517 @account.settings[:google_docs_domain] = google_docs_domain.present? ? google_docs_domain :
518 end
519
520 @account.enable_user_notes = enable_user_notes if enable_user_notes
521 @account.allow_sis_import = allow_sis_import if allow_sis_import && @account.root_account?
522 if @account.site_admin? && params[:account][:settings]
523 # these shouldn't get set for the site admin account
524 params[:account][:settings].delete(:enable_alerts)
525 params[:account][:settings].delete(:enable_eportfolios)
526 end
527 else
528 # must have :manage_site_settings to update these
529 [ :admins_can_change_passwords,
530 :admins_can_view_notifications,
531 :enable_alerts,
532 :enable_eportfolios,
533 :enable_profiles,
534 :show_scheduler,
535 :global_includes,
536 :gmail_domain
537 ].each do |key|
538 params[:account][:settings].try(:delete, key)
5
Infrastructure-as-Code](https://image.slidesharecdn.com/20191213-devopscomwebinar-terraform-191213203010/85/Best-Practices-of-Infrastructure-as-Code-with-Terraform-5-320.jpg)
![Need to
learn to
code?
CODE EDITOR
resource "google_compute_instance" "default" {
name = "test"
machine_type = "n1-standard-1"
zone = "us-central1-a"
tags = ["foo", "bar"]
boot_disk {
initialize_params {
image = "debian-cloud/debian-9"
}
}
// omitted for clarity
}
13](https://image.slidesharecdn.com/20191213-devopscomwebinar-terraform-191213203010/85/Best-Practices-of-Infrastructure-as-Code-with-Terraform-13-320.jpg)
![▪ Decouple with
data sources
▪ Run separately
CODE EDITOR
data "aws_vpc" "selected" {
filter {
name = "owner"
values = [var.owner]
}
}
resource "aws_subnet" "example" {
vpc_id = data.aws_vpc.selected.id
availability_zone = "us-west-2a"
cidr_block =
cidrsubnet(data.aws_vpc.selected.cidr_block, 4,
1)
}
34
sysadvent.blogspot.com/2019/12/day-5-break-up-your-terraform-project.html](https://image.slidesharecdn.com/20191213-devopscomwebinar-terraform-191213203010/85/Best-Practices-of-Infrastructure-as-Code-with-Terraform-34-320.jpg)
Best Practices of Infrastructure as Code with Terraform
- 1. Copyright © 2019 HashiCorp Best Practices of Infrastructure-as- Code with Terraform DevOps.com | December 13, 2019 1
- 2. Presenter Rosemary Wang Developer Advocate at HashiCorp she/her @joatmon08 joatmon08 linkedin.com/in/rosemarywang/ 2
- 3. The shift to provisioning dynamic infrastructure ⁄ USING TERRAFORM IN DYNAMIC INFRASTRUCTURE Copyright © 2018 HashiCorp ⁄ 3 Static Homogeneous, Private Dynamic Heterogeneous, Distributed
- 4. ⁄ USING TERRAFORM IN DYNAMIC INFRASTRUCTURE Copyright © 2018 HashiCorp ⁄ 4 Dynamic Heterogeneous, Distributed Static Homogeneous, PrivateThe shift to provisioning dynamic infrastructure
- 5. 475 def update 476 return update_api if api_request? 477 478 if authorized_action(@account, @current_user, :manage_account_settings) 479 respond_to do |format| 480 481 custom_help_links = params[:account].delete :custom_help_links 482 if custom_help_links 483 @account.settings[:custom_help_links] = custom_help_links.select{|k, h| h['state'] != 'delete 484 hash = index_with_hash[1] 485 hash.delete('state') 486 hash.assert_valid_keys ["text", "subtext", "url", "available_to"] 487 hash 488 end 489 end 490 491 params[:account][:turnitin_host] = validated_turnitin_host(params[:account][:turnitin_host]) 492 enable_user_notes = params[:account].delete :enable_user_notes 493 allow_sis_import = params[:account].delete :allow_sis_import 494 params[:account].delete :default_user_storage_quota_mb unless @account.root_account? && !@accou 495 unless @account.grants_right? @current_user, :manage_storage_quotas 496 [:storage_quota, :default_storage_quota, :default_storage_quota_mb, 497 :default_user_storage_quota, :default_user_storage_quota_mb, 498 :default_group_storage_quota, :default_group_storage_quota_mb].each { |key| params[:account] 499 end 500 if params[:account][:services] 501 params[:account][:services].slice(*Account.services_exposed_to_ui_hash(nil, @current_user, @a 502 @account.set_service_availability(key, value == '1') 503 end 504 params[:account].delete :services 505 end 506 if @account.grants_right?(@current_user, :manage_site_settings) 507 # If the setting is present (update is called from 2 different settings forms, one for notifi 508 if params[:account][:settings] && params[:account][:settings][:outgoing_email_default_name_op 509 # If set to default, remove the custom name so it doesn't get saved 510 params[:account][:settings][:outgoing_email_default_name] = '' if params[:account][:setting 511 end 512 513 google_docs_domain = params[:account][:settings].try(:delete, :google_docs_domain) 514 if @account.feature_enabled?(:google_docs_domain_restriction) && 515 @account.root_account? && 516 [email protected]_admin? 517 @account.settings[:google_docs_domain] = google_docs_domain.present? ? google_docs_domain : 518 end 519 520 @account.enable_user_notes = enable_user_notes if enable_user_notes 521 @account.allow_sis_import = allow_sis_import if allow_sis_import && @account.root_account? 522 if @account.site_admin? && params[:account][:settings] 523 # these shouldn't get set for the site admin account 524 params[:account][:settings].delete(:enable_alerts) 525 params[:account][:settings].delete(:enable_eportfolios) 526 end 527 else 528 # must have :manage_site_settings to update these 529 [ :admins_can_change_passwords, 530 :admins_can_view_notifications, 531 :enable_alerts, 532 :enable_eportfolios, 533 :enable_profiles, 534 :show_scheduler, 535 :global_includes, 536 :gmail_domain 537 ].each do |key| 538 params[:account][:settings].try(:delete, key) 5 Infrastructure-as-Code
- 6. Agenda Infrastructure-as-Code Challenges Solving Challenges with Terraform Collaboration & Scaling 6
- 8. Goals ▪ Unify the view of resources ▪ Support the modern data center (IaaS, PaaS, SaaS) ▪ Expose a way for individuals and teams to safely and predictably change infrastructure ▪ Provide a workflow that is technology agnostic ▪ Manage anything with an API 8
- 9. Initial Challenges ▪ Need to learn to code ▪ Can’t automate a resource ▪ Can’t track changes ▪ Don’t know change impact ▪ Need to revert a change 9
- 10. Scaling Challenges ▪ Multiple environments for infrastructure ▪ Duplicate code ▪ “Ball of Mud” configuration ▪ Too many working on code ▪ Dry run doesn’t reflect change impact ▪ Upgrades are disruptive 10
- 12. Initial Challenges ▪ Need to learn to code ▪ Can’t automate a resource ▪ Can’t track changes ▪ Don’t know change impact ▪ Need to revert a change 12
- 13. Need to learn to code? CODE EDITOR resource "google_compute_instance" "default" { name = "test" machine_type = "n1-standard-1" zone = "us-central1-a" tags = ["foo", "bar"] boot_disk { initialize_params { image = "debian-cloud/debian-9" } } // omitted for clarity } 13
- 14. Need to learn to code? ▪ HashiCorp Configuration Language ▪ Language describes intent ▪ Declarative (I declare, therefore I am.) ▪ Handles logic of calling APIs in proper order 14 terraform.io/docs/configuration/syntax.html
- 17. ▪ Many providers community- maintained ▪ Write your own with the Terraform Plugin SDK! CODE EDITOR # Create a new Datadog monitor resource "datadog_monitor" "foo" { name = "Name for monitor foo" type = "metric alert" message = "Monitor triggered." // omitted for clarity thresholds = { ok = 0 warning = 2 warning_recovery = 1 critical = 4 critical_recovery = 3 } // omitted for clarity } 17 hashicorp.com/resources/everything-as-code-with-terraform
- 19. Can't track changes? ▪ Track state of existing infrastructure resources ▪ State updates when changes applied IMPORTANT NOTE ▪ Non-Terraform resources not automatically added ▪ Configuration not automatically generated ▪ Manual changes get overwritten 19 terraform.io/docs/state/index.html
- 20. Don't know change impact? TERMINAL > terraform plan Terraform will perform the following actions: # aws_vpc.app_vpc will be created + resource "aws_vpc" "app_vpc" { + arn = (known after apply) + cidr_block = “10.128.0.0/25" // omitted for clarity } Plan: 1 to add, 0 to change, 0 to destroy. 20
- 22. TERMINAL + resource will be created - resource will be destroyed ~ resource will be updated in-place -/+ resources will be destroyed and re-created 22
- 23. Need to revert a change? CODE EDITOR terraform { backend "remote" { organization = “<tf cloud org>" workspaces { name = “<tf cloud workspace>” } } } 23
- 24. Need to revert a change? ▪ Version control working configuration ▪ Remote state and if possible, versioned ▪ Update to previous working version ▪ Add toggle for easier revert IMPORTANT NOTE ▪ More like “roll forward” 24 terraform.io/docs/backends/index.html
- 26. Scaling Challenges ▪ Multiple environments for infrastructure ▪ Duplicate code ▪ “Ball of Mud” configuration ▪ Too many working on code ▪ Dry run doesn’t reflect change impact ▪ Upgrades are disruptive 26
- 27. Multiple environ- ments? TERMINAL > terraform workspace list default dev * prod > tree terraform.tfstate.d terraform.tfstate.d ├── dev └── prod 27
- 28. Workspaces ▪ Each workspace isolates state ▪ Map environment to workspace prevents state contamination IMPORTANT NOTE ▪ More functionality for Terraform Cloud ▪ Manages state, access control, runs, etc. 28 terraform.io/docs/state/workspaces.html
- 29. TERMINAL > cd dev > terraform workspace dev > terraform init > terraform plan > terraform apply 29
- 30. Duplicate code? TERMINAL hello_world ├── dev │ ├── network.tf │ ├── kubernetes.tf │ ├── app.tf │ └── database.tf └── prod ├── network.tf ├── kubernetes.tf ├── app.tf └── database.tf 30 Evolving Your Infrastructure with Terraform (Nicki Watts)
- 31. ▪ Use modules ▪ Divide resource types into different files ▪ Other sources – Version Control (submodules) – Module registry TERMINAL hello_world ├── base // can be separately maintained │ ├── network │ │ ├── subnets.tf │ │ └── vpc.tf │ ├── kubernetes │ │ └── cluster.tf │ ├── database │ │ └── database.tf │ └── app │ └── app.tf ├── dev │ └── main.tf └── prod └── main.tf 31
- 32. When building modules… ▪ Set provider version in consumer ▪ Version with tagging CODE EDITOR provider "aws" { region = var.region version = "~> 2.41" } module "elb" { source = "terraform-aws-modules/elb/aws" version = "2.3.0" health_check = var.health_check listener = var.listener // omitted for clarity } output "dns" { value = module.elb.this_elb_dns_name } 32 terraform.io/docs/configuration/modules.html
- 33. “Ball of Mud” Config? TERMINAL > terraform plan Terraform will perform the following actions: // omitted for clarity Plan: 300 to add, 0 to change, 0 to destroy. 33
- 34. ▪ Decouple with data sources ▪ Run separately CODE EDITOR data "aws_vpc" "selected" { filter { name = "owner" values = [var.owner] } } resource "aws_subnet" "example" { vpc_id = data.aws_vpc.selected.id availability_zone = "us-west-2a" cidr_block = cidrsubnet(data.aws_vpc.selected.cidr_block, 4, 1) } 34 sysadvent.blogspot.com/2019/12/day-5-break-up-your-terraform-project.html
- 37. Establish Collaboration Patterns ▪ Adopt a software development pattern ▪ Put it in a CI pipeline ▪ Apply and audit changes based on code push ▪ Lock state during changes to prevent overrides 37 terraform.io/docs/state/locking.html
- 38. Dry run doesn’t reflect change impact? TERMINAL > kitchen test -----> Starting Kitchen (v2.3.3) … Waiting for SSH service on 54.93.35.169:22, retrying in 3 seconds Waiting for SSH service on 54.93.35.169:22, retrying in 3 seconds Waiting for SSH service on 54.93.35.169:22, retrying in 3 seconds Waiting for SSH service on 54.93.35.169:22, retrying in 3 seconds Waiting for SSH service on 54.93.35.169:22, retrying in 3 seconds 38
- 39. Integration Tests Contract Tests Unit Tests Infrastructure Testing Manual Testing Cost (Time, $$$) End-to-End Tests hashicorp.com/resources/test-driven-development-tdd-for-infrastructure
- 40. 40
- 41. Upgrades are disruptive? TERMINAL > terraform-0.7.13 apply Terraform doesn't allow running any operations against a state that was written by a future Terraform version. The state is reporting it is written by Terraform '0.8.8'. Please run at least that version of Terraform to continue 41
- 42. 42 0.8 0.9 0.10 0.11 0.12 CHANGELOG Upgrade Guide Template files & string interpolation changes AWS provider attribute deprecations CHANGELOG Upgrade Guide Migrating to Backends Deprecate Remote for Backend Configuration State Locking AWS provider changes may trigger recreation Providers separated as plugins from core repository & versioned Interactive approval for apply (breaks pipelines, add -auto- approve flag) CHANGELOG Upgrade Guide Changes to module inheritance of providers Always use splat (*) operator for count references CHANGELOG Upgrade Guide CHANGELOG Upgrade Guide Adds rich type system to a previously string-typed system Includes automated upgrade tool (with caveats) AWS Provider CHANGELOG AWS v2 Upgrade Guide speakerdeck.com/joatmon08/the-semi-ultimate-terraform-upgrade-guide
- 43. Ease Upgrade Path by… ▪ Pinning provider versions ▪ Using known functions and not creative hacks ▪ Decoupling configuration across providers (i.e., separate Kubernetes from GCP) ▪ Avoid provisioners or complicated lifecycle customizations 43 hashicorp.com/resources/closing-keynote-terraform-at-google
- 44. Resources ▪ Terraform Cloud | app.terraform.io/signup/account ▪ Learn Terraform | learn.hashicorp.com/terraform ▪ Community Forum | discuss.hashicorp.com 44
- 45. Rosemary Wang Developer Advocate at HashiCorp she/her @joatmon08 joatmon08 linkedin.com/in/rosemarywang/ 45 joatmon08.github.io