Modern Applications Web Day | Container Workloads on AWS
2 likes396 views
The document discusses Amazon Elastic Container Service (ECS) and AWS Fargate, highlighting their capabilities for managing container-based architectures on AWS, including features like cluster management, task scheduling, and scaling. It also introduces AWS Batch for running batch jobs using containers, explaining its architecture, job definitions, and the benefits of using managed vs. unmanaged compute environments. Additionally, it touches on integration with other AWS services and provides insights into container and batch processing strategies.
![© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
„Flip Image“ Job‘s Dockerfile
FROM amazonlinux:latest
RUN yum update -y
RUN yum install ImageMagick aws-cli -y
ADD flip.sh /usr/local/bin/flip.sh
WORKDIR /tmp
USER nobody
ENTRYPOINT ["/usr/local/bin/flip.sh"]](https://image.slidesharecdn.com/container-basedarchitecturesonaws-181030155311/85/Modern-Applications-Web-Day-Container-Workloads-on-AWS-46-320.jpg)
![© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Easily run massively parallel jobs
Efficient way to run:
• Parametric sweeps
• Monte Carlo simulations
• Processing a large collection of objects
Start up to 10,000 copies of an application with a single call using Array Jobs.
Get
File
List
flip.sh [1]
flip.sh [2]
flip.sh [n]](https://image.slidesharecdn.com/container-basedarchitecturesonaws-181030155311/85/Modern-Applications-Web-Day-Container-Workloads-on-AWS-51-320.jpg)
Modern Applications Web Day | Container Workloads on AWS
- 1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Container-based Architectures on AWS Sascha Möllering, Solutions Architect, @sascha242, Amazon Web Services EMEA SARL Steffen Grunwald, Solutions Architect, @steffeng, Amazon Web Services EMEA SARL
- 2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark We started in 2014
- 3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Customers running Docker on EC2 from the very beginning...
- 4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark But there were pain points. Things like scheduling, placing, managing and deploying containers were difficult. They wanted something to make those pain points better.
- 5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark So we built ECS Highly scalable, high performance container management system A managed platform Amazon Elastic Container Service Cluster management Container orchestration Deep AWS integration
- 6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark So we built ECS Highly scalable, high performance container management system AWS VPC networking mode Advanced task placement Deep integration with AWS platform ECS CLI…{ } Global footprint Powerful scheduling engines Auto scaling CloudWatch metrics Load balancers Amazon Elastic Container Service
- 7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Amazon ECS EC2 INSTANCES ECS AGENT TASK Containers TASK ECS AGENT TASK TASK AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT TASK TASK Internet LOAD BALANCER LOAD BALANCER Containers Containers
- 8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Cluster of hosts ECS AGENT TASK TASK ECS AGENT TASK TASK AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT TASK TASK Internet LOAD BALANCER LOAD BALANCER EC2 INSTANCES TASK TASK TASK TASK AGENT COMMUNICATION SERVICE API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE TASK TASK Internet Containers Containers Containers
- 9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Lightweight agent on each host EC2 INSTANCES ECS AGENT TASK TASK ECS AGENT TASK TASK AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT TASK TASK Internet LOAD BALANCER LOAD BALANCER TASK TASK TASK TASK AGENT COMMUNICATION SERVICE API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE TASK TASK Internet Containers Containers Containers
- 10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark API for launching containers on the cluster EC2 INSTANCES ECS AGENT TASK TASK ECS AGENT TASK TASK AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT TASK TASK Internet LOAD BALANCER LOAD BALANCER EC2 INSTANCES TASK TASK TASK TASK AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE TASK TASK Internet Containers Containers Containers
- 11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark EC2 INSTANCES ECS AGENT TASK TASK ECS AGENT TASK TASK AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT TASK TASK Internet LOAD BALANCER LOAD BALANCER Container task is placed on a host Containers Containers Containers
- 12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Traffic is sent to your host EC2 INSTANCES ECS AGENT TASK TASK ECS AGENT TASK TASK AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT TASK TASK Internet LOAD BALANCER LOAD BALANCER Containers Containers Containers
- 13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark ECS Optimized Amazon Machine Images (AMIs) • Optimized AMIs available for Linux & Windows • Bring your own images based on it • Expects ECS cluster name in user-data • Update images on SNS update notifications
- 14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Publish metrics Auto Scaling ECS service Availability Zone A Availability Zone B TASK A Add/Remove ECS tasks TASK C TASK BScaling Policies Amazon CloudWatch Amazon ECS Application Load Balancer Automatic Service Scaling
- 15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark ECS Cluster EC2 Instance EC2 Instance TASK A TASK B TASK B Amazon DynamoDB Amazon S3 IAM Roles For Tasks Amazon ECS
- 16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark •prod.app1.db-pass •general.license-code •prod.app2.user-name ECS Cluster EC2 Instance EC2 Instance TASK A TASK B TASK B System Manager – Parameter Store Secrets Management Amazon ECS
- 17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Amazon Elastic Container Registry (Amazon ECR) • Cloud-based Docker image registry • Fully managed • Secure – images encrypted at rest, integrated with IAM • Scalable and Highly Available • Integrated with Amazon ECS and the Docker CLI
- 18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS Fargate No cluster or infrastructure to manage or scale Everything is handled at the container level Scale seamlessly on demand Underlying technology for container management
- 19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark What does Fargate mean? Not worrying about scaling, underlying infrastructure, cluster resources, capacity, setup. Just give it a task definition or pod (in 2018), set some resource limits, and away you go.
- 20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Task Definitions Repository on GitHub
- 21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS CodePipeline AWS CodeBuild Developers 1 2 3 4 7 Continuous Deployment in Amazon ECS 5 6 AWS Fargate Amazon EC2 Amazon ECS Amazon ECR
- 22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark “Run Kubernetes for me.”
- 24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Amazon Elastic Container Service for Kubernetes: EKS Managed Kubernetes on AWS Highly available Automated version upgrades Integration with other AWS services Etcd Master Managed Kubernetes control plane CloudTrail, CloudWatch, ELB, IAM, VPC, PrivateLink
- 25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Elastic Container Service for Kubernetes • Platform for enterprises to run production grade workloads • Provides a native and upstream Kubernetes experience • Not forced to use additional AWS services, but offer seamless integration • EKS team actively contributes to the Kubernetes project
- 26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Elastic Container Service for Kubernetes
- 27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Availability Zone 1 Availability Zone 2 Availability Zone 3 Kubectl EKS Architecture
- 28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Cross-account Kubernetes Workers Masters Customer VPC EKS VPC Network Load Balancer ENI API Access Kubectl Exec/Logs TLS Static IPs
- 29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark EKS Cross-Account Networking: Availability Zones Availability Zone 1 Master Master Availability Zone 2 Availability Zone 3 Master Workers Workers Workers Customer VPC EKS VPC ENI ENI ENI
- 30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Jenkins – CI/CD with Kubernetes
- 31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS CodePipeline – CI/CD with Kubernetes AWS CodePipeline AWS CodeCommit AWS CodeBuild AWS Lambda Amazon ECR 1 2 4 3 5 6 Developer
- 32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
- 33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
- 34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
- 35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
- 36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Batch Processing with Containers
- 37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark What is batch computing? Run jobs asynchronously and automatically across one or more computers. Jobs may have dependencies, making the sequencing and scheduling of multiple jobs complex and challenging.
- 38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Cloud makes Sense for Batch • Scalable • Reliable • Choice: • Compute resources (GPUs, RAM- or CPU-bound) • Storage resources (fileshares, performance characteristics) • Downstream services (e.g. databases, streaming services) • Pricing models • Pay as you go (per second)
- 39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Anatomy of a Batch Event Changes in data state Requests to endpoints Services (anything) Scheduled triggers Compute Execution Your Code Auto Scaling Job Queue
- 40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Options for Batch Workloads on AWS AWS Lambda NEW: Amazon SQS as built-in event source AWS BatchAmazon Elastic Container Service (ECS)
- 41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Containers make Sense for Batch • Benefits from container development model • Polyglot • Do one thing well • Black Box – and easy to model
- 42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Basic Batch Workflow with ECS Input is put to S3 Bucket Output is put to S3 Bucket Event is put to Amazon SQS ECS provisions clusters and schedules tasks Containerized batch workers process files Queue depth is used for scaling Workers poll queue
- 43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Introducing AWS Batch • Fully managed batch primitives • Focus on your applications (shell scripts, Linux executables, Docker images) and their resource requirements • We take care of the rest!
- 44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Execution What you need to take care of... Auto Scaling Event Changes in data state Requests to endpoints Services (anything) Scheduled triggers Compute Your Code Job Queue
- 45. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Typical AWS Batch Job Architecture Input is put to S3 Bucket Output is put to S3 Bucket Job Queue with runnable jobs AWS Batch Compute Environment IAM role Job definition Application Image + config Scheduler
- 46. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark „Flip Image“ Job‘s Dockerfile FROM amazonlinux:latest RUN yum update -y RUN yum install ImageMagick aws-cli -y ADD flip.sh /usr/local/bin/flip.sh WORKDIR /tmp USER nobody ENTRYPOINT ["/usr/local/bin/flip.sh"]
- 47. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark flip.sh #!/bin/bash SRCBKT=$1 # SOURCE BUCKET OBJ=$2 # OBJECT KEY TRGBKT=$3 # TARGET BUCKET error_exit() { echo "${1}" >&2; exit 1; } tmpfile=$(mktemp /tmp/image.XXXXXX) aws s3 cp "s3://$SRCBKT/$OBJ" "$tmpfile" # DOWNLOAD || error_exit "Download failed $SRCBKT/$OBJ" convert -flip "$tmpfile" "$tmpfile" # FLIP w/ IMAGEMAGICK || error_exit "Failed to flip file" aws s3 cp "$tmpfile" "s3://$TRGBKT/$OBJ" # UPLOAD || error_exit "Upload failed $TRGBKT/$OBJ"
- 48. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Important AWS Batch Concepts • Job definitions • Jobs • Job queue • Compute environments
- 49. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Job Definitions AWS Batch job definitions specify how jobs are to be run. Some attributes in a job definition: • Container Image • IAM role associated with the job • vCPU and memory requirements • Mount points • Environment variables • Retry strategy
- 50. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Jobs are the unit of work executed by AWS Batch. Set/ overwrite Job Definition attributes, e.g.: • Command • Parameters • Dependencies Job command example: Jobs
- 51. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Easily run massively parallel jobs Efficient way to run: • Parametric sweeps • Monte Carlo simulations • Processing a large collection of objects Start up to 10,000 copies of an application with a single call using Array Jobs. Get File List flip.sh [1] flip.sh [2] flip.sh [n]
- 52. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS Batch supports up to 10 attempts per job: • errors in the AWS Batch job • termination of the Spot Instance The AWS_BATCH_JOB_ATTEMPT environment variable is set to the container's corresponding job attempt number. Job Retries
- 53. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Managed AWS scales and configures your instances for you. Optional choice: • On demand/ Spot • Instance Types/ Mix • Amazon Machine Image (AMI) Compute Environments Unmanaged You control and manage the instance configuration, provisioning, and scaling. Full control over scaling and instance provisioning for the ECS cluster used by AWS Batch.
- 54. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Bring your own AMIs Customer Provided AMIs let you set the AMI that is launched as part of a managed compute environment. Makes it possible to configure Docker settings, mount EBS/ EFS volumes, and configure drivers for GPU jobs. AMIs must be Linux-based, HVM and have a working ECS agent installation.
- 55. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Jobs are submitted to Job Queues. Job queues are assigned to one or more compute environments. Each job queue has a priority assigned. Jobs in queues with higher priority take precedence. Job Queues
- 56. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Use Spot Instances Get your jobs done faster or cheaper. AWS Batch retries jobs on instance terminations and selects from multiple instance types.
- 57. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark No additional charge for AWS Batch or Amazon ECS. You only pay for the underlying resources that you consume! Pricing
- 58. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Let’s build!
- 59. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Questions? Ask these guys!
- 60. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Thank you!