Microservice Architecture on AWS using AWS Lambda and Docker Containers
66 likes17,577 views
This document summarizes microservice architecture on AWS using AWS Lambda and Docker containers. It discusses why organizations adopt microservices and how to design smaller loosely coupled services. It then covers how to use AWS services like EC2 Container Service, Lambda, ECS, and others to build and deploy microservices. Specific topics covered include distributed systems, security, testing, monitoring, discovery, deployment pipelines and more. Examples of Netflix and Hailo architectures on AWS are also mentioned.
![Tasks are defined via Task Definitions
[
{
"image": "tutum/wordpress-stackable",
"name": "wordpress",
"cpu": 10,
"memory": 500,
"essential": true,
"links": [
"db"
],
"entryPoint": [
"/bin/sh",
"-c"
],
"environment": [
…
],
"portMappings": [
{
"containerPort": 80,
"hostPort": 80
}
]
},
{
"image": "mysql",
"name": "db",
"cpu": 10,
"memory": 500,
"essential": true,
"entryPoint": [
"/entrypoint.sh"
],
"environment": [
{
"name": "MYSQL_ROOT_PASSWORD",
"value": "pass"
}
],
"portMappings": []
}
]](https://image.slidesharecdn.com/01microservicesonawsshort-150330025225-conversion-gate01/85/Microservice-Architecture-on-AWS-using-AWS-Lambda-and-Docker-Containers-41-320.jpg)
![[
{
"image": "tutum/wordpress-stackable",
"name": "wordpress",
"cpu": 10,
"memory": 500,
"essential": true,
"links": [
"db"
],
"entryPoint": [
"/bin/sh",
"-c"
],
"environment": [
…
],
"portMappings": [
{
"containerPort": 80,
"hostPort": 80
}
]
},
]
{
"image": "mysql",
"name": "db",
"cpu": 10,
"memory": 500,
"essential": true,
"entryPoint": [
"/entrypoint.sh"
],
"environment": [
{
"name": "MYSQL_ROOT_PASSWORD",
"value": "pass"
}
],
"portMappings": []
}
]
From Docker Hub
10 CPU Units (1024 is full CPU),
500 Megabytes of Memory
Environment Variables
No external ports exposed
Tasks are defined via Task Definitions](https://image.slidesharecdn.com/01microservicesonawsshort-150330025225-conversion-gate01/85/Microservice-Architecture-on-AWS-using-AWS-Lambda-and-Docker-Containers-42-320.jpg)
![[
{
"image": "tutum/wordpress-stackable",
"name": "wordpress",
"cpu": 10,
"memory": 500,
"essential": true,
"links": [
"db"
],
"entryPoint": [
"/bin/sh",
"-c"
],
"environment": [
…
],
"portMappings": [
{
"containerPort": 80,
"hostPort": 80
}
]
},
[
{
"image": "mysql",
"name": "db",
"cpu": 10,
"memory": 500,
"essential": true,
"entryPoint": [
"/entrypoint.sh"
],
"environment": [
{
"name": "MYSQL_ROOT_PASSWORD",
"value": "pass"
}
],
"portMappings": []
}
]
Essential to our Task
Docker link to mysql container
Expose port 80 in container
to port 80 on host
Tasks are defined via Task Definitions](https://image.slidesharecdn.com/01microservicesonawsshort-150330025225-conversion-gate01/85/Microservice-Architecture-on-AWS-using-AWS-Lambda-and-Docker-Containers-43-320.jpg)
Microservice Architecture on AWS using AWS Lambda and Docker Containers
- 1. Microservice Architecture on AWS using AWS Lambda and Docker Containers Danilo Poccia ‒ AWS Technical Evangelist @danilop
- 4. As a Project scales Complexity arises “Complexity arises when the dependencies among the elements become important.” Complex Adaptive Systems: An Introduction to Computational Models of Social Life Scott E. Page, John H. Miller
- 5. How to design smaller services?
- 6. Business Domain + Loosely Coupled + Bounded Context
- 8. Microservices Choose the Right Tool
- 11. Adrian Cockcroft, Technology Fellow at Battery Ventures http://www.slideshare.net/adriancockcroft/goto-berlin
- 12. How small is small?
- 13. “something that could be rewritten in two weeks”
- 14. Two Pizza Teams
- 15. Conway’s law “organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations” Melvin Conway, 1968
- 17. Distributed Systems Independent Scalability Auto Scaling
- 18. Security Least Privileges Single Sign-On Confused Deputy Problem (for downstream calls after authentication)
- 19. Testing Automate Service Tests End-to-end Tests Synthetic Monitoring
- 20. Monitoring Expose Service Metrics Standard Log Format Correlation ID From Log collection to Real-Time Event Routing Amazon CloudWatch Logs Amazon Kinesis Understand your Trends
- 21. Discovery Self-Describing System Amazon Route 53 (DNS) AWS Resource Tagging
- 23. Developer Developer Developer Developer Developer Build Stage #1 Stage #2 … Stage #N Production Build Build Build Build
- 24. Developer Developer Developer Developer Developer Build Stage #1 Stage #2 … Stage #N Production Build Build Build Build Bug!
- 25. Developer Developer Developer Developer Developer Build Stage #1 Stage #2 … Stage #N Production Build Build Build Build Build Build Build Build Build Build Build Build Build Build
- 26. Developer Developer Developer Developer Developer Build Stage #1 Stage #2 … Stage #N Production Build Build Build Build Bug! Build Build Build Build Build Build Build Build Build Build
- 27. Design for Failure Degrade Functionality
- 28. "Jtecul" by own - Own work. Licensed under CC BY-SA 3.0 via Wikimedia Commons http://commons.wikimedia.org/wiki/File:Jtecul.jpg#/media/File:Jtecul.jpg Circuit Breakers
- 30. Amazon Route 53 Private DNS in VPC Health Checks Service Consumer(s) Consumer(s)
- 31. Amazon Route 53 Private DNS in VPC Elastic Load Balancing Health Checks Service Consumer(s) Consumer(s)
- 33. Why Docker?
- 35. Docker on AWS Amazon Linux A supported and maintained Linux image provided by Amazon Web Services Amazon EC2 Container Service Highly scalable, high performance container management service AWS Elastic Beanstalk For deploying and scaling web applications and services
- 36. Amazon EC2 Container Service Key Components Container Instances Clusters Tasks Task Definitions Amazon EC2 instances Docker daemon Amazon ECS agent
- 38. Amazon EC2 Container Service Key Components Container Instances Clusters Tasks Task Definitions Regional Resource pool Grouping of Container Instances Start empty, dynamically scalable
- 39. Amazon EC2 Container Service Key Components Container Instances Clusters Tasks Task Definitions Unit of work Grouping of related Containers Run on Container Instances
- 40. Amazon EC2 Container Service Key Components Container Instances Clusters Tasks Task Definitions [ { "image": "mysql", "name": "db", "cpu": 10, "memory": 500, …
- 41. Tasks are defined via Task Definitions [ { "image": "tutum/wordpress-stackable", "name": "wordpress", "cpu": 10, "memory": 500, "essential": true, "links": [ "db" ], "entryPoint": [ "/bin/sh", "-c" ], "environment": [ … ], "portMappings": [ { "containerPort": 80, "hostPort": 80 } ] }, { "image": "mysql", "name": "db", "cpu": 10, "memory": 500, "essential": true, "entryPoint": [ "/entrypoint.sh" ], "environment": [ { "name": "MYSQL_ROOT_PASSWORD", "value": "pass" } ], "portMappings": [] } ]
- 42. [ { "image": "tutum/wordpress-stackable", "name": "wordpress", "cpu": 10, "memory": 500, "essential": true, "links": [ "db" ], "entryPoint": [ "/bin/sh", "-c" ], "environment": [ … ], "portMappings": [ { "containerPort": 80, "hostPort": 80 } ] }, ] { "image": "mysql", "name": "db", "cpu": 10, "memory": 500, "essential": true, "entryPoint": [ "/entrypoint.sh" ], "environment": [ { "name": "MYSQL_ROOT_PASSWORD", "value": "pass" } ], "portMappings": [] } ] From Docker Hub 10 CPU Units (1024 is full CPU), 500 Megabytes of Memory Environment Variables No external ports exposed Tasks are defined via Task Definitions
- 43. [ { "image": "tutum/wordpress-stackable", "name": "wordpress", "cpu": 10, "memory": 500, "essential": true, "links": [ "db" ], "entryPoint": [ "/bin/sh", "-c" ], "environment": [ … ], "portMappings": [ { "containerPort": 80, "hostPort": 80 } ] }, [ { "image": "mysql", "name": "db", "cpu": 10, "memory": 500, "essential": true, "entryPoint": [ "/entrypoint.sh" ], "environment": [ { "name": "MYSQL_ROOT_PASSWORD", "value": "pass" } ], "portMappings": [] } ] Essential to our Task Docker link to mysql container Expose port 80 in container to port 80 on host Tasks are defined via Task Definitions
- 44. Why AWS Lambda?
- 45. AWS Lambda Event driven, fully managed compute
- 46. AWS Lambda Focus on business logic, not infrastructure Customer uploads code, AWS Lambda handles: Capacity Scaling Deployment Fault tolerance Monitoring Logging . . .
- 47. AWS Lambda Automatic scaling Customers pay only for what they use, no over/under provisioning
- 48. AWS Lambda Fine-grained pricing Price compute time by 100ms, even short jobs make sense Low request charge No hourly, daily, or monthly minimums Free tier
- 49. Events come in many different shapes & sizes S3 event notifications DynamoDB Streams Kinesis events Custom events
- 50. Amazon EC2 Containers AWS LambdaOn-Premises Weeks Minutes Seconds Milliseconds
- 51. Netflix Architecture Asgard, Amimator, Hystrix, Cassandra, JVM, Docker, … O n AW S
- 53. Hailo Architecture Hubot, Janky, Jenkins, Go, RabbitMQ, Cassandra, Docker, … O n AW S
- 55. Demo Architecture Content Management System Prototype O n AW S
- 56. Lambda Function S3 Bucket #1 S3 Bucket #2 DynamoDB Table ECS Task ECS Cluster Images + Medatada Upload Object + Metadata Event Notification Thumbnail Write Metadata Run Schedule Read Metadata Write “index.html” 1 2 3
- 57. <demo> … </demo>
- 59. @danilop