Containerised ASP.NET Core apps with Kubernetes
- 1. Containerised ASP.NET Core apps with Kubernetes Mete Atamel Developer Advocate for Google Cloud ` @meteatamel
- 2. Confidential & ProprietaryGoogle Cloud Platform 2 Mete Atamel Developer Advocate for Google Cloud @meteatamel [email protected] meteatamel.wordpress.com Please send talk feedback: bit.ly/atamel
- 3. Google Cloud Platform Agenda The .NET Revolution Convergence of the two worlds with .NET Core Containers What are containers? How do they help? Kubernetes What is Kubernetes? How does it help with management of containers? Kubernetes building blocks Deployments, pods, labels, selectors, services, replica sets and more @meteatamel
- 4. Google Cloud Platform The .NET Revolution @meteatamel
- 5. Google Cloud Platform The world of 2014 ASP.NET C# Visual Studio SQL Server PowerShell Java Apache MySQL Eclipse Bash @meteatamel
- 6. Google Cloud Platform Things are changing @meteatamel ASP.NET Core on Linux, Mac, Windows Microsoft joins Linux foundation Google joins .NET foundation 2014 .NET Goes Open Source OpenSSH on Windows Ubuntu, Debian on Azure 2015 SQL Server on Linux Bash on Windows PowerShell on Linux 2016
- 7. Google Cloud Platform The convergence ASP.NET C# Visual Studio SQL Server Java ApacheMySQL PowerShell Eclipse Bash
- 8. Confidential & ProprietaryGoogle Cloud Platform 8 Great time to be a .NET developer!
- 9. Google Cloud Platform Demo: Simple ASP.NET Core service @meteatamel
- 11. Google Cloud Platform What is a container? Lightweight Hermetically sealed Isolated Easily deployable Introspectable Composable Linux (or Windows) processes A lightweight way to virtualize applications @meteatamel Docker
- 12. Google Cloud Platform ✕ No isolation ✕ Common libs ✕ Highly coupled Apps & OS Why containers? app libs kernel libs app app kernel app libs libs kernel kernel libs app kernel libs app libs app libs app ✓ Isolation ✓ No Common Libs ✕ Expensive and Inefficient ✕ Hard to manage ✓ Isolation ✓ No Common Libs ✓ Less overhead ✕ Less Dependency on Host OS kernel libs app app app app Physical Machine Virtual Machines Containers @meteatamel
- 13. Google Cloud Platform Google has been developing and using containers to manage our applications for over 12 years. Images by Connie Zhou @meteatamel
- 14. Google Cloud Platform Everything at Google runs in containers Gmail, Web Search, Maps, ... MapReduce, batch, ... GFS, Colossus, ... Even Google’s Cloud Platform: our VMs run in containers! We launch over 2 billion containers per week @meteatamel
- 15. Google Cloud Platform Demo: Containerised Microservice @meteatamel
- 16. Google Cloud Platform Containers not enough @meteatamel Containers help to create a lightweight and consistent environment for apps ● Resiliency ● Scaling up and down ● Deploying a new version of your app reliably ● Rolling back a version ● Health checks ● Graceful shutdown ● Etc. But you still need to manage your app in production
- 18. Google Cloud Platform Greek for “Helmsman”; also the root of the words “governor” and “cybernetic” • Manages container clusters • Inspired and informed by Google’s experiences and internal systems (borg) • Supports multiple cloud and bare-metal environments • Supports multiple container runtimes • 100% Open source, written in Go Manage applications, not machines Kubernetes @meteatamel
- 19. Google Cloud Platform kubelet UI kubeletCLI API users master nodes etcd kubelet scheduler controllers apiserver The 10000 foot view @meteatamel
- 20. Google Cloud Platform UI API Container Cluster All you really care about @meteatamel
- 21. Google Cloud Platform 1. Setting up the cluster • Choose a cloud: GCE, AWS, Azure, Rackspace, on-premises, ... • Choose a node OS: CoreOS, Atomic, RHEL, Debian, CentOS, Ubuntu, ... • Provision machines: Boot VMs, install and run kube components, ... • Configure networking: IP ranges for Pods, Services, SDN, ... • Start cluster services: DNS, logging, monitoring, ... • Manage nodes: kernel upgrades, OS updates, hardware failures... Not the easy or fun part, but unavoidable This is where things like Google Container Engine (GKE) really help Container clusters: A story in two parts @meteatamel
- 22. Google Cloud Platform Kubernetes cluster on GKE @meteatamel
- 23. Google Cloud Platform Demo: Create Kubernetes cluster @meteatamel
- 24. Google Cloud Platform 2. Using the cluster • Run Pods & Containers • Replica Sets • Services • Volumes This is the fun part! A distinct set of problems from cluster setup and management Don’t make developers deal with cluster administration! Accelerate development by focusing on the applications, not the cluster Container clusters: A story in two parts @meteatamel
- 25. Google Cloud Platform Kubernetes Building Blocks @meteatamel
- 26. Container cluster Service Pods Each pod containers one or more containers Nodes Role: frontend Role: frontend Role: frontend Role: frontend Replication controller Replicas: 3 Env: prod microservice labels Service communication channel Blueprint “pod template” Env: prod Env: prod Env: prod registry containers @meteatamel
- 28. Google Cloud Platform A Deployment provides declarative updates for Pods and Replica Sets Describe the desired state and the Deployment controller will change the actual state to the desired state at a controlled rate for you. Deployment manages replica changes for you • stable object name • updates are configurable, done server-side • kubectl edit or kubectl apply ... Deployments @meteatamel
- 29. Google Cloud Platform Demo: Create Deployment @meteatamel
- 30. Google Cloud Platform Pods and Volumes @meteatamel
- 31. Google Cloud Platform Small group of containers & volumes Tightly coupled The atom of scheduling & placement Shared namespace • share IP address & localhost • share IPC, etc. Managed lifecycle • bound to a node, restart in place • can die, cannot be reborn with same ID Example: data puller & web server Consumers Content Manager File Puller Web Server Volume Pod Pods @meteatamel
- 32. Google Cloud Platform Pod-scoped storage Support many types of volume plugins • Empty dir (and tmpfs) • Host path • Git repository • GCE Persistent Disk • AWS Elastic Block Store • Azure File Storage • iSCSI • Flocker • NFS • vSphere • GlusterFS • Ceph File and RBD • Cinder • FibreChannel • Secret, ConfigMap, DownwardAPI • Flex (exec a binary) • ... Volumes @meteatamel
- 33. Google Cloud Platform Labels & Selectors @meteatamel
- 34. Google Cloud Platform Arbitrary metadata Attached to any API object Generally represent identity Queryable by selectors • think SQL ‘select ... where ...’ The only grouping mechanism • pods under a ReplicationController • pods in a Service • capabilities of a node (constraints) Labels @meteatamel
- 35. Google Cloud Platform App: MyApp Phase: prod Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE Selectors @meteatamel
- 36. Google Cloud Platform App: MyApp Phase: prod Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE App = MyApp Selectors @meteatamel
- 37. Google Cloud Platform App: MyApp Phase: prod Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE App = MyApp, Role = FE Selectors @meteatamel
- 38. Google Cloud Platform App: MyApp Phase: prod Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE App = MyApp, Role = BE Selectors @meteatamel
- 39. Google Cloud Platform App: MyApp Phase: prod Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE App = MyApp, Phase = prod Selectors @meteatamel
- 40. Google Cloud Platform App: MyApp Phase: prod Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE App = MyApp, Phase = test Selectors @meteatamel
- 42. Google Cloud Platform A simple control loop Runs out-of-process wrt API server One job: ensure N copies of a pod • grouped by a selector • too few? start some • too many? kill some Layered on top of the public Pod API Replicated pods are fungible • No implied order or identity * The evolution of ReplicationControllers ReplicaSet - name = “my-rc” - selector = {“App”: “MyApp”} - template = { ... } - replicas = 4 API Server How many? 3 Start 1 more OK How many? 4 ReplicaSets* @meteatamel
- 43. Google Cloud Platform ReplicaSets Replication Controller Pod frontend Pod frontend app = demo app = demo app = demo ReplicaSet #pods = 3 app = demo color in (blue,grey) show: version = v2 color = blue color = blue color = grey Behavior Benefits ● Keeps Pods running ● Gives direct control of Pod #s ● Grouped by Label Selector ➔ Recreates Pods, maintains desired state ➔ Fine-grained control for scaling ➔ Standard grouping semantics Pod Pod Pod @meteatamel
- 44. Google Cloud Platform Demo: Replication @meteatamel
- 46. Google Cloud Platform Services Client Pod Container Pod Container Pod Container A logical grouping of pods that perform the same function (the Service’s endpoints) • grouped by label selector Load balances incoming requests across constituent pods Choice of pod is random but supports session affinity (ClientIP) Gets a stable virtual IP and port • also a DNS nametype = Service Label selector: type = FE VIP type = FE type = FE type = FE @meteatamel
- 47. Google Cloud Platform Demo: Services @meteatamel
- 48. Google Cloud Platform A general purpose, web-based UI to view/manage Kubernetes clusters Kubernetes Dashboard
- 49. Google Cloud Platform Demo: Kubernetes Dashboard
- 51. Google Cloud Platform Scaling @meteatamel Service Label selectors: version = 1.0 type = Frontend Service name = frontend Label selector: type = BE Replication Controller Pod frontend Pod version= v1 version = v1 ReplicaSet version = v1 #pods = 1 show: version = v2 type = FE type = FE Pod frontend Pod version = v1 type = FE ReplicaSet version = v1 #pods = 2 show: version = v2 Pod ReplicaSet version = v1 type = FE #pods = 3 show: version = v2
- 52. Google Cloud Platform Scaling @meteatamel Service Label selectors: version = 1.0 type = Frontend Service name = frontend Label selector: type = BE Replication Controller Pod frontend Pod version= v1 version = v1 ReplicaSet version = v1 #pods = 1 show: version = v2 type = FE type = FE Pod frontend Pod version = v1 type = FE ReplicaSet version = v1 #pods = 2 show: version = v2 Pod Pod ReplicaSet version = v1 type = FE #pods = 4 show: version = v2 version = v1 type = FE
- 53. Google Cloud Platform Demo: Scaling @meteatamel
- 54. Google Cloud Platform Rolling Update @meteatamel
- 55. Google Cloud Platform ReplicaSet - name: my-app-v1 - replicas: 3 - selector: - app: MyApp - version: v1 Service - app: MyApp Rolling Update @meteatamel
- 56. Google Cloud Platform ReplicaSet - name: my-app-v1 - replicas: 3 - selector: - app: MyApp - version: v1 ReplicaSet - name: my-app-v2 - replicas: 0 - selector: - app: MyApp - version: v2 Service - app: MyApp Rolling Update @meteatamel
- 57. Google Cloud Platform ReplicaSet - name: my-app-v1 - replicas: 3 - selector: - app: MyApp - version: v1 ReplicaSet - name: my-app-v2 - replicas: 1 - selector: - app: MyApp - version: v2 Service - app: MyApp Rolling Update @meteatamel
- 58. Google Cloud Platform ReplicaSet - name: my-app-v1 - replicas: 2 - selector: - app: MyApp - version: v1 ReplicaSet - name: my-app-v2 - replicas: 1 - selector: - app: MyApp - version: v2 Service - app: MyApp Rolling Update @meteatamel
- 59. Google Cloud Platform ReplicaSet - name: my-app-v1 - replicas: 2 - selector: - app: MyApp - version: v1 ReplicaSet - name: my-app-v2 - replicas: 2 - selector: - app: MyApp - version: v2 Service - app: MyApp Rolling Update @meteatamel
- 60. Google Cloud Platform ReplicaSet - name: my-app-v1 - replicas: 1 - selector: - app: MyApp - version: v1 ReplicaSet - name: my-app-v2 - replicas: 2 - selector: - app: MyApp - version: v2 Service - app: MyApp Rolling Update @meteatamel
- 61. Google Cloud Platform ReplicaSet - name: my-app-v1 - replicas: 1 - selector: - app: MyApp - version: v1 ReplicaSet - name: my-app-v2 - replicas: 3 - selector: - app: MyApp - version: v2 Service - app: MyApp Rolling Update @meteatamel
- 62. Google Cloud Platform ReplicaSet - name: my-app-v1 - replicas: 0 - selector: - app: MyApp - version: v1 ReplicaSet - name: my-app-v2 - replicas: 3 - selector: - app: MyApp - version: v2 Service - app: MyApp Rolling Update @meteatamel
- 63. Google Cloud Platform ReplicaSet - name: my-app-v2 - replicas: 3 - selector: - app: MyApp - version: v2 Service - app: MyApp Rolling Update @meteatamel
- 64. Google Cloud Platform Demo: Rolling Update @meteatamel
- 65. Google Cloud Platform Canary Deployments Replication Controller ReplicaSet version = v2 type = BE #pods = 1 show: version = v2 Pod frontend Pod version = v2 type = BE @meteatamel Pod frontend Service Label selectors: version = 1.0 type = Frontend Service name = backend Label selector: type = BE Replication Controller Pod version= v1 ReplicaSet version = v1 type = BE #pods = 2 show: version = v2 type = BE type = BE Pod version = v1
- 66. Google Cloud Platform Autoscaling Replication Controller Pod frontend Pod name=locust name=locust ReplicaSet name=locust role=worker #pods = 1 show: version = v2 Pod frontend Pod name=locust ReplicaSet name=locust role=worker #pods = 2 show: version = v2 Pod Pod name=locust role=worker role=worker role=worker role=worker ReplicaSet name=locust role=worker #pods = 4 Heapster 70% CPU 40% CPU Scale CPU Target% = 50 > 50% CPU @meteatamel
- 68. Google Cloud Platform Problem: how to run a Pod on every node? • or a subset of nodes Similar to ReplicaSet • principle: do one thing, don’t overload “Which nodes?” is a selector Use familiar tools and patterns Pod DaemonSets @meteatamel
- 70. Google Cloud Platform Run-to-completion, as opposed to run-forever • Express parallelism vs. required completions • Workflow: restart on failure • Build/test: don’t restart on failure Aggregates success/failure counts Built for batch and big-data work ... Jobs @meteatamel
- 72. Google Cloud Platform Goal: enable clustered software on Kubernetes • mysql, redis, zookeeper, ... Clustered apps need “identity” and sequencing guarantees • stable hostname, available in DNS • an ordinal index • stable storage: linked to the ordinal & hostname • discovery of peers for quorum • startup/teardown ordering StatefulSets @meteatamel
- 74. Google Cloud Platform Goal: manage app configuration • ...without making overly-brittle container images 12-factor says config comes from the environment • Kubernetes is the environment Manage config via the Kubernetes API Inject config as a virtual volume into your Pods • late-binding, live-updated (atomic) • also available as env vars node API Pod Config Map ConfigMaps @meteatamel
- 76. Google Cloud Platform Goal: grant a pod access to a secured something • don’t put secrets in the container image! 12-factor says config comes from the environment • Kubernetes is the environment Manage secrets via the Kubernetes API Inject secrets as virtual volumes into your Pods • late-binding, tmpfs - never touches disk • also available as env vars node API Pod Secret Secrets @meteatamel
- 77. Confidential & ProprietaryGoogle Cloud Platform 77 There is more! @meteatamel
- 78. Confidential & ProprietaryGoogle Cloud Platform 78 kubernetes.io cloud.google.com/container-engine Mete Atamel @meteatamel [email protected] meteatamel.wordpress.com Thank You @meteatamel Send talk feedback bit.ly/atamel