Containerizing MongoDB with kubernetes
1 like1,565 views
The document discusses the use of Kubernetes for containerizing MongoDB, emphasizing the journey of FuboTV in scaling their services. It covers the fundamentals of Kubernetes, high availability in MongoDB, challenges of managing stateful services, and solutions such as persistent volumes and service endpoints. The presentation acknowledges the strengths of both Kubernetes and MongoDB while highlighting the importance of testing and adapting workflows for successful implementation.
Containerizing MongoDB with kubernetes
- 1. Containerizing MongoDB with Kubernetes Dan Worth (fuboTV) Brian McNamara (CloudyOps)
- 2. Who are we? Dan Worth ● Software Engineer ● Currently at fuboTV ● @djworth on the Internet
- 3. Who are we? Brian McNamara ● Ops Engineer ● @mcnamarabrian / @cloudyops ● Enjoys learning new things and quoting corny 80s comedies whenever possible
- 4. What are we going to talk about? ● In the beginning of fuboTV ● Fundamentals of Kubernetes (Dan) ● Fundamentals of MongoDB high availability (Brian) ● Challenges of running stateful services on Kubernetes (Brian)
- 5. Let’s Start with a Story...
- 6. fubo.tv Business Overview ● Sports first virtual multichannel video programming distributor (MVPD) ● Partnership with 21st Century Fox and Sky ● Create the best live events platform ● Subscription based ● Build communities around sports (teams, matches, players)
- 7. fubo.tv Started Taking Over Streaming World ● Needed scalable offering ● Bursty demand patterns around soccer / futbol matches ● Lots of in-house development experience ● Not a lot of in-house operational experience ● Didn’t want to maintain hardware
- 8. fubo.tv v1 ● Node.js hosted with well-known PaaS provider ● MongoDB hosted with a separate provider ● Focus was on introducing application features, iterating quickly
- 9. Things Changed in Philly
- 10. Enter Kubernetes ● Presentation by Kelsey Hightower ● Sysadmin who can code ● 2015 PhillyETE: Managing Containers at Scale with CoreOS and Kubernetes ● Container cluster manager ● Don’t sweat the scheduling of containers in your cluster ● Live demo of rolling application updates
- 11. Minds = Blown
- 12. How Can We Do That? ● Google’s infrastructure for everyone else ● Loved demo ● Saw the possibilities of Kubernetes ● But….still didn’t want to maintain hardware
- 13. Enter Google Container Engine ● No need to run a Kubernetes cluster in-house ● Google provided service ● Fully managed
- 14. Fundamentals of Kubernetes ● Open Source container cluster manager by Google ● Run Anywhere (GKE, GCE, AWS, Bare metal) ● Self-healing when using the right primitives ● Service discovery and load balancing ● Secret and configuration management ● Key high level domain objects ○ Pods ○ Replication Controllers ○ Services
- 15. Fundamentals of Kubernetes (Pods) ● Pods ○ Unit of Scheduling ○ One or more containers ○ Define environment ○ Pods get their own IP addresses spec: containers: - name: mongo image: mongo:3.2 ports: - containerPort: 27017 resources: limits: cpu: 4
- 16. Fundamentals of Kubernetes (RC) ● Replication Controller ● All the goodness of the Pod ● Additional benefit of defining count of pods
- 17. Replication Controller Example apiVersion: v1 kind: ReplicationController metadata: name: mongo1 spec: replicas: 1 selector: name: mongo1 template: metadata: labels: name: mongo1 spec: containers: - name: mongo1 image: mongo:3.2 volumeMounts: - name: mongo1-data mountPath: /data/db resources: Limits: cpu: 4 memory: 4Gi ports: - name: "mongo" containerPort: 27017 protocol: TCP command: - ...
- 18. Fundamentals of Kubernetes (Services) ● Provides stable endpoint to pods / replication controllers ● Uses metadata like ports and selectors to identify how to map endpoint to pod
- 19. Service Example apiVersion: v1 kind: Service metadata: name: mongo1-service labels: name: mongo1-service spec: ports: - port: 27017 targetPort: 27017 protocol: TCP selector: name: mongo1 type: LoadBalancer
- 20. Fundamentals of Kubernetes Service Replication Controller Pod Pod Pod
- 21. Fundamentals of MongoDB High Availability ● Possible to scale reads and writes ○ Scaling reads: use replica sets ○ Scaling writes: use shards ● Clients can do things to take advantage of availability primitives ● We’ll focus on scaling reads using replica sets
- 22. Fundamentals of MongoDB High Availability (cont) MongoDB Replica Set
- 23. Fundamentals of MongoDB High Availability (cont) Heartbeat among replica set members
- 24. Fundamentals of MongoDB High Availability (cont) Automated election of Primary in the event of failure
- 25. Fundamentals of MongoDB High Availability (cont) > rs.config() { "_id" : "replica_set_name", "version" : 105978, "protocolVersion" : NumberLong(1), "members" : [ { "_id" : 0, "host" : "ip_or_hostname:port_number", "arbiterOnly" : false, "buildIndexes" : true, "hidden" : false, "priority" : 1, "tags" : {}, "slaveDelay" : NumberLong(0), "votes" : 1 }, { … } }
- 26. Challenges of Running Stateful Services on Kubernetes ● Kubernetes is amazing at running and rescheduling containers with Pods, Replication Controllers, and Services ● Stateless services are easiest to manage but... ● Sometimes we need things to maintain state
- 27. Challenges of Running Stateful Services MongoDB on Kubernetes ● Kubernetes Replication Controller ○ Ensure requisite number of Pods are scheduled ○ Don’t guarantee consistent hostname or IP address ● MongoDB replica set configuration uses well defined endpoints ○ Remember that rs.conf() output? ○ Updating replica set configuration by hand feels dirty and you’re a bad person if you want to do that. ● MongoDB data should persist ○ If not, when a new replica set member comes up there will be a full sync ○ Kubernetes manages the scheduling but who needs the full sync?
- 29. Built-in Primitives to the Rescue ● Kubernetes ○ Replication Controller + Service ○ Label selector allows for consistent association between pod and service hostname / IP ○ Persistent volume can be defined ■ Allows GCE volumes to move with Pod ■ Result: No need to do expensive resync of data ● MongoDB ○ Take advantage of service endpoint when defining replica set ■ Service IP or DNS
- 30. Kubernetes - Code Samples Replication Controller Code Service Code
- 31. Parting Thoughts ● MongoDB has good resilience in the face of failure but be sure to test different failure scenarios. ● Docker is great to work with, but make sure your development workflows, tools and harnesses are adapted to build and run apps with it. ● Kubernetes is still relatively young, but maturing quickly. You need to carefully evaluate whether you want to roll your own platform with it, or instead rely on a hosted service like Google Container Engine
- 32. Questions? … and thanks! Dan Worth (@djworth) Brian McNamara (@mcnamarabrian / @cloudyops)
- 33. References Google Container Engine (GKE) Kubernetes Docs MongoDB Read Preference MongoDB Replication Introduction MongoDB Replica Set Tag Sets MongoDB Leaf in the Wild Post CloudyOps Deploying a 3-node MongoDB Replica Set with Kubernetes 2015 PhillyETE: Managing Containers at Scale with CoreOS and Kubernetes