Ofir Makmal - Intro To Kubernetes Operators - Google Cloud Summit 2018 Tel Aviv
- 1. Kubernetes Operators Introduction to Ofir Makmal | CTO | SELA Group
- 2. Ku·ber·ne·tes Kubernetes can be many things… Its a scalable, self-healing and resilient ● Container orchestration engine ● Cluster management system ● Mesos, Swarm, Rancher, _______ replacement ● Microservices, CI/CD, Machine Learning platforms ● Portable ‘cloud’ But, It’s NOT just another way to run containers...
- 3. In just three years time… Recent surveys from medium & large US-based companies shows that over 60% are currently using Kubernetes to run workloads ● Trials/POC stages through development to production stages It has a huge community: ● ~36K Stars ● ~1600 Contributors! ● ~13K Forks ● ~1000 Pending pull requests!
- 5. We all know Kubernetes Building Blocks
- 6. And we all know how to run a stateless service... Creating a Namespace, defining a Deployment to manage a ReplicaSet of Pods, exposing them as Services and Ingress, maybe mounting Persistent Volumes into their containers, injecting ConfigMaps and Secrets as environment variables. After deployment, we expect that Kubernetes will take care the rest.
- 7. What about stateful applications?
- 8. Stateless is easy, stateful is hard i.e - Cassandra, a Distributed Database. After deployment, comes operations. ● Backups and Restore ● Upgrades ● Auto-scaling ● Redistribution of data when cluster size changes ● Leader election in case of failure and preserving a Quorum These kind of stateful applications are usually used as a dependency for other stateless applications, and needed to be discovered and consumed.
- 9. Custom Resource Definition (CRD) “A resource is an endpoint in the Kubernetes API that stores a collection of API objects of a certain kind. For example, the built-in pods resource contains a collection of Pod objects.” *Prior to K8S 1.7 this feature was called Third Party Resource (TPR) First class citizens in Kubernetes API and Command Line Interface $ kubectl get CassandraClusters $ kubectl get CassandraBackups
- 10. apiVersion: cassandra.d/v1alpha1 kind: CassandraCluster metadata: name: cassandracluster spec: statefulsetName: cassandracluster replicas: 4 apiVersion: krallistic.github.com/v1 kind: KafkaCluster metadata: name: orders-kafka spec: brokerCount: 3 topics: - name: newOrdersTopic replicationFactor: 1 partitions: 1
- 11. Custom Controller ● A controller interprets the CRD data as user’s desired state ● Then, it continually takes action to achieve and maintain this state. ● Custom Resources combined with a custom controller becomes a true declarative API “Make sure we have X instances on different machines” “Make sure we’ll do backup every X minutes” “Make sure to re-distribute keys between nodes in case of X event”
- 12. = OperatorController + CRD
- 13. An Operator ● Is used for adding new kind of functionality to a Kubernetes Cluster ● Can Automates administration and operations ○ Upgrades, Backups, Reshuffling data, etc.. ● Using a declarative approach - it’s not a script! ● Kube-Controller-Manager does this exactly for the native building blocks ○ Deployment, ReplicaSets, Pod, Service, etc.. ○ Theres a job title exactly for that. Site Reliability Engineer (SRE) as a service ®
- 14. The Operator Pattern Observe Watches CustomResource state using Kubernetes API Act Create a plan to transition from Current State to Desired State Analyze Compare Current State to Desired State 03 01 02
- 15. The Operator Pattern for { currentState := kubernetesApi.getCurrentState() desiredState := kubernetesApi.getDesiredState() takeMeThere(currentState, desiredState) }
- 18. Demo
- 19. Operator Framework by CoreOS ● Consists on: ● Operator Lifecycle Manager ● Operator Metering (Soon) ● Operator SDK ● Use Operator SDK to easily create new Operators ● Code Generated CRD’s and Custom Controller ● Using Go ● https://github.com/operator-framework
- 20. Stateful applications requires domain-specific operation knowledge that needs to be automated ________ CRDs and Operator Framework opens whole new ways for writing apps natively on top of Kubernetes
Editor's Notes
- #5: Example of solutions around Kuberntes: Hosting Platforms Kubernetes Package Manager Operations Micro-Services Debuggers Workflow platforms CI/CD platforms Gateways and more
- #7: Also mention: DaemonSet Job + CronJob StatefulSet – and it’s limitations.
- #8: Define State, Stateful. Order of creation, destruction, Persistent Data, Master & Slave, Read only replicas, etc..
- #9: In Stateful apps, deployment is easy part. Operation is the hard part.
- #12: If CRD is the Model, Custom Controller is BL
- #13: Model + BL = Kubernetes Operator