![The Core Namespace
(ns clojure-berlin-2016.core
(:require [aleph.http :as http]
[clojure.core.async :refer [<!! chan]]))
(defn -main [& args]
(-> (fn [_]
{:status 200
:body "Clojure Berlin 2016"})
(http/start-server {:port 8080}))
(<!! (chan)))
• A simple web server returning "Clojure Berlin 2016"](https://image.slidesharecdn.com/cdclojureberlin2016-161214234318/85/Continuous-Deployment-with-Kubernetes-Docker-and-GitLab-CI-15-320.jpg)
![The Leiningen Project File
(defproject clojure-berlin-2016 "<VERSION>"
:dependencies [[aleph "0.4.1"]
[org.clojure/clojure "1.8.0"]
[org.clojure/core.async "0.2.395"]]
:main clojure-berlin-2016.core)
• <VERSION> is replaced at build time by the Git SHA
• :main is for lein run to work](https://image.slidesharecdn.com/cdclojureberlin2016-161214234318/85/Continuous-Deployment-with-Kubernetes-Docker-and-GitLab-CI-16-320.jpg)
![Docker file
FROM clojure:lein-2.7.1
COPY src /app/src
COPY project.clj /app/
WORKDIR /app
RUN lein with-profile production deps
EXPOSE 80
CMD ["lein", "with-profile", "production", "run"]
• Just copy the sources into the container
• Use Leiningen itself to run in production](https://image.slidesharecdn.com/cdclojureberlin2016-161214234318/85/Continuous-Deployment-with-Kubernetes-Docker-and-GitLab-CI-21-320.jpg)
Continuous Deployment with Kubernetes, Docker and GitLab CI
- 1. Continuous Deployment with Kubernetes, Docker and GitLab CI @alexander_kiel Clojure Berlin 2016
- 3. Outline • Continuous Deployment why? • Docker • Kubernetes • Sample Clojure Service • Deploy with GitLabCI
- 4. Continuous Deployment • What do we want? • Increase responsiveness • Decrease time to market • Gain confidence by deploying often in small amounts • How to achieve that? • Automate everything • Always deploy the master into production • Use feature toggles when needed
- 5. Simple Git Workflow • Works for in-house apps • not for libs or shipping apps • No versions, no tags, just SHA’s • Latest commit on master is always deployed to production • Feature/fix branches are merged when ready master feature/fix branches 1ebb95d be61dda 6e4010d
- 6. Docker • Like VM’s but much more light-weight and shippable • Runs on Linux, executes processes in an isolated environment (resource limitation, filesystem, network) • Container principle: Can contain everything, but looks the same from the outside • A container platform can run every container • Developers have max. freedom what to do • In contrast: PaaS like Heroku - has to support the language
- 7. Kubernetes • Container runtime platform • Originally designed by Google - now Open Source • One of the most active projects on GitHub - 20,000 stars, 40,000 commits, 15,000 issues, 200 releases • Alternatives: Apache Mesos, Docker Swarm (lacks features)
- 8. Kubernetes Architecture k8s-master-1 k8s-master-2 k8s-master-3 load-balancer-1 load-balancer-2 DNS RR k8s-worker-1 proxy app-1 k8s-worker-2 proxy app-2 k8s-worker-n proxy app-k etcd cluster quorum HAProxy • Runs on VMware ESX • CoreOS Linux • Single YAML file as configuration • Everything in containers
- 9. Kubernetes - Pods • A Pod is a deployable unit in Kubernetes • Pods can contain multiple containers • Containers inside a Pod share on port space, can use localhost and can communicate via IPC and shared memory • Idea: one process per container - many cooperating processes in one Pod apiVersion: v1 kind: Pod metadata: name: <pod-name> labels: <key>: <value> spec: containers: - name: <container-name> image: <container-image> ports: - containerPort: 80 env: - name: <key> value: <value>
- 10. Kubernetes - Deployments • A Deployment ensures that certain number of Pods are always running • It consists of a Pod template and the number of replicas • It supports hot-redeployments by changing parts of the Pod template • Horizontal scaling is possible apiVersion: extensions/v1beta1 kind: Deployment metadata: name: <deployment-name> spec: replicas: 2 template: metadata: labels: <key>: <value> spec: containers: - name: <container-name> image: <container-image> ports: - containerPort: 80 env: - name: <key> value: <value>
- 11. Kubernetes - Services • Kubernetes uses an overlay network to provide different address spaces (we use flannel) • Every Pod has an IP address - but it changes every time one is created • Services provide a stable IP address for groups of Pods • Service names are resolvable by an internal DNS • Service selectors are used to match Pods according to there labels apiVersion: v1 kind: Service metadata: name: clojure-berlin-2016 labels: app: lens spec: type: NodePort ports: - port: 80 targetPort: 80 protocol: TCP selector: service: clojure- berlin-2016
- 12. Kubernetes - External Access • Kubernetes networks are internal only • External access through load balancers necessary • Certain Platforms like Google Compute Engine provide load balancer integration with Kubernetes • We have our own solution as a combination of HAProxy and Kubernetes NodePort • Kubernetes Services with type NodePort are exposed on every worker under a certain port frontend http bind 0.0.0.0:80 mode http option httplog acl host_clj hdr(host) clj.<domain> use_backend clj if host_clj backend clj mode http balance roundrobin option httplog server worker-1 <ip>:32599 check server worker-2 <ip>:32599 check
- 13. Deployment Lifecycle GitLab CI Source Code build test Kubernetes Test Cluster Kubernetes Prod Cluster automatic deployment manual deployment git push
- 14. Sample Clojure Service • .gitlab-ci.yml • Like .travis.yml contains instructions for GitLabCI how to test, build and deploy • Dockerfile • Instructions for Docker how to build the image of the app • Artifact of the build is a docker image - not uberjar • kube-deployment.yml • Kubernetes deployment instructions • kube-svc.yml • Kubernetes service description https://github.com/alexanderkiel/clojure-berlin-2016
- 15. The Core Namespace (ns clojure-berlin-2016.core (:require [aleph.http :as http] [clojure.core.async :refer [<!! chan]])) (defn -main [& args] (-> (fn [_] {:status 200 :body "Clojure Berlin 2016"}) (http/start-server {:port 8080})) (<!! (chan))) • A simple web server returning "Clojure Berlin 2016"
- 16. The Leiningen Project File (defproject clojure-berlin-2016 "<VERSION>" :dependencies [[aleph "0.4.1"] [org.clojure/clojure "1.8.0"] [org.clojure/core.async "0.2.395"]] :main clojure-berlin-2016.core) • <VERSION> is replaced at build time by the Git SHA • :main is for lein run to work
- 17. .gitlab-ci.yml - test/build image: clojure:lein-2.7.1 stages: - test - build - deploy test: stage: test tags: - docker script: - lein test build: stage: build tags: - docker script: - sed -i "s/<VERSION>/$CI_BUILD_REF/" project.clj - docker build -t clojure-berlin-2016:$CI_BUILD_REF . - docker push clojure-berlin-2016:$CI_BUILD_REF
- 18. .gitlab-ci.yml - deploy branch deploy-branch: stage: deploy environment: test image: dreg.life.uni-leipzig.local/kubectl:0.4 tags: - docker script: - sed -i "s/<VERSION>/$CI_BUILD_REF/" kube-deployment.yml - kubectl config use-context gitlab-ci-test - kubectl apply -f kube-deployment.yml except: - master when: manual • Used to test a feature/fix branch in a full environment
- 19. .gitlab-ci.yml - deploy test deploy-master: stage: deploy environment: test image: dreg.life.uni-leipzig.local/kubectl:0.4 tags: - docker script: - sed -i "s/<VERSION>/$CI_BUILD_REF/" kube-deployment.yml - kubectl config use-context gitlab-ci-test - kubectl apply -f kube-deployment.yml only: - master
- 20. .gitlab-ci.yml - deploy prod deploy-prod: stage: deploy environment: prod image: dreg.life.uni-leipzig.local/kubectl:0.4 tags: - docker script: - sed -i "s/<VERSION>/$CI_BUILD_REF/" kube-deployment.yml - kubectl config use-context gitlab-ci-prod-a - kubectl apply -f kube-deployment.yml only: - master when: manual
- 21. Docker file FROM clojure:lein-2.7.1 COPY src /app/src COPY project.clj /app/ WORKDIR /app RUN lein with-profile production deps EXPOSE 80 CMD ["lein", "with-profile", "production", "run"] • Just copy the sources into the container • Use Leiningen itself to run in production
- 22. kube-deployment.yml apiVersion: extensions/v1beta1 kind: Deployment metadata: name: clojure-berlin-2016 spec: replicas: 2 template: metadata: labels: app: lens service: clojure-berlin-2016 spec: containers: - name: clojure-berlin-2016 image: dreg.life.uni-leipzig.local/clojure-berlin-2016:<VERSION> ports: - containerPort: 8080 resources: requests: cpu: "125m" memory: "1Gi" limits: cpu: 1 memory: "2Gi"
- 23. kube-svc.yml apiVersion: v1 kind: Service metadata: name: clojure-berlin-2016 labels: app: lens spec: type: NodePort ports: - port: 80 targetPort: 8080 protocol: TCP selector: service: clojure-berlin-2016
- 24. Steps to Follow • Create the Kubernetes Service • kubectl create -f kube-svc.yml • Edit HAProxy Config • add rules and backend for the service • Push to GitLab • git push
- 25. Pipeline in GitLab CI
- 27. Environments in GitLabCI • Very good visibility of wich commit is deployed in which environment right now • Manual deployment to prod possible
- 28. Environment History • Easy to see when what commit was deployed • Rollback possible
- 29. Numbers • Our team has 4 developers • We run 2 Kubernetes clusters (test and prod) with about 96 GB RAM and and 24 vCPU’s each • We run about 60 pods in production • We have other services like central log aggregation running using Fluentd and Elasticsearch/Kibana
- 30. Thank You • Sample Project on Github https://github.com/alexanderkiel/clojure-berlin-2016 • Twitter @alexander_kiel • Mail [email protected]