DevOps of Python applications using OpenShift (Italian version)
2 likes559 views
The document presents a comprehensive overview of using OpenShift for deploying Python applications, covering key topics such as architecture, setup, and integration with CI/CD tools like Jenkins. It details the use of Source-to-Image (S2I) for building applications and explains concepts like application lifecycle management, deployment strategies including blue-green and canary deployments. Additionally, it offers practical examples of using templates and automation to streamline application setup and management.
DevOps of Python applications using OpenShift (Italian version)
- 1. DevOps di applicazioni Python (e non solo) su OpenShift Francesco Fiore, System Architect PyCon Nove, 20 aprile 2018
- 2. 2 Agenda • Cos’è OpenShift • Cenni sull’architettura • Setup di una semplice applicazione • OpenShift per applicazioni Python: source-to-image • Dietro le quinte: API objects • Parametrizzare il setup di applicazioni usando i Template • Git, OpenShift e Jenkins: integrazione nativa per la CI/CD • Application lifecycle management • Application promotion • Strategie di deployment
- 3. 3 OpenShift overview • Platform as a Service di Red Hat • OpenShift Origin è un progetto opensource • Basato su: – Kubernetes (CaaS) – Docker – Atomic • Add-on rispetto a k8s – API objects – Web console – Software Defined Network – Docker registry – logging centralizzato • Focus su application lifecycle management e automation • Red Hat OpenShift Container Platform
- 4. 4 OpenShift vs Kubernetes Source to Image (S2I) Integrated Registry Build Configuration Image Stream Deployment Configuration Persistent Volume Claims Replication Controller Replica Sets Daemon Sets Persistent Volumes Secrets Config Maps Services Routes Software Defined Network (SDN) Web console Users and Groups Projects Namespaces OpenShift Kubernetes
- 6. 6 Setup di una applicazione oc new-project demo oc new-app https://github.com/ffiore/devops-on-openshift-demo.git --context-dir samples/flask --name flask-sample • OpenShift rileva automaticamente qual è la strategy di build – if Jenkinsfile -> Pipeline strategy – elif Dockerfile -> Docker strategy – else -> Source strategy File Linguaggio requirements.txt, setup.py python pom.xml jee app.json, package.json nodejs Godeps, main.go golang cpanfile, index.pl perl composer.json, index.php php Gemfile, Rakefile, config.ru ruby build.sbt scala • Se Source strategy, OpenShift rileva il linguaggio
- 7. 7 Setup di una applicazione oc new-app https://github.com/ffiore/devops-on-openshift-demo.git --context-dir samples/flask --name flask-sample --env WELCOME_MESSAGE=‘Hello Pycon9!’ --build-env HTTP_PROXY=‘http://x.y.z.w:8080’ oc new-app python:2.7~https://github.com/ffiore/devops-on- openshift-demo.git --context-dir samples/flask --name flask-sample-python27 --env WELCOME_MESSAGE=‘Hello Pycon9!’ oc expose svc flask-sample • Per esporre l’applicazione all’esterno (Route)
- 8. 8 Source-to-image: Python • s2i è la modalità utilizzata con Source strategy • Nessuna modifica all’applicazione – sufficiente requirements.txt • 4 modalità di esecuzione – Gunicorn • se in requirements.txt o in setup.py (install_requires) • entrypoint: wsgi.py (wsgi:application) o APP_MODULE – Django development server – Python script • APP_FILE, default app.py – Script file • APP_SCRIPT, default app.sh
- 9. 9 Setup di una applicazione: API objects • BuildConfig – Build • ImageStream – ImageStreamTag • DeploymentConfig – ReplicationController • Pod • Service spec: output: to: kind: ImageStreamTag name: flask-sample:latest source: git: uri: https://github.com/ffiore/devops-on- openshift-demo contextDir: samples/flask type: Git strategy: sourceStrategy: from: kind: ImageStreamTag name: python:3.5 namespace: openshift type: Source triggers: - github: secret: iopmuY9undV5grr7Z8zV type: GitHub - generic: secret: QOhldIxeE2ciwomAgoY9 type: Generic - type: ConfigChange - type: ImageChange apiVersion: v1 kind: BuildConfig metadata: annotations: openshift.io/generated-by: OpenShiftNewApp labels: app: flask-sample name: flask-sample namespace: demo ...
- 10. 10 Setup di una applicazione: API objects • BuildConfig – Build • ImageStream – ImageStreamTag • DeploymentConfig – ReplicationController • Pod • Service apiVersion: v1 kind: ImageStream metadata: annotations: openshift.io/generated-by: OpenShiftNewApp labels: app: flask-sample name: flask-sample namespace: demo spec: {} status: dockerImageRepository: 172.30.0.128:5000/demo/flask-sample
- 11. 11 Setup di una applicazione: API objects • BuildConfig – Build • ImageStream – ImageStreamTag • DeploymentConfig – ReplicationController • Pod • Service template: metadata: ... labels: app: flask-sample deploymentconfig: flask-sample spec: containers: - image: flask-sample:latest imagePullPolicy: Always name: flask-sample ports: - containerPort: 8080 protocol: TCP dnsPolicy: ClusterFirst restartPolicy: Always terminationGracePeriodSeconds: 30 triggers: - type: ConfigChange - imageChangeParams: automatic: true containerNames: - flask-sample from: kind: ImageStreamTag name: flask-sample:latest namespace: demo type: ImageChange apiVersion: v1 kind: DeploymentConfig metadata: name: flask-sample namespace: demo spec: replicas: 1 strategy: rollingParams: ... type: Rolling
- 12. 12 Setup di una applicazione: API objects • BuildConfig – Build • ImageStream – ImageStreamTag • DeploymentConfig – ReplicationController • Pod • Service apiVersion: v1 kind: Service metadata: annotations: openshift.io/generated-by: OpenShiftNewApp labels: app: flask-sample name: flask-sample namespace: demo spec: clusterIP: 172.30.82.11 ports: - name: 8080-tcp port: 8080 protocol: TCP targetPort: 8080 selector: app: flask-sample deploymentconfig: flask-sample type: ClusterIP
- 13. 13 Creare un Template • Aggiungere altri API object – ConfigMap – Secret – PersistentVolumeClaim – ... • Personalizzare gli oggetti – consumare ConfigMap, Secret, PVC, ecc. – creare legami tra componenti applicativi (es. applicazione + database) • Parametrizzare gli oggetti • Riutilizzare il tutto – su project/ambienti diversi – per applicazioni diverse
- 14. 14 Usare i Template oc process –f python-flask-sample-s2i.yaml -p APPLICATION_NAME=‘welcome-app’ -p SOURCE_REPOSITORY_URL=‘https://github.com/ffiore/devops-on-openshift- demo’ -p CONTEXT_DIR=‘samples/flask’ -p WELCOME_MESSAGE=‘Hello Pycon9!’ –o yaml | oc create –f - • Creare l’applicazione usando il Template • Inserire un Template «a catalogo» oc –n devops create –f django-postgresql-persistent.yaml • Usare un Template «a catalogo» oc –n devops process django-psql-persistent -p NAME=‘django-app’ -p SOURCE_REPOSITORY_URL=‘https://github.com/sclorg/django-ex’ | oc –n demo create –f -
- 15. 15 Automatizzare con Jenkins • BuildConfig possono avere strategy jenkinsPipelineStrategy – jenkinsfile – webhook per il triggering da Git • OpenShift si aspetta di trovare una istanza di Jenkins – creazione della pipeline in Jenkins – creazione di un Build • In alternativa, jenkins-ephemeral • Jenkins – autoconfigurazione al primo deploy • puntamenti a Kubernetes • provisioning pipeline esistenti – openshift jenkins plugin (deprecato da OpenShift 3.7) – openshift jenkins client plugin (GA da OpenShift 3.7) • Parametrizzare usando Template – automatic setup e CI/CD
- 16. 16 Lifecycle management • git flow/gitlab flow/... • github/generic webhook git push/merge • build applicazione • unit test (postCommit) • build image build • setup/update database (pre) • deploy applicazione • integration test (post) deploy
- 17. 17 Lifecycle management e software promotion git push/merge •git flow/gitlab flow/... •github/generic webhook build •build applicazione •unit test (postCommit) •build image deploy DEV •setup/update database (pre) •deploy applicazione •functional/integration test (post) deploy STAGING •setup/update database (pre) •deploy applicazione •smoke test (post) deploy PRODUCTION •setup/update database (pre) •deploy applicazione •smoke test (post)
- 18. 18 Rolling deployment • Rimpiazzare le istanze della vecchia versione con la nuova – readiness check – canary deployment – rollback automatico • builtin in OpenShift • quando: – no downtime – N-1 compatibility • lifecycle hooks – pre – post pre hook scale in old rc scale out new rc repeat scaling post hook git push build deploy
- 19. 19 Recreate deployment • Rimpiazzare tutte le istanze della vecchia versione con la nuova – readiness check – rollback automatico • builtin in OpenShift • quando: – migrazione dati – no N-1 compatibility – RWO volume • downtime • lifecycle hooks – pre – mid – post pre hook scale in to 0 mid hook scale out to N post hook git push build deploy
- 20. 20 Blue-green deployment • Obiettivo: testing prima dello switch del traffico – smoke test – integration test • 2 versioni dell’applicazione – production – internal (release candidate) • ‘internal’ può essere pubblica o privata • 2 service • 2 deployment configuration • switch gestito da Jenkins • alternativa, API manager – <public, staging, private> URL • N-1 compatibility git push build deploy test go live
- 21. 21 Blue-green deployment Pod Service Route app app-blue app-blue-1-x app-blue-1-y app-green app-green-2-z app-green-2-k app.os.local.int
- 22. 22 Blue-green deployment Pod Service Route app app-blue app-blue-1-x app-blue-1-y app-green app-green-2-z app-green-2-k app.os.local.int oc patch route app –p ‘{“spec”: {“to”: {“name”: “app-green”}}}’
- 23. 23 Canary deployment • Obiettivo: ridurre il rischio di deploy di una nuova versione • come blue/green, ma entrambe versioni online • 1 service • 2 deployment configuration • molteplici shard • proxy shard – scale out/in deployment configs -> traffic splitter (%) • N-1 compatibility • gestito da Jenkins git push build deploy go live test scale out/in
- 24. 24 Canary deployment Pod Service Route app app app1-1-x app1-1-y app1-1-z app2-1-k app.os.local.int
- 25. 25 Canary deployment Pod Service Route app app app1-1-x app1-1-y app2-1-k app2-1-t app.os.local.int oc scale --replicas=2 dc app2 && oc scale --replicas=2 dc app1
- 26. 26 Canary deployment Pod Service Route app app app2-1-k app2-1-t app2-1-j app2-1-w app.os.local.int oc scale --replicas=4 dc app2 && oc scale --replicas=0 dc app1
- 27. 27 A/B deployment • Obiettivi: – testing di 2 (o più) versioni contemporaneamente – 2 (o più) configurazioni, stessa versione (es. più region) • come A/B testing, ma codice + config • 1 service • 2 (o più) deployment configuration • molteplici shard • proxy shard – scale out/in deployment configs -> traffic splitter (%) • N-1 compatibility • gestito da Jenkins git push build deploy go live test scale out/in
- 28. 28 A/B deployment Pod Service Route app app app-1-x app-1-k app-a-1-y app-b-2-z app.os.local.int
- 29. 29 A/B deployment Pod Service Route app app app-1-x app-a-1-y app-b-2-z app-b-2-k oc scale --replicas=2 dc app-b && oc scale --replicas=1 dc app app.os.local.int
- 30. 30 A/B deployment Pod Service Route app app app-a-1-y app-a-1-t app-b-2-z app-b-2-k oc scale --replicas=2 dc app-a && oc scale --replicas=0 dc app app.os.local.int
- 31. 31 A/B deployment Pod Service Route app app app-b-2-z app-b-2-k app-b-2-t app-b-2-w oc scale --replicas=4 dc app-b && oc scale --replicas=0 dc app-a app.os.local.int
- 32. Sede Legale e Unità Operativa Via Alfredo Campanini, 6 20124 Milano Tel: +39 02.66732.1 – Fax: +39 02.66732.300 Unità Operativa Via Cristoforo Colombo, 163 00147 Roma Tel: +39 06.9826.9600 – Fax: +39 06.9826.9680 Grazie per l’attenzione! [email protected] | @ffiore81 https://www.par-tec.it