Deploying Apache Spark Jobs on Kubernetes with Helm and Spark Operator
2 likes895 views
The document discusses deploying Apache Spark jobs on Kubernetes using Helm and the Spark Operator. It outlines the process of building, running, and deploying applications, highlighting the importance of Docker images and Helm templates. Additional resources and links for further information are provided, along with a call for feedback on the session.
1 of 25
Downloaded 31 times
More Related Content
What's hot (20)
Ad
Similar to Deploying Apache Spark Jobs on Kubernetes with Helm and Spark Operator (20)
Ad
More from Databricks (20)
Deploying Apache Spark Jobs on Kubernetes with Helm and Spark Operator
- 2. Deploying Apache Spark Jobs on Kubernetes with Helm and Spark Operator Tom Lous Freelance Data Engineer @ Shell @tomlous
- 3. Why?
- 4. Bad Idea?
- 5. 1. Build it! 2. Run it?
- 6. Challenge!
- 7. Solution?
- 8. Kubernetes!?
- 10. Application
- 12. Base Image
- 13. Dockerize
- 14. Deploy?
- 15. Spark Operator!
- 16. Helm Template
- 17. Helm Values
- 18. Chart Museum
- 19. Deploy!
- 20. Success!
- 21. Next Steps
- 22. Links ▪ HowTo: https://medium.com/@tomlous/deploying-apache-spark- jobs-on-kubernetes-with-helm-and-spark-operator-eb1455930435 ▪ SparkOperator: https://github.com/GoogleCloudPlatform/spark-on- k8s-operator ▪ SparkOperator Helm: https://github.com/helm/charts/tree/master/incubator/sparkoperat or ▪ Code: https://github.com/TomLous/medium-spark-k8s ▪ Chart Museum: https://github.com/helm/chartmuseum
- 23. Questions?
- 24. Feedback Your feedback is important to us. Don’t forget to rate and review the sessions.