SlideShare a Scribd company logo

Containers and Kubernetes without limits

Antje Barth, profile picture
Antje Barth

The document outlines the integration of stateful applications with Kubernetes and containers, focusing on achieving reliability, scalability, and security in a containerized environment. It discusses the evolution of containers, their benefits, such as being lightweight and stateless, and the challenges they face, particularly in handling persistent data. Additionally, it highlights the capabilities of Kubernetes, including orchestration, management, and the need for persistent volumes to address state-related issues in containerized applications.

Software
1 of 71
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
CONTAINERS AND KUBERNETES WITHOUT LIMITS How to make stateless containers reliable and available for stateful applications! Copenhagen, September 18th, 2018
2 © 2018 MapR Technologies, Inc. // MapR Confidential Today‘s Goal Learn how to build a reliable, scalable, and secure containerized platform, and to handle persistent data in large containerized environments across multiple data centers or geographic locations. (DATA)
4 © 2018 MapR Technologies, Inc. // MapR Confidential Agenda Introduction to Containers and Kubernetes • Architectural concepts • Use cases for containers • Container challenges How to build a Reliable, Scalable, and Secure Containerized Platform • Kubernetes • Challenges for stateful applications • #Kubernetes4Data DEMO SUMMARY Q&A SWAG J FOR YOU
Introduction to Containers and Kubernetes
6 © 2018 MapR Technologies, Inc. // MapR Confidential Virtual Machines are Computers in a Box Containers are Applications in a Box
7 © 2018 MapR Technologies, Inc. // MapR Confidential hardware os hypervisor vm os libs app vm os libs app hardware os container libs app container libs app container libs app VM vs Container
8 © 2018 MapR Technologies, Inc. // MapR Confidential Pets vs Cattle - long lived - name them - care for them - ephemeral - brand them with #’s - well.. vets are expensive
9 © 2018 MapR Technologies, Inc. // MapR Confidential Container characteristics Light-weight, easy and quick to deploy Portable Break complex monolithic applications into smaller, modules
10 © 2018 MapR Technologies, Inc. // MapR Confidential Typical use cases for containers ENTERPRISE APPLICATION MODERNIZATION All Applications that can be run in containers MACHINE LEARNING Deploy portable, smart applications MICROSERVICES Create distributed applications by using containers to creating independent tasks CI/CD Continuous Integration/ Development
11 © 2018 MapR Technologies, Inc. // MapR Confidential Source: Marko Luksa (2017). Kubernetes in Action. Shelter Island, NY: Manning Publications Co. • Docker combined several Linux technologies into a single set of tools • Docker uses Linux ‘cgroups’ (process isolation), overlay file systems (OverlayFS/aufs), virtual networks & bridges, and virtual directory mount points to create “containers” • These tools separately have been in the Linux community for many years: Docker put them all together in a single, simple set of commands Containers & Docker
12 © 2018 MapR Technologies, Inc. // MapR Confidential cgroups ● cpu ● memory ● network ● etc. namespaces ● pids ● mnts ● etc. Chroot (filesystem) Isolation
13 © 2018 MapR Technologies, Inc. // MapR Confidential File File Read-only Layer Container Images
14 © 2018 MapR Technologies, Inc. // MapR Confidential File File File Read-only Layer Read-only Layer Container Images
15 © 2018 MapR Technologies, Inc. // MapR Confidential File File File Read-only Layer Read-only Layer Writable Layer Container Images
16 © 2018 MapR Technologies, Inc. // MapR Confidential File File File Container Image chroot cgroups ● cpu ● memory ● network ● etc. namespaces ● pids ● mnts ● etc. Container = Image + Isolation
17 © 2018 MapR Technologies, Inc. // MapR Confidential Why are containers called stateless? CONTAINER A APP BIN/LIBs APP BIN/LIBs CONTAINER B SHARED RESOURCES • Build containers with what is minimally needed for the application • Infrastructure resources are shared and assigned for each container • Container does not hold data
18 © 2018 MapR Technologies, Inc. // MapR Confidential What is the impact of containers being stateless? APP • Application server run as a service in container • Updating containers is not applicable. Update the docker image, stop and restart containers • Backing up a container is not needed
19 © 2018 MapR Technologies, Inc. // MapR Confidential Container growth – Trend towards production Source: CNCF Survey, 2018. https://www.cncf.io/blog/2018/08/29/cncf-survey-use-of-cloud-native-technologies-in-production-has-grown-over-200-percent/ • 73% of respondents are currently using containers in production today, with the remaining 27% planning to use them in the future. • 89% of respondents are currently using containers for proof of concepts, as well as testing (85%) and development (86%).
20 © 2018 MapR Technologies, Inc. // MapR Confidential Containers – What we know so far! • Are lightweight • Are stateless • Are portable • Targeted for developing applications • Surely moving towards production • Docker made it popular … and added a whole lot of jargon for us to learn! J
21 © 2018 MapR Technologies, Inc. // MapR Confidential But… Containers have a problem
22 © 2018 MapR Technologies, Inc. // MapR Confidential Challenges in using / deploying containers Source: CNCF Survey, 2018. https://www.cncf.io/blog/2018/08/29/cncf-survey-use-of-cloud-native-technologies-in-production-has-grown-over-200-percent/ • Cultural Changes with Development Team • Complexity • Lack of Training • Security • Monitoring • Storage • Networking
23 © 2018 MapR Technologies, Inc. // MapR Confidential Some of the things Docker can’t do • Monitor running containers • Handle dead containers • Move containers so utilization improves • Auto-scale container instances to handle load • Solve port mapping hell • …
24 © 2018 MapR Technologies, Inc. // MapR Confidential You can never get away from pets unless: • You have an environment to support cattle • You handle the problem of container state
Kubernetes kubernetes (n.) - greek word for pilot or helm
26 © 2018 MapR Technologies, Inc. // MapR Confidential Now home at the CNCF! Large-scale cluster management at Google with Borg, 2015. https://ai.google/research/pubs/pub43438 Kubernetes started life as a successor to Google’s Borg project... https://www.cncf.io/ https://kubernetes.io/
27 © 2018 MapR Technologies, Inc. // MapR Confidential Kubernetes is an API and agents The Kubernetes API provides containers with a scheduling, configuration, network, and storage The Kubernetes runtime manages the containers
28 © 2018 MapR Technologies, Inc. // MapR Confidential Kubernetes – Why is it so popular? • There are many management software solutions to create, manage & delete containers with newer vendors emerging everyday • Kubernetes remains the leader with 83% (up from 77%) • The ecosystem and developer community augmented by Google’s support gave Kubernetes the edge over others Source: CNCF Survey, 2018. https://www.cncf.io/blog/2018/08/29/cncf-survey-use-of-cloud-native-technologies-in-production-has-grown-over-200-percent/
29 © 2018 MapR Technologies, Inc. // MapR Confidential Key K8s features that help you orchestrate containers at scale • Creating, managing multiple containers in a single entity, a POD. • Container Scheduling on same host or on particular hosts, and sharing IP address allows the containers to communicate with each other • Auto scaling, replication and recovery of containers • cAdvisor resource usage monitoring module • Network pluggable architecture • Load balancer • Health check mechanisms
30 © 2018 MapR Technologies, Inc. // MapR Confidential *Pods are not containers! IP address: 1.2.3.4 container #1 container #2 pod • Pods are the atomic unit of scheduling/mgmt in k8s • Pods can have multiple containers • Containers in pod have same IP • Containers in pod share IPC address space • Many advanced management features directed at pods
31 © 2018 MapR Technologies, Inc. // MapR Confidential api server scheduler controller manager etcd node 1 node 2 node 3 node 4 node 5 kubelet kube-proxy docker Control Plane Workers
32 © 2018 MapR Technologies, Inc. // MapR Confidential kubectl yaml pod pod pod pod etcd pods plans The API server turns YAML into pods and plans. api server
33 © 2018 MapR Technologies, Inc. // MapR Confidential Kubernetes – an open, pluggable framework
34 © 2018 MapR Technologies, Inc. // MapR Confidential Kubernetes – an open, pluggable framework Source: CNCF landscape, see https://github.com/cncf/landscape and http://l.cncf.io
35 © 2018 MapR Technologies, Inc. // MapR Confidential • From eval to prod: How a Service Mesh Helped Us Build Production Cloud-Native Services - Israel Sotomayor, Moltin o Session recording: https://youtu.be/VoYAtCVz3Ig • What Does “Production Ready” Really Mean for a Kubernetes Cluster? - Lucas Käldström, Individual o Session recording: https://youtu.be/EjSiZgGdRqk • Keynote: Running with Scissors - Liz Rice, Technology Evangelist, Aqua Security • Session recording: https://youtu.be/ltrV-Qmh3oY KubeCon: Learn from the community how to use the services
36 © 2018 MapR Technologies, Inc. // MapR Confidential We still have a problem
37 © 2018 MapR Technologies, Inc. // MapR Confidential State!
38 © 2018 MapR Technologies, Inc. // MapR Confidential Problem with Containers and State What about stateful applications? Real applications are stateful - they need to persist data somewhere! Pod and ReplicaSet abstract compute and memory. 1. Containers are ephemeral: no way to persist state • Container termination / crashes result in loss of data • Can’t run stateful applications 2. Containers can’t share data between each other
39 © 2018 MapR Technologies, Inc. // MapR Confidential Give Containers Persistent Volumes Source: CNCF Webinar - Introduction to Cloud Native Storage, https://www.youtube.com/watch?v=S9PnpnrHdIs ü Data survives (persists) beyond container, pod, host ü Workload can choose its best storage
40 © 2018 MapR Technologies, Inc. // MapR Confidential Persistence & Portability Source: CNCF Webinar - Introduction to Cloud Native Storage, https://www.youtube.com/watch?v=S9PnpnrHdIs
41 © 2018 MapR Technologies, Inc. // MapR Confidential Pod to Persistence Volume Source: CNCF Webinar - Introduction to Cloud Native Storage, https://www.youtube.com/watch?v=S9PnpnrHdIs 3
How does MapR fit into this?
43 © 2018 MapR Technologies, Inc. // MapR Confidential Cloud native storage takes many forms Source: CNCF Webinar - Introduction to Cloud Native Storage, https://www.youtube.com/watch?v=S9PnpnrHdIs
44 © 2018 MapR Technologies, Inc. // MapR Confidential Cloud native storage takes many forms Source: CNCF Webinar - Introduction to Cloud Native Storage, https://www.youtube.com/watch?v=S9PnpnrHdIs MAPR DATA PLATFORM FILES / OBJECTS / TABLES / STREAMS APIs: NFS, POSIX, REST, S3, HDFS, HBASE, JSON, KAFKA DATA CENTER CLOUD MULTI-CLOUD EDGE KUBERNETES COMMODITY SERVER VIRTUAL MACHINE IoT & Edge AI / ML ADV. ANALYTICS ENTERPRISE APPLICATIONS Pod Pod Pod Pod
45 © 2018 MapR Technologies, Inc. // MapR Confidential Scale. It distributes data across the cluster and offers a global namespace for a unified view of data regardless of its physical location High Availability. Offers configurable levels of replication to ensure data durability. In event of a failure, all nodes participate to self-heal and reconstruct data automatically Data Protection. End-to-end security, per volume Access control expressions, space efficient snapshots, volume mirroring, offers several choices to build a data protection strategy Intelligent Data Placement. Offers three different storage tiers with automated storage policies to place data based on their SLAs Edge, on-premises, Cloud: Can be deployed in on-premises datacenters, edge and on the cloud Modern Data Platform
46 © 2018 MapR Technologies, Inc. // MapR Confidential MapR Data Container Architecture Built For Speed, Scale, Reliability Data & metadata fully distributed A A A B B B C C C D D D E E E 32 GB 256 MB 8 KB Hierarchical organization of data No single point of failure Fast parallel access Exabyte scale Full read-write
47 © 2018 MapR Technologies, Inc. // MapR Confidential MapR Volumes Volumes are logical units of management, holding files, directories, tables, messages. WHAT CAN YOU DO WITH VOLUMES? • Schedule snapshots • Schedule mirrors • Control data placement • Access permissions • Enforce volume quotas • Manage performance • Specify replication factor Volumes: Shared MapR Cluster r : user:sally | (group:research & group:managers) MAPR ACCESS CONTROL EXPRESSIONS /mktg /finance /projectx
48 © 2018 MapR Technologies, Inc. // MapR Confidential MapR Volumes Volumes are used for easy control of access, multi-tenancy, data locality & DR
49 © 2018 MapR Technologies, Inc. // MapR Confidential ü Global data view in a single namespace ü Distributed data processing ü Unified Security ü Global Replication For Data Distribution & DR ü Bandwidth-aware to manage global data flows ü Simplify cross cloud application development & deployment Global Namespace – common path to connect to any data Globally Protected Globally Accessible Globally Managed Globally Replicated Across Locations Across Clouds /mapr /us.mapr.com /eu_cloud.mapr.com /asia.mapr.com /us_cloud.mapr.com
50 © 2018 MapR Technologies, Inc. // MapR Confidential Automatically Synchronized Globally Distributed Data Topic Topic Topic On-Premises S3 EDGE DATA PLATFORM DATA PLATFORM DATA PLATFORM DATA PLATFORM Multi-Cloud Data Movement & Application Portability Enabling Application and Data Portability
51 © 2018 MapR Technologies, Inc. // MapR Confidential “Kubernetes is our platform for apps. We view MapR as our Kubernetes for data.” “The combination of the two allows us to support future & current workloads on any infrastructure. With SLA enforcement and the ability to move apps & data seamlessly across clouds.” CUSTOMERS #Kubernetes4Data
52 © 2018 MapR Technologies, Inc. // MapR Confidential MapR Persistent Application Client Container (PACC) • Pre-built, certified container image for connecting to MapR services • Secure authentication at container level, secure connection • Extensible support for application layers • Available in Docker Hub, Dockerfile for customizability MapR POSIX Client for Containers MapR Converged Client for Containers Space for Customer Application MapR PACC MAPR DATA PLATFORM EVENT DATA STREAMS ANALYTICS & ML ENGINES OPERATIONAL DATABASE CLOUD-SCALE FILE AND OBJECT STORE
53 © 2018 MapR Technologies, Inc. // MapR Confidential Containerized Microservices have real-time access to files/tables/streams Microservices Databases/files Microservices Databases/files Microservices Databases/files Microservices Microservices Microservices Microservices Microservices Microservices MAPR DATA PLATFORM Stream Stream
MapR Data Fabric for Kubernetes
55 © 2018 MapR Technologies, Inc. // MapR Confidential MapR Data Fabric for Kubernetes node Pod Pod node Pod Pod node Pod Pod MAPR MAPR MAPR MAPR DATA PLATFORM • Integration with Kubernetes APIs, packaged and run as a POSIX client on each Kubernetes host • MapR Volumes are mounted for containers • Persist data for containerized applications • Scale data and performance as containers grow • Highly available by leveraging replicas, snapshots, mirroring of data • Benefit from MapR tickets, for end-to- end security • Multi-tenant deployment and access
56 © 2018 MapR Technologies, Inc. // MapR Confidential There are two ways to provision a volume: 1. Kubernetes Volume with Static Provisioner This is used to mount an existing MapR Volume to Containers managed by Kubernetes. 2. Kubernetes Persistent Volumes with Dynamic Provisioner This is used to create and mount a new MapR Volume to Containers managed by Kubernetes. Kubernetes Integration via Volume Driver Plugin
57 © 2018 MapR Technologies, Inc. // MapR Confidential pod kubelet docker plugin mapr fuse Example 1: You have a Postgres container that needs persistent storage. Plugin mounts MapR path via fuse Static Provisioning • Kdf volume plugin • Admin provisions • Fast, uses Posix drivers • Secured with MapR tickets • MapR cluster can be external to K8s
58 © 2018 MapR Technologies, Inc. // MapR Confidential KUBERNETES (CLIENT HOST) YOUR CONTAINER MAPR VOLUME PLUGIN - POSIX CLIENT K8S PERSISTANT VOLUME MAPR DATA PLATFORM 1. Request Volume 2. Mount POSIX Volume Mounting An Existing MapR Volume (Static Provisioner)
59 © 2018 MapR Technologies, Inc. // MapR Confidential pod kubelet docker plugin mapr fuse provisioner rest Example 2: You are testing a new container. You want the storage in MapR automatically allocated for the container. Dynamic provisioning • Kdf provisioner • Uses MapR REST API’s to allocate/delete MapR volumes • Mounting is the same as static provisioning
60 © 2018 MapR Technologies, Inc. // MapR Confidential KUBERNETES (CLIENT HOST) YOUR CONTAINER MAPR VOLUME PLUGIN - POSIX CLIENT K8S PERSISTANT VOLUME MAPR DATA PLATFORM 1. Request Volume 5. Mount POSIX Volume PERSISTENT VOLUME CLAIM STORAGE CLASS DYNAMIC PROVISIONER 2. Request Volume 3. Request Volume Creating A New MapR Volume (Dynamic Provisioner) Volume Claim binds the volume created to the container(s) Storage Classes used by Administrators express the type, size and other characteristics that the volume should contain
Demo
62 © 2018 MapR Technologies, Inc. // MapR Confidential Demo: Run database server container in high availability Run PostgreSQL in Container Dynamically create and store the Postgres data on MapR as the persistent data store.
63 © 2018 MapR Technologies, Inc. // MapR Confidential Demo: Run database server container in high availability Dynamically create MapR Volume Use the Volume Driver Plugin to dynamically create a MapR Volume as the PostgreSQL data store. 1 1 Launch PostgreSQL container The PostgreSQL uses the Volume Driver plugin to locate the volume. 2 2 Simulate PostgreSQL failover The PostgreSQL container can failover without having effect on the data stored on MapR. 3 3 4 Relaunch PostgreSQL container A new PostgreSQL container will leverage the existing data on MapR. 4
Summary
65 © 2018 MapR Technologies, Inc. // MapR Confidential RECAP: Today‘s Goal Learn how to build a reliable, scalable, and secure containerized platform, and to handle persistent data in large containerized environments across multiple data centers or geographic locations. (DATA)
66 © 2018 MapR Technologies, Inc. // MapR Confidential COMPUTE AGILITY MAPR DATA PLATFORM Containers & Kubernetes without limits! MAPR KUBERNETES VOLUME PLUGIN TENANT N Application….. TENANT 1 Application APP AGILITY DATA AGILITY DATA CENTER CLOUD MULTI-CLOUD KUBERNETES EDGE ACROSS INFRASTRUCTURES ENTERPRISE APPLICATIONS AI AND ML ADV. ANALYTICS Stateful app container MAPR POSIX CLIENT FOR CONTAINERS Application
More resources
68 © 2018 MapR Technologies, Inc. // MapR Confidential Kubernetes (e)books
69 © 2018 MapR Technologies, Inc. // MapR Confidential MapR Data Fabric for Kubernetes https://mapr.com/solutions/data-fabric/kubernetes/ MapR Data Fabric for Kubernetes - Documentation https://mapr.com/docs/60/PersistentStorage/kdf_overview.html MapR Data Fabric for Kubernetes - GitHub https://github.com/mapr/KubernetesDataFabric MapR Tutorial: How to Install and Deploy Applications at Scale on K8s Part 1: https://mapr.com/blog/making-data-actionable-at-scale-part-1-of-3 Part 2: https://mapr.com/blog/making-data-actionable-at-scale-part-2-of-3 Part 3: https://mapr.com/blog/making-data-actionable-at-scale-part-3-of-3 MapR Data Fabric for Kubernetes
70 © 2018 MapR Technologies, Inc. // MapR Confidential MapR Academy Free Online Training: learn.mapr.com
71 © 2018 MapR Technologies, Inc. // MapR Confidential New O’Reilly Book! by Ted Dunning and Ellen Friedman Just released at Strata New York, September 2018 GRAB YOUR FREE COPY HERE! or download the e-book here: https://mapr.com/ebook/ai-and-analytics-in- production/
THANK YOU! #MapR #Kubernetes4Data

More Related Content

PDF
E bpf and profilers
LibbySchulze
 
PDF
Hardening Your CI/CD Pipelines with GitOps and Continuous Security
Weaveworks
 
PDF
Making your app soar without a container manifest
LibbySchulze
 
PDF
Continuous Security for GitOps
Weaveworks
 
PDF
Security: The Value of SBOMs
Weaveworks
 
PDF
Secure GitOps pipelines for Kubernetes with Snyk & Weaveworks
Weaveworks
 
PDF
GitOps for Helm Users by Scott Rigby
Weaveworks
 
PDF
Is your kubernetes negative or positive
LibbySchulze
 
E bpf and profilers
LibbySchulze
 
Hardening Your CI/CD Pipelines with GitOps and Continuous Security
Weaveworks
 
Making your app soar without a container manifest
LibbySchulze
 
Continuous Security for GitOps
Weaveworks
 
Security: The Value of SBOMs
Weaveworks
 
Secure GitOps pipelines for Kubernetes with Snyk & Weaveworks
Weaveworks
 
GitOps for Helm Users by Scott Rigby
Weaveworks
 
Is your kubernetes negative or positive
LibbySchulze
 

What's hot (20)

PDF
WTF is GitOps and Why You Should Care?
Weaveworks
 
PDF
GitOps is the best modern practice for CD with Kubernetes
Volodymyr Shynkar
 
PDF
GitOps with ArgoCD
CloudOps2005
 
PDF
Intro to Kubernetes & GitOps Workshop
Weaveworks
 
PPTX
C++ Webinar "Why Should You Learn C++ in 2021-22?"
GlobalLogic Ukraine
 
PDF
The Power of GitOps with Flux & GitOps Toolkit
Weaveworks
 
PDF
Designing a complete ci cd pipeline using argo events, workflow and cd products
Julian Mazzitelli
 
PPTX
GitOps w/argocd
Jean-Philippe Bélanger
 
PDF
Continuous Deployment for Staging and Production Environments
OlyaSurits
 
PPTX
GitOps - Modern best practices for high velocity app dev using cloud native t...
Weaveworks
 
PDF
CI:CD in Lightspeed with kubernetes and argo cd
Billy Yuen
 
PDF
Security, Automation and the Software Supply Chain
OlyaSurits
 
PDF
Hybrid and Multi-Cloud Strategies for Kubernetes with GitOps
Sonja Schweigert
 
PDF
Observe and command your fleets across any kubernetes with weave git ops
Weaveworks
 
PPTX
[Konveyor] migrate and modernize your application portfolio to kubernetes wit...
Konveyor Community
 
PDF
Kubecon seattle 2018 workshop slides
Weaveworks
 
PDF
Leveraging Gradle @ Netflix (Madrid GUG Feb 2, 2021)
Roberto Pérez Alcolea
 
PDF
Delivering Quality at Speed with GitOps
Weaveworks
 
PDF
Dev ops
Eman Abdelmohsen
 
PPTX
[Konveyor] address technical risks when implementing workload modernization u...
Konveyor
 
WTF is GitOps and Why You Should Care?
Weaveworks
 
GitOps is the best modern practice for CD with Kubernetes
Volodymyr Shynkar
 
GitOps with ArgoCD
CloudOps2005
 
Intro to Kubernetes & GitOps Workshop
Weaveworks
 
C++ Webinar "Why Should You Learn C++ in 2021-22?"
GlobalLogic Ukraine
 
The Power of GitOps with Flux & GitOps Toolkit
Weaveworks
 
Designing a complete ci cd pipeline using argo events, workflow and cd products
Julian Mazzitelli
 
GitOps w/argocd
Jean-Philippe Bélanger
 
Continuous Deployment for Staging and Production Environments
OlyaSurits
 
GitOps - Modern best practices for high velocity app dev using cloud native t...
Weaveworks
 
CI:CD in Lightspeed with kubernetes and argo cd
Billy Yuen
 
Security, Automation and the Software Supply Chain
OlyaSurits
 
Hybrid and Multi-Cloud Strategies for Kubernetes with GitOps
Sonja Schweigert
 
Observe and command your fleets across any kubernetes with weave git ops
Weaveworks
 
[Konveyor] migrate and modernize your application portfolio to kubernetes wit...
Konveyor Community
 
Kubecon seattle 2018 workshop slides
Weaveworks
 
Leveraging Gradle @ Netflix (Madrid GUG Feb 2, 2021)
Roberto Pérez Alcolea
 
Delivering Quality at Speed with GitOps
Weaveworks
 
Dev ops
Eman Abdelmohsen
 
[Konveyor] address technical risks when implementing workload modernization u...
Konveyor
 
Ad

Similar to Containers and Kubernetes without limits (20)

PDF
Container and Kubernetes without limits
Antje Barth
 
PPTX
How to Get Going with Kubernetes
Ted Dunning
 
PDF
Big Data LDN 2018: PROGRESS FOR BIG DATA IN KUBERNETES
Matt Stubbs
 
PDF
The rise of microservices
Cloud Technology Experts
 
PPTX
MapR Product Update - Spring 2017
MapR Technologies
 
PDF
Http Services in Rust on Containers
Anton Whalley
 
PDF
Container Attached Storage (CAS) with OpenEBS - SDC 2018
OpenEBS
 
PPTX
Rancher k8s Training - Oct 4 - Rajashree Mandaogane.pptx
satzzqwerty
 
PPTX
Progress for big data in Kubernetes
Ted Dunning
 
PDF
Navigating a Mesh of Microservices in the new Cloud-Native World with Istio
Gary Arora
 
PDF
Big Data LDN 2018: 7 SUCCESSFUL HABITS FOR DATA-INTENSIVE APPLICATIONS IN PRO...
Matt Stubbs
 
PDF
7 Habits for Big Data in Production - keynote Big Data London Nov 2018
Ellen Friedman
 
PDF
Tampere Docker meetup - Happy 5th Birthday Docker
Sakari Hoisko
 
PPTX
Bahrain ch9 introduction to docker 5th birthday
Walid Shaari
 
PDF
A hitchhiker‘s guide to the cloud native stack
QAware GmbH
 
PDF
A Hitchhiker’s Guide to the Cloud Native Stack. #CDS17
Mario-Leander Reimer
 
PDF
Containers and Kubernetes
Altoros
 
PPTX
Cisco Connect 2018 Indonesia - Building container ready infrastructure
NetworkCollaborators
 
PDF
Modern big data and machine learning in the era of cloud, docker and kubernetes
Slim Baltagi
 
PPTX
Media processing with serverless architecture
Kensaku Komatsu
 
Container and Kubernetes without limits
Antje Barth
 
How to Get Going with Kubernetes
Ted Dunning
 
Big Data LDN 2018: PROGRESS FOR BIG DATA IN KUBERNETES
Matt Stubbs
 
The rise of microservices
Cloud Technology Experts
 
MapR Product Update - Spring 2017
MapR Technologies
 
Http Services in Rust on Containers
Anton Whalley
 
Container Attached Storage (CAS) with OpenEBS - SDC 2018
OpenEBS
 
Rancher k8s Training - Oct 4 - Rajashree Mandaogane.pptx
satzzqwerty
 
Progress for big data in Kubernetes
Ted Dunning
 
Navigating a Mesh of Microservices in the new Cloud-Native World with Istio
Gary Arora
 
Big Data LDN 2018: 7 SUCCESSFUL HABITS FOR DATA-INTENSIVE APPLICATIONS IN PRO...
Matt Stubbs
 
7 Habits for Big Data in Production - keynote Big Data London Nov 2018
Ellen Friedman
 
Tampere Docker meetup - Happy 5th Birthday Docker
Sakari Hoisko
 
Bahrain ch9 introduction to docker 5th birthday
Walid Shaari
 
A hitchhiker‘s guide to the cloud native stack
QAware GmbH
 
A Hitchhiker’s Guide to the Cloud Native Stack. #CDS17
Mario-Leander Reimer
 
Containers and Kubernetes
Altoros
 
Cisco Connect 2018 Indonesia - Building container ready infrastructure
NetworkCollaborators
 
Modern big data and machine learning in the era of cloud, docker and kubernetes
Slim Baltagi
 
Media processing with serverless architecture
Kensaku Komatsu
 
Ad

Recently uploaded (20)

PDF
Which alternative to Crystal Reports is best for small or large businesses.pdf
Varsha Nayak
 
PDF
Why TechBuilder is the Future of Pickup and Delivery App Development (1).pdf
TechBuilder
 
PDF
Digital Strategies for Manufacturing Companies
Virendra Rai, PMP
 
PDF
EN-Survey-Report-SAP-LeanIX-EA-Insights-2025.pdf
deepakkhaitan3
 
PPTX
Operating system designcfffgfgggggggvggggggggg
rmskumara09
 
PDF
iTop VPN Free 5.6.0.5262 Crack latest version 2025
itskinga12
 
PPTX
assetexplorer- product-overview - presentation
nonameist3434
 
PDF
wealthsignaloriginal-com-DS-text-... (1).pdf
skmiani786
 
PPTX
Odoo POS Development Services by CandidRoot Solutions
CandidRoot Solutions Private Limited
 
PPTX
Introduction to Artificial Intelligence
lohedi9363
 
PDF
medical staffing services at VALiNTRY
Tiyan Grayson
 
PDF
top salesforce developer skills in 2025.pdf
CloudMetic
 
PDF
Designing Intelligence for the Shop Floor.pdf
nikglvk
 
PDF
Odoo Companies in India – Driving Business Transformation.pdf
azhuhardeen1968
 
PDF
Internet Downloader Manager (IDM) Crack 6.42 Build 42 Updates Latest 2025
itskinga12
 
PDF
Understanding Forklifts - TECH EHS Solution
TECH EHS Solution
 
PDF
Product Update: Alluxio AI 3.7 Now with Sub-Millisecond Latency
Alluxio, Inc.
 
PPTX
Embracing Complexity in Serverless! GOTO Serverless Bengaluru
SheenBrisals
 
PDF
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
OnePlan Solutions
 
PPTX
Reimagine Home Health with the Power of Agentic AI​
AutomationEdge Technologies
 
Which alternative to Crystal Reports is best for small or large businesses.pdf
Varsha Nayak
 
Why TechBuilder is the Future of Pickup and Delivery App Development (1).pdf
TechBuilder
 
Digital Strategies for Manufacturing Companies
Virendra Rai, PMP
 
EN-Survey-Report-SAP-LeanIX-EA-Insights-2025.pdf
deepakkhaitan3
 
Operating system designcfffgfgggggggvggggggggg
rmskumara09
 
iTop VPN Free 5.6.0.5262 Crack latest version 2025
itskinga12
 
assetexplorer- product-overview - presentation
nonameist3434
 
wealthsignaloriginal-com-DS-text-... (1).pdf
skmiani786
 
Odoo POS Development Services by CandidRoot Solutions
CandidRoot Solutions Private Limited
 
Introduction to Artificial Intelligence
lohedi9363
 
medical staffing services at VALiNTRY
Tiyan Grayson
 
top salesforce developer skills in 2025.pdf
CloudMetic
 
Designing Intelligence for the Shop Floor.pdf
nikglvk
 
Odoo Companies in India – Driving Business Transformation.pdf
azhuhardeen1968
 
Internet Downloader Manager (IDM) Crack 6.42 Build 42 Updates Latest 2025
itskinga12
 
Understanding Forklifts - TECH EHS Solution
TECH EHS Solution
 
Product Update: Alluxio AI 3.7 Now with Sub-Millisecond Latency
Alluxio, Inc.
 
Embracing Complexity in Serverless! GOTO Serverless Bengaluru
SheenBrisals
 
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
OnePlan Solutions
 
Reimagine Home Health with the Power of Agentic AI​
AutomationEdge Technologies
 

Containers and Kubernetes without limits

  • 1. CONTAINERS AND KUBERNETES WITHOUT LIMITS How to make stateless containers reliable and available for stateful applications! Copenhagen, September 18th, 2018
  • 2. 2 © 2018 MapR Technologies, Inc. // MapR Confidential Today‘s Goal Learn how to build a reliable, scalable, and secure containerized platform, and to handle persistent data in large containerized environments across multiple data centers or geographic locations. (DATA)
  • 3. 4 © 2018 MapR Technologies, Inc. // MapR Confidential Agenda Introduction to Containers and Kubernetes • Architectural concepts • Use cases for containers • Container challenges How to build a Reliable, Scalable, and Secure Containerized Platform • Kubernetes • Challenges for stateful applications • #Kubernetes4Data DEMO SUMMARY Q&A SWAG J FOR YOU
  • 4. Introduction to Containers and Kubernetes
  • 5. 6 © 2018 MapR Technologies, Inc. // MapR Confidential Virtual Machines are Computers in a Box Containers are Applications in a Box
  • 6. 7 © 2018 MapR Technologies, Inc. // MapR Confidential hardware os hypervisor vm os libs app vm os libs app hardware os container libs app container libs app container libs app VM vs Container
  • 7. 8 © 2018 MapR Technologies, Inc. // MapR Confidential Pets vs Cattle - long lived - name them - care for them - ephemeral - brand them with #’s - well.. vets are expensive
  • 8. 9 © 2018 MapR Technologies, Inc. // MapR Confidential Container characteristics Light-weight, easy and quick to deploy Portable Break complex monolithic applications into smaller, modules
  • 9. 10 © 2018 MapR Technologies, Inc. // MapR Confidential Typical use cases for containers ENTERPRISE APPLICATION MODERNIZATION All Applications that can be run in containers MACHINE LEARNING Deploy portable, smart applications MICROSERVICES Create distributed applications by using containers to creating independent tasks CI/CD Continuous Integration/ Development
  • 10. 11 © 2018 MapR Technologies, Inc. // MapR Confidential Source: Marko Luksa (2017). Kubernetes in Action. Shelter Island, NY: Manning Publications Co. • Docker combined several Linux technologies into a single set of tools • Docker uses Linux ‘cgroups’ (process isolation), overlay file systems (OverlayFS/aufs), virtual networks & bridges, and virtual directory mount points to create “containers” • These tools separately have been in the Linux community for many years: Docker put them all together in a single, simple set of commands Containers & Docker
  • 11. 12 © 2018 MapR Technologies, Inc. // MapR Confidential cgroups ● cpu ● memory ● network ● etc. namespaces ● pids ● mnts ● etc. Chroot (filesystem) Isolation
  • 12. 13 © 2018 MapR Technologies, Inc. // MapR Confidential File File Read-only Layer Container Images
  • 13. 14 © 2018 MapR Technologies, Inc. // MapR Confidential File File File Read-only Layer Read-only Layer Container Images
  • 14. 15 © 2018 MapR Technologies, Inc. // MapR Confidential File File File Read-only Layer Read-only Layer Writable Layer Container Images
  • 15. 16 © 2018 MapR Technologies, Inc. // MapR Confidential File File File Container Image chroot cgroups ● cpu ● memory ● network ● etc. namespaces ● pids ● mnts ● etc. Container = Image + Isolation
  • 16. 17 © 2018 MapR Technologies, Inc. // MapR Confidential Why are containers called stateless? CONTAINER A APP BIN/LIBs APP BIN/LIBs CONTAINER B SHARED RESOURCES • Build containers with what is minimally needed for the application • Infrastructure resources are shared and assigned for each container • Container does not hold data
  • 17. 18 © 2018 MapR Technologies, Inc. // MapR Confidential What is the impact of containers being stateless? APP • Application server run as a service in container • Updating containers is not applicable. Update the docker image, stop and restart containers • Backing up a container is not needed
  • 18. 19 © 2018 MapR Technologies, Inc. // MapR Confidential Container growth – Trend towards production Source: CNCF Survey, 2018. https://www.cncf.io/blog/2018/08/29/cncf-survey-use-of-cloud-native-technologies-in-production-has-grown-over-200-percent/ • 73% of respondents are currently using containers in production today, with the remaining 27% planning to use them in the future. • 89% of respondents are currently using containers for proof of concepts, as well as testing (85%) and development (86%).
  • 19. 20 © 2018 MapR Technologies, Inc. // MapR Confidential Containers – What we know so far! • Are lightweight • Are stateless • Are portable • Targeted for developing applications • Surely moving towards production • Docker made it popular … and added a whole lot of jargon for us to learn! J
  • 20. 21 © 2018 MapR Technologies, Inc. // MapR Confidential But… Containers have a problem
  • 21. 22 © 2018 MapR Technologies, Inc. // MapR Confidential Challenges in using / deploying containers Source: CNCF Survey, 2018. https://www.cncf.io/blog/2018/08/29/cncf-survey-use-of-cloud-native-technologies-in-production-has-grown-over-200-percent/ • Cultural Changes with Development Team • Complexity • Lack of Training • Security • Monitoring • Storage • Networking
  • 22. 23 © 2018 MapR Technologies, Inc. // MapR Confidential Some of the things Docker can’t do • Monitor running containers • Handle dead containers • Move containers so utilization improves • Auto-scale container instances to handle load • Solve port mapping hell • …
  • 23. 24 © 2018 MapR Technologies, Inc. // MapR Confidential You can never get away from pets unless: • You have an environment to support cattle • You handle the problem of container state
  • 24. Kubernetes kubernetes (n.) - greek word for pilot or helm
  • 25. 26 © 2018 MapR Technologies, Inc. // MapR Confidential Now home at the CNCF! Large-scale cluster management at Google with Borg, 2015. https://ai.google/research/pubs/pub43438 Kubernetes started life as a successor to Google’s Borg project... https://www.cncf.io/ https://kubernetes.io/
  • 26. 27 © 2018 MapR Technologies, Inc. // MapR Confidential Kubernetes is an API and agents The Kubernetes API provides containers with a scheduling, configuration, network, and storage The Kubernetes runtime manages the containers
  • 27. 28 © 2018 MapR Technologies, Inc. // MapR Confidential Kubernetes – Why is it so popular? • There are many management software solutions to create, manage & delete containers with newer vendors emerging everyday • Kubernetes remains the leader with 83% (up from 77%) • The ecosystem and developer community augmented by Google’s support gave Kubernetes the edge over others Source: CNCF Survey, 2018. https://www.cncf.io/blog/2018/08/29/cncf-survey-use-of-cloud-native-technologies-in-production-has-grown-over-200-percent/
  • 28. 29 © 2018 MapR Technologies, Inc. // MapR Confidential Key K8s features that help you orchestrate containers at scale • Creating, managing multiple containers in a single entity, a POD. • Container Scheduling on same host or on particular hosts, and sharing IP address allows the containers to communicate with each other • Auto scaling, replication and recovery of containers • cAdvisor resource usage monitoring module • Network pluggable architecture • Load balancer • Health check mechanisms
  • 29. 30 © 2018 MapR Technologies, Inc. // MapR Confidential *Pods are not containers! IP address: 1.2.3.4 container #1 container #2 pod • Pods are the atomic unit of scheduling/mgmt in k8s • Pods can have multiple containers • Containers in pod have same IP • Containers in pod share IPC address space • Many advanced management features directed at pods
  • 30. 31 © 2018 MapR Technologies, Inc. // MapR Confidential api server scheduler controller manager etcd node 1 node 2 node 3 node 4 node 5 kubelet kube-proxy docker Control Plane Workers
  • 31. 32 © 2018 MapR Technologies, Inc. // MapR Confidential kubectl yaml pod pod pod pod etcd pods plans The API server turns YAML into pods and plans. api server
  • 32. 33 © 2018 MapR Technologies, Inc. // MapR Confidential Kubernetes – an open, pluggable framework
  • 33. 34 © 2018 MapR Technologies, Inc. // MapR Confidential Kubernetes – an open, pluggable framework Source: CNCF landscape, see https://github.com/cncf/landscape and http://l.cncf.io
  • 34. 35 © 2018 MapR Technologies, Inc. // MapR Confidential • From eval to prod: How a Service Mesh Helped Us Build Production Cloud-Native Services - Israel Sotomayor, Moltin o Session recording: https://youtu.be/VoYAtCVz3Ig • What Does “Production Ready” Really Mean for a Kubernetes Cluster? - Lucas Käldström, Individual o Session recording: https://youtu.be/EjSiZgGdRqk • Keynote: Running with Scissors - Liz Rice, Technology Evangelist, Aqua Security • Session recording: https://youtu.be/ltrV-Qmh3oY KubeCon: Learn from the community how to use the services
  • 35. 36 © 2018 MapR Technologies, Inc. // MapR Confidential We still have a problem
  • 36. 37 © 2018 MapR Technologies, Inc. // MapR Confidential State!
  • 37. 38 © 2018 MapR Technologies, Inc. // MapR Confidential Problem with Containers and State What about stateful applications? Real applications are stateful - they need to persist data somewhere! Pod and ReplicaSet abstract compute and memory. 1. Containers are ephemeral: no way to persist state • Container termination / crashes result in loss of data • Can’t run stateful applications 2. Containers can’t share data between each other
  • 38. 39 © 2018 MapR Technologies, Inc. // MapR Confidential Give Containers Persistent Volumes Source: CNCF Webinar - Introduction to Cloud Native Storage, https://www.youtube.com/watch?v=S9PnpnrHdIs ü Data survives (persists) beyond container, pod, host ü Workload can choose its best storage
  • 39. 40 © 2018 MapR Technologies, Inc. // MapR Confidential Persistence & Portability Source: CNCF Webinar - Introduction to Cloud Native Storage, https://www.youtube.com/watch?v=S9PnpnrHdIs
  • 40. 41 © 2018 MapR Technologies, Inc. // MapR Confidential Pod to Persistence Volume Source: CNCF Webinar - Introduction to Cloud Native Storage, https://www.youtube.com/watch?v=S9PnpnrHdIs 3
  • 41. How does MapR fit into this?
  • 42. 43 © 2018 MapR Technologies, Inc. // MapR Confidential Cloud native storage takes many forms Source: CNCF Webinar - Introduction to Cloud Native Storage, https://www.youtube.com/watch?v=S9PnpnrHdIs
  • 43. 44 © 2018 MapR Technologies, Inc. // MapR Confidential Cloud native storage takes many forms Source: CNCF Webinar - Introduction to Cloud Native Storage, https://www.youtube.com/watch?v=S9PnpnrHdIs MAPR DATA PLATFORM FILES / OBJECTS / TABLES / STREAMS APIs: NFS, POSIX, REST, S3, HDFS, HBASE, JSON, KAFKA DATA CENTER CLOUD MULTI-CLOUD EDGE KUBERNETES COMMODITY SERVER VIRTUAL MACHINE IoT & Edge AI / ML ADV. ANALYTICS ENTERPRISE APPLICATIONS Pod Pod Pod Pod
  • 44. 45 © 2018 MapR Technologies, Inc. // MapR Confidential Scale. It distributes data across the cluster and offers a global namespace for a unified view of data regardless of its physical location High Availability. Offers configurable levels of replication to ensure data durability. In event of a failure, all nodes participate to self-heal and reconstruct data automatically Data Protection. End-to-end security, per volume Access control expressions, space efficient snapshots, volume mirroring, offers several choices to build a data protection strategy Intelligent Data Placement. Offers three different storage tiers with automated storage policies to place data based on their SLAs Edge, on-premises, Cloud: Can be deployed in on-premises datacenters, edge and on the cloud Modern Data Platform
  • 45. 46 © 2018 MapR Technologies, Inc. // MapR Confidential MapR Data Container Architecture Built For Speed, Scale, Reliability Data & metadata fully distributed A A A B B B C C C D D D E E E 32 GB 256 MB 8 KB Hierarchical organization of data No single point of failure Fast parallel access Exabyte scale Full read-write
  • 46. 47 © 2018 MapR Technologies, Inc. // MapR Confidential MapR Volumes Volumes are logical units of management, holding files, directories, tables, messages. WHAT CAN YOU DO WITH VOLUMES? • Schedule snapshots • Schedule mirrors • Control data placement • Access permissions • Enforce volume quotas • Manage performance • Specify replication factor Volumes: Shared MapR Cluster r : user:sally | (group:research & group:managers) MAPR ACCESS CONTROL EXPRESSIONS /mktg /finance /projectx
  • 47. 48 © 2018 MapR Technologies, Inc. // MapR Confidential MapR Volumes Volumes are used for easy control of access, multi-tenancy, data locality & DR
  • 48. 49 © 2018 MapR Technologies, Inc. // MapR Confidential ü Global data view in a single namespace ü Distributed data processing ü Unified Security ü Global Replication For Data Distribution & DR ü Bandwidth-aware to manage global data flows ü Simplify cross cloud application development & deployment Global Namespace – common path to connect to any data Globally Protected Globally Accessible Globally Managed Globally Replicated Across Locations Across Clouds /mapr /us.mapr.com /eu_cloud.mapr.com /asia.mapr.com /us_cloud.mapr.com
  • 49. 50 © 2018 MapR Technologies, Inc. // MapR Confidential Automatically Synchronized Globally Distributed Data Topic Topic Topic On-Premises S3 EDGE DATA PLATFORM DATA PLATFORM DATA PLATFORM DATA PLATFORM Multi-Cloud Data Movement & Application Portability Enabling Application and Data Portability
  • 50. 51 © 2018 MapR Technologies, Inc. // MapR Confidential “Kubernetes is our platform for apps. We view MapR as our Kubernetes for data.” “The combination of the two allows us to support future & current workloads on any infrastructure. With SLA enforcement and the ability to move apps & data seamlessly across clouds.” CUSTOMERS #Kubernetes4Data
  • 51. 52 © 2018 MapR Technologies, Inc. // MapR Confidential MapR Persistent Application Client Container (PACC) • Pre-built, certified container image for connecting to MapR services • Secure authentication at container level, secure connection • Extensible support for application layers • Available in Docker Hub, Dockerfile for customizability MapR POSIX Client for Containers MapR Converged Client for Containers Space for Customer Application MapR PACC MAPR DATA PLATFORM EVENT DATA STREAMS ANALYTICS & ML ENGINES OPERATIONAL DATABASE CLOUD-SCALE FILE AND OBJECT STORE
  • 52. 53 © 2018 MapR Technologies, Inc. // MapR Confidential Containerized Microservices have real-time access to files/tables/streams Microservices Databases/files Microservices Databases/files Microservices Databases/files Microservices Microservices Microservices Microservices Microservices Microservices MAPR DATA PLATFORM Stream Stream
  • 53. MapR Data Fabric for Kubernetes
  • 54. 55 © 2018 MapR Technologies, Inc. // MapR Confidential MapR Data Fabric for Kubernetes node Pod Pod node Pod Pod node Pod Pod MAPR MAPR MAPR MAPR DATA PLATFORM • Integration with Kubernetes APIs, packaged and run as a POSIX client on each Kubernetes host • MapR Volumes are mounted for containers • Persist data for containerized applications • Scale data and performance as containers grow • Highly available by leveraging replicas, snapshots, mirroring of data • Benefit from MapR tickets, for end-to- end security • Multi-tenant deployment and access
  • 55. 56 © 2018 MapR Technologies, Inc. // MapR Confidential There are two ways to provision a volume: 1. Kubernetes Volume with Static Provisioner This is used to mount an existing MapR Volume to Containers managed by Kubernetes. 2. Kubernetes Persistent Volumes with Dynamic Provisioner This is used to create and mount a new MapR Volume to Containers managed by Kubernetes. Kubernetes Integration via Volume Driver Plugin
  • 56. 57 © 2018 MapR Technologies, Inc. // MapR Confidential pod kubelet docker plugin mapr fuse Example 1: You have a Postgres container that needs persistent storage. Plugin mounts MapR path via fuse Static Provisioning • Kdf volume plugin • Admin provisions • Fast, uses Posix drivers • Secured with MapR tickets • MapR cluster can be external to K8s
  • 57. 58 © 2018 MapR Technologies, Inc. // MapR Confidential KUBERNETES (CLIENT HOST) YOUR CONTAINER MAPR VOLUME PLUGIN - POSIX CLIENT K8S PERSISTANT VOLUME MAPR DATA PLATFORM 1. Request Volume 2. Mount POSIX Volume Mounting An Existing MapR Volume (Static Provisioner)
  • 58. 59 © 2018 MapR Technologies, Inc. // MapR Confidential pod kubelet docker plugin mapr fuse provisioner rest Example 2: You are testing a new container. You want the storage in MapR automatically allocated for the container. Dynamic provisioning • Kdf provisioner • Uses MapR REST API’s to allocate/delete MapR volumes • Mounting is the same as static provisioning
  • 59. 60 © 2018 MapR Technologies, Inc. // MapR Confidential KUBERNETES (CLIENT HOST) YOUR CONTAINER MAPR VOLUME PLUGIN - POSIX CLIENT K8S PERSISTANT VOLUME MAPR DATA PLATFORM 1. Request Volume 5. Mount POSIX Volume PERSISTENT VOLUME CLAIM STORAGE CLASS DYNAMIC PROVISIONER 2. Request Volume 3. Request Volume Creating A New MapR Volume (Dynamic Provisioner) Volume Claim binds the volume created to the container(s) Storage Classes used by Administrators express the type, size and other characteristics that the volume should contain
  • 60. Demo
  • 61. 62 © 2018 MapR Technologies, Inc. // MapR Confidential Demo: Run database server container in high availability Run PostgreSQL in Container Dynamically create and store the Postgres data on MapR as the persistent data store.
  • 62. 63 © 2018 MapR Technologies, Inc. // MapR Confidential Demo: Run database server container in high availability Dynamically create MapR Volume Use the Volume Driver Plugin to dynamically create a MapR Volume as the PostgreSQL data store. 1 1 Launch PostgreSQL container The PostgreSQL uses the Volume Driver plugin to locate the volume. 2 2 Simulate PostgreSQL failover The PostgreSQL container can failover without having effect on the data stored on MapR. 3 3 4 Relaunch PostgreSQL container A new PostgreSQL container will leverage the existing data on MapR. 4
  • 64. 65 © 2018 MapR Technologies, Inc. // MapR Confidential RECAP: Today‘s Goal Learn how to build a reliable, scalable, and secure containerized platform, and to handle persistent data in large containerized environments across multiple data centers or geographic locations. (DATA)
  • 65. 66 © 2018 MapR Technologies, Inc. // MapR Confidential COMPUTE AGILITY MAPR DATA PLATFORM Containers & Kubernetes without limits! MAPR KUBERNETES VOLUME PLUGIN TENANT N Application….. TENANT 1 Application APP AGILITY DATA AGILITY DATA CENTER CLOUD MULTI-CLOUD KUBERNETES EDGE ACROSS INFRASTRUCTURES ENTERPRISE APPLICATIONS AI AND ML ADV. ANALYTICS Stateful app container MAPR POSIX CLIENT FOR CONTAINERS Application
  • 67. 68 © 2018 MapR Technologies, Inc. // MapR Confidential Kubernetes (e)books
  • 68. 69 © 2018 MapR Technologies, Inc. // MapR Confidential MapR Data Fabric for Kubernetes https://mapr.com/solutions/data-fabric/kubernetes/ MapR Data Fabric for Kubernetes - Documentation https://mapr.com/docs/60/PersistentStorage/kdf_overview.html MapR Data Fabric for Kubernetes - GitHub https://github.com/mapr/KubernetesDataFabric MapR Tutorial: How to Install and Deploy Applications at Scale on K8s Part 1: https://mapr.com/blog/making-data-actionable-at-scale-part-1-of-3 Part 2: https://mapr.com/blog/making-data-actionable-at-scale-part-2-of-3 Part 3: https://mapr.com/blog/making-data-actionable-at-scale-part-3-of-3 MapR Data Fabric for Kubernetes
  • 69. 70 © 2018 MapR Technologies, Inc. // MapR Confidential MapR Academy Free Online Training: learn.mapr.com
  • 70. 71 © 2018 MapR Technologies, Inc. // MapR Confidential New O’Reilly Book! by Ted Dunning and Ellen Friedman Just released at Strata New York, September 2018 GRAB YOUR FREE COPY HERE! or download the e-book here: https://mapr.com/ebook/ai-and-analytics-in- production/