SlideShare a Scribd company logo

Open stack architecture overview-meetup-6-6_2013

Mirantis, profile picture
Mirantis

This document provides an overview of OpenStack architecture and components. It discusses the goals of OpenStack, including understanding its purpose, ecosystem, definition, history, and projects. It describes the logical architecture and components of OpenStack like Nova, Glance, Swift, Cinder, Quantum, Keystone, and Horizon. It explains how a virtual machine provisioning request flows through different OpenStack components.

Technology
1 of 188
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
© MIRANTIS 2013 PAGE 1© MIRANTIS 2013 Module 1: OpenStack Architecture Overview Mirantis, 2013
© MIRANTIS 2013 PAGE 2 Goals
© MIRANTIS 2013 PAGE 3 Goals • Understand OpenStack purpose and use cases Understand the OpenStack ecosystem • Definition • History • Projects
© MIRANTIS 2013 PAGE 4 Goals • Understand OpenStack purpose and use cases Understand the OpenStack ecosystem • Definition • History • Projects BACKGROUND
© MIRANTIS 2013 PAGE 5 Goals • Understand OpenStack purpose and use cases Understand the OpenStack ecosystem • Definition • History • Projects • Understand OpenStack architecture • Logical architecture • Provision VM request flow • Components details BACKGROUND
© MIRANTIS 2013 PAGE 6 Goals • Understand OpenStack purpose and use cases Understand the OpenStack ecosystem • Definition • History • Projects • Understand OpenStack architecture • Logical architecture • Provision VM request flow • Components details BACKGROUND THEORY
© MIRANTIS 2013 PAGE 7 What is OpenStack? As described by the OpenStack Foundation: "Open source software for building private and public clouds“
© MIRANTIS 2013 PAGE 8 OpenStack as IaaS
© MIRANTIS 2013 PAGE 9 OpenStack Capabilities
© MIRANTIS 2013 PAGE 10 OpenStack Capabilities • VMs on demand • provisioning • snapshotting
© MIRANTIS 2013 PAGE 11 OpenStack Capabilities • VMs on demand • provisioning • snapshotting • Volumes
© MIRANTIS 2013 PAGE 12 OpenStack Capabilities • VMs on demand • provisioning • snapshotting • Volumes • Networks
© MIRANTIS 2013 PAGE 13 OpenStack Capabilities • VMs on demand • provisioning • snapshotting • Volumes • Networks • Object storage for VM images and arbitrary files
© MIRANTIS 2013 PAGE 14 OpenStack Capabilities • VMs on demand • provisioning • snapshotting • Volumes • Networks • Object storage for VM images and arbitrary files • Multi-tenancy • quotas for different tenants • user can be associated with multiple tenants
© MIRANTIS 2013 PAGE 15 OpenStack History Date Rel Projects Type Note Jul 2010 N/A PoC * Rackspace Hosting & NASA joint launch Oct 2010 Austin Nova, Swift PoC Feb 2011 Bexar Nova, Glance, Swift PoC Apr 2011 Cactus Nova, Glance, Swift PoC ** 6 month development cycle starts Sep 2011 Diablo Nova, Glance, Swift Prod 1st production release (Cactus) at Internap (10/27) Apr 2012 Essex Nova, Glance, Swift, Horizon, Keystone Prod Common web UI and shared authentication mechanism added Sep 2012 Folsom Nova, Glance, Swift, Horizon, Keystone, Quantum, Cinder, Oslo Prod OpenStack Foundation Established Apr 2013 Grizzly Nova, Glance, Swift, Horizon, Keystone, Quantum, Cinder, Oslo Prod Ceilometer and Heat integration projects added Oct 2013 Havana Nova, Glance, Swift, Horizon, Keystone, Quantum, Cinder, Oslo, Heat, Ceilometer Prod Coming Soon! ** Follows similar Ubuntu 6 month release cycle * Pre-July 2010 is predicated by Rackspace Cloud Files project (Swift), NASA Nebula project (Nova)
© MIRANTIS 2013 PAGE 16 OpenStack Grizzly Projects • Core Projects: • Nova (Compute Service) • Glance (Image Service) • Quantum (Network Service) • Cinder (Block Storage Service) • Swift (Object Store Service) • Common Projects: • Keystone (Identity Service) • Horizon (Dashboard) • Library Projects: • Oslo (Shared Infrastructure Code) • Incubated Projects (Coming in Havana) • Ceilometer (Metering/Monitoring) • Heat (Orchestration)
© MIRANTIS 2013 PAGE 17 Each OpenStack Project
© MIRANTIS 2013 PAGE 18 Each OpenStack Project • Is also a “top-level” OpenStack component
© MIRANTIS 2013 PAGE 19 Each OpenStack Project • Is also a “top-level” OpenStack component • Has an elected “Project Technical Lead” (PTL)
© MIRANTIS 2013 PAGE 20 Each OpenStack Project • Is also a “top-level” OpenStack component • Has an elected “Project Technical Lead” (PTL) • Has separate developers and design teams
© MIRANTIS 2013 PAGE 21 Each OpenStack Project • Is also a “top-level” OpenStack component • Has an elected “Project Technical Lead” (PTL) • Has separate developers and design teams • Has a well defined public API • With the exception of Horizon, which is the Web GUI, all other projects have a RESTfull (JSON/HTTP) API
© MIRANTIS 2013 PAGE 22 Each OpenStack Project • Is also a “top-level” OpenStack component • Has an elected “Project Technical Lead” (PTL) • Has separate developers and design teams • Has a well defined public API • With the exception of Horizon, which is the Web GUI, all other projects have a RESTfull (JSON/HTTP) API • Has a separate database and isolated persistent layer
© MIRANTIS 2013 PAGE 23 OpenStack Projects: Begining (Cactus)
© MIRANTIS 2013 PAGE 24 OpenStack Projects: Now (Grizzly)
© MIRANTIS 2013 PAGE 25 Dev Trends • Decoupling of features (e.g. Nova-volume became Cinder) • API-s to communicate • Common generic API/Infrastructure (Oslo) • Backends & drivers (everything’s pluggable)
© MIRANTIS 2013 PAGE 26 OpenStack Projects Relationships
© MIRANTIS 2013 PAGE 27 OpenStack Projects Relationships
© MIRANTIS 2013 PAGE 28 OpenStack Projects Relationships
© MIRANTIS 2013 PAGE 29 OpenStack Projects Relationships MySQL Database Not a project, but important to understand the relationship
© MIRANTIS 2013 PAGE 30 OpenStack Projects Relationships MySQL Database Not a project, but important to understand the relationship RabbitMQ Not a project, but important to understand the relationship
© MIRANTIS 2013 PAGE 31 Deployment – Pick up What You Want • The components can be mixed & matched • Base: • Nova • Keystone • Dashboard • Glance • Mutually exclusive scenarios • Some components can conflict
© MIRANTIS 2013 PAGE 32 OpenStack: “Typical” Deployment Topology Control Cluster Horizon Keystone GlanceQuantum Cinder Cloud Ctrl. (nova) Compute Cluster compute node compute node compute node compute node compute node compute node Storage Cluster (Swift) storage node storage node storage node storage node storage node storage node
© MIRANTIS 2013 PAGE 33 OpenStack: “Typical” Deployment Topology Control Cluster Horizon Keystone GlanceQuantum Cinder Cloud Ctrl. (nova) Compute Cluster compute node compute node compute node compute node compute node compute node Storage Cluster (Swift) storage node storage node storage node storage node storage node storage node Heavy CPU and RAM
© MIRANTIS 2013 PAGE 34 OpenStack: “Typical” Deployment Topology Control Cluster Horizon Keystone GlanceQuantum Cinder Cloud Ctrl. (nova) Compute Cluster compute node compute node compute node compute node compute node compute node Storage Cluster (Swift) storage node storage node storage node storage node storage node storage node Heavy CPU and RAM Heavy Disk and I/O
© MIRANTIS 2013 PAGE 35 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell OpenStack Projects: Communication Types HTTP AMQP SQL Native API iSCSI Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI
© MIRANTIS 2013 PAGE 36 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell OpenStack Projects: Communication Types HTTP AMQP SQL Native API iSCSI Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Every OpenStack service exposes access to restful API via HTTP UI: Horizon or CLI
© MIRANTIS 2013 PAGE 37 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell OpenStack Projects: Communication Types HTTP AMQP SQL Native API iSCSI Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Every OpenStack service exposes access to restful API via HTTP UI: Horizon or CLI Each action treated as distributed transaction, state built as MQ messages
© MIRANTIS 2013 PAGE 38 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell OpenStack Projects: Communication Types HTTP AMQP SQL Native API iSCSI Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Every OpenStack service exposes access to restful API via HTTP Each service updates it’s own DB with state information as actions are performed UI: Horizon or CLI Each action treated as distributed transaction, state built as MQ messages
© MIRANTIS 2013 PAGE 39 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell OpenStack Projects: Communication Types HTTP AMQP SQL Native API iSCSI Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Every OpenStack service exposes access to restful API via HTTP Each service updates it’s own DB with state information as actions are performed UI: Horizon or CLI Each action treated as distributed transaction, state built as MQ messages Direct access calls, ex. Plugins, NetApp, Nicira, etc.
© MIRANTIS 2013 PAGE 40 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell OpenStack Projects: Communication Types HTTP AMQP SQL Native API iSCSI Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Cinder Block storage provided as iSCSI storage to VMs Every OpenStack service exposes access to restful API via HTTP Each service updates it’s own DB with state information as actions are performed UI: Horizon or CLI Each action treated as distributed transaction, state built as MQ messages Direct access calls, ex. Plugins, NetApp, Nicira, etc.
© MIRANTIS 2013 PAGE 41 OpenStack REST API • OpenStack public API is a RESTful API
© MIRANTIS 2013 PAGE 42 OpenStack REST API • OpenStack public API is a RESTful API • REST stands for Representational State Transfer
© MIRANTIS 2013 PAGE 43 OpenStack REST API • OpenStack public API is a RESTful API • REST stands for Representational State Transfer • REST is a stateless client/server protocol with a uniform interface for accessing the object model
© MIRANTIS 2013 PAGE 44 OpenStack REST API • OpenStack public API is a RESTful API • REST stands for Representational State Transfer • REST is a stateless client/server protocol with a uniform interface for accessing the object model • OpenStack RESTful API is implemented using HTTP GET/PUT/POST/DELETE in combination with JSON for data
© MIRANTIS 2013 PAGE 45 Part 1 Recap
© MIRANTIS 2013 PAGE 46 Part 1 Recap • OpenStack – open source software for building IaaS
© MIRANTIS 2013 PAGE 47 Part 1 Recap • OpenStack – open source software for building IaaS • OpenStack release cycle is every 6 month
© MIRANTIS 2013 PAGE 48 Part 1 Recap • OpenStack – open source software for building IaaS • OpenStack release cycle is every 6 month • OpenStack is an umbrella over multiple independent projects (components)
© MIRANTIS 2013 PAGE 49 Part 1 Recap • OpenStack – open source software for building IaaS • OpenStack release cycle is every 6 month • OpenStack is an umbrella over multiple independent projects (components) • All OpenStack Components talk RESTful API
© MIRANTIS 2013 PAGE 50 Part 1 Recap • OpenStack – open source software for building IaaS • OpenStack release cycle is every 6 month • OpenStack is an umbrella over multiple independent projects (components) • All OpenStack Components talk RESTful API • Most OpenStack Components have dedicated DB (SQL) and MQ (QP), some talk to 3rd party components using their native APIs
© MIRANTIS 2013 PAGE 51 Use case: Provision VM • Most common and complex process • Interacts with all OpenStack components
© MIRANTIS 2013 PAGE 52 Initial State Assumes Tenant is created, provisioning quota is available, user has an access to Horizon/CLICloud Operator, DevOp, etc. Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
© MIRANTIS 2013 PAGE 53 Step 1: Request Provisioning – From UI
© MIRANTIS 2013 PAGE 54 Step 1: Request Provisioning – From UI • Login to Horizon
© MIRANTIS 2013 PAGE 55 Step 1: Request Provisioning – From UI • Login to Horizon • Specify params of VM • VM Name • Image (OS type) • Flavor (specifies CPU, Memory, Disk) • Network (required for Folsom or later) • Optional (SSH Keys, Persistent volumes, comments, etc.)
© MIRANTIS 2013 PAGE 56 Step 1: Request Provisioning – From UI • Login to Horizon • Specify params of VM • VM Name • Image (OS type) • Flavor (specifies CPU, Memory, Disk) • Network (required for Folsom or later) • Optional (SSH Keys, Persistent volumes, comments, etc.) • Hit "Create" button
© MIRANTIS 2013 PAGE 57 What is Horizon "The OpenStack Dashboard (Horizon) provides a baseline user interface for managing OpenStack services.“
© MIRANTIS 2013 PAGE 58 Horizon Notes
© MIRANTIS 2013 PAGE 59 Horizon Notes • "Stateless“, no DB
© MIRANTIS 2013 PAGE 60 Horizon Notes • "Stateless“, no DB • Error handling is delegated to back-end
© MIRANTIS 2013 PAGE 61 Horizon Notes • "Stateless“, no DB • Error handling is delegated to back-end • Doesn't support all API functions
© MIRANTIS 2013 PAGE 62 Horizon Notes • "Stateless“, no DB • Error handling is delegated to back-end • Doesn't support all API functions • Can use memcached or database to store sessions
© MIRANTIS 2013 PAGE 63 Horizon Notes • "Stateless“, no DB • Error handling is delegated to back-end • Doesn't support all API functions • Can use memcached or database to store sessions • Gets updated via Nova API polling
© MIRANTIS 2013 PAGE 64 Horizon Internals • Subprojects • Horizon – generic Python Django libraries and components to work with REST-based back-end / restful web service • Openstack Dashboard - web app itself, exposes UI for OpenStack with styles, locale, etc. • Dashboard for each component = Individual nested Django app • Easily modifiable • Modularly developed
© MIRANTIS 2013 PAGE 65 Step 1: Request VM Provisioning via UI/CLI Cloud Operator, DevOp, etc. Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI User logs in to UI Specifies VM params: name, flavor, keys, etc. and hits "Create" button Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
© MIRANTIS 2013 PAGE 66 Step 1: Request Provisioning – Under the Hood
© MIRANTIS 2013 PAGE 67 Step 1: Request Provisioning – Under the Hood • Form params are converted to POST data
© MIRANTIS 2013 PAGE 68 Step 1: Request Provisioning – Under the Hood • Form params are converted to POST data • "Create" request initiate HTTP POST request to back-end • To Keystone if auth token is not cached – step 2
© MIRANTIS 2013 PAGE 69 What is Keystone? "Keystone provides Identity, Token, Catalog and Policy services for use specifically by projects in the OpenStack family.“ Keystone was developed by the OpenStack community but is written as a “generic” authentication / authorization mechanism for any 2 or more restful API services to communicate
© MIRANTIS 2013 PAGE 70 Keystone Architecture Deploys with it’s own DB but can also be integrated with LDAP or other EAS Contains user, role, and tenant data Contains temporary tokens Rule management interface and rule- based authorization Contains endpoint registry
© MIRANTIS 2013 PAGE 71 Keystone Data Model • User: has account credentials, is associated with one or more tenants • Tenant: unit of ownership in OpenStack, contains one or more users • Role: a first-class piece of metadata associated with many user-tenant pairs • Token: identifying credential associated with a user or user and tenant • Extras: bucket of key-value metadata associated with a user- tenant pair • Rule: describes a set of requirements for performing an action
© MIRANTIS 2013 PAGE 72 Keystone Key Concept • What service exposes • http://myservice/instances/* - GET/POST/PUT • http://myservice/images/* - GET • How RBAC mapping works • Role X in Tenant Y can do actions A,B,C • User: GET/POST/PUT on instances, images • Admin: GET/POST/PUT on tenants, users, quotas • Each API Service has it’s own RBAC enforcement through policy files
© MIRANTIS 2013 PAGE 73 Step 2: Validate Auth Data Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI Horizon sends HTTP request to Keystone. Auth info is specified in HTTP headers. Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
© MIRANTIS 2013 PAGE 74 Step 2: Validate Auth Data • Horizon sends HTTP request to Keystone • Keystone parses HTTP header info and verifies that • The credentials are valid (Authentication) • User-Tenant-Role mapping is valid (Access Control) • The requested action is available for this user (Authorization)
© MIRANTIS 2013 PAGE 75 Step 2: Validate Auth Data - Success Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI Keystone sends temporary token back to Horizon via HTTP. Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
© MIRANTIS 2013 PAGE 76 Step 1: Request Provisioning – Under the Hood • Form params are converted to POST data • "Create" request initiate HTTP POST request to back-end • To Keystone if auth token is not cached – step 2
© MIRANTIS 2013 PAGE 77 Step 1: Request Provisioning – Under the Hood • Form params are converted to POST data • "Create" request initiate HTTP POST request to back-end • To Keystone if auth token is not cached – step 2 • To Nova API if auth token hasn't expired yet – step 3
© MIRANTIS 2013 PAGE 78 Nova API “Nova API is a RESTful API web service which is used to interact with Nova"
© MIRANTIS 2013 PAGE 79 Nova API Characteristics
© MIRANTIS 2013 PAGE 80 Nova API Characteristics • Exposes REST API via HTTP
© MIRANTIS 2013 PAGE 81 Nova API Characteristics • Exposes REST API via HTTP • Provides system for managing multiple APIs on different sub-domains • EC2-compatible – Starting to be deprecated • Compute API – all innovation happens here
© MIRANTIS 2013 PAGE 82 Nova API Characteristics • Exposes REST API via HTTP • Provides system for managing multiple APIs on different sub-domains • EC2-compatible – Starting to be deprecated • Compute API – all innovation happens here • The only "allowed" way to interact with Nova
© MIRANTIS 2013 PAGE 83 Nova API Characteristics • Exposes REST API via HTTP • Provides system for managing multiple APIs on different sub-domains • EC2-compatible – Starting to be deprecated • Compute API – all innovation happens here • The only "allowed" way to interact with Nova • Stateless - HA-ready
© MIRANTIS 2013 PAGE 84 Nova API Clients Active effort in the community to make one CLI to “rule them all”, currently multiple CLIs available OpenStack dashboard is currently the only “unified” OpenStack API client
© MIRANTIS 2013 PAGE 85 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 3: Send API Request to Nova API Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI Horizon sends POST request to Nova API (signed with given token).
© MIRANTIS 2013 PAGE 86 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 4: Validate API Token Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI Nova API sends HTTP request to validate API token to Keystone.
© MIRANTIS 2013 PAGE 87 Step 4: Validate Token – Keystone API
© MIRANTIS 2013 PAGE 88 Keystone /w PKI - Token Validation • User gets one-time-password on creation • User uses it to establish a key-pair • Public key is signed and stored on Keystone • From this point user uses client certificate to login • Nova API performs offline check of the validity of token using CA&Cert it has from Keystone Keystone Key generate_cms_token( meta, keystone_key)user/pass/tenant signed_cms_token signed_cms_token Nova CA&Cert from Keystone verify(signed_cms_toke n, Cafile,certfile)
© MIRANTIS 2013 PAGE 89 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 4: Validate API Token - Sucess Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI Keystone validates API token and sends HTTP response with token acceptance/rejection info.
© MIRANTIS 2013 PAGE 90 Step 5: Process API Request Process
© MIRANTIS 2013 PAGE 91 Step 5: Process API Request Process • Validate request params • Typographical errors are verified on code level • Cloud-related params are validated via DB requests
© MIRANTIS 2013 PAGE 92 Step 5: Process API Request Process • Validate request params • Typographical errors are verified on code level • Cloud-related params are validated via DB requests • If request cannot be processed then throw an exception
© MIRANTIS 2013 PAGE 93 Step 5: Process API Request Process • Validate request params • Typographical errors are verified on code level • Cloud-related params are validated via DB requests • If request cannot be processed then throw an exception • If request can be processed • Save initial state to the Database
© MIRANTIS 2013 PAGE 94 Nova Database “Nova Database stores current state of all objects in compute cluster."
© MIRANTIS 2013 PAGE 95 Nova Database • In theory can be any relational database • Most of the deployments are done with MySQL or PostgreSQL • Nova API talks to DB via SQLAlchemy (python ORM (Object Related Mapper)) • DB HA should be done via external tools (like Galera or Multi-Master replication Model for MySQL)
© MIRANTIS 2013 PAGE 96 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 5: Process API Request Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI Nova API parses request to python object model and validates it by fetching data from Nova DB. If request is valid, it saves initia db entry about VM to the database.
© MIRANTIS 2013 PAGE 97 Step 5: Process API Request Process • Validate request params • Typographical errors are verified on code level • Cloud-related params are validated via DB requests • If request cannot be processed then throw an exception • If request can be processed • Save initial state to the database
© MIRANTIS 2013 PAGE 98 Step 5: Process API Request Process • Validate request params • Typographical errors are verified on code level • Cloud-related params are validated via DB requests • If request cannot be processed then throw an exception • If request can be processed • Save initial state to the database • Send message with next actions to MQ – step 6
© MIRANTIS 2013 PAGE 99 Message Queue "Message Queue is a unified way for collaboration between nova components."
© MIRANTIS 2013 PAGE 100 Messaging Process Example • 2 modes: • rpc.cast - don't wait for result (fire and forget) • rpc.call - wait for result (when there is something to return)
© MIRANTIS 2013 PAGE 101 Messaging Process Example • 2 modes: • rpc.cast - don't wait for result (fire and forget) • rpc.call - wait for result (when there is something to return) Ex. Nova API Ex. Nova Scheduler
© MIRANTIS 2013 PAGE 102 Messaging Process Example • 2 modes: • rpc.cast - don't wait for result (fire and forget) • rpc.call - wait for result (when there is something to return) Ex. Nova API Ex. Nova Scheduler
© MIRANTIS 2013 PAGE 103 Messaging Process Example • 2 modes: • rpc.cast - don't wait for result (fire and forget) • rpc.call - wait for result (when there is something to return) Ex. Nova API Ex. Nova Scheduler
© MIRANTIS 2013 PAGE 104 Messaging Process Example • 2 modes: • rpc.cast - don't wait for result (fire and forget) • rpc.call - wait for result (when there is something to return) Ex. Nova API Ex. Nova Scheduler
© MIRANTIS 2013 PAGE 105 Messaging Process Example • 2 modes: • rpc.cast - don't wait for result (fire and forget) • rpc.call - wait for result (when there is something to return) Ex. Nova API Ex. Nova Scheduler
© MIRANTIS 2013 PAGE 106 OpenStack Messagings Notes
© MIRANTIS 2013 PAGE 107 OpenStack Messagings Notes • Uses multiple queues within single RabbitMQ instance • Used by services to build machine state • Each compute node has a queue for scheduling
© MIRANTIS 2013 PAGE 108 OpenStack Messagings Notes • Uses multiple queues within single RabbitMQ instance • Used by services to build machine state • Each compute node has a queue for scheduling • Messages traffic is not intensive
© MIRANTIS 2013 PAGE 109 OpenStack Messagings Notes • Uses multiple queues within single RabbitMQ instance • Used by services to build machine state • Each compute node has a queue for scheduling • Messages traffic is not intensive • Doesn't send broadcast messages, e.g. for monitoring, uses API polling instead
© MIRANTIS 2013 PAGE 110 OpenStack Messagings Notes • Uses multiple queues within single RabbitMQ instance • Used by services to build machine state • Each compute node has a queue for scheduling • Messages traffic is not intensive • Doesn't send broadcast messages, e.g. for monitoring, uses API polling instead • HA should be configured separately, e.g. mirrored queues, not handled by OpenStack
© MIRANTIS 2013 PAGE 111 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 6a: Publish Provisioning Request Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Nova API makes rpc.call to Scheduler. It publishes a short message to scheduler queue with VM info. UI: Horizon or CLI Request has been validated, but no action has been taken yet, i.e. which host, IP address, etc.
© MIRANTIS 2013 PAGE 112 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 7: Pick up Provisioning Request Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Scheduler picks up the message from MQ. UI: Horizon or CLI
© MIRANTIS 2013 PAGE 113 Nova Scheduler “Nova Scheduler is a daemon, which determines, on which compute host the request should run.“ • Only provisioning time component, i.e. not like VMware’s DRS • Typically co-located with the Cloud Controller
© MIRANTIS 2013 PAGE 114 VM Scheduling: Typical Requirements • provision VM to particular host • provision VMs of the particular tenant to isolated hosts • provision all VMs on different hosts • provision VMs to "higher density" hosts
© MIRANTIS 2013 PAGE 115 Nova Scheduler: Available Schedulers Scheduler Description Behavior Chance Picks a host that is up Random Simple Picks a host that is up and has the fewest running instances Least Loaded Filter Picks the best-suited host which satisfies selected filter Custom JSON Filters Multi (Deprecated, to be replaced by cells) A scheduler that holds multiple sub-schedulers Collection of filters commonly used for multi-site or customized deployments
© MIRANTIS 2013 PAGE 116 Nova Scheduler: Filtering Affinity, Anti-affinity, etc. Eliminate inapplicable hosts
© MIRANTIS 2013 PAGE 117 Nova Scheduler: Filters Filter Description affinity Same host or different host availability zone Least cost inside selected availability zone core Least CPU core utilization ram Only return hosts with sufficient RAM json Allows simple JSON based grammar. Can be used to build custom schedulers. i/o filter out hosts with too many concurrent I/O operations compute capabilities match attributes with compute node's capabilities (e.g. CPU arch.)
© MIRANTIS 2013 PAGE 118 Nova Scheduler: Filters Filter Description aggregate specs match the attributes for the instance with those provided by aggregate image properties find compute nodes with capabilities matching image specification from glance isolated host match given image with a group of compute nodes trusted host (by Intel) finds only "attested" hosts type find only compute nodes which do not run any instances … A lot more
© MIRANTIS 2013 PAGE 119 Nova Scheduler: Filters • Filters are statically configured in nova.conf • Multiple filters can be specified • It is possible to create custom filter • Inherit from BaseHostFilter class • Override host_passes(self, host_state, filter_properties)
© MIRANTIS 2013 PAGE 120 Nova Scheduler: Filtering RAM, CPU, etc. Integer values
© MIRANTIS 2013 PAGE 121 Nova Scheduler: Weights and Costs • Cost - integer value • Every compute host can have several cost functions associated with it • If no cost functions associated - use default from nova.conf • weight = sum(costi + weigth_fni)
© MIRANTIS 2013 PAGE 122 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 8a: Schedule Provisioning Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Scheduler fetches information about the whole cluster from database, filters, and selects compute node and updates DB with its ID UI: Horizon or CLI
© MIRANTIS 2013 PAGE 123 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 8b: Provision Scheduled Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Scheduler publishes message to the compute queue (based on host ID) to trigger VM provisioning UI: Horizon or CLI
© MIRANTIS 2013 PAGE 124 VM Provisioning Algorithm • Step 9 – Nova Compute gets message from MQ and asks Nova Conductor for VM info from database • Step 10 – Nova Compute queries Quantum (previously Nova- Network in Essex) to allocate networking information • Step 11 – Nova Compute queries Cinder to allocate volume information (optional step for persistent data) • Steps 12-14 – Nova Compute fetches VM image (base OS) from Glance • Step 15 – Nova Compute passes all information about VM (in a single message) to Hypervisor and Hypervisor (KVM / Xen) creates an instance
© MIRANTIS 2013 PAGE 125 Nova Compute “Nova Compute is a worker daemon, which primarily creates and terminates VMs via Hypervisor API."
© MIRANTIS 2013 PAGE 126 Nova Compute Drivers Nova Compute XCP VM VM VMWare VM VM HyperV VM VM LPAR VM VM libvirt KVM VM VM Xen VM VM Qemu VM VM LXC VM VM Today only 1 hypervisor type per cloud instance. Libvirt / KVM is most common deployment Maintained by Citrix Maintained by VMWare Maintained by Microsoft Maintained by IBM Native or through libvirt Bare Metal VM VM Experimental at this point
© MIRANTIS 2013 PAGE 127 Nova Compute Drivers (Continued) • Functionality is not 100% similar • Exact "run_instance" flow depends on driver implementation • Most of the features are developed and tested on KVM
© MIRANTIS 2013 PAGE 128 Nova Compute Config (nova.conf on each host) • --libvirt_type • Hypervisor being used. In this deployment ‘kvm’ is specified. • --libvirt_uri • URI to use for connection to hypervisor. In this deployment ‘qemu+tcp:///system’ is specified. • --sql_connection • Database connection string in SQLAlchemy format. This is used for connecting to state database (if Nova Conductor is not used) • --rabbit_host • IP address for RabbitMQ host. Non-standard port also can be specified • --glance_host • IP address and port of Glance Image Service host. This is needed for streaming virtual boot images. • --glance_api_server • IP address and port of Glance API server. This is needed for getting virtual boot images meta-data.
© MIRANTIS 2013 PAGE 129 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 9a: Start VM Provisioning Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Nova Compute gets message from MQ UI: Horizon or CLI
© MIRANTIS 2013 PAGE 130 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 9b: Start VM Provisioning Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Nova Compute makes rpc.call to Nova Conductor for Information on VM from DB UI: Horizon or CLI
© MIRANTIS 2013 PAGE 131 Nova Conductor “The Nova Conductor service is key to completing no-db-compute"
© MIRANTIS 2013 PAGE 132 Nova Conductor Notes controller node DB nova-conductor compute node nova-compute rpc.call()
© MIRANTIS 2013 PAGE 133 Nova Conductor Notes • Eliminites remote DB access (security) controller node DB nova-conductor compute node nova-compute rpc.call()
© MIRANTIS 2013 PAGE 134 Nova Conductor Notes • Eliminites remote DB access (security) • Horizontal scalability (performance) controller node DB nova-conductor compute node nova-compute rpc.call()
© MIRANTIS 2013 PAGE 135 Nova Conductor Notes • Eliminites remote DB access (security) • Horizontal scalability (performance) • Hides DB implementation/schema from the Nova Compute (upgrades) controller node DB nova-conductor compute node nova-compute rpc.call()
© MIRANTIS 2013 PAGE 136 Nova Conductor Notes • Eliminites remote DB access (security) • Horizontal scalability (performance) • Hides DB implementation/schema from the Nova Compute (upgrades) • Possible offloading of long-running operations from other services, not just Nova Compute controller node DB nova-conductor compute node nova-compute rpc.call()
© MIRANTIS 2013 PAGE 137 Nova Conductor Notes • Eliminites remote DB access (security) • Horizontal scalability (performance) • Hides DB implementation/schema from the Nova Compute (upgrades) • Possible offloading of long-running operations from other services, not just Nova Compute • Beneficial for operations that cross multiple compute nodes (migration, resizes) controller node DB nova-conductor compute node nova-compute rpc.call()
© MIRANTIS 2013 PAGE 138 Nova Conductor Notes • Eliminites remote DB access (security) • Horizontal scalability (performance) • Hides DB implementation/schema from the Nova Compute (upgrades) • Possible offloading of long-running operations from other services, not just Nova Compute • Beneficial for operations that cross multiple compute nodes (migration, resizes) • “This is just one (major) step along the path” controller node DB nova-conductor compute node nova-compute rpc.call()
© MIRANTIS 2013 PAGE 139 VM Provisioning Algorithm • Step 9 – Nova Compute gets message from MQ and asks Nova Conductor for VM info from Nova Conductor • Step 10 – Nova Compute queries Quantum (previously Nova- Network in Essex) to allocate networking information • Step 11 – Nova Compute queries Cinder to allocate volume information (optional step for persistent data) • Steps 12-14 – Nova Compute fetches VM image (base OS) from Glance • Step 15 – Nova Compute passes all information about VM (in a single message) to Hypervisor and Hypervisor (KVM / Xen) creates an instance
© MIRANTIS 2013 PAGE 140 Quantum "network-as-a-service"
© MIRANTIS 2013 PAGE 141 Quantum Notes
© MIRANTIS 2013 PAGE 142 Quantum Notes • Provides a flexible API (POST / GET) for service providers or their tenants to manage OpenStack network topologies • Create networks, associate VMs, set routers, etc.
© MIRANTIS 2013 PAGE 143 Quantum Notes • Provides a flexible API (POST / GET) for service providers or their tenants to manage OpenStack network topologies • Create networks, associate VMs, set routers, etc. • Presents a logical API and a corresponding plug-in architecture that separates the description of network connectivity from its implementationion
© MIRANTIS 2013 PAGE 144 Quantum Notes • Provides a flexible API (POST / GET) for service providers or their tenants to manage OpenStack network topologies • Create networks, associate VMs, set routers, etc. • Presents a logical API and a corresponding plug-in architecture that separates the description of network connectivity from its implementationion • API evolves independently of the compute API, allowing to introduce more advanced network capabilities (e.g. QoS, ACLs, etc.)
© MIRANTIS 2013 PAGE 145 Quantum Notes • Provides a flexible API (POST / GET) for service providers or their tenants to manage OpenStack network topologies • Create networks, associate VMs, set routers, etc. • Presents a logical API and a corresponding plug-in architecture that separates the description of network connectivity from its implementationion • API evolves independently of the compute API, allowing to introduce more advanced network capabilities (e.g. QoS, ACLs, etc.) • In Folsom/Grizzly one can choose to stay with nova-network (Essex approach) or to go with Quantum
© MIRANTIS 2013 PAGE 146 Quantum Architecture - “Bird's Eye" View 3rd Party plug-in, networking data is stored outside of OpenStack / Quantum Quantum native functionality
© MIRANTIS 2013 PAGE 147 Network Configuration Flow • Allocate MAC addresses • Allocate IPs (for each network) • Associate IP and MAC with VM (DB) • Setup network - L2: • configure L2 via a quantum plugin • actual action can be variable, depending on the plugin used (with OVS plugin the action is: plugging an instance into the integration bridge on the hypervisor) • Setup network - L3 • Update DHCP config • Initialize gateway Allocation during cloud setup Association and Setup during VM provisioning
© MIRANTIS 2013 PAGE 148 Available Quantum Plugins • Linux Bridge • OpenVSwitch (most common) • Nicira NVP • Cisco (UCS Blade + Nexus) • Ryu OpenFlow controller • NEC ProgrammableFlow Controller
© MIRANTIS 2013 PAGE 149 Step 10: Configure Network Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Nova Compute makes a call to Quantum API to provision network for the instance UI: Horizon or CLI Keystone Keystone Server Keystone DB Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
© MIRANTIS 2013 PAGE 150 Step 10: Configure Network (Continued) Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Quantum configures IP, gateway, DNS name, L2 connectivity, etc. UI: Horizon or CLI Keystone Keystone Server Keystone DB Static Dynamic Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
© MIRANTIS 2013 PAGE 151 VM Provisioning Algorithm • Step 9 – Nova Compute gets message from MQ and asks Nova Conductor for VM info from database • Step 10 – Nova Compute queries Quantum (previously Nova- Network in Essex) to allocate networking information • Step 11 – Nova Compute queries Cinder to allocate volume information (optional step for persistent data) • Steps 12-14 – Nova Compute fetches VM image (base OS) from Glance • Step 15 – Nova Compute passes all information about VM (in a single message) to Hypervisor and Hypervisor (KVM / Xen) creates an instance
© MIRANTIS 2013 PAGE 152 Cinder "block storage as-a-service"
© MIRANTIS 2013 PAGE 153 Cinder Notes
© MIRANTIS 2013 PAGE 154 Cinder Notes • Optional
© MIRANTIS 2013 PAGE 155 Cinder Notes • Optional • iSCSI solution which can plug into a number of storage backends
© MIRANTIS 2013 PAGE 156 Cinder Notes • Optional • iSCSI solution which can plug into a number of storage backends • Volume can be attached only to 1 instance at a time
© MIRANTIS 2013 PAGE 157 Cinder Notes • Optional • iSCSI solution which can plug into a number of storage backends • Volume can be attached only to 1 instance at a time • Persistent volumes keep their state independent of instances
© MIRANTIS 2013 PAGE 158 Cinder Architecture
© MIRANTIS 2013 PAGE 159 Cinder Drivers • iSCSI • Fibre Channel • Xen Storage Manager • Nexenta • NetApp • Zadara VPSA • SAN • NFS (volumes as sparse files) • RBD Ceph • IBM Storwize / XIV • HP 3PAR • Coraid • Huawei • Scality SOFS • GlusterFS • LVM thin provisioning support • Mirrored LVM • XenAPINFS • EMC VNX/VMAX arrays • Solidfire
© MIRANTIS 2013 PAGE 160 Step 11: Request Volume Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI Keystone Keystone Server Keystone DB Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell It is assumed a volume is already created. Nova Compute contacts Cinder to get volume data. Can also attach volumes after VM is built.
© MIRANTIS 2013 PAGE 161 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 11: Request volume (Continued) Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Nova Compute sets up iSCSI initiator & instructs the Hypervisor to mount iSCSI vol. as a new block device UI: Horizon or CLI Keystone Keystone Server Keystone DB
© MIRANTIS 2013 PAGE 162 VM Provisioning Algorithm • Step 9 – Nova Compute gets message from MQ and asks Nova Conductor for VM info from database • Step 10 – Nova Compute queries Quantum (previously Nova- Network in Essex) to allocate networking information • Step 11 – Nova Compute queries Cinder to allocate volume information (optional step for persistent data) • Steps 12-14 – Nova Compute fetches VM image (base OS) from Glance • Step 15 – Nova Compute passes all information about VM (in a single message) to Hypervisor and Hypervisor (KVM / Xen) creates an instance
© MIRANTIS 2013 PAGE 163 Glance "The Glance project provides services for discovering, registering, and retrieving virtual machine images."
© MIRANTIS 2013 PAGE 164 Glance Summary • Images-as-a-Service • Can use multiple back-ends for image storage • Supports multiple image formats
© MIRANTIS 2013 PAGE 165 Glance Architecture
© MIRANTIS 2013 PAGE 166 Glance Capabilities
© MIRANTIS 2013 PAGE 167 Glance Capabilities • CRUD images (Create, Read, Update, Delete)
© MIRANTIS 2013 PAGE 168 Glance Capabilities • CRUD images (Create, Read, Update, Delete) • Search images via filters • name • container format • disk format • size_min, size_max • status
© MIRANTIS 2013 PAGE 169 Glance Capabilities • CRUD images (Create, Read, Update, Delete) • Search images via filters • name • container format • disk format • size_min, size_max • status • Caches images • uses SQLite or FS that supports xattrs for caching • queues images for prefetching • prefetches images • prunes images • cleans invalid cache entries
© MIRANTIS 2013 PAGE 170 Glance Image Formats Disk Format Description raw This is an unstructured disk image format vhd This is the VHD disk format, a common disk format used by virtual machine monitors from VMWare, Xen, Microsoft, VirtualBox, and others vmdk Another common disk format supported by many common virtual machine monitors vdi A disk format supported by VirtualBox virtual machine monitor and the QEMU emulator iso An archive format for the data contents of an optical disc (e.g. CDROM). qcow2 A disk format supported by the QEMU emulator that can expand dynamically and supports Copy on Write aki This indicates what is stored in Glance is an Amazon kernel image ari This indicates what is stored in Glance is an Amazon ramdisk image ami This indicates what is stored in Glance is an Amazon machine image
© MIRANTIS 2013 PAGE 171 Custom Image Creation • Get installation ISO • Create VM (qemu-img create) • Start VM and connect to it via VNC console • Install image without LVM • Create default iptables rules • Install and configure cloud-init • With cloud-init configure image • Prepare image for OpenStack • Extract root partition, kernel and ramdisk • cleanup • package
© MIRANTIS 2013 PAGE 172 Step 12: Request VM Image from Glance Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Nova Compute requests VM image from Glance via Image ID UI: Horizon or CLI Keystone Keystone Server Keystone DB Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
© MIRANTIS 2013 PAGE 173 Step 13: Get Image URI from Glance Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW If image with given image ID can be found - return URI – HTTP Get URI UI: Horizon or CLI Keystone Keystone Server Keystone DB Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
© MIRANTIS 2013 PAGE 174 Step 14: Download Image from Swift Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Nova Compute downloads image using URI, given by Glance, from Swif (or Glance's back-end) UI: Horizon or CLI Keystone Keystone Server Keystone DB Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
© MIRANTIS 2013 PAGE 175 VM Provisioning Algorithm • Step 9 – Nova Compute gets message from MQ and asks Nova Conductor for VM info from database • Step 10 – Nova Compute queries Quantum (previously Nova- Network in Essex) to allocate networking information • Step 11 – Nova Compute queries Cinder to allocate volume information (optional step for persistent data) • Steps 12-14 – Nova Compute fetches VM image (base OS) from Glance • Step 15 – Nova Compute passes all information about VM (in a single message) to Hypervisor and Hypervisor (KVM / Xen) creates an instance
© MIRANTIS 2013 PAGE 176 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 15: Start VM Rendering via Hypervisor Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Nova Compute fetches information about VM from DB, creates a command to Hypervisor and delegates VM rendering to Hypervisor. UI: Horizon or CLI Keystone Keystone Server Keystone DB In case of KVM / libvirtd this is a single XML VM config file
© MIRANTIS 2013 PAGE 177 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 16: VM is UP Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW iSCSI communication begins for volume UI: Horizon or CLI Keystone Keystone Server Keystone DB Nova Compute sends a message to Nova Conductor to update DB with VM state
© MIRANTIS 2013 PAGE 178 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 17: User is Happy Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Horizon polls Nova API for VM status and power state, which is taken from Database. UI: Horizon or CLI Keystone Keystone Server Keystone DB
© MIRANTIS 2013 PAGE 179 Recap:
© MIRANTIS 2013 PAGE 180 Recap: • Users logs into Horizon and initiates a VM create
© MIRANTIS 2013 PAGE 181 Recap: • Users logs into Horizon and initiates a VM create • Keystone authorizes
© MIRANTIS 2013 PAGE 182 Recap: • Users logs into Horizon and initiates a VM create • Keystone authorizes • Nova initiates provisioning and saves state to DB
© MIRANTIS 2013 PAGE 183 Recap: • Users logs into Horizon and initiates a VM create • Keystone authorizes • Nova initiates provisioning and saves state to DB • Nova Scheduler finds appropriate host
© MIRANTIS 2013 PAGE 184 Recap: • Users logs into Horizon and initiates a VM create • Keystone authorizes • Nova initiates provisioning and saves state to DB • Nova Scheduler finds appropriate host • Quantum configures networking
© MIRANTIS 2013 PAGE 185 Recap: • Users logs into Horizon and initiates a VM create • Keystone authorizes • Nova initiates provisioning and saves state to DB • Nova Scheduler finds appropriate host • Quantum configures networking • Cinder provides block device
© MIRANTIS 2013 PAGE 186 Recap: • Users logs into Horizon and initiates a VM create • Keystone authorizes • Nova initiates provisioning and saves state to DB • Nova Scheduler finds appropriate host • Quantum configures networking • Cinder provides block device • Image URI is looked up through Glance
© MIRANTIS 2013 PAGE 187 Recap: • Users logs into Horizon and initiates a VM create • Keystone authorizes • Nova initiates provisioning and saves state to DB • Nova Scheduler finds appropriate host • Quantum configures networking • Cinder provides block device • Image URI is looked up through Glance • Image is retrieved via Swift
© MIRANTIS 2013 PAGE 188 Recap: • Users logs into Horizon and initiates a VM create • Keystone authorizes • Nova initiates provisioning and saves state to DB • Nova Scheduler finds appropriate host • Quantum configures networking • Cinder provides block device • Image URI is looked up through Glance • Image is retrieved via Swift • VM is rendered

More Related Content

PPTX
Why to Cloud Native
Karthik Gaekwad
 
PDF
Webinar "Introduction to OpenStack"
CREATE-NET
 
PDF
Openstack 101
Kamesh Pemmaraju
 
PPSX
Cloud Architecture - Multi Cloud, Edge, On-Premise
Araf Karsh Hamid
 
PDF
Autoscaling Kubernetes
craigbox
 
PPTX
Cloud Computing
Naveed Farooq
 
PDF
Cloud native principles
Diego Pacheco
 
PDF
VMware NSX 101: What, Why & How
Aniekan Akpaffiong
 
Why to Cloud Native
Karthik Gaekwad
 
Webinar "Introduction to OpenStack"
CREATE-NET
 
Openstack 101
Kamesh Pemmaraju
 
Cloud Architecture - Multi Cloud, Edge, On-Premise
Araf Karsh Hamid
 
Autoscaling Kubernetes
craigbox
 
Cloud Computing
Naveed Farooq
 
Cloud native principles
Diego Pacheco
 
VMware NSX 101: What, Why & How
Aniekan Akpaffiong
 

What's hot (20)

PDF
Red Hat OpenShift -- Innovation without limitation.pdf
ssuser1490e8
 
PDF
Présentation docker et kubernetes
Kiwi Backup
 
PDF
Red Hat Insights
Alessandro Silva
 
PDF
[OpenStack Days Korea 2016] Track1 - Monasca를 이용한 Cloud 모니터링
OpenStack Korea Community
 
PPTX
An Intrudction to OpenStack 2017
Haim Ateya
 
PPTX
Cloud Computing
International School Of Management Excellence
 
PDF
Storage 101: Rook and Ceph - Open Infrastructure Denver 2019
Sean Cohen
 
PPTX
Issues of OpenStack multi-region mode
Joe Huang
 
PDF
High Availability for OpenStack
Kamesh Pemmaraju
 
PDF
Api observability
Red Hat
 
PDF
OpenShift Virtualization- Technical Overview.pdf
ssuser1490e8
 
PDF
Starting Your Modern DataOps Journey
CloverDX
 
PDF
Docker & kubernetes
NexThoughts Technologies
 
PDF
Introduction to OpenStack
Edureka!
 
PDF
Open shift 4-update
SaeidVarmazyar
 
PPTX
cloud computing Multi cloud
Dr.Neeraj Kumar Pandey
 
PDF
Ericsson NFVi solution
Ericsson
 
PDF
Introduction to Google Cloud Platform
Sujai Prakasam
 
PPTX
How to Easily Build SCADA & HMI HTML5 Web Applications
Inductive Automation
 
PDF
Data Warehouse - Incremental Migration to the Cloud
Michael Rainey
 
Red Hat OpenShift -- Innovation without limitation.pdf
ssuser1490e8
 
Présentation docker et kubernetes
Kiwi Backup
 
Red Hat Insights
Alessandro Silva
 
[OpenStack Days Korea 2016] Track1 - Monasca를 이용한 Cloud 모니터링
OpenStack Korea Community
 
An Intrudction to OpenStack 2017
Haim Ateya
 
Cloud Computing
International School Of Management Excellence
 
Storage 101: Rook and Ceph - Open Infrastructure Denver 2019
Sean Cohen
 
Issues of OpenStack multi-region mode
Joe Huang
 
High Availability for OpenStack
Kamesh Pemmaraju
 
Api observability
Red Hat
 
OpenShift Virtualization- Technical Overview.pdf
ssuser1490e8
 
Starting Your Modern DataOps Journey
CloverDX
 
Docker & kubernetes
NexThoughts Technologies
 
Introduction to OpenStack
Edureka!
 
Open shift 4-update
SaeidVarmazyar
 
cloud computing Multi cloud
Dr.Neeraj Kumar Pandey
 
Ericsson NFVi solution
Ericsson
 
Introduction to Google Cloud Platform
Sujai Prakasam
 
How to Easily Build SCADA & HMI HTML5 Web Applications
Inductive Automation
 
Data Warehouse - Incremental Migration to the Cloud
Michael Rainey
 
Ad

Viewers also liked (20)

PPT
Swift Architecture and Practice, by Alex Yang
Hui Cheng
 
PDF
OpenStack keystone identity service
openstackindia
 
PPTX
OpenStack Quantum
openstackindia
 
PDF
OpenStack Quantum
openstackindia
 
PPTX
OpenStack Quantum Network Service
Lew Tucker
 
PPTX
OpenStack Icehouse Overview
OpenStack Foundation
 
PDF
State of the Stack April 2013
Randy Bias
 
PDF
Mirantis OpenStack-DC-Meetup 17 Sept 2014
Mirantis
 
PDF
OPENSTACK CERTIFICATION BY MIRANTIS OCM50_100-375
Hosam Al Ali
 
PDF
KUBERNETES CERTIFICATION BY MIRANTIS KCM100_100-982-720
Hosam Al Ali
 
PDF
OpenStack Foundation - Certified OpenStack Administrator
Hosam Al Ali
 
PDF
OPENSTACK CERTIFICATION BY MIRANTIS OCM100_200-004-835
Hosam Al Ali
 
PPTX
QNAP COSCUP Container Station
Wu Fan-Cheng
 
PPTX
QNAP MOPCON 2015 - 輕鬆打造持續整合開發環境，使用 QNAP Docker
Wu Fan-Cheng
 
PDF
Open-E DSS V7 Active-Active Load Balanced iSCSI HA Cluster (with bonding)
open-e
 
PDF
Atlanta OpenStack 2014 Chef for OpenStack Deployment Workshop
Matt Ray
 
PDF
Pycon 2008: Python Command-line Tools *Nix
Lecturer UC Davis & Northwestern
 
PPTX
Red Hat Storage Day Seattle: Why Software-Defined Storage Matters
Red_Hat_Storage
 
PDF
OpenStack Deployment with Chef Workshop
Matt Ray
 
PDF
Modern Web App Development using ClojureScript & React.js / Baishampayan “BG”...
Ontico
 
Swift Architecture and Practice, by Alex Yang
Hui Cheng
 
OpenStack keystone identity service
openstackindia
 
OpenStack Quantum
openstackindia
 
OpenStack Quantum
openstackindia
 
OpenStack Quantum Network Service
Lew Tucker
 
OpenStack Icehouse Overview
OpenStack Foundation
 
State of the Stack April 2013
Randy Bias
 
Mirantis OpenStack-DC-Meetup 17 Sept 2014
Mirantis
 
OPENSTACK CERTIFICATION BY MIRANTIS OCM50_100-375
Hosam Al Ali
 
KUBERNETES CERTIFICATION BY MIRANTIS KCM100_100-982-720
Hosam Al Ali
 
OpenStack Foundation - Certified OpenStack Administrator
Hosam Al Ali
 
OPENSTACK CERTIFICATION BY MIRANTIS OCM100_200-004-835
Hosam Al Ali
 
QNAP COSCUP Container Station
Wu Fan-Cheng
 
QNAP MOPCON 2015 - 輕鬆打造持續整合開發環境，使用 QNAP Docker
Wu Fan-Cheng
 
Open-E DSS V7 Active-Active Load Balanced iSCSI HA Cluster (with bonding)
open-e
 
Atlanta OpenStack 2014 Chef for OpenStack Deployment Workshop
Matt Ray
 
Pycon 2008: Python Command-line Tools *Nix
Lecturer UC Davis & Northwestern
 
Red Hat Storage Day Seattle: Why Software-Defined Storage Matters
Red_Hat_Storage
 
OpenStack Deployment with Chef Workshop
Matt Ray
 
Modern Web App Development using ClojureScript & React.js / Baishampayan “BG”...
Ontico
 
Ad

Similar to Open stack architecture overview-meetup-6-6_2013 (20)

PDF
OpenStack Overview and History
Mirantis
 
PPTX
Openstack: starter level
Alessandro Martellone
 
PPTX
Introduction To OpenStack
Haim Ateya
 
PDF
What is OpenStack and the added value of IBM solutions
Sasha Lazarevic
 
PPT
OpenStack - An Overview
graziol
 
PPTX
Cloud and OpenStack
Seyed Ehsan Beheshtian
 
PPT
Openstack presentation
Sankalp Jain
 
PPTX
Oct meetup open stack 101 clean
benrodrigue
 
PDF
OpenStack 101 update
Kamesh Pemmaraju
 
PDF
Openstack Pakistan intro
Affan Syed
 
PDF
Cloud Architect Alliance #15: Openstack
Microsoft
 
PPTX
Introduction to openstack
Yaniv Zadka
 
PDF
Chef and OpenStack Workshop from ChefConf 2013
Matt Ray
 
PPTX
Cloud computing and OpenStack
Edgar Magana
 
PPTX
OpenStack Architecture and Use Cases
Jalal Mostafa
 
PPTX
Open stack Architecture and Use Cases
Ahmad Tfaily
 
PDF
Bitnami Bootcamp. OpenStack
Alberto Molina Coballes
 
PPTX
Power of OpenStack & Hadoop
Tuan Yang
 
PPTX
Quick overview of Openstack architecture
Toni Ramirez
 
PDF
OpenStack Workshop - WECode Harvard Conference
Iccha Sethi
 
OpenStack Overview and History
Mirantis
 
Openstack: starter level
Alessandro Martellone
 
Introduction To OpenStack
Haim Ateya
 
What is OpenStack and the added value of IBM solutions
Sasha Lazarevic
 
OpenStack - An Overview
graziol
 
Cloud and OpenStack
Seyed Ehsan Beheshtian
 
Openstack presentation
Sankalp Jain
 
Oct meetup open stack 101 clean
benrodrigue
 
OpenStack 101 update
Kamesh Pemmaraju
 
Openstack Pakistan intro
Affan Syed
 
Cloud Architect Alliance #15: Openstack
Microsoft
 
Introduction to openstack
Yaniv Zadka
 
Chef and OpenStack Workshop from ChefConf 2013
Matt Ray
 
Cloud computing and OpenStack
Edgar Magana
 
OpenStack Architecture and Use Cases
Jalal Mostafa
 
Open stack Architecture and Use Cases
Ahmad Tfaily
 
Bitnami Bootcamp. OpenStack
Alberto Molina Coballes
 
Power of OpenStack & Hadoop
Tuan Yang
 
Quick overview of Openstack architecture
Toni Ramirez
 
OpenStack Workshop - WECode Harvard Conference
Iccha Sethi
 

More from Mirantis (20)

PDF
How to Accelerate Your Application Delivery Process on Top of Kubernetes Usin...
Mirantis
 
PDF
Kubernetes Security Workshop
Mirantis
 
PDF
Using Kubernetes to make cellular data plans cheaper for 50M users
Mirantis
 
PDF
How to Build a Basic Edge Cloud
Mirantis
 
PDF
Securing Your Containers is Not Enough: How to Encrypt Container Data
Mirantis
 
PDF
What's New in Kubernetes 1.18 Webinar Slides
Mirantis
 
PDF
Comparison of Current Service Mesh Architectures
Mirantis
 
PDF
Your Application Deserves Better than Kubernetes Ingress: Istio vs. Kubernetes
Mirantis
 
PDF
Demystifying Cloud Security Compliance
Mirantis
 
PDF
Mirantis life
Mirantis
 
PDF
OpenStack and the IoT: Where we are, where we're going, what we need to get t...
Mirantis
 
PDF
Boris Renski: OpenStack Summit Keynote Austin 2016
Mirantis
 
PPTX
Digital Disciplines: Attaining Market Leadership through the Cloud
Mirantis
 
PPTX
Decomposing Lithium's Monolith with Kubernetes and OpenStack
Mirantis
 
PPTX
OpenStack: Changing the Face of Service Delivery
Mirantis
 
PPTX
Accelerating the Next 10,000 Clouds
Mirantis
 
PPTX
Containers for the Enterprise: It's Not That Simple
Mirantis
 
PPTX
Protecting Yourself from the Container Shakeout
Mirantis
 
PPTX
It's Not the Technology, It's You
Mirantis
 
PDF
OpenStack as the Platform for Innovation
Mirantis
 
How to Accelerate Your Application Delivery Process on Top of Kubernetes Usin...
Mirantis
 
Kubernetes Security Workshop
Mirantis
 
Using Kubernetes to make cellular data plans cheaper for 50M users
Mirantis
 
How to Build a Basic Edge Cloud
Mirantis
 
Securing Your Containers is Not Enough: How to Encrypt Container Data
Mirantis
 
What's New in Kubernetes 1.18 Webinar Slides
Mirantis
 
Comparison of Current Service Mesh Architectures
Mirantis
 
Your Application Deserves Better than Kubernetes Ingress: Istio vs. Kubernetes
Mirantis
 
Demystifying Cloud Security Compliance
Mirantis
 
Mirantis life
Mirantis
 
OpenStack and the IoT: Where we are, where we're going, what we need to get t...
Mirantis
 
Boris Renski: OpenStack Summit Keynote Austin 2016
Mirantis
 
Digital Disciplines: Attaining Market Leadership through the Cloud
Mirantis
 
Decomposing Lithium's Monolith with Kubernetes and OpenStack
Mirantis
 
OpenStack: Changing the Face of Service Delivery
Mirantis
 
Accelerating the Next 10,000 Clouds
Mirantis
 
Containers for the Enterprise: It's Not That Simple
Mirantis
 
Protecting Yourself from the Container Shakeout
Mirantis
 
It's Not the Technology, It's You
Mirantis
 
OpenStack as the Platform for Innovation
Mirantis
 

Recently uploaded (20)

PDF
Accuracy of neural networks in brain wave diagnosis of schizophrenia
IAESIJAI
 
PDF
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
PDF
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
PPTX
A Presentation on Artificial Intelligence
memembruh336
 
PDF
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
IAESIJAI
 
PPT
Teaching material agriculture food technology
LiaRayya
 
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
muhammadmuneebnaeem7
 
PDF
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
PPTX
Group 1 Presentation -Planning and Decision Making .pptx
MatthewLewis227954
 
PPTX
Programs and apps: productivity, graphics, security and other tools
4mqw9zch22
 
PDF
Encapsulation theory and applications.pdf
gurumoop
 
PDF
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
PDF
Getting Started with Data Integration: FME Form 101
Safe Software
 
PDF
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
PDF
Assigned Numbers - 2025 - Bluetooth® Document
Bloombase
 
Accuracy of neural networks in brain wave diagnosis of schizophrenia
IAESIJAI
 
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
A Presentation on Artificial Intelligence
memembruh336
 
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
Diabetes mellitus diagnosis method based random forest with bat algorithm
IAESIJAI
 
Teaching material agriculture food technology
LiaRayya
 
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
muhammadmuneebnaeem7
 
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
Group 1 Presentation -Planning and Decision Making .pptx
MatthewLewis227954
 
Programs and apps: productivity, graphics, security and other tools
4mqw9zch22
 
Encapsulation theory and applications.pdf
gurumoop
 
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
Getting Started with Data Integration: FME Form 101
Safe Software
 
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
Assigned Numbers - 2025 - Bluetooth® Document
Bloombase
 

Open stack architecture overview-meetup-6-6_2013

  • 1. © MIRANTIS 2013 PAGE 1© MIRANTIS 2013 Module 1: OpenStack Architecture Overview Mirantis, 2013
  • 2. © MIRANTIS 2013 PAGE 2 Goals
  • 3. © MIRANTIS 2013 PAGE 3 Goals • Understand OpenStack purpose and use cases Understand the OpenStack ecosystem • Definition • History • Projects
  • 4. © MIRANTIS 2013 PAGE 4 Goals • Understand OpenStack purpose and use cases Understand the OpenStack ecosystem • Definition • History • Projects BACKGROUND
  • 5. © MIRANTIS 2013 PAGE 5 Goals • Understand OpenStack purpose and use cases Understand the OpenStack ecosystem • Definition • History • Projects • Understand OpenStack architecture • Logical architecture • Provision VM request flow • Components details BACKGROUND
  • 6. © MIRANTIS 2013 PAGE 6 Goals • Understand OpenStack purpose and use cases Understand the OpenStack ecosystem • Definition • History • Projects • Understand OpenStack architecture • Logical architecture • Provision VM request flow • Components details BACKGROUND THEORY
  • 7. © MIRANTIS 2013 PAGE 7 What is OpenStack? As described by the OpenStack Foundation: "Open source software for building private and public clouds“
  • 8. © MIRANTIS 2013 PAGE 8 OpenStack as IaaS
  • 9. © MIRANTIS 2013 PAGE 9 OpenStack Capabilities
  • 10. © MIRANTIS 2013 PAGE 10 OpenStack Capabilities • VMs on demand • provisioning • snapshotting
  • 11. © MIRANTIS 2013 PAGE 11 OpenStack Capabilities • VMs on demand • provisioning • snapshotting • Volumes
  • 12. © MIRANTIS 2013 PAGE 12 OpenStack Capabilities • VMs on demand • provisioning • snapshotting • Volumes • Networks
  • 13. © MIRANTIS 2013 PAGE 13 OpenStack Capabilities • VMs on demand • provisioning • snapshotting • Volumes • Networks • Object storage for VM images and arbitrary files
  • 14. © MIRANTIS 2013 PAGE 14 OpenStack Capabilities • VMs on demand • provisioning • snapshotting • Volumes • Networks • Object storage for VM images and arbitrary files • Multi-tenancy • quotas for different tenants • user can be associated with multiple tenants
  • 15. © MIRANTIS 2013 PAGE 15 OpenStack History Date Rel Projects Type Note Jul 2010 N/A PoC * Rackspace Hosting & NASA joint launch Oct 2010 Austin Nova, Swift PoC Feb 2011 Bexar Nova, Glance, Swift PoC Apr 2011 Cactus Nova, Glance, Swift PoC ** 6 month development cycle starts Sep 2011 Diablo Nova, Glance, Swift Prod 1st production release (Cactus) at Internap (10/27) Apr 2012 Essex Nova, Glance, Swift, Horizon, Keystone Prod Common web UI and shared authentication mechanism added Sep 2012 Folsom Nova, Glance, Swift, Horizon, Keystone, Quantum, Cinder, Oslo Prod OpenStack Foundation Established Apr 2013 Grizzly Nova, Glance, Swift, Horizon, Keystone, Quantum, Cinder, Oslo Prod Ceilometer and Heat integration projects added Oct 2013 Havana Nova, Glance, Swift, Horizon, Keystone, Quantum, Cinder, Oslo, Heat, Ceilometer Prod Coming Soon! ** Follows similar Ubuntu 6 month release cycle * Pre-July 2010 is predicated by Rackspace Cloud Files project (Swift), NASA Nebula project (Nova)
  • 16. © MIRANTIS 2013 PAGE 16 OpenStack Grizzly Projects • Core Projects: • Nova (Compute Service) • Glance (Image Service) • Quantum (Network Service) • Cinder (Block Storage Service) • Swift (Object Store Service) • Common Projects: • Keystone (Identity Service) • Horizon (Dashboard) • Library Projects: • Oslo (Shared Infrastructure Code) • Incubated Projects (Coming in Havana) • Ceilometer (Metering/Monitoring) • Heat (Orchestration)
  • 17. © MIRANTIS 2013 PAGE 17 Each OpenStack Project
  • 18. © MIRANTIS 2013 PAGE 18 Each OpenStack Project • Is also a “top-level” OpenStack component
  • 19. © MIRANTIS 2013 PAGE 19 Each OpenStack Project • Is also a “top-level” OpenStack component • Has an elected “Project Technical Lead” (PTL)
  • 20. © MIRANTIS 2013 PAGE 20 Each OpenStack Project • Is also a “top-level” OpenStack component • Has an elected “Project Technical Lead” (PTL) • Has separate developers and design teams
  • 21. © MIRANTIS 2013 PAGE 21 Each OpenStack Project • Is also a “top-level” OpenStack component • Has an elected “Project Technical Lead” (PTL) • Has separate developers and design teams • Has a well defined public API • With the exception of Horizon, which is the Web GUI, all other projects have a RESTfull (JSON/HTTP) API
  • 22. © MIRANTIS 2013 PAGE 22 Each OpenStack Project • Is also a “top-level” OpenStack component • Has an elected “Project Technical Lead” (PTL) • Has separate developers and design teams • Has a well defined public API • With the exception of Horizon, which is the Web GUI, all other projects have a RESTfull (JSON/HTTP) API • Has a separate database and isolated persistent layer
  • 23. © MIRANTIS 2013 PAGE 23 OpenStack Projects: Begining (Cactus)
  • 24. © MIRANTIS 2013 PAGE 24 OpenStack Projects: Now (Grizzly)
  • 25. © MIRANTIS 2013 PAGE 25 Dev Trends • Decoupling of features (e.g. Nova-volume became Cinder) • API-s to communicate • Common generic API/Infrastructure (Oslo) • Backends & drivers (everything’s pluggable)
  • 26. © MIRANTIS 2013 PAGE 26 OpenStack Projects Relationships
  • 27. © MIRANTIS 2013 PAGE 27 OpenStack Projects Relationships
  • 28. © MIRANTIS 2013 PAGE 28 OpenStack Projects Relationships
  • 29. © MIRANTIS 2013 PAGE 29 OpenStack Projects Relationships MySQL Database Not a project, but important to understand the relationship
  • 30. © MIRANTIS 2013 PAGE 30 OpenStack Projects Relationships MySQL Database Not a project, but important to understand the relationship RabbitMQ Not a project, but important to understand the relationship
  • 31. © MIRANTIS 2013 PAGE 31 Deployment – Pick up What You Want • The components can be mixed & matched • Base: • Nova • Keystone • Dashboard • Glance • Mutually exclusive scenarios • Some components can conflict
  • 32. © MIRANTIS 2013 PAGE 32 OpenStack: “Typical” Deployment Topology Control Cluster Horizon Keystone GlanceQuantum Cinder Cloud Ctrl. (nova) Compute Cluster compute node compute node compute node compute node compute node compute node Storage Cluster (Swift) storage node storage node storage node storage node storage node storage node
  • 33. © MIRANTIS 2013 PAGE 33 OpenStack: “Typical” Deployment Topology Control Cluster Horizon Keystone GlanceQuantum Cinder Cloud Ctrl. (nova) Compute Cluster compute node compute node compute node compute node compute node compute node Storage Cluster (Swift) storage node storage node storage node storage node storage node storage node Heavy CPU and RAM
  • 34. © MIRANTIS 2013 PAGE 34 OpenStack: “Typical” Deployment Topology Control Cluster Horizon Keystone GlanceQuantum Cinder Cloud Ctrl. (nova) Compute Cluster compute node compute node compute node compute node compute node compute node Storage Cluster (Swift) storage node storage node storage node storage node storage node storage node Heavy CPU and RAM Heavy Disk and I/O
  • 35. © MIRANTIS 2013 PAGE 35 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell OpenStack Projects: Communication Types HTTP AMQP SQL Native API iSCSI Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI
  • 36. © MIRANTIS 2013 PAGE 36 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell OpenStack Projects: Communication Types HTTP AMQP SQL Native API iSCSI Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Every OpenStack service exposes access to restful API via HTTP UI: Horizon or CLI
  • 37. © MIRANTIS 2013 PAGE 37 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell OpenStack Projects: Communication Types HTTP AMQP SQL Native API iSCSI Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Every OpenStack service exposes access to restful API via HTTP UI: Horizon or CLI Each action treated as distributed transaction, state built as MQ messages
  • 38. © MIRANTIS 2013 PAGE 38 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell OpenStack Projects: Communication Types HTTP AMQP SQL Native API iSCSI Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Every OpenStack service exposes access to restful API via HTTP Each service updates it’s own DB with state information as actions are performed UI: Horizon or CLI Each action treated as distributed transaction, state built as MQ messages
  • 39. © MIRANTIS 2013 PAGE 39 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell OpenStack Projects: Communication Types HTTP AMQP SQL Native API iSCSI Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Every OpenStack service exposes access to restful API via HTTP Each service updates it’s own DB with state information as actions are performed UI: Horizon or CLI Each action treated as distributed transaction, state built as MQ messages Direct access calls, ex. Plugins, NetApp, Nicira, etc.
  • 40. © MIRANTIS 2013 PAGE 40 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell OpenStack Projects: Communication Types HTTP AMQP SQL Native API iSCSI Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Cinder Block storage provided as iSCSI storage to VMs Every OpenStack service exposes access to restful API via HTTP Each service updates it’s own DB with state information as actions are performed UI: Horizon or CLI Each action treated as distributed transaction, state built as MQ messages Direct access calls, ex. Plugins, NetApp, Nicira, etc.
  • 41. © MIRANTIS 2013 PAGE 41 OpenStack REST API • OpenStack public API is a RESTful API
  • 42. © MIRANTIS 2013 PAGE 42 OpenStack REST API • OpenStack public API is a RESTful API • REST stands for Representational State Transfer
  • 43. © MIRANTIS 2013 PAGE 43 OpenStack REST API • OpenStack public API is a RESTful API • REST stands for Representational State Transfer • REST is a stateless client/server protocol with a uniform interface for accessing the object model
  • 44. © MIRANTIS 2013 PAGE 44 OpenStack REST API • OpenStack public API is a RESTful API • REST stands for Representational State Transfer • REST is a stateless client/server protocol with a uniform interface for accessing the object model • OpenStack RESTful API is implemented using HTTP GET/PUT/POST/DELETE in combination with JSON for data
  • 45. © MIRANTIS 2013 PAGE 45 Part 1 Recap
  • 46. © MIRANTIS 2013 PAGE 46 Part 1 Recap • OpenStack – open source software for building IaaS
  • 47. © MIRANTIS 2013 PAGE 47 Part 1 Recap • OpenStack – open source software for building IaaS • OpenStack release cycle is every 6 month
  • 48. © MIRANTIS 2013 PAGE 48 Part 1 Recap • OpenStack – open source software for building IaaS • OpenStack release cycle is every 6 month • OpenStack is an umbrella over multiple independent projects (components)
  • 49. © MIRANTIS 2013 PAGE 49 Part 1 Recap • OpenStack – open source software for building IaaS • OpenStack release cycle is every 6 month • OpenStack is an umbrella over multiple independent projects (components) • All OpenStack Components talk RESTful API
  • 50. © MIRANTIS 2013 PAGE 50 Part 1 Recap • OpenStack – open source software for building IaaS • OpenStack release cycle is every 6 month • OpenStack is an umbrella over multiple independent projects (components) • All OpenStack Components talk RESTful API • Most OpenStack Components have dedicated DB (SQL) and MQ (QP), some talk to 3rd party components using their native APIs
  • 51. © MIRANTIS 2013 PAGE 51 Use case: Provision VM • Most common and complex process • Interacts with all OpenStack components
  • 52. © MIRANTIS 2013 PAGE 52 Initial State Assumes Tenant is created, provisioning quota is available, user has an access to Horizon/CLICloud Operator, DevOp, etc. Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
  • 53. © MIRANTIS 2013 PAGE 53 Step 1: Request Provisioning – From UI
  • 54. © MIRANTIS 2013 PAGE 54 Step 1: Request Provisioning – From UI • Login to Horizon
  • 55. © MIRANTIS 2013 PAGE 55 Step 1: Request Provisioning – From UI • Login to Horizon • Specify params of VM • VM Name • Image (OS type) • Flavor (specifies CPU, Memory, Disk) • Network (required for Folsom or later) • Optional (SSH Keys, Persistent volumes, comments, etc.)
  • 56. © MIRANTIS 2013 PAGE 56 Step 1: Request Provisioning – From UI • Login to Horizon • Specify params of VM • VM Name • Image (OS type) • Flavor (specifies CPU, Memory, Disk) • Network (required for Folsom or later) • Optional (SSH Keys, Persistent volumes, comments, etc.) • Hit "Create" button
  • 57. © MIRANTIS 2013 PAGE 57 What is Horizon "The OpenStack Dashboard (Horizon) provides a baseline user interface for managing OpenStack services.“
  • 58. © MIRANTIS 2013 PAGE 58 Horizon Notes
  • 59. © MIRANTIS 2013 PAGE 59 Horizon Notes • "Stateless“, no DB
  • 60. © MIRANTIS 2013 PAGE 60 Horizon Notes • "Stateless“, no DB • Error handling is delegated to back-end
  • 61. © MIRANTIS 2013 PAGE 61 Horizon Notes • "Stateless“, no DB • Error handling is delegated to back-end • Doesn't support all API functions
  • 62. © MIRANTIS 2013 PAGE 62 Horizon Notes • "Stateless“, no DB • Error handling is delegated to back-end • Doesn't support all API functions • Can use memcached or database to store sessions
  • 63. © MIRANTIS 2013 PAGE 63 Horizon Notes • "Stateless“, no DB • Error handling is delegated to back-end • Doesn't support all API functions • Can use memcached or database to store sessions • Gets updated via Nova API polling
  • 64. © MIRANTIS 2013 PAGE 64 Horizon Internals • Subprojects • Horizon – generic Python Django libraries and components to work with REST-based back-end / restful web service • Openstack Dashboard - web app itself, exposes UI for OpenStack with styles, locale, etc. • Dashboard for each component = Individual nested Django app • Easily modifiable • Modularly developed
  • 65. © MIRANTIS 2013 PAGE 65 Step 1: Request VM Provisioning via UI/CLI Cloud Operator, DevOp, etc. Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI User logs in to UI Specifies VM params: name, flavor, keys, etc. and hits "Create" button Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
  • 66. © MIRANTIS 2013 PAGE 66 Step 1: Request Provisioning – Under the Hood
  • 67. © MIRANTIS 2013 PAGE 67 Step 1: Request Provisioning – Under the Hood • Form params are converted to POST data
  • 68. © MIRANTIS 2013 PAGE 68 Step 1: Request Provisioning – Under the Hood • Form params are converted to POST data • "Create" request initiate HTTP POST request to back-end • To Keystone if auth token is not cached – step 2
  • 69. © MIRANTIS 2013 PAGE 69 What is Keystone? "Keystone provides Identity, Token, Catalog and Policy services for use specifically by projects in the OpenStack family.“ Keystone was developed by the OpenStack community but is written as a “generic” authentication / authorization mechanism for any 2 or more restful API services to communicate
  • 70. © MIRANTIS 2013 PAGE 70 Keystone Architecture Deploys with it’s own DB but can also be integrated with LDAP or other EAS Contains user, role, and tenant data Contains temporary tokens Rule management interface and rule- based authorization Contains endpoint registry
  • 71. © MIRANTIS 2013 PAGE 71 Keystone Data Model • User: has account credentials, is associated with one or more tenants • Tenant: unit of ownership in OpenStack, contains one or more users • Role: a first-class piece of metadata associated with many user-tenant pairs • Token: identifying credential associated with a user or user and tenant • Extras: bucket of key-value metadata associated with a user- tenant pair • Rule: describes a set of requirements for performing an action
  • 72. © MIRANTIS 2013 PAGE 72 Keystone Key Concept • What service exposes • http://myservice/instances/* - GET/POST/PUT • http://myservice/images/* - GET • How RBAC mapping works • Role X in Tenant Y can do actions A,B,C • User: GET/POST/PUT on instances, images • Admin: GET/POST/PUT on tenants, users, quotas • Each API Service has it’s own RBAC enforcement through policy files
  • 73. © MIRANTIS 2013 PAGE 73 Step 2: Validate Auth Data Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI Horizon sends HTTP request to Keystone. Auth info is specified in HTTP headers. Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
  • 74. © MIRANTIS 2013 PAGE 74 Step 2: Validate Auth Data • Horizon sends HTTP request to Keystone • Keystone parses HTTP header info and verifies that • The credentials are valid (Authentication) • User-Tenant-Role mapping is valid (Access Control) • The requested action is available for this user (Authorization)
  • 75. © MIRANTIS 2013 PAGE 75 Step 2: Validate Auth Data - Success Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI Keystone sends temporary token back to Horizon via HTTP. Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
  • 76. © MIRANTIS 2013 PAGE 76 Step 1: Request Provisioning – Under the Hood • Form params are converted to POST data • "Create" request initiate HTTP POST request to back-end • To Keystone if auth token is not cached – step 2
  • 77. © MIRANTIS 2013 PAGE 77 Step 1: Request Provisioning – Under the Hood • Form params are converted to POST data • "Create" request initiate HTTP POST request to back-end • To Keystone if auth token is not cached – step 2 • To Nova API if auth token hasn't expired yet – step 3
  • 78. © MIRANTIS 2013 PAGE 78 Nova API “Nova API is a RESTful API web service which is used to interact with Nova"
  • 79. © MIRANTIS 2013 PAGE 79 Nova API Characteristics
  • 80. © MIRANTIS 2013 PAGE 80 Nova API Characteristics • Exposes REST API via HTTP
  • 81. © MIRANTIS 2013 PAGE 81 Nova API Characteristics • Exposes REST API via HTTP • Provides system for managing multiple APIs on different sub-domains • EC2-compatible – Starting to be deprecated • Compute API – all innovation happens here
  • 82. © MIRANTIS 2013 PAGE 82 Nova API Characteristics • Exposes REST API via HTTP • Provides system for managing multiple APIs on different sub-domains • EC2-compatible – Starting to be deprecated • Compute API – all innovation happens here • The only "allowed" way to interact with Nova
  • 83. © MIRANTIS 2013 PAGE 83 Nova API Characteristics • Exposes REST API via HTTP • Provides system for managing multiple APIs on different sub-domains • EC2-compatible – Starting to be deprecated • Compute API – all innovation happens here • The only "allowed" way to interact with Nova • Stateless - HA-ready
  • 84. © MIRANTIS 2013 PAGE 84 Nova API Clients Active effort in the community to make one CLI to “rule them all”, currently multiple CLIs available OpenStack dashboard is currently the only “unified” OpenStack API client
  • 85. © MIRANTIS 2013 PAGE 85 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 3: Send API Request to Nova API Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI Horizon sends POST request to Nova API (signed with given token).
  • 86. © MIRANTIS 2013 PAGE 86 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 4: Validate API Token Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI Nova API sends HTTP request to validate API token to Keystone.
  • 87. © MIRANTIS 2013 PAGE 87 Step 4: Validate Token – Keystone API
  • 88. © MIRANTIS 2013 PAGE 88 Keystone /w PKI - Token Validation • User gets one-time-password on creation • User uses it to establish a key-pair • Public key is signed and stored on Keystone • From this point user uses client certificate to login • Nova API performs offline check of the validity of token using CA&Cert it has from Keystone Keystone Key generate_cms_token( meta, keystone_key)user/pass/tenant signed_cms_token signed_cms_token Nova CA&Cert from Keystone verify(signed_cms_toke n, Cafile,certfile)
  • 89. © MIRANTIS 2013 PAGE 89 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 4: Validate API Token - Sucess Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI Keystone validates API token and sends HTTP response with token acceptance/rejection info.
  • 90. © MIRANTIS 2013 PAGE 90 Step 5: Process API Request Process
  • 91. © MIRANTIS 2013 PAGE 91 Step 5: Process API Request Process • Validate request params • Typographical errors are verified on code level • Cloud-related params are validated via DB requests
  • 92. © MIRANTIS 2013 PAGE 92 Step 5: Process API Request Process • Validate request params • Typographical errors are verified on code level • Cloud-related params are validated via DB requests • If request cannot be processed then throw an exception
  • 93. © MIRANTIS 2013 PAGE 93 Step 5: Process API Request Process • Validate request params • Typographical errors are verified on code level • Cloud-related params are validated via DB requests • If request cannot be processed then throw an exception • If request can be processed • Save initial state to the Database
  • 94. © MIRANTIS 2013 PAGE 94 Nova Database “Nova Database stores current state of all objects in compute cluster."
  • 95. © MIRANTIS 2013 PAGE 95 Nova Database • In theory can be any relational database • Most of the deployments are done with MySQL or PostgreSQL • Nova API talks to DB via SQLAlchemy (python ORM (Object Related Mapper)) • DB HA should be done via external tools (like Galera or Multi-Master replication Model for MySQL)
  • 96. © MIRANTIS 2013 PAGE 96 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 5: Process API Request Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI Nova API parses request to python object model and validates it by fetching data from Nova DB. If request is valid, it saves initia db entry about VM to the database.
  • 97. © MIRANTIS 2013 PAGE 97 Step 5: Process API Request Process • Validate request params • Typographical errors are verified on code level • Cloud-related params are validated via DB requests • If request cannot be processed then throw an exception • If request can be processed • Save initial state to the database
  • 98. © MIRANTIS 2013 PAGE 98 Step 5: Process API Request Process • Validate request params • Typographical errors are verified on code level • Cloud-related params are validated via DB requests • If request cannot be processed then throw an exception • If request can be processed • Save initial state to the database • Send message with next actions to MQ – step 6
  • 99. © MIRANTIS 2013 PAGE 99 Message Queue "Message Queue is a unified way for collaboration between nova components."
  • 100. © MIRANTIS 2013 PAGE 100 Messaging Process Example • 2 modes: • rpc.cast - don't wait for result (fire and forget) • rpc.call - wait for result (when there is something to return)
  • 101. © MIRANTIS 2013 PAGE 101 Messaging Process Example • 2 modes: • rpc.cast - don't wait for result (fire and forget) • rpc.call - wait for result (when there is something to return) Ex. Nova API Ex. Nova Scheduler
  • 102. © MIRANTIS 2013 PAGE 102 Messaging Process Example • 2 modes: • rpc.cast - don't wait for result (fire and forget) • rpc.call - wait for result (when there is something to return) Ex. Nova API Ex. Nova Scheduler
  • 103. © MIRANTIS 2013 PAGE 103 Messaging Process Example • 2 modes: • rpc.cast - don't wait for result (fire and forget) • rpc.call - wait for result (when there is something to return) Ex. Nova API Ex. Nova Scheduler
  • 104. © MIRANTIS 2013 PAGE 104 Messaging Process Example • 2 modes: • rpc.cast - don't wait for result (fire and forget) • rpc.call - wait for result (when there is something to return) Ex. Nova API Ex. Nova Scheduler
  • 105. © MIRANTIS 2013 PAGE 105 Messaging Process Example • 2 modes: • rpc.cast - don't wait for result (fire and forget) • rpc.call - wait for result (when there is something to return) Ex. Nova API Ex. Nova Scheduler
  • 106. © MIRANTIS 2013 PAGE 106 OpenStack Messagings Notes
  • 107. © MIRANTIS 2013 PAGE 107 OpenStack Messagings Notes • Uses multiple queues within single RabbitMQ instance • Used by services to build machine state • Each compute node has a queue for scheduling
  • 108. © MIRANTIS 2013 PAGE 108 OpenStack Messagings Notes • Uses multiple queues within single RabbitMQ instance • Used by services to build machine state • Each compute node has a queue for scheduling • Messages traffic is not intensive
  • 109. © MIRANTIS 2013 PAGE 109 OpenStack Messagings Notes • Uses multiple queues within single RabbitMQ instance • Used by services to build machine state • Each compute node has a queue for scheduling • Messages traffic is not intensive • Doesn't send broadcast messages, e.g. for monitoring, uses API polling instead
  • 110. © MIRANTIS 2013 PAGE 110 OpenStack Messagings Notes • Uses multiple queues within single RabbitMQ instance • Used by services to build machine state • Each compute node has a queue for scheduling • Messages traffic is not intensive • Doesn't send broadcast messages, e.g. for monitoring, uses API polling instead • HA should be configured separately, e.g. mirrored queues, not handled by OpenStack
  • 111. © MIRANTIS 2013 PAGE 111 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 6a: Publish Provisioning Request Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Nova API makes rpc.call to Scheduler. It publishes a short message to scheduler queue with VM info. UI: Horizon or CLI Request has been validated, but no action has been taken yet, i.e. which host, IP address, etc.
  • 112. © MIRANTIS 2013 PAGE 112 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 7: Pick up Provisioning Request Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Scheduler picks up the message from MQ. UI: Horizon or CLI
  • 113. © MIRANTIS 2013 PAGE 113 Nova Scheduler “Nova Scheduler is a daemon, which determines, on which compute host the request should run.“ • Only provisioning time component, i.e. not like VMware’s DRS • Typically co-located with the Cloud Controller
  • 114. © MIRANTIS 2013 PAGE 114 VM Scheduling: Typical Requirements • provision VM to particular host • provision VMs of the particular tenant to isolated hosts • provision all VMs on different hosts • provision VMs to "higher density" hosts
  • 115. © MIRANTIS 2013 PAGE 115 Nova Scheduler: Available Schedulers Scheduler Description Behavior Chance Picks a host that is up Random Simple Picks a host that is up and has the fewest running instances Least Loaded Filter Picks the best-suited host which satisfies selected filter Custom JSON Filters Multi (Deprecated, to be replaced by cells) A scheduler that holds multiple sub-schedulers Collection of filters commonly used for multi-site or customized deployments
  • 116. © MIRANTIS 2013 PAGE 116 Nova Scheduler: Filtering Affinity, Anti-affinity, etc. Eliminate inapplicable hosts
  • 117. © MIRANTIS 2013 PAGE 117 Nova Scheduler: Filters Filter Description affinity Same host or different host availability zone Least cost inside selected availability zone core Least CPU core utilization ram Only return hosts with sufficient RAM json Allows simple JSON based grammar. Can be used to build custom schedulers. i/o filter out hosts with too many concurrent I/O operations compute capabilities match attributes with compute node's capabilities (e.g. CPU arch.)
  • 118. © MIRANTIS 2013 PAGE 118 Nova Scheduler: Filters Filter Description aggregate specs match the attributes for the instance with those provided by aggregate image properties find compute nodes with capabilities matching image specification from glance isolated host match given image with a group of compute nodes trusted host (by Intel) finds only "attested" hosts type find only compute nodes which do not run any instances … A lot more
  • 119. © MIRANTIS 2013 PAGE 119 Nova Scheduler: Filters • Filters are statically configured in nova.conf • Multiple filters can be specified • It is possible to create custom filter • Inherit from BaseHostFilter class • Override host_passes(self, host_state, filter_properties)
  • 120. © MIRANTIS 2013 PAGE 120 Nova Scheduler: Filtering RAM, CPU, etc. Integer values
  • 121. © MIRANTIS 2013 PAGE 121 Nova Scheduler: Weights and Costs • Cost - integer value • Every compute host can have several cost functions associated with it • If no cost functions associated - use default from nova.conf • weight = sum(costi + weigth_fni)
  • 122. © MIRANTIS 2013 PAGE 122 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 8a: Schedule Provisioning Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Scheduler fetches information about the whole cluster from database, filters, and selects compute node and updates DB with its ID UI: Horizon or CLI
  • 123. © MIRANTIS 2013 PAGE 123 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 8b: Provision Scheduled Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Scheduler publishes message to the compute queue (based on host ID) to trigger VM provisioning UI: Horizon or CLI
  • 124. © MIRANTIS 2013 PAGE 124 VM Provisioning Algorithm • Step 9 – Nova Compute gets message from MQ and asks Nova Conductor for VM info from database • Step 10 – Nova Compute queries Quantum (previously Nova- Network in Essex) to allocate networking information • Step 11 – Nova Compute queries Cinder to allocate volume information (optional step for persistent data) • Steps 12-14 – Nova Compute fetches VM image (base OS) from Glance • Step 15 – Nova Compute passes all information about VM (in a single message) to Hypervisor and Hypervisor (KVM / Xen) creates an instance
  • 125. © MIRANTIS 2013 PAGE 125 Nova Compute “Nova Compute is a worker daemon, which primarily creates and terminates VMs via Hypervisor API."
  • 126. © MIRANTIS 2013 PAGE 126 Nova Compute Drivers Nova Compute XCP VM VM VMWare VM VM HyperV VM VM LPAR VM VM libvirt KVM VM VM Xen VM VM Qemu VM VM LXC VM VM Today only 1 hypervisor type per cloud instance. Libvirt / KVM is most common deployment Maintained by Citrix Maintained by VMWare Maintained by Microsoft Maintained by IBM Native or through libvirt Bare Metal VM VM Experimental at this point
  • 127. © MIRANTIS 2013 PAGE 127 Nova Compute Drivers (Continued) • Functionality is not 100% similar • Exact "run_instance" flow depends on driver implementation • Most of the features are developed and tested on KVM
  • 128. © MIRANTIS 2013 PAGE 128 Nova Compute Config (nova.conf on each host) • --libvirt_type • Hypervisor being used. In this deployment ‘kvm’ is specified. • --libvirt_uri • URI to use for connection to hypervisor. In this deployment ‘qemu+tcp:///system’ is specified. • --sql_connection • Database connection string in SQLAlchemy format. This is used for connecting to state database (if Nova Conductor is not used) • --rabbit_host • IP address for RabbitMQ host. Non-standard port also can be specified • --glance_host • IP address and port of Glance Image Service host. This is needed for streaming virtual boot images. • --glance_api_server • IP address and port of Glance API server. This is needed for getting virtual boot images meta-data.
  • 129. © MIRANTIS 2013 PAGE 129 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 9a: Start VM Provisioning Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Nova Compute gets message from MQ UI: Horizon or CLI
  • 130. © MIRANTIS 2013 PAGE 130 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 9b: Start VM Provisioning Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Keystone Keystone Server Keystone DB Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Nova Compute makes rpc.call to Nova Conductor for Information on VM from DB UI: Horizon or CLI
  • 131. © MIRANTIS 2013 PAGE 131 Nova Conductor “The Nova Conductor service is key to completing no-db-compute"
  • 132. © MIRANTIS 2013 PAGE 132 Nova Conductor Notes controller node DB nova-conductor compute node nova-compute rpc.call()
  • 133. © MIRANTIS 2013 PAGE 133 Nova Conductor Notes • Eliminites remote DB access (security) controller node DB nova-conductor compute node nova-compute rpc.call()
  • 134. © MIRANTIS 2013 PAGE 134 Nova Conductor Notes • Eliminites remote DB access (security) • Horizontal scalability (performance) controller node DB nova-conductor compute node nova-compute rpc.call()
  • 135. © MIRANTIS 2013 PAGE 135 Nova Conductor Notes • Eliminites remote DB access (security) • Horizontal scalability (performance) • Hides DB implementation/schema from the Nova Compute (upgrades) controller node DB nova-conductor compute node nova-compute rpc.call()
  • 136. © MIRANTIS 2013 PAGE 136 Nova Conductor Notes • Eliminites remote DB access (security) • Horizontal scalability (performance) • Hides DB implementation/schema from the Nova Compute (upgrades) • Possible offloading of long-running operations from other services, not just Nova Compute controller node DB nova-conductor compute node nova-compute rpc.call()
  • 137. © MIRANTIS 2013 PAGE 137 Nova Conductor Notes • Eliminites remote DB access (security) • Horizontal scalability (performance) • Hides DB implementation/schema from the Nova Compute (upgrades) • Possible offloading of long-running operations from other services, not just Nova Compute • Beneficial for operations that cross multiple compute nodes (migration, resizes) controller node DB nova-conductor compute node nova-compute rpc.call()
  • 138. © MIRANTIS 2013 PAGE 138 Nova Conductor Notes • Eliminites remote DB access (security) • Horizontal scalability (performance) • Hides DB implementation/schema from the Nova Compute (upgrades) • Possible offloading of long-running operations from other services, not just Nova Compute • Beneficial for operations that cross multiple compute nodes (migration, resizes) • “This is just one (major) step along the path” controller node DB nova-conductor compute node nova-compute rpc.call()
  • 139. © MIRANTIS 2013 PAGE 139 VM Provisioning Algorithm • Step 9 – Nova Compute gets message from MQ and asks Nova Conductor for VM info from Nova Conductor • Step 10 – Nova Compute queries Quantum (previously Nova- Network in Essex) to allocate networking information • Step 11 – Nova Compute queries Cinder to allocate volume information (optional step for persistent data) • Steps 12-14 – Nova Compute fetches VM image (base OS) from Glance • Step 15 – Nova Compute passes all information about VM (in a single message) to Hypervisor and Hypervisor (KVM / Xen) creates an instance
  • 140. © MIRANTIS 2013 PAGE 140 Quantum "network-as-a-service"
  • 141. © MIRANTIS 2013 PAGE 141 Quantum Notes
  • 142. © MIRANTIS 2013 PAGE 142 Quantum Notes • Provides a flexible API (POST / GET) for service providers or their tenants to manage OpenStack network topologies • Create networks, associate VMs, set routers, etc.
  • 143. © MIRANTIS 2013 PAGE 143 Quantum Notes • Provides a flexible API (POST / GET) for service providers or their tenants to manage OpenStack network topologies • Create networks, associate VMs, set routers, etc. • Presents a logical API and a corresponding plug-in architecture that separates the description of network connectivity from its implementationion
  • 144. © MIRANTIS 2013 PAGE 144 Quantum Notes • Provides a flexible API (POST / GET) for service providers or their tenants to manage OpenStack network topologies • Create networks, associate VMs, set routers, etc. • Presents a logical API and a corresponding plug-in architecture that separates the description of network connectivity from its implementationion • API evolves independently of the compute API, allowing to introduce more advanced network capabilities (e.g. QoS, ACLs, etc.)
  • 145. © MIRANTIS 2013 PAGE 145 Quantum Notes • Provides a flexible API (POST / GET) for service providers or their tenants to manage OpenStack network topologies • Create networks, associate VMs, set routers, etc. • Presents a logical API and a corresponding plug-in architecture that separates the description of network connectivity from its implementationion • API evolves independently of the compute API, allowing to introduce more advanced network capabilities (e.g. QoS, ACLs, etc.) • In Folsom/Grizzly one can choose to stay with nova-network (Essex approach) or to go with Quantum
  • 146. © MIRANTIS 2013 PAGE 146 Quantum Architecture - “Bird's Eye" View 3rd Party plug-in, networking data is stored outside of OpenStack / Quantum Quantum native functionality
  • 147. © MIRANTIS 2013 PAGE 147 Network Configuration Flow • Allocate MAC addresses • Allocate IPs (for each network) • Associate IP and MAC with VM (DB) • Setup network - L2: • configure L2 via a quantum plugin • actual action can be variable, depending on the plugin used (with OVS plugin the action is: plugging an instance into the integration bridge on the hypervisor) • Setup network - L3 • Update DHCP config • Initialize gateway Allocation during cloud setup Association and Setup during VM provisioning
  • 148. © MIRANTIS 2013 PAGE 148 Available Quantum Plugins • Linux Bridge • OpenVSwitch (most common) • Nicira NVP • Cisco (UCS Blade + Nexus) • Ryu OpenFlow controller • NEC ProgrammableFlow Controller
  • 149. © MIRANTIS 2013 PAGE 149 Step 10: Configure Network Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Nova Compute makes a call to Quantum API to provision network for the instance UI: Horizon or CLI Keystone Keystone Server Keystone DB Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
  • 150. © MIRANTIS 2013 PAGE 150 Step 10: Configure Network (Continued) Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Quantum configures IP, gateway, DNS name, L2 connectivity, etc. UI: Horizon or CLI Keystone Keystone Server Keystone DB Static Dynamic Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
  • 151. © MIRANTIS 2013 PAGE 151 VM Provisioning Algorithm • Step 9 – Nova Compute gets message from MQ and asks Nova Conductor for VM info from database • Step 10 – Nova Compute queries Quantum (previously Nova- Network in Essex) to allocate networking information • Step 11 – Nova Compute queries Cinder to allocate volume information (optional step for persistent data) • Steps 12-14 – Nova Compute fetches VM image (base OS) from Glance • Step 15 – Nova Compute passes all information about VM (in a single message) to Hypervisor and Hypervisor (KVM / Xen) creates an instance
  • 152. © MIRANTIS 2013 PAGE 152 Cinder "block storage as-a-service"
  • 153. © MIRANTIS 2013 PAGE 153 Cinder Notes
  • 154. © MIRANTIS 2013 PAGE 154 Cinder Notes • Optional
  • 155. © MIRANTIS 2013 PAGE 155 Cinder Notes • Optional • iSCSI solution which can plug into a number of storage backends
  • 156. © MIRANTIS 2013 PAGE 156 Cinder Notes • Optional • iSCSI solution which can plug into a number of storage backends • Volume can be attached only to 1 instance at a time
  • 157. © MIRANTIS 2013 PAGE 157 Cinder Notes • Optional • iSCSI solution which can plug into a number of storage backends • Volume can be attached only to 1 instance at a time • Persistent volumes keep their state independent of instances
  • 158. © MIRANTIS 2013 PAGE 158 Cinder Architecture
  • 159. © MIRANTIS 2013 PAGE 159 Cinder Drivers • iSCSI • Fibre Channel • Xen Storage Manager • Nexenta • NetApp • Zadara VPSA • SAN • NFS (volumes as sparse files) • RBD Ceph • IBM Storwize / XIV • HP 3PAR • Coraid • Huawei • Scality SOFS • GlusterFS • LVM thin provisioning support • Mirrored LVM • XenAPINFS • EMC VNX/VMAX arrays • Solidfire
  • 160. © MIRANTIS 2013 PAGE 160 Step 11: Request Volume Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW UI: Horizon or CLI Keystone Keystone Server Keystone DB Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell It is assumed a volume is already created. Nova Compute contacts Cinder to get volume data. Can also attach volumes after VM is built.
  • 161. © MIRANTIS 2013 PAGE 161 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 11: Request volume (Continued) Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Nova Compute sets up iSCSI initiator & instructs the Hypervisor to mount iSCSI vol. as a new block device UI: Horizon or CLI Keystone Keystone Server Keystone DB
  • 162. © MIRANTIS 2013 PAGE 162 VM Provisioning Algorithm • Step 9 – Nova Compute gets message from MQ and asks Nova Conductor for VM info from database • Step 10 – Nova Compute queries Quantum (previously Nova- Network in Essex) to allocate networking information • Step 11 – Nova Compute queries Cinder to allocate volume information (optional step for persistent data) • Steps 12-14 – Nova Compute fetches VM image (base OS) from Glance • Step 15 – Nova Compute passes all information about VM (in a single message) to Hypervisor and Hypervisor (KVM / Xen) creates an instance
  • 163. © MIRANTIS 2013 PAGE 163 Glance "The Glance project provides services for discovering, registering, and retrieving virtual machine images."
  • 164. © MIRANTIS 2013 PAGE 164 Glance Summary • Images-as-a-Service • Can use multiple back-ends for image storage • Supports multiple image formats
  • 165. © MIRANTIS 2013 PAGE 165 Glance Architecture
  • 166. © MIRANTIS 2013 PAGE 166 Glance Capabilities
  • 167. © MIRANTIS 2013 PAGE 167 Glance Capabilities • CRUD images (Create, Read, Update, Delete)
  • 168. © MIRANTIS 2013 PAGE 168 Glance Capabilities • CRUD images (Create, Read, Update, Delete) • Search images via filters • name • container format • disk format • size_min, size_max • status
  • 169. © MIRANTIS 2013 PAGE 169 Glance Capabilities • CRUD images (Create, Read, Update, Delete) • Search images via filters • name • container format • disk format • size_min, size_max • status • Caches images • uses SQLite or FS that supports xattrs for caching • queues images for prefetching • prefetches images • prunes images • cleans invalid cache entries
  • 170. © MIRANTIS 2013 PAGE 170 Glance Image Formats Disk Format Description raw This is an unstructured disk image format vhd This is the VHD disk format, a common disk format used by virtual machine monitors from VMWare, Xen, Microsoft, VirtualBox, and others vmdk Another common disk format supported by many common virtual machine monitors vdi A disk format supported by VirtualBox virtual machine monitor and the QEMU emulator iso An archive format for the data contents of an optical disc (e.g. CDROM). qcow2 A disk format supported by the QEMU emulator that can expand dynamically and supports Copy on Write aki This indicates what is stored in Glance is an Amazon kernel image ari This indicates what is stored in Glance is an Amazon ramdisk image ami This indicates what is stored in Glance is an Amazon machine image
  • 171. © MIRANTIS 2013 PAGE 171 Custom Image Creation • Get installation ISO • Create VM (qemu-img create) • Start VM and connect to it via VNC console • Install image without LVM • Create default iptables rules • Install and configure cloud-init • With cloud-init configure image • Prepare image for OpenStack • Extract root partition, kernel and ramdisk • cleanup • package
  • 172. © MIRANTIS 2013 PAGE 172 Step 12: Request VM Image from Glance Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Nova Compute requests VM image from Glance via Image ID UI: Horizon or CLI Keystone Keystone Server Keystone DB Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
  • 173. © MIRANTIS 2013 PAGE 173 Step 13: Get Image URI from Glance Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW If image with given image ID can be found - return URI – HTTP Get URI UI: Horizon or CLI Keystone Keystone Server Keystone DB Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
  • 174. © MIRANTIS 2013 PAGE 174 Step 14: Download Image from Swift Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Nova Compute downloads image using URI, given by Glance, from Swif (or Glance's back-end) UI: Horizon or CLI Keystone Keystone Server Keystone DB Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell
  • 175. © MIRANTIS 2013 PAGE 175 VM Provisioning Algorithm • Step 9 – Nova Compute gets message from MQ and asks Nova Conductor for VM info from database • Step 10 – Nova Compute queries Quantum (previously Nova- Network in Essex) to allocate networking information • Step 11 – Nova Compute queries Cinder to allocate volume information (optional step for persistent data) • Steps 12-14 – Nova Compute fetches VM image (base OS) from Glance • Step 15 – Nova Compute passes all information about VM (in a single message) to Hypervisor and Hypervisor (KVM / Xen) creates an instance
  • 176. © MIRANTIS 2013 PAGE 176 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 15: Start VM Rendering via Hypervisor Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Nova Compute fetches information about VM from DB, creates a command to Hypervisor and delegates VM rendering to Hypervisor. UI: Horizon or CLI Keystone Keystone Server Keystone DB In case of KVM / libvirtd this is a single XML VM config file
  • 177. © MIRANTIS 2013 PAGE 177 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 16: VM is UP Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW iSCSI communication begins for volume UI: Horizon or CLI Keystone Keystone Server Keystone DB Nova Compute sends a message to Nova Conductor to update DB with VM state
  • 178. © MIRANTIS 2013 PAGE 178 Nova Nova DB Queue Nova: ControllerNova API Scheduler Conductor Nova Cell Step 17: User is Happy Swift Object Store Proxy Server Quantum Quantum DB Plugin / Agent Quantum Server Cinder Cinder API Cinder DB Scheduler Queue Cinder Vol Glance Glance API Glance Registry Glance DB Compute Node Hypervisor Network VM Nova: Computenova-compute Block Storage Node Storage Network Node DHCP / IPAM Router / GW Horizon polls Nova API for VM status and power state, which is taken from Database. UI: Horizon or CLI Keystone Keystone Server Keystone DB
  • 179. © MIRANTIS 2013 PAGE 179 Recap:
  • 180. © MIRANTIS 2013 PAGE 180 Recap: • Users logs into Horizon and initiates a VM create
  • 181. © MIRANTIS 2013 PAGE 181 Recap: • Users logs into Horizon and initiates a VM create • Keystone authorizes
  • 182. © MIRANTIS 2013 PAGE 182 Recap: • Users logs into Horizon and initiates a VM create • Keystone authorizes • Nova initiates provisioning and saves state to DB
  • 183. © MIRANTIS 2013 PAGE 183 Recap: • Users logs into Horizon and initiates a VM create • Keystone authorizes • Nova initiates provisioning and saves state to DB • Nova Scheduler finds appropriate host
  • 184. © MIRANTIS 2013 PAGE 184 Recap: • Users logs into Horizon and initiates a VM create • Keystone authorizes • Nova initiates provisioning and saves state to DB • Nova Scheduler finds appropriate host • Quantum configures networking
  • 185. © MIRANTIS 2013 PAGE 185 Recap: • Users logs into Horizon and initiates a VM create • Keystone authorizes • Nova initiates provisioning and saves state to DB • Nova Scheduler finds appropriate host • Quantum configures networking • Cinder provides block device
  • 186. © MIRANTIS 2013 PAGE 186 Recap: • Users logs into Horizon and initiates a VM create • Keystone authorizes • Nova initiates provisioning and saves state to DB • Nova Scheduler finds appropriate host • Quantum configures networking • Cinder provides block device • Image URI is looked up through Glance
  • 187. © MIRANTIS 2013 PAGE 187 Recap: • Users logs into Horizon and initiates a VM create • Keystone authorizes • Nova initiates provisioning and saves state to DB • Nova Scheduler finds appropriate host • Quantum configures networking • Cinder provides block device • Image URI is looked up through Glance • Image is retrieved via Swift
  • 188. © MIRANTIS 2013 PAGE 188 Recap: • Users logs into Horizon and initiates a VM create • Keystone authorizes • Nova initiates provisioning and saves state to DB • Nova Scheduler finds appropriate host • Quantum configures networking • Cinder provides block device • Image URI is looked up through Glance • Image is retrieved via Swift • VM is rendered