Deploying OpenStack Object Storage (Swift)
5 likes3,235 views
This document provides an overview of OpenStack Object Storage (Swift). Swift is open source software for creating scalable object storage using clusters of standardized servers to store petabytes of accessible data. It uses a RESTful HTTP API to store and retrieve blobs of data and metadata. The Swift architecture uses three different entities - accounts, containers, and objects - distributed across multiple zones and partitions for redundancy. Services like the proxy server, storage nodes, and replicators work together to provide a redundant and scalable storage solution.
Deploying OpenStack Object Storage (Swift)
- 1. Deploying OpenStack Object Storage Juan J. Martínez <[email protected]> 10-11 September 2011 http://www.usebox.net/jjm/ University Club, Oxford
- 2. What is OpenStack? Open Source software for “building clouds” ● OpenStack Object Storage (Swift) Storage ● OpenStack Compute (Nova) ● OpenStack Image Service (Glance) IaaS ● Incubating projects: Keystone, Dashboard, Crowbar, ... Image Service PaaS?
- 3. What is Swift? Open Source software for creating redundant, scalable object storage using clusters of standardized servers to store petabytes of accessible data. Object Storage Real File System ● Not a real file system ● open, read, write, seek, close ● RESTful HTTP API ● chdir, mkdir, rmdir ● Blob and metadata (name, content ● remove, rename type, date-time, size, MD5) ● chmod, chown, access ● Support for pseudo-hierarchical ● etc directories
- 4. Swift Architecture 3 Different Listing entities: Zone 3 Replicas In DIFFERENT Zones Accounts Containers Objects Partitions Zone Listing The Ring: location of an entity in the cluster Zone ● Three rings (were intended for the Elves) ● Weights can be used to balance the distribution ● Three different logical levels Drives
- 5. Swift Services ● proxy server ● account server ● container server ● object server Proxy Server ● replicators (rsync based) ● updaters (container, object) ● auditors ● reappers (account, container) Storage Node Storage Node Storage Node Storage Node
- 6. Swift Authentication (pluggable) Client Request (Token) User/Password Token Different solutions: Public Auth Proxy Server ● devauth (obsolete) ● swauth (external project) Private Auth ● tempauth (testing) ● keystone (the future!) Storage Node Storage Node Token? Storage Node Storage Node DMZ
- 7. Swift Interfaces Client OpenStack API HTTP (Rackspace Cloud Files) FTP/SFTP CDN FTP/SFTP Public Auth Proxy Server Private Auth Other clients: Storage Node Storage Node S3 Proxy Storage Node Storage Node https://github.com/chmouel/ftp-cloudfs https://github.com/Memset/sftpcloudfs DMZ
- 8. Swift Operations ● Managing the rings (adding/removing devices, zones, search for devices, rebalance the ring) ● Upgrading services (one zone at a time) ● Handling driver failure (unmount; optionally remove it from the ring, mount a new EMPTY drive) ● Zone failure (temporal: nothing!) ● Detecting failing disks (device audit) ● Object auditor (manually after a system crash) What could possibly go wrong?
- 9. Any questions? Thanks for your attention! http://openstack.org/ Juan J. Martínez <[email protected]> 10-11 September 2011 http://www.usebox.net/jjm/ University Club, Oxford