Shootout at the AWS Corral
Download as ODP, PDF4 likes3,297 views
The document summarizes the results of benchmarking and comparing the performance of PostgreSQL databases hosted on Amazon EC2, RDS, and Heroku. It finds that EC2 provides the most configuration options but requires more management, RDS offers simplified deployment but less configuration options, and Heroku requires no management but has limited configuration and higher costs. Benchmark results show EC2 performing best for raw performance while RDS and Heroku trade off some performance for manageability. Heroku was the most expensive option.
Shootout at the AWS Corral
- 1. Shootout at the AWS Corral EC2 RDS Heroku Josh Berkus PostgreSQL Experts Inc. SCALE 13
- 4. Thanks!
- 5. What is Amazon Web Services?
- 6. Magic? Image by Sperlingsmaedchen. Free for non-profit use only.
- 7. a bunch of servers
- 10. … and a great API with stuff
- 11. The Good ● Fast deployment – new servers in minutes, with a script ● Easy scale-out – add replicas in minutes ● Minimize ops staff – no HW wranglers
- 12. ¢heap (at the low end)
- 13. ex$pensive (at the high end)
- 14. The Bad ● Low system resources – VMs are small/slow ● Security – more attack vectors
- 16. The Ugly ● Everything is shared – Network – IO / Storage – CPU (partly) Your performance depends on somebody else's peak load.
- 19. ephemeral cloud ● DR is not optional ● your virtual DB server will go away ● you need replicas & backup
- 20. why Postgres?
- 21. transactional DB workout ● works CPU ● works RAM ● works IO ● works network IO at the same time, in parallel
- 22. Cast of Three The Gunslinger Roll-Your-Own The Rancher RDS The Dandy Heroku
- 23. The Gunslinger The Gunslinger Roll Your Own on EC2
- 24. Become a Gunslinger I. Create an EC2 instance II. Install PostgreSQL on it III. Configure PostgreSQL
- 25. Roll-Your-Own ++ ● cheapest option ● highly configurable ● install whatever you want – version – extensions
- 26. Roll-Your-Own -- ● you still do all the admin – installation – backup/redundancy – updates – OS updates ● configuration required
- 27. A Fistful of Services ● AMIs – “clone” your database server setup ● AWS's other services – caching, queueing, s3 storage, etc.
- 28. Instance Types ● m3.* general-purpose ● c3.* small, CPU-bound DBs $$ ● r3.* maximize caching $$$ ● i2.* Data warehousing $$$$
- 29. instance tips ● Use m3.* if you don't know what to use ● Get enough RAM to cache your database if you can
- 30. Storage Types ● EBS + Provisioned IOPS – large size, latency issues – reliable, snapshots – choose “EBS optimized” ● General SSD – better for bursts (about 20% better) – high variability
- 31. Storage Types ● Instance Storage + SSD option – low-latency, limited size – risky: data loss, data corruption – just for “running with scissors”
- 33. PrIOPS fallacy ● not a guarantee, a limit – but mostly pretty consistent ● each IOP is no more than 8K ● random access → each page is an IOP – no real prefetch ● PrIOPS ~ rows/second
- 34. Stuff to Set Up ● Backup: WAL-E to S3 ● Replication: not optional – in another Availbility Zone ● Monitoring for instance failure ● Secure your instance – SSL – pg_hba.conf
- 35. Configuration Tips ● random_page_cost = 1.5 ● wal_buffers = 32 to 64MB ● stats_temp_directory = /mnt/tmpfs/stats ● synchronous_commit = off – if you can afford it
- 36. Junior Gunslinger ● “small” “economical” ● m3.medium ● 1 core 3.75GB RAM ● 40GB + 1000 PrIOPS
- 37. Senior Gunslinger ● “large” “performance” ● r3.2xlarge ● 8 cores 61GB RAM ● 200GB + 4000 PrIOPS
- 38. Junior Gunslinger Cost instance: $36.50/month EBS PrIOPS: $105/month S3 Archive: $5/month X2 for replica == $288.00 a month (+ misc charges)
- 39. Cheaper Gunslinger instance: $36.50/month EBS PrIOPS: $105/month S3 Archive: $5/month no replica == $146.50 a month (+ misc charges)
- 40. Senior Gunslinger archive-only: $760.70/month with replica: $1509.40/month (+ misc charges)
- 42. Ranching 101 1. Go to AWS RDS 2. Choose “PostgreSQL” 3. Select instance size and storage 4. Launch 5. Connect over port 5432
- 43. RDS ++ ● Simpler deployment ● AWS manages updates, uptime ● Easy replicas ● Double redundancy – multizone warm standby OR replicas – regular DB snapshots
- 44. RDS -- ● Limited extensions ● 9.3 only – and not promptly updated ● No shell access ● Still might have to configure Postgres
- 45. Rancher equipment ● Integration with some AWS sevices – caching – S3 snapshotting – plus regular access to other services ● 2 dozen extensions available
- 46. RDS Options ● Instance types: – same m3.* and r3.* options – no c3 or i3 instances currently ● again, get enough RAM ● All storage is EBS – take PrIOPS storage options
- 47. RDS redundancy ● Do Multi-AZ instances or replication – Multi-AZ: automated failover – Replication: better performance ● Set up auto DB snapshots – automatically deleted snapshots?
- 48. RDS configuration ● Same as Roll-Your-Own – except: be cautious, defaults are OK ● Except fewer security options
- 49. RDS Cost Small, single: $ 184.35 Small, redundant: $ 358.70 Large, single: $1119.35 Large, redundant: $2342.70 (+ misc charges)
- 51. Doing the Dandy 1. Choose “Create Database” 2. Pick a size 3. Launch 4. Connect using supplied credentials
- 52. Heroku ++ ● Heroku manages everything – updates, backups, availability, configuration ● really no Ops staff ● Heroku-only features ● Latest Postgres stuff – sometimes feature previews
- 53. Heroku -- ● No configurability – webapp assumed – can't control AZ, etc. ● Limited extensions & versions ● Costs escalate ● No shell
- 54. Dandy Bling ● git-based instance management – works really well with Rails/Django ● Dataclips – web-sharable matviews! ● Followers == replicas
- 55. Dandy Bling ● About 20 extensions ● Heroku addons and apps ● encryption ● Access all AWS services
- 57. Heroku options ● 5 database “sizes” ● 3 levels of HA/uptime ● that's it
- 58. Heroku Sizing ● Small Standard 2: 3.5 GB RAM shared hosting ● Large Standard 6: 60GB RAM dedicated instance
- 59. Heroku Sizing ● Small archive: $200 HA: $350 ● Large archive: $2000 HA: $3500
- 60. the shootout
- 61. pgbench++ ● ships with Postgres ● microbenchmark – very simple “bank trade” workload ● fast to set up and run
- 62. pgbench-- ● doesn't do complex queries ● pure random data / access ● unrealistic balance of work – too reliant on single-row write speed ● not very tunable
- 63. pgbench sizing 1. memory read-write (RW): – 50% of RAM, write transactions 2. memory read-only (RO): – 50% of RAM, read-only queries 3. disk read-write (RW): – 200% of RAM, write transactions
- 64. pgbench small ● memory RW ● pgbench -i -s 100 --foreign-keys ● pgbench -c 4 -T 900 ● memory RO ● pgbench -i -s 100 --foreign-keys ● pgbench -c 4 -T 900 -S ● disk RW ● pgbench -i -s 400 --foreign-keys ● pgbench -c 4 -T 900
- 65. pgbench large ● memory RW ● pgbench -i -s 1000 --foreign-keys ● pgbench -c 16 -T 900 ● memory RO ● pgbench -i -s 1000 --foreign-keys ● pgbench -c 16 -T 900 -S ● disk RW ● pgbench -i -s 7000 --foreign-keys ● pgbench -c 16 -T 900
- 66. metrics ● TPS: transactions-per-second – measures multiple things ● Load Time: time to build the database
- 67. run many many timesrun many many times
- 68. Box PlotBox Plot
- 69. 495 TPS 587 TPS 1685 TPS 2537 TPS 6156 TPS 50 90 10 Min Max
- 71. when the smoke clears ...
- 72. features Versions Extensions Superuser Replication EC2 Any All Yes Yes RDS 9.3 only Some No Yes Heroku 9.3, 9.4, betas Some No Yes Auto- Failover Snapshots Extras Support EC2 No DIY DIY No RDS Yes* Yes No No Heroku Yes Yes Yes Yes
- 73. EC2 RDS Heroku $0.00 $100.00 $200.00 $300.00 $400.00 Small Instance Pricing Archive HA costpermonth
- 74. EC2 RDS Heroku $0.00 $500.00 $1,000.00 $1,500.00 $2,000.00 $2,500.00 $3,000.00 $3,500.00 $4,000.00 Large Instance Pricing Archive HA costpermonth
- 75. EC2 Heroku RDS RDS HA 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Small Node Load Time In-memory DB (smaller is faster) median 90% Minutes
- 76. EC2 Heroku RDS RDS HA 0 5 10 15 20 25 30 Small Node Load Time On-Disk DB (smaller is faster) median 90% Minutes
- 77. EC2 Heroku RDS RDS HA 0 5 10 15 20 25 30 Large Node Load Time In-Memory DB (smaller is faster) median 90% Minutes
- 78. EC2 Heroku RDS RDS HA 0 20 40 60 80 100 120 140 160 180 Large Node Load Time On-Disk DB (smaller is faster) median 90% Minutes
- 79. EC2 Heroku RDS RDS HA 0 100 200 300 400 500 600 700 800 In-Memory RW Test Small Node (taller is faster) median 90% TPS
- 80. EC2 Heroku RDS RDS HA 0 500 1000 1500 2000 2500 3000 In-Memory RW Test Large Node (taller is faster) median 90% TPS
- 81. EC2 Heroku RDS RDS HA 0 1000 2000 3000 4000 5000 6000 7000 In-Memory RO Test Small Node (taller is faster) median 90% TPS
- 82. EC2 Heroku RDS RDS HA 0 5000 10000 15000 20000 25000 30000 In-Memory RO Test Large Node (taller is faster) median 90% TPS
- 83. EC2 Heroku RDS RDS HA 0 50 100 150 200 250 300 350 400 On-Disk RW Test Small Node (taller is faster) median 90% TPS
- 84. EC2 Heroku RDS RDS HA 0 200 400 600 800 1000 1200 1400 1600 On-Disk RW Test Large Node (taller is faster) median 90% TPS
- 85. What's Next
- 86. More Clouds ● Rackspace ● Digital Ocean ● OpenShift ● Google Compute Engine
- 87. More Benchmarks ● OLTPBench? – Wikipedia, Auctionmark, Epinions ● DVDStore? ● New benchmark? – really need something more “webby” ● NoSQLish benchmark?
- 88. Better Visualizations ● better graphs ● automated graph generation ● detailed response times and time graphs
- 89. “running with scissors” ● test for pure ephemeral instances ● no transaction log ● local SSD ● just for RO load-balancing
- 90. more shooting ● Josh Berkus: [email protected] – www.pgexperts.com ● More Shootouts – www.databasesoup.com – https://github.com/manageacloud/cloud- benchmark-postgres/ – pgConf.US NYC, pgCon Ottawa Copyright 2015 PostgreSQL Experts Inc. Released under the Creative Commons Share-Alike 3.0 License. All images and trademarks are the property of their respective owners.