TechTalk_Cloud Performance Testing_0.6
1 like167 views
The document provides an overview of a presentation on cloud performance testing. The presentation agenda includes cloud 101 concepts, cloud offerings and deployment models, challenges of cloud computing, and tools for cloud performance testing. It also summarizes a proof of concept that was conducted to compare the performance and costs of using a commercial tool versus an open source tool for load testing on cloud infrastructure. The results showed comparable response times between the tools and significantly lower costs when using the cloud versus maintaining physical infrastructure.
TechTalk_Cloud Performance Testing_0.6
- 1. TechTalk – Knowledge Sharing Session Cloud Performance Testing October 19, 2011 Sravanthi Naga
- 2. AppLabs Confidential 2 Purpose & Pre-requisites Purpose of this presentation: The purpose of the presentation is to give an overview of the ubiquitous Cloud and explain the latest trends and techniques in Cloud Performance Testing. Pre-requisites: It helps if the intended audience has a basic understanding of Cloud terminologies Virtualization Traditional Performance Testing Life Cycle (PTLC)
- 3. AppLabs Confidential 3 Agenda Cloud 101 Cloud offerings Cloud deployment models Cloud vs. Non-cloud solutions Cloud Performance Testing Challenges of Cloud computing Tools at each phase of PTLC AppCloud AppCloud - PoC AppCloud - The road ahead Key takeaways Recommended reading
- 4. AppLabs Confidential 4 Cloud 101 Characteristics of a Cloud » On-demand self service » Elasticity » Broad geographical network access » Pool of resources » Pay-as-you-go » Customizable platforms » Universal access » Multi-tenancy Cloud On-demand Software NetworkInfrastructure
- 5. AppLabs Confidential 5 Cloud offerings 1. Software as a Service (SaaS) - On-demand software that is accessed via a web browser and is paid on a subscription basis. Allows users to run existing online applications. 2. Infrastructure as a Service (IaaS) Infrastructure (which includes storage, hardware, servers and networking components) is provided as a Service to the end-user on a Pay-per-use model. 3. Platform as a Service (PaaS): Externally hosted service providing a complete platform to create, run, and operate users own applications.
- 6. AppLabs Confidential 6 Cloud deployment models 1. Public Cloud (Amazon EC2, Intel, Cisco) Resources are available to general public on a self-service billing model 2. Private Cloud (Amazon EC2) Resources are available solely to a specific organization 3. Community Cloud (Healthcare & Media ) Created for a specific community catering to the needs of a group 4. Hybrid Cloud (Microsoft Azure, GoGrid) Combination of Public , Private or Community Clouds
- 7. AppLabs Confidential 7 Cloud vs. Non-cloud solutions Cloud Solutions Pay-by-usage Access anywhere Customizable Infrastructure Reduced human costs Eliminates depreciable expenses Multi tenancy Self service model Effective hardware utilization Non-cloud Solutions Huge cost and effort Geographical location limitation Physical server maintenance Human resources management Static machines Static platforms
- 8. AppLabs Confidential 8 Cloud Performance Testing As shown above, there are 3 possibilities of Performance Testing in the Cloud.
- 9. AppLabs Confidential 9 Why to choose Cloud Performance Testing? Increased Expectations Faster LG’s setup Isolate Perf bottlenecks faster Elastic Load Generation On-demand environment Pay-per-use model Customizabl e SLA’s No geo. limitation for load generation Performance Testing over a Cloud offers vital benefits such as: 1. Faster Load Generators setup 2. Load can be generated from any location 3. Elastic load generation 4. Reduced costs 5. Performance bottlenecks can be isolated faster 6. No physical maintenance of the servers
- 10. AppLabs Confidential 10 Challenges of a Cloud While there are many tangible benefits with the cloud, there are some challenges in adopting a cloud, some of the key problems faced by a cloud include - Privacy Security ManageabilityAvailability Performance 1. Data Integrity 2. Data Theft 3. Privacy issues 4. Infected Application 5. Data loss 6. Data Location 7. Security on Vendor level 8. Security on user level
- 11. AppLabs Confidential 11 Tools at each phase of PTLC Phase Applicable Tools Discovery phase AppCloud Manager web application Test Script development phase OpenSource Apache JMeter, HP LR or Borland SilkPerformer Test Execution phase AppCloud Manager, Amazon cloud, Rackspace or CSC cloud Test Reporting phase AppCloud Manager, Amazon Cloudwatch, Native Performance Monitoring tools or Cloud Analytics
- 12. AppCloud
- 13. AppLabs Confidential 13 AppCloud – An Introduction • Requirements gathering through questionnaire • Discover performance characteristics of the application • Establish performance goals and team Discovery Phase • Setup test environment • Setup master data • Create performance test scripts • Identify & Deploy monitoring tools Test Design & Execution Phase • Collect performance metrics • Analyze performance metrics • Prepare performance test report Test Analysis & Reporting Phase Discovery + Planning + Test Executions + Test Analysis & Reporting = AppCloudoffered as a TaaS
- 14. AppLabs Confidential 14 AppCloud management console Discovery Phase • Questionnaire module Test Design & Execution Phase • Execution module Reporting Phase • Reporting module AppCloud Management Console has the following 3 modules
- 15. AppLabs Confidential 15 Features Benefits AppCloud console – Questionnaire module AppCloud management console’s Questionnaire module helps uncover the Customer requirements during the initial discovery phase of the project. • Helps the customer to fill the questionnaire form over the internet • Accordingly the team can plan for the Project resources and take it forward henceforth • Provides customized questionnaire to each customer • Globally accessible without any limitations • Immediately notifies the Customer and Admin on the status of filled-in questionnaire • An easy to understandable GUI • One can create, edit and customize the questions as per the need Reporting module
- 16. AppLabs Confidential 16 Features Benefits AppCloud console – Reporting module AppCloud management console’s Reporting module generates elegant final Performance Test summary reports within a fraction of seconds from the huge log files • The major benefit is one can drastically reduce the time consumed in generation of elegant final summary reports. • Globally accessible web application • User friendly GUI. • The tool can spawn any number of records, transcending Excel’s 32 lakh row limitation. • Generates elegant MS-Word reports. •Allows the user to print, save, merge and zoom the graphs. • Saves productive time. Reporting module
- 17. AppCloud Demo
- 18. AppCloud – Proof of Concept
- 19. AppLabs Confidential 19 Proof of Concept (PoC) Overview » An Open Source e-commerce application available under GNU General Public License was tested with both Commercial tool and an Open Source tool Apache JMeter on the same Cloud infrastructure for 10,000 users » Both Load Runner and JMeter used the Load Generators available in the Amazon Cloud. » Both the tests were designed to simulate equal number of transactions. JMeter HPS LoadRunner HPS
- 20. AppLabs Confidential 20 PoC – Tool Setup Apache JMeter Setup HP Load Runner Setup
- 21. AppLabs Confidential 21 PoC - Key metrics Regardless of the tools, both the tool response times were comparable and within the acceptable limit (The variation is in fraction of milliseconds). S.No. Parameter Name HP Load Runner Apache JMeter 1 Ramp-up 4 vusers every 2 seconds (83 minutes) 4 vusers every 2 seconds (83 minutes) 2 Vusers 10000 9996 3 Ramp-down 15 vusers every 2 seconds (22 minutes) 15 vusers every 2 seconds (22 minutes) 4 Test Duration 2 hours, 20 minutes 2 hours, 20 minutes 5 Average Throughput (Bytes per second) 43,806 42,118 6 Total transactions 154030 156194 7 Errors 0 0 8 Think time 60 seconds 60 seconds 9 Average response time (all transactions) in the final test 0.027 0.027
- 22. AppLabs Confidential 22 PoC - Performance Testing Costs 0 50000 100000 1 Physical Data Center 67213 Amazon Cloud 48 Cost(in$) Cost - Physical Data Center vs. Amazon Cloud Source: Amazon Cloud Environment Parameter Cost per unit Total Cost Physical Data Center (LR) Machines $10 per hour 900 Bandwidth $677 per 100Mbps 677 Tool License For 10,000 users license 65636 Amazon Cloud (JMeter) Machines $ 0.48per hour 48 Bandwidth $0.10 per GB usage 0 Tool License 0 0
- 23. AppLabs Confidential 23 AppCloud - The road ahead Integrate the Test execution interface into AppCloud console Integrate AppCloud console with CSC cloud Market in Pre-sales brochures
- 24. AppLabs Confidential 24 Key takeaways » Cloud is here to stay – Get tuned to it » By 2012, 80% of Fortune 1000 enterprises will pay for some cloud-computing service (Gartner) » By 2011, CIOs Must Answer the Question, “Why Not Run In The Cloud” (Forrester) » Customers has to pay only for what they use » Enroll in Amazon sandbox » Market about the AppCloud services in the Pre-sales brochures AppCloud On Demand Zero tool licensing cost Reduced infra costs Managed testing
- 25. AppLabs Confidential 25 Recommended reading Wikipedia http://en.wikipedia.org/wiki/Cl oud_computing Refer to Cloud webinars and latest happenings http://aws.amazon.com/ec2/p ricing/
- 27. AppLabs Confidential 27 Thank You!