SlideShare a Scribd company logo
PERFORMANCE TESTING STRATEGY
FOR
CLOUD-BASED SYSTEM USING
OPEN SOURCE TESTING TOOL

Assoc. Prof. Dr. Suhaimi Ibrahim
Advanced Inforamtics School
Universiti Teknologi Malaysia
Outline
•
•
•
•
•
•

Introduction & Related Works
Overview of System Under Test
Performance Testing Strategy
Performance Test Results
Findings & Discussion
Conclusion
Introduction & Related Works
Performance requirements that should be
specified are throughput level, latency, capacity
and ability to operate in lower mode level
(Hussain and Abdulsalam, 2012].

This study illustrates the
strategy employed for
conducting performance
testing for a cloud-based
software by using open
source testing tool via
distributed performance
testing approach

Important tasks during and after the performance
test execution of the cloud should involve
environment deployment, monitoring of
environment status, controller-agent model
implementation, logs analysis, analysis of slow
queries, errors visualization as well as statistics
comparison (Lynch, Cerqueus and Thorpe, 2012)

A framework for testing scalability of web
application on the cloud should include software
that serves as HTTP server, load balancer,
database, cache and benchmark tool (Vasar,
Srirama and Dumas, 2012)
Testing cloud application shall also deal with
quality of workload generation, complexity in data
analysis and service level agreement (Zhou, Zhang
and Ye, 2013)
Overview of System Under Test
Performance Testing Strategy (1)

Sub-System

Scenario

Number of Virtual Users
(VU)

Client System

Response time to load services
home page

1

Server System
(Intelligent Services
Portal + Content
Aggregator +
Location Aware)

Response time to load services
home page (via client test page)

1, 100, 200, 300, 400, 500,
1000, 1500, 3000
Performance Testing Strategy (2)
Performance Test Results (1)
Performance Test Results (2)
Client Internet
Connection

Server Load

Response Time (seconds)

LAN speed

High

7.01

LAN speed

Low

4.81

Wireless LAN

Low

5.33

1 Mbps wired broadband

High

8.57

1 Mbps wired broadband

Low

5.33

0.44 Mbps wireless
broadband

Low

16.52 – 24.88
Findings & Discussion
•
•
•
•
•
•
•
•

The online services system hosted in the cloud platform could response to specified
performance requirement of five seconds from 1 user to 1500 concurrent users without any
break.
At 3000 concurrent users, the system still can response to huge requests without fail but with
delay in response time.
When the server is under heavy load or many concurrent users are accessing the online services
system, a particular end-user may experience slower response time accessing the online
services via the client system.
End-user with fast Internet connection will access the online services faster than those with
slow speed in any server load condition.
Cloud platform allows the virtual web servers to be scaled up in order to meet the performance
requirements for faster response time
Performance testing activities should be planned earlier in the life cycle and not during
execution or after execution completed
Performance test cases should mimic the real-world scenarios to ensure the reliability of test
results
Understanding of the performance requirements will help in selection of suitable performance
test tools to accommodate different usage profile, loads, strategy and environment setup
Conclusion
• The study has demonstrated the success of the test strategy in measuring the
performance of cloud-base system with the support open source testing tool
• it is important to have proper planning and come out with realistic scenarios for
performance testing
• Performance testing should not be treated as “when required” or “when
requested” basis. When performance test is done using the right strategy, right
scenarios and appropriate testing tool, any system would be able to deal with
increasing loads
• Future works?
 Host the performance testing tool in the cloud too
 Expanded the performance analysis further to not just on response time,
but also other aspects such as CPU utilization, memory utilization, disk I/O
and network latency
 Explore other possible real-world scenarios for the system usage to ensure
better performance testing coverage
Ad

Recommended

Cloud-based performance testing
Cloud-based performance testing
abhinavm
 
Cloud based Performance Testing
Cloud based Performance Testing
Impetus Technologies
 
Performance Testing Cloud-Based Systems
Performance Testing Cloud-Based Systems
TechWell
 
Performance Testing : Cloud Deployments
Performance Testing : Cloud Deployments
Shreyas Chaudhari
 
Top 7 Mistakes in Performance Testing
Top 7 Mistakes in Performance Testing
stuartmoncrieff
 
Agile Testing in the Cloud
Agile Testing in the Cloud
Cygnet Infotech
 
Cloud Performance Testing with LoadRunner
Cloud Performance Testing with LoadRunner
Richard Bishop
 
Performance Testing webinar
Performance Testing webinar
Abhinav Das
 
Soasta Cloud Test
Soasta Cloud Test
MatthewSOASTA
 
Automate across Platform, OS, Technologies with TaaS
Automate across Platform, OS, Technologies with TaaS
Anand Bagmar
 
Load Testing with JMeter, BlazeMeter, New Relic
Load Testing with JMeter, BlazeMeter, New Relic
Giedrius Krisciukaitis, PMP
 
Greg Maxey - Electric Cloud - Process as Code: An Introduction to the Electri...
Greg Maxey - Electric Cloud - Process as Code: An Introduction to the Electri...
DevOps Enterprise Summit
 
SOASTA CloudTest Lite
SOASTA CloudTest Lite
Fred Beringer
 
Cloud Testing - A New Age Approach to Testing
Cloud Testing - A New Age Approach to Testing
Software Testing Solution
 
Creating Complete Test Environments in the Cloud: Skytap & Parasoft Webinar
Creating Complete Test Environments in the Cloud: Skytap & Parasoft Webinar
Skytap Cloud
 
BOSE - Josh Steckler - Automating Automation: Build environments, on-demand
BOSE - Josh Steckler - Automating Automation: Build environments, on-demand
DevOps Enterprise Summit
 
Webinar: Automating the Creation and Use of Virtual Testing Environments
Webinar: Automating the Creation and Use of Virtual Testing Environments
Skytap Cloud
 
Sam Fell - Electric Cloud - Faster Continuous Integration with ElectricAccele...
Sam Fell - Electric Cloud - Faster Continuous Integration with ElectricAccele...
DevOps Enterprise Summit
 
Agile application delivery trio webinar
Agile application delivery trio webinar
Skytap Cloud
 
Sam Fell - Electric Cloud - Automating Continuous Delivery with ElectricFlow
Sam Fell - Electric Cloud - Automating Continuous Delivery with ElectricFlow
DevOps Enterprise Summit
 
Performance Testing for Mobile Apps & Sites using Apache JMeter
Performance Testing for Mobile Apps & Sites using Apache JMeter
Alon Girmonsky
 
Cloud testing: challenges and opportunities, TaaS, Integration Testing
Cloud testing: challenges and opportunities, TaaS, Integration Testing
Dr Ganesh Iyer
 
Skytap parasoft webinar new years resolution- accelerate sdlc
Skytap parasoft webinar new years resolution- accelerate sdlc
Skytap Cloud
 
A Year of “Testing” the Cloud for Development and Test
A Year of “Testing” the Cloud for Development and Test
TechWell
 
Taking Testing to the Cloud
Taking Testing to the Cloud
Cognizant
 
Webinar: Skytap & Jenkins
Webinar: Skytap & Jenkins
Skytap Cloud
 
Demystifying Operational Features for Product Owners - AgileCam - SkeltonThat...
Demystifying Operational Features for Product Owners - AgileCam - SkeltonThat...
Skelton Thatcher Consulting Ltd
 
Rohit Jainendra - Electric Cloud - Enabling DevOps Adoption with Electric Cloud
Rohit Jainendra - Electric Cloud - Enabling DevOps Adoption with Electric Cloud
DevOps Enterprise Summit
 
Tectonic Summit 2016: It's Go Time
Tectonic Summit 2016: It's Go Time
CoreOS
 
How to Make Test Automation for Cloud-based System
How to Make Test Automation for Cloud-based System
Nick Babich
 

More Related Content

What's hot (20)

Soasta Cloud Test
Soasta Cloud Test
MatthewSOASTA
 
Automate across Platform, OS, Technologies with TaaS
Automate across Platform, OS, Technologies with TaaS
Anand Bagmar
 
Load Testing with JMeter, BlazeMeter, New Relic
Load Testing with JMeter, BlazeMeter, New Relic
Giedrius Krisciukaitis, PMP
 
Greg Maxey - Electric Cloud - Process as Code: An Introduction to the Electri...
Greg Maxey - Electric Cloud - Process as Code: An Introduction to the Electri...
DevOps Enterprise Summit
 
SOASTA CloudTest Lite
SOASTA CloudTest Lite
Fred Beringer
 
Cloud Testing - A New Age Approach to Testing
Cloud Testing - A New Age Approach to Testing
Software Testing Solution
 
Creating Complete Test Environments in the Cloud: Skytap & Parasoft Webinar
Creating Complete Test Environments in the Cloud: Skytap & Parasoft Webinar
Skytap Cloud
 
BOSE - Josh Steckler - Automating Automation: Build environments, on-demand
BOSE - Josh Steckler - Automating Automation: Build environments, on-demand
DevOps Enterprise Summit
 
Webinar: Automating the Creation and Use of Virtual Testing Environments
Webinar: Automating the Creation and Use of Virtual Testing Environments
Skytap Cloud
 
Sam Fell - Electric Cloud - Faster Continuous Integration with ElectricAccele...
Sam Fell - Electric Cloud - Faster Continuous Integration with ElectricAccele...
DevOps Enterprise Summit
 
Agile application delivery trio webinar
Agile application delivery trio webinar
Skytap Cloud
 
Sam Fell - Electric Cloud - Automating Continuous Delivery with ElectricFlow
Sam Fell - Electric Cloud - Automating Continuous Delivery with ElectricFlow
DevOps Enterprise Summit
 
Performance Testing for Mobile Apps & Sites using Apache JMeter
Performance Testing for Mobile Apps & Sites using Apache JMeter
Alon Girmonsky
 
Cloud testing: challenges and opportunities, TaaS, Integration Testing
Cloud testing: challenges and opportunities, TaaS, Integration Testing
Dr Ganesh Iyer
 
Skytap parasoft webinar new years resolution- accelerate sdlc
Skytap parasoft webinar new years resolution- accelerate sdlc
Skytap Cloud
 
A Year of “Testing” the Cloud for Development and Test
A Year of “Testing” the Cloud for Development and Test
TechWell
 
Taking Testing to the Cloud
Taking Testing to the Cloud
Cognizant
 
Webinar: Skytap & Jenkins
Webinar: Skytap & Jenkins
Skytap Cloud
 
Demystifying Operational Features for Product Owners - AgileCam - SkeltonThat...
Demystifying Operational Features for Product Owners - AgileCam - SkeltonThat...
Skelton Thatcher Consulting Ltd
 
Rohit Jainendra - Electric Cloud - Enabling DevOps Adoption with Electric Cloud
Rohit Jainendra - Electric Cloud - Enabling DevOps Adoption with Electric Cloud
DevOps Enterprise Summit
 
Automate across Platform, OS, Technologies with TaaS
Automate across Platform, OS, Technologies with TaaS
Anand Bagmar
 
Greg Maxey - Electric Cloud - Process as Code: An Introduction to the Electri...
Greg Maxey - Electric Cloud - Process as Code: An Introduction to the Electri...
DevOps Enterprise Summit
 
SOASTA CloudTest Lite
SOASTA CloudTest Lite
Fred Beringer
 
Cloud Testing - A New Age Approach to Testing
Cloud Testing - A New Age Approach to Testing
Software Testing Solution
 
Creating Complete Test Environments in the Cloud: Skytap & Parasoft Webinar
Creating Complete Test Environments in the Cloud: Skytap & Parasoft Webinar
Skytap Cloud
 
BOSE - Josh Steckler - Automating Automation: Build environments, on-demand
BOSE - Josh Steckler - Automating Automation: Build environments, on-demand
DevOps Enterprise Summit
 
Webinar: Automating the Creation and Use of Virtual Testing Environments
Webinar: Automating the Creation and Use of Virtual Testing Environments
Skytap Cloud
 
Sam Fell - Electric Cloud - Faster Continuous Integration with ElectricAccele...
Sam Fell - Electric Cloud - Faster Continuous Integration with ElectricAccele...
DevOps Enterprise Summit
 
Agile application delivery trio webinar
Agile application delivery trio webinar
Skytap Cloud
 
Sam Fell - Electric Cloud - Automating Continuous Delivery with ElectricFlow
Sam Fell - Electric Cloud - Automating Continuous Delivery with ElectricFlow
DevOps Enterprise Summit
 
Performance Testing for Mobile Apps & Sites using Apache JMeter
Performance Testing for Mobile Apps & Sites using Apache JMeter
Alon Girmonsky
 
Cloud testing: challenges and opportunities, TaaS, Integration Testing
Cloud testing: challenges and opportunities, TaaS, Integration Testing
Dr Ganesh Iyer
 
Skytap parasoft webinar new years resolution- accelerate sdlc
Skytap parasoft webinar new years resolution- accelerate sdlc
Skytap Cloud
 
A Year of “Testing” the Cloud for Development and Test
A Year of “Testing” the Cloud for Development and Test
TechWell
 
Taking Testing to the Cloud
Taking Testing to the Cloud
Cognizant
 
Webinar: Skytap & Jenkins
Webinar: Skytap & Jenkins
Skytap Cloud
 
Demystifying Operational Features for Product Owners - AgileCam - SkeltonThat...
Demystifying Operational Features for Product Owners - AgileCam - SkeltonThat...
Skelton Thatcher Consulting Ltd
 
Rohit Jainendra - Electric Cloud - Enabling DevOps Adoption with Electric Cloud
Rohit Jainendra - Electric Cloud - Enabling DevOps Adoption with Electric Cloud
DevOps Enterprise Summit
 

Viewers also liked (20)

Tectonic Summit 2016: It's Go Time
Tectonic Summit 2016: It's Go Time
CoreOS
 
How to Make Test Automation for Cloud-based System
How to Make Test Automation for Cloud-based System
Nick Babich
 
Adding Value in the Cloud with Performance Test
Adding Value in the Cloud with Performance Test
Rodolfo Kohn
 
Cloud Computing System models for Distributed and cloud computing & Performan...
Cloud Computing System models for Distributed and cloud computing & Performan...
hrmalik20
 
Testing Framework on AWS Cloud - Solution Set
Testing Framework on AWS Cloud - Solution Set
Blazeclan Technologies Private Limited
 
Cloud based testing
Cloud based testing
Muhammadu Isa
 
Dessi docker kubernetes paas cloud
Dessi docker kubernetes paas cloud
Massimiliano Dessì
 
Cloud Testing
Cloud Testing
Binnate E Hawwa
 
Cloud Strategy
Cloud Strategy
Richard Harvey
 
Performance Testing - Keytorc Approach
Performance Testing - Keytorc Approach
Keytorc Software Testing Services
 
Performance Testing in the Cloud
Performance Testing in the Cloud
stuartmoncrieff
 
Shakr - Container CI/CD with Google Cloud Platform
Shakr - Container CI/CD with Google Cloud Platform
Minku Lee
 
Clipping Famtour Gastronomico 2010
Clipping Famtour Gastronomico 2010
Agência DUE
 
Mobile marketing
Mobile marketing
DianaGarcia7
 
ASP.net WPF Projects
ASP.net WPF Projects
Keyideas Infotech Private Limited
 
For mwc media mhw 2010 images
For mwc media mhw 2010 images
Greg Millan
 
2.10.10 khn afdeling kansen voor horecaondernemers
2.10.10 khn afdeling kansen voor horecaondernemers
Al Sauerfield
 
2011 Edelman Health Barometer
2011 Edelman Health Barometer
Edelman Japan
 
Clipping Vinicola Garibaldi - Setembro-Dezembro 2009
Clipping Vinicola Garibaldi - Setembro-Dezembro 2009
Agência DUE
 
Sample presentation
Sample presentation
Sarah Rainey
 
Tectonic Summit 2016: It's Go Time
Tectonic Summit 2016: It's Go Time
CoreOS
 
How to Make Test Automation for Cloud-based System
How to Make Test Automation for Cloud-based System
Nick Babich
 
Adding Value in the Cloud with Performance Test
Adding Value in the Cloud with Performance Test
Rodolfo Kohn
 
Cloud Computing System models for Distributed and cloud computing & Performan...
Cloud Computing System models for Distributed and cloud computing & Performan...
hrmalik20
 
Dessi docker kubernetes paas cloud
Dessi docker kubernetes paas cloud
Massimiliano Dessì
 
Performance Testing in the Cloud
Performance Testing in the Cloud
stuartmoncrieff
 
Shakr - Container CI/CD with Google Cloud Platform
Shakr - Container CI/CD with Google Cloud Platform
Minku Lee
 
Clipping Famtour Gastronomico 2010
Clipping Famtour Gastronomico 2010
Agência DUE
 
For mwc media mhw 2010 images
For mwc media mhw 2010 images
Greg Millan
 
2.10.10 khn afdeling kansen voor horecaondernemers
2.10.10 khn afdeling kansen voor horecaondernemers
Al Sauerfield
 
2011 Edelman Health Barometer
2011 Edelman Health Barometer
Edelman Japan
 
Clipping Vinicola Garibaldi - Setembro-Dezembro 2009
Clipping Vinicola Garibaldi - Setembro-Dezembro 2009
Agência DUE
 
Sample presentation
Sample presentation
Sarah Rainey
 
Ad

Similar to Performance Testing Strategy for Cloud-Based System using Open Source Testing Tool (20)

Achieve Maximum Performance_ Top 10 Cloud Performance Testing Tools in 2024.pdf
Achieve Maximum Performance_ Top 10 Cloud Performance Testing Tools in 2024.pdf
kalichargn70th171
 
Achieve Maximum Performance_ Top 10 Cloud Performance Testing Tools in 2024.pdf
Achieve Maximum Performance_ Top 10 Cloud Performance Testing Tools in 2024.pdf
flufftailshop
 
6 Best Practices for Cloud Performance Testing in 2025.pdf
6 Best Practices for Cloud Performance Testing in 2025.pdf
flufftailshop
 
6 Best Practices for Cloud Performance Testing in 2024.pdf
6 Best Practices for Cloud Performance Testing in 2024.pdf
kalichargn70th171
 
Performance Testing at Scale Techniques for High-Volume Services
Performance Testing at Scale Techniques for High-Volume Services
Knoldus Inc.
 
Cloud-enabled Performance Testing vis-à-vis On-premise- Impetus White Paper
Cloud-enabled Performance Testing vis-à-vis On-premise- Impetus White Paper
Impetus Technologies
 
Performance testing methodologies and tools
Performance testing methodologies and tools
Alexander Decker
 
11.performance testing methodologies and tools
11.performance testing methodologies and tools
Alexander Decker
 
Alexander Podelko - Context-Driven Performance Testing
Alexander Podelko - Context-Driven Performance Testing
Neotys_Partner
 
Harnessing the Cloud for Performance Testing- Impetus White Paper
Harnessing the Cloud for Performance Testing- Impetus White Paper
Impetus Technologies
 
Software Testing Company in India.pptx
Software Testing Company in India.pptx
SakshiPatel82
 
Effektives Consulting - Performance Engineering
Effektives Consulting - Performance Engineering
hitdhits
 
Reactive Performance Testing
Reactive Performance Testing
Lilit Yenokyan
 
Performance testing in scope of migration to cloud by Serghei Radov
Performance testing in scope of migration to cloud by Serghei Radov
Valeriia Maliarenko
 
Performance testing and rpt
Performance testing and rpt
SivaprasanthRentala1975
 
Context-Driven Performance Testing
Context-Driven Performance Testing
Alexander Podelko
 
Performance Testing Overview
Performance Testing Overview
James Venetsanakos
 
performance testing training in hyderabad
performance testing training in hyderabad
aparna3zen
 
performance testing training in hyderabad
performance testing training in hyderabad
madhupriya3zen
 
Performance Testing.3zen.pdf
Performance Testing.3zen.pdf
swathi3zen
 
Achieve Maximum Performance_ Top 10 Cloud Performance Testing Tools in 2024.pdf
Achieve Maximum Performance_ Top 10 Cloud Performance Testing Tools in 2024.pdf
kalichargn70th171
 
Achieve Maximum Performance_ Top 10 Cloud Performance Testing Tools in 2024.pdf
Achieve Maximum Performance_ Top 10 Cloud Performance Testing Tools in 2024.pdf
flufftailshop
 
6 Best Practices for Cloud Performance Testing in 2025.pdf
6 Best Practices for Cloud Performance Testing in 2025.pdf
flufftailshop
 
6 Best Practices for Cloud Performance Testing in 2024.pdf
6 Best Practices for Cloud Performance Testing in 2024.pdf
kalichargn70th171
 
Performance Testing at Scale Techniques for High-Volume Services
Performance Testing at Scale Techniques for High-Volume Services
Knoldus Inc.
 
Cloud-enabled Performance Testing vis-à-vis On-premise- Impetus White Paper
Cloud-enabled Performance Testing vis-à-vis On-premise- Impetus White Paper
Impetus Technologies
 
Performance testing methodologies and tools
Performance testing methodologies and tools
Alexander Decker
 
11.performance testing methodologies and tools
11.performance testing methodologies and tools
Alexander Decker
 
Alexander Podelko - Context-Driven Performance Testing
Alexander Podelko - Context-Driven Performance Testing
Neotys_Partner
 
Harnessing the Cloud for Performance Testing- Impetus White Paper
Harnessing the Cloud for Performance Testing- Impetus White Paper
Impetus Technologies
 
Software Testing Company in India.pptx
Software Testing Company in India.pptx
SakshiPatel82
 
Effektives Consulting - Performance Engineering
Effektives Consulting - Performance Engineering
hitdhits
 
Reactive Performance Testing
Reactive Performance Testing
Lilit Yenokyan
 
Performance testing in scope of migration to cloud by Serghei Radov
Performance testing in scope of migration to cloud by Serghei Radov
Valeriia Maliarenko
 
Context-Driven Performance Testing
Context-Driven Performance Testing
Alexander Podelko
 
performance testing training in hyderabad
performance testing training in hyderabad
aparna3zen
 
performance testing training in hyderabad
performance testing training in hyderabad
madhupriya3zen
 
Performance Testing.3zen.pdf
Performance Testing.3zen.pdf
swathi3zen
 
Ad

More from MIMOS Berhad/Open University Malaysia/Universiti Teknologi Malaysia (10)

An Alternative of Secured Online Shopping System via Point-Based Contactless ...
An Alternative of Secured Online Shopping System via Point-Based Contactless ...
MIMOS Berhad/Open University Malaysia/Universiti Teknologi Malaysia
 
A Proposal of Postgraduate Programme for Software Testing Specialization
A Proposal of Postgraduate Programme for Software Testing Specialization
MIMOS Berhad/Open University Malaysia/Universiti Teknologi Malaysia
 
A Method for Predicting Defects in System Testing for V-Model
A Method for Predicting Defects in System Testing for V-Model
MIMOS Berhad/Open University Malaysia/Universiti Teknologi Malaysia
 
A Regression Analysis Approach for Building a Prediction Model for System Tes...
A Regression Analysis Approach for Building a Prediction Model for System Tes...
MIMOS Berhad/Open University Malaysia/Universiti Teknologi Malaysia
 
Performance Testing: Analyzing Differences of Response Time between Performan...
Performance Testing: Analyzing Differences of Response Time between Performan...
MIMOS Berhad/Open University Malaysia/Universiti Teknologi Malaysia
 
Adopting Six Sigma Approach in Predicting Functional Defects for System Testing
Adopting Six Sigma Approach in Predicting Functional Defects for System Testing
MIMOS Berhad/Open University Malaysia/Universiti Teknologi Malaysia
 
Establishing A Defect Prediction Model Using A Combination of Product Metrics...
Establishing A Defect Prediction Model Using A Combination of Product Metrics...
MIMOS Berhad/Open University Malaysia/Universiti Teknologi Malaysia
 
Testing Experience Magazine Vol.14 June 2011
Testing Experience Magazine Vol.14 June 2011
MIMOS Berhad/Open University Malaysia/Universiti Teknologi Malaysia
 
Testing Experience Magazine Vol.12 Dec 2010
Testing Experience Magazine Vol.12 Dec 2010
MIMOS Berhad/Open University Malaysia/Universiti Teknologi Malaysia
 
Breaking the Software - A Topic on Software Engineering & Testing
Breaking the Software - A Topic on Software Engineering & Testing
MIMOS Berhad/Open University Malaysia/Universiti Teknologi Malaysia
 

Recently uploaded (20)

EIS-Webinar-Engineering-Retail-Infrastructure-06-16-2025.pdf
EIS-Webinar-Engineering-Retail-Infrastructure-06-16-2025.pdf
Earley Information Science
 
Techniques for Automatic Device Identification and Network Assignment.pdf
Techniques for Automatic Device Identification and Network Assignment.pdf
Priyanka Aash
 
Lessons Learned from Developing Secure AI Workflows.pdf
Lessons Learned from Developing Secure AI Workflows.pdf
Priyanka Aash
 
"Database isolation: how we deal with hundreds of direct connections to the d...
"Database isolation: how we deal with hundreds of direct connections to the d...
Fwdays
 
AI vs Human Writing: Can You Tell the Difference?
AI vs Human Writing: Can You Tell the Difference?
Shashi Sathyanarayana, Ph.D
 
"How to survive Black Friday: preparing e-commerce for a peak season", Yurii ...
"How to survive Black Friday: preparing e-commerce for a peak season", Yurii ...
Fwdays
 
GenAI Opportunities and Challenges - Where 370 Enterprises Are Focusing Now.pdf
GenAI Opportunities and Challenges - Where 370 Enterprises Are Focusing Now.pdf
Priyanka Aash
 
"Scaling in space and time with Temporal", Andriy Lupa.pdf
"Scaling in space and time with Temporal", Andriy Lupa.pdf
Fwdays
 
Cluster-Based Multi-Objective Metamorphic Test Case Pair Selection for Deep N...
Cluster-Based Multi-Objective Metamorphic Test Case Pair Selection for Deep N...
janeliewang985
 
Smarter Aviation Data Management: Lessons from Swedavia Airports and Sweco
Smarter Aviation Data Management: Lessons from Swedavia Airports and Sweco
Safe Software
 
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik
 
WebdriverIO & JavaScript: The Perfect Duo for Web Automation
WebdriverIO & JavaScript: The Perfect Duo for Web Automation
digitaljignect
 
Curietech AI in action - Accelerate MuleSoft development
Curietech AI in action - Accelerate MuleSoft development
shyamraj55
 
Cyber Defense Matrix Workshop - RSA Conference
Cyber Defense Matrix Workshop - RSA Conference
Priyanka Aash
 
UserCon Belgium: Honey, VMware increased my bill
UserCon Belgium: Honey, VMware increased my bill
stijn40
 
Using the SQLExecutor for Data Quality Management: aka One man's love for the...
Using the SQLExecutor for Data Quality Management: aka One man's love for the...
Safe Software
 
MuleSoft for AgentForce : Topic Center and API Catalog
MuleSoft for AgentForce : Topic Center and API Catalog
shyamraj55
 
2025_06_18 - OpenMetadata Community Meeting.pdf
2025_06_18 - OpenMetadata Community Meeting.pdf
OpenMetadata
 
The Future of Technology: 2025-2125 by Saikat Basu.pdf
The Future of Technology: 2025-2125 by Saikat Basu.pdf
Saikat Basu
 
Python Conference Singapore - 19 Jun 2025
Python Conference Singapore - 19 Jun 2025
ninefyi
 
EIS-Webinar-Engineering-Retail-Infrastructure-06-16-2025.pdf
EIS-Webinar-Engineering-Retail-Infrastructure-06-16-2025.pdf
Earley Information Science
 
Techniques for Automatic Device Identification and Network Assignment.pdf
Techniques for Automatic Device Identification and Network Assignment.pdf
Priyanka Aash
 
Lessons Learned from Developing Secure AI Workflows.pdf
Lessons Learned from Developing Secure AI Workflows.pdf
Priyanka Aash
 
"Database isolation: how we deal with hundreds of direct connections to the d...
"Database isolation: how we deal with hundreds of direct connections to the d...
Fwdays
 
AI vs Human Writing: Can You Tell the Difference?
AI vs Human Writing: Can You Tell the Difference?
Shashi Sathyanarayana, Ph.D
 
"How to survive Black Friday: preparing e-commerce for a peak season", Yurii ...
"How to survive Black Friday: preparing e-commerce for a peak season", Yurii ...
Fwdays
 
GenAI Opportunities and Challenges - Where 370 Enterprises Are Focusing Now.pdf
GenAI Opportunities and Challenges - Where 370 Enterprises Are Focusing Now.pdf
Priyanka Aash
 
"Scaling in space and time with Temporal", Andriy Lupa.pdf
"Scaling in space and time with Temporal", Andriy Lupa.pdf
Fwdays
 
Cluster-Based Multi-Objective Metamorphic Test Case Pair Selection for Deep N...
Cluster-Based Multi-Objective Metamorphic Test Case Pair Selection for Deep N...
janeliewang985
 
Smarter Aviation Data Management: Lessons from Swedavia Airports and Sweco
Smarter Aviation Data Management: Lessons from Swedavia Airports and Sweco
Safe Software
 
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik
 
WebdriverIO & JavaScript: The Perfect Duo for Web Automation
WebdriverIO & JavaScript: The Perfect Duo for Web Automation
digitaljignect
 
Curietech AI in action - Accelerate MuleSoft development
Curietech AI in action - Accelerate MuleSoft development
shyamraj55
 
Cyber Defense Matrix Workshop - RSA Conference
Cyber Defense Matrix Workshop - RSA Conference
Priyanka Aash
 
UserCon Belgium: Honey, VMware increased my bill
UserCon Belgium: Honey, VMware increased my bill
stijn40
 
Using the SQLExecutor for Data Quality Management: aka One man's love for the...
Using the SQLExecutor for Data Quality Management: aka One man's love for the...
Safe Software
 
MuleSoft for AgentForce : Topic Center and API Catalog
MuleSoft for AgentForce : Topic Center and API Catalog
shyamraj55
 
2025_06_18 - OpenMetadata Community Meeting.pdf
2025_06_18 - OpenMetadata Community Meeting.pdf
OpenMetadata
 
The Future of Technology: 2025-2125 by Saikat Basu.pdf
The Future of Technology: 2025-2125 by Saikat Basu.pdf
Saikat Basu
 
Python Conference Singapore - 19 Jun 2025
Python Conference Singapore - 19 Jun 2025
ninefyi
 

Performance Testing Strategy for Cloud-Based System using Open Source Testing Tool

  • 1. PERFORMANCE TESTING STRATEGY FOR CLOUD-BASED SYSTEM USING OPEN SOURCE TESTING TOOL Assoc. Prof. Dr. Suhaimi Ibrahim Advanced Inforamtics School Universiti Teknologi Malaysia
  • 2. Outline • • • • • • Introduction & Related Works Overview of System Under Test Performance Testing Strategy Performance Test Results Findings & Discussion Conclusion
  • 3. Introduction & Related Works Performance requirements that should be specified are throughput level, latency, capacity and ability to operate in lower mode level (Hussain and Abdulsalam, 2012]. This study illustrates the strategy employed for conducting performance testing for a cloud-based software by using open source testing tool via distributed performance testing approach Important tasks during and after the performance test execution of the cloud should involve environment deployment, monitoring of environment status, controller-agent model implementation, logs analysis, analysis of slow queries, errors visualization as well as statistics comparison (Lynch, Cerqueus and Thorpe, 2012) A framework for testing scalability of web application on the cloud should include software that serves as HTTP server, load balancer, database, cache and benchmark tool (Vasar, Srirama and Dumas, 2012) Testing cloud application shall also deal with quality of workload generation, complexity in data analysis and service level agreement (Zhou, Zhang and Ye, 2013)
  • 4. Overview of System Under Test
  • 5. Performance Testing Strategy (1) Sub-System Scenario Number of Virtual Users (VU) Client System Response time to load services home page 1 Server System (Intelligent Services Portal + Content Aggregator + Location Aware) Response time to load services home page (via client test page) 1, 100, 200, 300, 400, 500, 1000, 1500, 3000
  • 8. Performance Test Results (2) Client Internet Connection Server Load Response Time (seconds) LAN speed High 7.01 LAN speed Low 4.81 Wireless LAN Low 5.33 1 Mbps wired broadband High 8.57 1 Mbps wired broadband Low 5.33 0.44 Mbps wireless broadband Low 16.52 – 24.88
  • 9. Findings & Discussion • • • • • • • • The online services system hosted in the cloud platform could response to specified performance requirement of five seconds from 1 user to 1500 concurrent users without any break. At 3000 concurrent users, the system still can response to huge requests without fail but with delay in response time. When the server is under heavy load or many concurrent users are accessing the online services system, a particular end-user may experience slower response time accessing the online services via the client system. End-user with fast Internet connection will access the online services faster than those with slow speed in any server load condition. Cloud platform allows the virtual web servers to be scaled up in order to meet the performance requirements for faster response time Performance testing activities should be planned earlier in the life cycle and not during execution or after execution completed Performance test cases should mimic the real-world scenarios to ensure the reliability of test results Understanding of the performance requirements will help in selection of suitable performance test tools to accommodate different usage profile, loads, strategy and environment setup
  • 10. Conclusion • The study has demonstrated the success of the test strategy in measuring the performance of cloud-base system with the support open source testing tool • it is important to have proper planning and come out with realistic scenarios for performance testing • Performance testing should not be treated as “when required” or “when requested” basis. When performance test is done using the right strategy, right scenarios and appropriate testing tool, any system would be able to deal with increasing loads • Future works?  Host the performance testing tool in the cloud too  Expanded the performance analysis further to not just on response time, but also other aspects such as CPU utilization, memory utilization, disk I/O and network latency  Explore other possible real-world scenarios for the system usage to ensure better performance testing coverage