Mastering Distributed Performance Testing
Download as pptx, pdf0 likes108 views
The document provides a comprehensive guide on mastering distributed load testing, emphasizing the significance of performance testing to evaluate application performance under stress and load conditions. It covers distributed/remote performance testing techniques, architecture setup for master and slave nodes, creating test plans in JMeter, executing tests, and analyzing results, while addressing potential challenges and offering alternative solutions like parallel testing. Additionally, it highlights the major disadvantages of neglecting performance testing, such as poor user experience and increased operational costs.
Mastering Distributed Performance Testing
- 1. Mastering Distributed Load Testing Abhishek Sharma Technical Lead Test Automation Competency
- 2. 1. Performance Testing Introduction Significance of Performance Testing 2. Distributed/Remote Performance Testing Introduction Significance of Distributed/Remote Performance Testing 3. Pre-requisites 4. Architecture - Distributed Testing Components 5. Configuration § Master End § Slave End 6. Create Test plan in JMeter 7. Generate Large Volumes of Data for Performance Testing 8. Executing a test on Distributed Testing Environment Execution in GUI & Non-GUI Confirmation of successful test start 9. Live Monitoring § Configuration § Dashboard 10. Results Analysis 11. Challenges 12. Alternate Approach – Parallel Testing 13. DEMO 14. Major Disadvantages of Not Conducting Performance Testing
- 3. Performance Testing Introduction : “Performance testing is a systematic testing approach to validate the performance of an application under load.” Significance of Performance Testing : Performance testing is done to evaluate the performance of an application under some stress and load conditions. Generally, this is measured in terms of the user activity’s response time. Performance Test scenarios are designed to test the performance of the whole system at high stress and high load conditions. Performance Testing gives confidence to the developer as well as the client that the application can handle X number of users without any degradation in the performance. It also helps to identify the break-point of the application. 1.Reduce Setbacks 2.Create High Standards 3.Drive Innovation 4.Measure Stability 5.Compare 2 Systems
- 4. Distributed Performance Testing Introduction : Distributed load testing is a performance testing technique in which the workload is distributed across multiple machines or nodes to simulate many virtual users or concurrent connections. This method is employed to evaluate the performance of a system or application when subjected to intense loads and simultaneous user activity. The objective of distributed load testing is to pinpoint performance limitations, gauge system scalability, and verify the application's capability to manage anticipated user loads. Significance of Performance Testing : When conducting load testing, if the test requires execution with 10,000 users, it prompts the consideration of Distributed Load Testing. This method becomes relevant particularly when real-time simulation is unfeasible. In such cases, opting for distributed load testing instead of conventional load testing becomes essential. 1.Identifying Bottlenecks 2.Scalability Assessment 3.User Experience Assurance 4.Preventing Downtime:
- 5. • Ensure Java and JMeter version matches between Master and Slave machine • All machines are to be under the same subnet (IP address) • Any plugins/ jars/ reporting saving jtl file configurations are to be similar in all machines. It is best to have the same JMeter tool folder installed on all machines involved in the testing to avoid discrepancies • Save the ‘jmx File (Test Scripts)’ and ‘CSV files used in the jmx scripts’ explicitly on Slave machines • Firewalls on the systems are turned off, or required ports are opened for connection • JMeter on each system can access the target server • For simplicity, disable the SSL for RMI
- 6. Architecture Architecture : Components Master: The system running JMeter GUI/non-GUI, controls each slave and receives test execution information (collected metrics, threads/virtual Users activities, errors, etc.) from them. Slave: The system running JMeter- Server(jmeter-server.bat) receives a command from the master and sends a request to the server which is the application under test. Target: The Application or server under test. Master Target
- 7. Distributed Testing Configurations At Master End : Configuring the master node is simpler than doing that for the slave nodes. We just need to open the jmeter.properties file, find the remote_hosts property, uncomment it and type the IP addresses (without port number) of all the slaves that you have set up. Add the IP address of Slave machines separated by comma. Once you performed all the above-mentioned steps then restart JMeter. Cont ….
- 8. Distributed Testing Configurations At Slave End : Now, we need to generate the rmi-keystore.jks file in our bin folder of JMeter. First, we need to open cmd and go to the JMeter bin. Open the create-rmi-keystore.bat file in cmd and fill in all the details. Note : We are configuring 2 slaves for our Distributed Load Testing.
- 9. Distributed Testing Configurations Now, copy the generated rmi_keystore.jks file to the JMeterbin folder and copy past “this” rmi_keystore.jks file to your slave's systems. Open the JMeter-server.bat file in both our slave and master system. Restart the JMeter in both the slave and master system. Note : There is one more way to do it without rmi_keystore.jks file. Open the jmeter.properties file, add this “server.rmi.ssl.disable=true” line and save it. Now you don’t need to create rmi-keystore.jks file every time.
- 10. Create Test Plan in JMeter • For creating a test plan in JMeter, first, we required to create a JMeter Thread Group in the machine. • After the allocation of threads. Before starting the test in a distributed environment, the number of threads calculation is required. • Set target user load is 100 and there are 10 slaves then ‘Number of Threads (users)’ field will have the input as 10 (=100/10). • Master machine sends a copy of the JMeter script to each slave machine and according to the script input each slave generates the load and executes the test. • Accumulative generated load is equal to the target load. Master machine JMeter does not have the capability to divide the number of threads according to the available slaves. • Manual intervention is required for the number of threads allocation. • Now, give the source URL and port number to all the config elements. • Consider that all the tested URLs are supported by the specific path. Note : Add Backend Listener also if needs to configure live monitoring for JMeter.
- 11. Generate Large Volumes of Test Data for Performance Testing Synthetic Test Data •We can directly generate it running Query at Database side. •We can generate data with help of code or shell and then seed the same in Database. •We can create a script to generate data at Application side only. •We can generate a large amount generalized data with the help of online tools like •Open-Source Test Data Generation Tools: Faker, Jfairy, Jailer. •Commercial Test Data Generation Tools: Informatica Test Data Management, Talend Data Preparation •Cloud Based Test Data Generation Tools: Mockaroo, Tricentis Tosca, Data Factory Existing Production Data •We can ask DB team or manual testing team for Data. •If Client agrees, we can ask them to copy Production Data in Perf DB
- 12. Executing a test on Distributed Testing Environment • GUI Mode : In GUI mode, JMeter UI is launched at the master machine. If the test needs to be started using specific slaves, then those slaves need to be chosen from the Run menu. JMeter Menu -> Run -> Remote Start -> Click the name of the host (s) JMeter Menu -> Run -> Remote Start All Cont ….
- 13. Executing a test on Distributed Testing Environment •Non-GUI Mode : To execute the test in non-GUI mode, first navigate to the /bin folder of JMeter and run the following command: •For Windows: •jmeter -n -t <script path> -l <log file path> -r •For Unix: •./jmeter.sh -n -t <script path> -l <log file path> -r •The test can also be executed by selecting specific slave machines. •For Windows: •jmeter -n -t <script path> -l <log file path> -R server1,server2,server3, •For Unix: •./jmeter.sh -n -t <script path> -l <log file path> -R server1,server2,server3, Note : To run Load tests, Non-GUI Mode is recommended.
- 14. Confirmation of successful test start At Master End At Slave End
- 15. Live Monitoring While running JMeter distributed test, we can start test and capture results once test completed. We don't have option to monitor running test and analyze errors in between tests i.e., we don't have live monitoring in JMeter. But we can make live monitoring available for JMeter tests with Jmeter Backend Listener, Influx DB and Grafana.
- 16. Live Monitoring Components : JMeter Backend Listener : JMeter Elasticsearch Backend Listener is a JMeter plugin enabling us to send test results to an Elasticsearch engine. It is meant as an alternative live-monitoring tool to the built-in "InfluxDB" backend listener of JMeter. Influx DB : To store data(Test results) send by JMeter Backend Listener. Grafana : Grafana is a multi-platform open-source analytics and interactive visualization web application. It provides charts, graphs, and alerts for the web when connected to supported data sources. Add Data Source as Influx DB and details of Influx DB. After that, Create Dashboard for JMeter Metrics with Dashboard ID.
- 17. Results Analysis The Results file (xyz.jtl) will be available on the master machine. All you need to do after the test is finished is to grab the test results (xyz.jtl) and generate the reports via JMeter Listeners as per requirement. We can use below listeners to analyze JMeter test results. Aggregate Report Summary Report Aggregate Graph Graph Results We can also generate HTML reports which provides a complete overview of JMeter Tests for Analysis We can generate HTML Report with Results.jtl file. Go to Jmeter GUI > Click Tools menu > Click Generate HTML report > Mention Details (results.jtl file, Output Directory, User Properties) and Click Generate Report.
- 18. Challenges Data File Adjustment : When a unique test data scenario needs to execute then test data in the CSV data set config file placed at different slave; must be unique. The same copy of the CSV file will produce duplicate data. Connectivity Issues : • If we are unable to start the test on slaves running jmeter-server due to Connection time-out errors Try using ping or telnet commands to ensure Master can ping to the IP address of Slave machines • Check if the slaves’ firewalls are configured to allow incoming connections on the TCP ports that are configured in the JMeter properties: server.rmi.localport and server.rmi.port property.
- 19. Alternate Approach – Parallel Testing Despite the efforts, sometimes due to the company’s network/firewall policy, sometimes setting up the connectivity between master and slaves is not possible. At times like these, as an interim solution, we can use parallel testing. This type of testing approach involves running the test scenario across multiple machines simultaneously, thereby aggregating the preferred total number of users against the application under test (AUT). For instance, if the requirement is to test for 1000 users, we configure the test run in two different machines with 500 users each at the same time. Under Distributed architecture, all machines are synchronized and in constant communication. Master commands the slave machines on when to start and stop the test, and Slave machines, in turn, send test results to the Master machine in near real-time. For Parallel Testing, the two or many testing systems used are independent entities and focus on injecting load against the AUT. Jmeter Node 1 Jmeter Node 2 Jmeter Node n Application Under Test Results file1 Results filen Results file2 : : : : :
- 21. Major Disadvantages of Not Conduction Performance Testing Now, User expectations are at an all-time high. Here the importance of performance testing cannot be overstated. Here are the major disadvantages of not conducting reliable performance testing of an application, website, or system. • Poor User Experience • Brand Reputation Damage • Lost Revenue Opportunities • Competitive Disadvantage • Inability to Scale Effectively • Increased Operational Costs • Low-Confidence • Increased Support Costs • Decreased Productivity • Legal & Compliance Issues st