Performance Testing Strategy for Cloud-Based System using Open Source Testing Tool

Download Performance Testing Strategy for Cloud-Based System using Open Source Testing Tool

Post on 05-Dec-2014




0 download

Embed Size (px)


Paper presented at Malaysian Software Engineering Conference 2013 (MySEC2013) at Malacca


<ul><li> 1. PERFORMANCE TESTING STRATEGY FOR CLOUD-BASED SYSTEM USING OPEN SOURCE TESTING TOOL Assoc. Prof. Dr. Suhaimi Ibrahim Advanced Inforamtics School Universiti Teknologi Malaysia </li> <li> 2. Outline Introduction &amp; Related Works Overview of System Under Test Performance Testing Strategy Performance Test Results Findings &amp; Discussion Conclusion </li> <li> 3. Introduction &amp; Related Works Performance requirements that should be specified are throughput level, latency, capacity and ability to operate in lower mode level (Hussain and Abdulsalam, 2012]. This study illustrates the strategy employed for conducting performance testing for a cloud-based software by using open source testing tool via distributed performance testing approach Important tasks during and after the performance test execution of the cloud should involve environment deployment, monitoring of environment status, controller-agent model implementation, logs analysis, analysis of slow queries, errors visualization as well as statistics comparison (Lynch, Cerqueus and Thorpe, 2012) A framework for testing scalability of web application on the cloud should include software that serves as HTTP server, load balancer, database, cache and benchmark tool (Vasar, Srirama and Dumas, 2012) Testing cloud application shall also deal with quality of workload generation, complexity in data analysis and service level agreement (Zhou, Zhang and Ye, 2013) </li> <li> 4. Overview of System Under Test </li> <li> 5. Performance Testing Strategy (1) Sub-System Scenario Number of Virtual Users (VU) Client System Response time to load services home page 1 Server System (Intelligent Services Portal + Content Aggregator + Location Aware) Response time to load services home page (via client test page) 1, 100, 200, 300, 400, 500, 1000, 1500, 3000 </li> <li> 6. Performance Testing Strategy (2) </li> <li> 7. Performance Test Results (1) </li> <li> 8. Performance Test Results (2) Client Internet Connection Server Load Response Time (seconds) LAN speed High 7.01 LAN speed Low 4.81 Wireless LAN Low 5.33 1 Mbps wired broadband High 8.57 1 Mbps wired broadband Low 5.33 0.44 Mbps wireless broadband Low 16.52 24.88 </li> <li> 9. Findings &amp; Discussion The online services system hosted in the cloud platform could response to specified performance requirement of five seconds from 1 user to 1500 concurrent users without any break. At 3000 concurrent users, the system still can response to huge requests without fail but with delay in response time. When the server is under heavy load or many concurrent users are accessing the online services system, a particular end-user may experience slower response time accessing the online services via the client system. End-user with fast Internet connection will access the online services faster than those with slow speed in any server load condition. Cloud platform allows the virtual web servers to be scaled up in order to meet the performance requirements for faster response time Performance testing activities should be planned earlier in the life cycle and not during execution or after execution completed Performance test cases should mimic the real-world scenarios to ensure the reliability of test results Understanding of the performance requirements will help in selection of suitable performance test tools to accommodate different usage profile, loads, strategy and environment setup </li> <li> 10. Conclusion The study has demonstrated the success of the test strategy in measuring the performance of cloud-base system with the support open source testing tool it is important to have proper planning and come out with realistic scenarios for performance testing Performance testing should not be treated as when required or when requested basis. When performance test is done using the right strategy, right scenarios and appropriate testing tool, any system would be able to deal with increasing loads Future works? Host the performance testing tool in the cloud too Expanded the performance analysis further to not just on response time, but also other aspects such as CPU utilization, memory utilization, disk I/O and network latency Explore other possible real-world scenarios for the system usage to ensure better performance testing coverage </li> </ul>


View more >