PERFORMANCE TESTING STRATEGY FOR

CLOUD-BASED SYSTEM USING OPEN SOURCE TESTING TOOL

Assoc. Prof. Dr. Suhaimi IbrahimAdvanced Inforamtics SchoolUniversiti Teknologi Malaysia

Outline

• Introduction & Related Works• Overview of System Under

Test• Performance Testing Strategy• Performance Test Results• Findings & Discussion • Conclusion

Introduction & Related Works

This study illustrates the strategy employed for

conducting performance testing for a cloud-based software by using open

source testing tool via distributed performance

testing approach

Performance requirements that should be specified are throughput level, latency, capacity and ability to operate in lower mode level (Hussain and Abdulsalam, 2012].

Important tasks during and after the performance test execution of the cloud should involve environment deployment, monitoring of environment status, controller-agent model implementation, logs analysis, analysis of slow queries, errors visualization as well as statistics comparison (Lynch, Cerqueus and Thorpe, 2012)

Testing cloud application shall also deal with quality of workload generation, complexity in data analysis and service level agreement (Zhou, Zhang and Ye, 2013)

A framework for testing scalability of web application on the cloud should include software that serves as HTTP server, load balancer, database, cache and benchmark tool (Vasar, Srirama and Dumas, 2012)

Overview of System Under Test

Performance Testing Strategy (1)

Sub-System ScenarioNumber of Virtual Users

(VU)

Client System Response time to load services home page 1

Server System (Intelligent Services Portal + Content Aggregator + Location Aware)

Response time to load services home page (via client test page)

1, 100, 200, 300, 400, 500, 1000, 1500, 3000

Performance Testing Strategy (2)

Performance Test Results (1)

Performance Test Results (2)

Client Internet Connection Server Load Response Time (seconds)

LAN speed High 7.01

LAN speed Low 4.81

Wireless LAN Low 5.33

1 Mbps wired broadband High 8.57

1 Mbps wired broadband Low 5.33

0.44 Mbps wireless broadband Low 16.52 – 24.88

Findings & Discussion The online services system hosted in the cloud platform could response to specified

performance requirement of five seconds from 1 user to 1500 concurrent users without any break.

At 3000 concurrent users, the system still can response to huge requests without fail but with delay in response time.

When the server is under heavy load or many concurrent users are accessing the online services system, a particular end-user may experience slower response time accessing the online services via the client system.

End-user with fast Internet connection will access the online services faster than those with slow speed in any server load condition.

Cloud platform allows the virtual web servers to be scaled up in order to meet the performance requirements for faster response time

Performance testing activities should be planned earlier in the life cycle and not during execution or after execution completed

Performance test cases should mimic the real-world scenarios to ensure the reliability of test results

Understanding of the performance requirements will help in selection of suitable performance test tools to accommodate different usage profile, loads, strategy and environment setup

Conclusion

• The study has demonstrated the success of the test strategy in measuring the performance of cloud-base system with the support open source testing tool

• it is important to have proper planning and come out with realistic scenarios for performance testing

• Performance testing should not be treated as “when required” or “when requested” basis. When performance test is done using the right strategy, right scenarios and appropriate testing tool, any system would be able to deal with increasing loads

• Future works? Host the performance testing tool in the cloud too Expanded the performance analysis further to not just on response time, but

also other aspects such as CPU utilization, memory utilization, disk I/O and network latency

Explore other possible real-world scenarios for the system usage to ensure better performance testing coverage