Many large corporations are moving their mission critical applications to the Cloud. Some are in the progress of automating deployments while doing so, and automate testing as well. These activities represent massive changes in how the applications are developed, deployed, managed, and tested.
The challenges of traditional load testing
Traditional load testing usually is not done using a browser. In stead, the most popular load test tools, also the commercial ones, try to simulate the network traffic that browsers produce, but do not use a browser for that. That poses a number of disadvantages.
Firstly. the reported response time does not cover the browser based response time contribution. As we saw above, we might, and we will, miss a substantial part of the response time.
Secondly, even though network simulation capabilities is usually a feature in load testing tools, this simulation hardly ever represent how the browser reacts to slower networks. This is due to the fact that the behavior of browsers handling connections, caches and SSL encryption, to name just a few, is very hard to simulate. That means, slow networks are hard to simulate using traditional load test tooling: this is probably the main reason, why these features of the load test tool are hardly ever used.
Thirdly, since the traditional load test tooling do not use a browser, they cannot determine the difference between the different browsers, or between different browser versions. With the move of more and more logic to the browser, small differences in how the browsers deal with code can have a high impact on the load time of the page: An important reason why some browsers are very popular, as they “feel” fast, while others are more and more unpopular, as they are perceived as unbearably slow.
Finally, modern web pages include a dynamic build up of the web page, to show the most meaningful information as soon as possible to the end user, while other, less meaningful information, is shown later: the time to the first meaningful paintis an important denotation of how fast a site feels to the end-user. Only a real browser can show the first meaningful paint: traditional load testing tools that only simulate the network traffic of the browser cannot and will not.
Moving load testing to the Cloud
You could wonder, if load testing using a real browser is so much more production-like, why has is not been done before? Well, because of many reasons, but one of the main ones has to do with the fact that if you want to use a real browser to simulate an end user, then you need many browsers. And they use a lot of CPU and memory, that was not available until now. Could solutions, whether extern with a well-known Cloud provider like Microsoft Azure, AWS, Pivotal Cloud Foundry or Google Cloud, or intern in the company using a private-version of the before or using Kubernetes, provide this kind of processing power, especially if used only during the load test, and freed after the load test. So, in a sense, Front-End testing and scalable Cloud solutions go hand in hand.
Introducing browser-based load tests
So, combining Browsers in a Cloud solution is the way to go, in fact: I see it as the future of load testing. They provide the much needed real user experience, open the way for shared scripting efforts of functional and load testers, and simplify load testing complex Front End web applications. How to achieve this?
There are a number of tool vendors that now include Selenium integration in their load testing solutions. In fact, some vendors like Microfocus, offer a commercial alternative to Selenium specialized in performance testing the end user experience in the browser, called TruClient. However, TruClient was rarely employed to produce high load, as it requires a lot of hardware resources. Instead, during load testing , TruClient was traditionally used with one or two instances, to measure the end users view on performance, while protocol based load testing tools produced the bulk of the load. Now, with Cloud scalability, browser based tooling can produce the load as well, eliminating the need for both a protocol-based and a browser-based script.
Indeed, there are freeware solutions out there as well, that, with some experimenting, can be put to use. JMETER has a Selenium Webdriver sampler that can be used in a Selenium GRID configuration. Combined with a Cloud solution, the Selenium Grid can scale almost unrestricted, only being limited by the available underlying hardware.
Conclusion and a recommendation
About the author
Arthur Luimes is a Senior Performance Consultant at Altersis Performance, with more than twenty five years of experience in performance management, and is leading the way to embrace new technologies to enable performance optimization.