It does not seem that a day goes by without another company offering to test your web application from the cloud. Some of these companies are established players while others seem to be emerging from garages across the world.
So what is driving this testing pandemic and how might it change the performance test tool market place? I think the cloud may be removing the last barrier of entry to small companies offering performance test services. At this stage a quick caveat is needed. This reduction of the barrier to companies offering to test systems accessed via web browsers. This is because over the years we have seen the growth in commodity-based performance tools. The key commodity is that more and more companies have offered business critical services via a browser and the web. Hence, there is more of a need to test these applications. So, a growth has also occurred in tools that can test web applications. Example of these tools include WebTester by Reflective Solutions, QTest by Quotium and WAPT by SoftLogica.
By concentrating on the web alone companies can make the tools simpler to use and cheaper to produce and lower development and support costs are generally passed on to customers.
These commodity tools share many features, such as alignment to web technology, user interfaces that simplify the test process and a low sticker price even when emulating an eye-wateringly high number of virtual users. The more established model of pricing for performance test tools has been based on how many users you wish to emulate during the test. Some of the commodity-based tools change this model, Microsoft Visual Studio Team Test, for example, which is based on the number of CPUs on the machines used to generate the test workload.
If this is not enough to encourage you, you also have open source products concentrating in the web market. These tools are maturing and some are capable of supporting a commercial test offering. So if you don’t mind flying without any formal support, an Open Source tool like OpenSTA, JMeter or WebLoad may be for you. Added to this is the cloud where virtual servers can be configured in minutes to act as the workload generators for a large load test. It is easy to see that with a PC, an open source tool and a web connection you can quickly be in business to run a web-based performance test.
So what about the big boys? Well they don’t want to miss out on the action and are making sure that their tools will work from the Cloud. However, I think we will see these guys concentrate more on the enterprise market where their customers have to performance test on many different technologies such as Oracle, Siebel, Citrix, etc. The complexity of the systems under test will mean that only more complex tools are acceptable. Also, enterprise customers have design and test processes supported by an array of tools and having a performance test tool that can integrate with these test management tools is an additional advantage. With these requirements these big players can charge a premium for their product.
However, we have also see a consolidation in tools with vendors being bought and integrated into larger, more diverse software vendors. For example, Mercury has been incorporated into HP and SilkPerformer, originally bought by Borland and then sold on to MicroFocus in their bid to become part of the software testing market. Furthermore, at the same time as buying SilkPerformer, MicroFocus also bought QA Load from Compuware and currently the company is offering both tools, but I wonder whether, as the development costs rise, we shall see an integration of the two tools or the domination of one?
So, the dominance of web technologies requiring performance testing, the maturing of open source and commodity load test tools coupled, finally, with access to cheap servers via the Cloud is leading to load testing being offered by so many companies. It will be interesting to see how the big boys react. However, we must bear in mind that a tool is still only as good as the person who uses it. There is still no substitute for proper test planning and skilful execution of a performance test exercise. Maybe the simplification of performance test scripts means we will spend more time on the test process rather than writing the scripts!