In the QA forums I frequent, there are often questions about how to properly load test when you don’t have access to production or an identically built environment. Most companies won’t spring the cash to build an environment that is identical to production; generally, testing environments are made up of hand-me-down servers that used to be in production. Of course, there is also the cost of test suite licensing to produce a productional load, and the near impossibility of mimicking real production traffic.
Though a production clone would be ideal, a watered down environment can be sufficient, and in some ways better. Bottlenecks are achieved faster, without having to push through 50 Mbps of data. Additionally, a “lesser” environment will be more sensitive to changes; your transaction may take 0.5 seconds on production-grade servers, and a defect that doubles it to 1.0 seconds is hardly noticeable, but on a lesser environment where that transaction takes 6.0 seconds, doubling it to twelve throws up red flags.
For a watered-down environment, try to lessen the horsepower of your system while matching the architecture. If your productional environment is eight web servers that are all quad 3.2 Ghz Xeons running Windows Server 2003 Web Edition, and all load balanced through a hardware load balancer, you can bring it down to two web servers with less horsepower–perhaps dual 700Mhz P3s–but the servers should still run Windows Server 2003 Web Edition and be balanced with a hardware balancer. Do not drop below two web servers because you will still want a load balanced environment, and do not switch to Windows 2000 or use Microsoft’s NLB (Network Load Balancing). If your production web environment uses Windows 2000 and NLB, obviously use that technology in your testing environment; do not switch to Windows 2003 or a hardware load balancer.
Additionally, try to reduce equally throughout your environment. Don’t drop your web servers from Pentium 4s to Pentium 3s while dropping your database servers from Pentium 4s to an old 486 desktop. Equal reductions maintain your continuity, and in the end, your sanity. Unequal reductions introduce new problems that don’t exist in production, but will still happily waste your time and money. A major bottleneck might exist on your web servers, but the defect could be masked because you were database-bound by using that old 486.
The idea behind this is that many bugs can be introduced by a specific revision of your OS (Think of the problems from Windows XP SP2), from your style of network infrastructure, the version of your graphics driver, etc. You want as many common points as possible between your testing and production environments to eliminate any surprises when you launch your application. Ideally, your testing environment is an exact replica of your production environment, but unless you are making desktop applications, it is only a fantasy, so just try to get as close as you can. Use the same OS version, including the same service pack and the same installed hot fixes. Use the same driver versions, and configure the same settings on your web server software. You are trying to create a miniature version of your production environment, like a model car or a ship in a bottle. Pay attention to the details and you will be okay. To your application, the environments should be exactly the same; one is just a little snug.
Remember Me