How to Extend Load Tests with PDQ

Suppose you want to assess the scalability of a web application based on measurements from a test rig using a load-test tool like LR, WAS or Grinder. There's one slight problem. The load-test tools are limited to running a finite number (N) of client-side load generators, and that number will always be much much smaller than the number of actual web users submitting transactions when the application goes live. How can you bridge that gap?

You can use PDQ to extrapolate from your load-test data to a large-N analysis using the following steps:

• Generate your load test data in the usual way, with think time (Z), up to your client limit (N).
  
• Construct a CLOSED PDQ model. See e.g., Chaps. 9 and 10 in my Perl::PDQ book for examples.
  
• Use the approximate solution technique, i.e., pdq::Solve($pdq::APPROX); to calibrate your model.
  
• Increase the PDQ parameter N up to the desired level while keeping the ratio N/Z constant.
  

A simple PDQ model written in Perl code would look like this:

#!/usr/bin/perl
use pdq;

$workload_name = "httpGets";
$server_name   = "WebLogic";
$service_time  = 1.00;

#--------------- Load Test Model -----------
#
# CLOSED model that originally matches the test rig data

$scale_factor = 1000;                   # adjust upward from 1, 10, ...
$Vusers       = 7.5 * $scale_factor;    # example starting value
$thinktime    = 10  * $scale_factor;    # example starting value

pdq::Init("Load Test Model");
pdq::SetWUnit("Cnx");
pdq::SetTUnit("Sec");

$s = pdq::CreateClosed($workload_name, $pdq::TERM, $Vusers, $thinktime);
$n = pdq::CreateNode($server_name, $pdq::CEN, $pdq::FCFS);
pdq::SetDemand($server_name, $workload_name, $service_time);

pdq::Solve($pdq::APPROX);
pdq::Report();

#--------------- Web Application Model -----------
#
# This OPEN model has the same performance statistics 
# as the above CLOSED model

$arrival_rate  = 0.75;

pdq::Init("Web App Model");
pdq::SetWUnit("Cnx");
pdq::SetTUnit("Sec");

$s = pdq::CreateOpen($workload_name, $arrival_rate);
$n = pdq::CreateNode($server_name, $pdq::CEN, $pdq::FCFS);
pdq::SetDemand($server_name, $workload_name, $service_time);
pdq::Solve($pdq::CANON);
pdq::Report();

Here's why it works. The test rig can only have a finite number of client requests circulating at any given time and is therefore equivalent to a CLOSED queueing sytem (in PDQ parlance), while web apps are better approximated by an OPEN queueing model because the number of users can be unbounded. For a closed queueing system, the effective arrival rate is a variable given by the equation:< λ(Q) = (N - Q) / Z                        (1) This equation defines something known as the "load line". It simply says that the rate of arrivals into the queueing system is a decreasing function of the number of requests (Q) that are already enqueued. If all N clients have issued requests, then the effective arrival rate will be zero for the next period. For an open queueing system, the arrival rate is treated as a constant. λ(Q) = λ == const.                        (2) How do we get from (1) to (2)? To explain, I'll refer to the following plot. If we simply increase N to some very large number (much greater than anything used in the load tests), the PDQ model will eventually saturate. In the plot, I've shown what happens if N is increased from say, 100 users (blue solid) to 200 users (blue dashed) while leaving Z unchanged. The load line maintains its slope but shifts in the upward direction. This happens because both the x-intercept and y-intercept are being scaled by the same factor (N). However, if we fix the position of the y-intercept in the plot, by keeping the ratio N/Z constant in eqn.(1), then the x-intercept will move off to the right as we increase N. Eventually, the load line will tend toward the horizontal (red line), which is equivalent to a constant arrival rate for all values of Q.