Using Think Times to Determine Arrival Rates

This question came up at the NorCal CMG meeting last week. Hugh S. asked me: Is there is a relationship between the choice of think time (Z) in a load-test client script and the rate at which requests will arrive into the system under test? The answer is, yes, and it's easy to understand how by using the preceding blog post about mapping virtual users to real users.

As Hugh was well aware, a load-test system has a finite number of requests being serviced in the SUT at each load level (N). In queueing theory parlance, it's a closed system with all queue lengths bounded. Moreover, the effective arrival rate λ is not constant, but a function of N. The relationship between λ and N looks like this:

λN  =  (N − Q)/Z .

(1)

This makes sense because, the more vusers N, the more arrivals into the SUT you can expect to see in any measurement period. Conversely, the bigger you make Z, the longer will be the delay between the successive submission of requests from the client script and that will make the arrival rate smaller. The only fly in the ointment is, Q. What is that?

Here, Q is the number of outstanding requests in the SUT. At any given load level, N, we can know the throughput X and the corresponding response time R, as measured by the load test tool, e.g., LoadRunner or JMeter. As I discussed in the preceding post, Litttle's law tells us the number of active requests in the SUT since Q = XR. Therefore, I can rewrite eqn.(1) as:

λN  =  (N − XR)/Z ,

(2)

which can be rearranged to give Z directly,

Z  =  (N − XR)/λN .

(3)

Once you know what arrival rate you would like, this equation tells you what Z value to assign in your client script.

Now, the values of X and R are those measured in steady state (between ramp-up and ramp-down) or, they should be. But in steady state, λ_N = X. Therefore, eqn.(3) can be simplified to

Z  =  (N/X) − R .

(4)

As a example of how to apply this result, suppose the load level of N = 1000 vusers produces a measured throughput of X = 50 TPS and a response time of R = 0.250 seconds. The following R code:

> N <- 1000 
> X <- 50 
> R <- 0.250
> (N/X)-R
[1] 19.75

shows that mean value of the think time should be set to 19.75 seconds in the client script.