Crib Sheet for Emulating Web Traffic

Our paper entitled, How to Emulate Web Traffic Using Standard Load Testing Tools (PDF) is now available online and will be presented at the upcoming CMG conference in November.

Presenter: James Brady (co-author: Neil Gunther)
Session Number: 436
Subject Area: APM
Session Date: Wed, November 9, 2016
Session Time: 1:00 PM - 2:00 PM
Session Room: PortofinoB

The motivation for this work harks back to a Guerrilla forum in 2014 that essentially centered on the same topic as the title of our paper. It was clear from that discussion that commenters were talking at cross purposes because of misunderstandings on many levels. I had already written a much earlier blog post on the key queue-theoretic concept, viz., holding the $N/Z$ ratio constant as the load $N$ is increased, but I was incapable of describing how that concept should be implemented in a real load-testing environment.

On the other hand, I knew that Jim Brady had presented a similar implementation in his 2012 CMG paper, based on a statistical analysis of the load-generation traffic. There were a few details that I couldn't quite reconcile in Jim's paper but, at the CMG 2015 conference in San Antonia, I suggested that we should combine our separate approaches and aim at a definitive work on the subject. After nine months gestation (ugh!), this 30-page paper is the result.

Although our paper doesn't contain any new invention, per se, the novelty lies in how we needed to bring together so many disparate and subtle concepts in precisely the correct way to reach a complete and consistent methodology. The complexity of this task was far greater than either of us had imagined at the outset. The hyperlinked Glossary should help with the terminology, but because there are so many interrelated parts, I've put together the following crib notes in an effort to help performance engineers get through it (since they're the ones that most stand to benefit). The key results of our paper are indicated by ◀

1. Standard load-test tools only allow finite number of virtual users
2. Web traffic is characterized by an unlimited number of real users
3. Attention usually focused on SUT (system under test) performance
4. We focus on the GEN (load generator) characteristics
5. Measure distribution of web requests and their mean arrival rate
6. Web traffic should be a Poisson process (cf. A.K. Erlang 1909) ◀
7. Requires statistically independent arrivals, i.e., no correlations
8. Independent requests should arrive asynchronously into the SUT
9. But virtual-user requests are synchronized (correlated) in SUT queues ◀
10. De-correlate arrivals by shrinking SUT queues (Principle A): ◀
    • Shrink by increasing think delay $Z$ with user load as a fixed ratio
    • Traffic rate $\lambda$ approaches a constant $\lambda = N/Z$ as SUT queues shrink
11. Check requests are Poisson by measuring the coefficient of variation ($CoV$)
12. Require $CoV \approx 1$ for a Poisson process (Principle B) ◀

Originally, I assumed the paper would be no more than a third it's current length. Wrong! My only defense is: it's all there, you just need to read it. (tl;dr doesn't apply) Apologies in advance, but hopefully, these crib notes will help you.