Apdex Meets Apex

The Apdex Alliance has defined a performance metric, called the Apdex index, which rates the measured response times of distributed applications from an Internet user perspective. The Apdex index is constructed from three categories which are defined by partitioning the total number of sample counts (C) according to an agreed upon threshold time (τ):

1. Satisfied (0 < S < τ)
2. Tolerating (τ < T < 4τ)
3. Frustrated (F > 4τ)
   

It therefore follows that:

S + T + F = C
(S/C) + (T/C) + (F/C) = 1

which can be written more simply as

s + t + c = 1

where the lower-case metrics can be read as "percent", since they must sum to 100%. In this notation, the Apdex metric is defined as:

A_τ = s + t/2

The value of A_τ can be further color coded. One limitation of its design is that A_τ only rates the response time for a single application instance measured from one geographical location. That limitation is actually intentional; keep it simple. But there is no convenient way to "drill down" for further information without examining the raw data. Suppose I had 5 different Keynote or Gomez measurements for an application and I wanted to compare them? Typically, this would require some form of tabulation. Suppose I wanted to compare 5 samples for 5 applications? Things start to get messy. Worse yet, suppose I sample the remote response times twice every hour and I want to see how the values change? How can I represent all these Apdex metrics in a compact visual way? Enter Barry-3 ... (click on the image to see the animation) This diagram is based on barycentric (triangular) coordinates, which I developed for multiprocessor CPU performance in 1992. The apex of the triangle (labeled S) represents maximal satisfaction (ideally where you'd like all your apps to be). The base of the triangle is labeled T and F respectively. The term "barry-3" indicates that we are displaying 3 performance metrics in the 2-dimensional triangular coordinate system. This information compression is a key benefit of our approach. The Apdex color encoding is superimposed as diagonal bands on the barycentric coordinate system. The black dots represent 5 geographic samples for a single application. We see immediately, that 4 of the samples are clustered near the apex while one straggler is near the centroid of the triangle, which means that sample is comprised of roughly one-third s, one-third t and one-third f values. Using this basic scheme, it is not hard to imagine generalizing it to display multiple apps using differently shaped marks or colored dots for each app type as well as having them change their location in time via animation, e.g., twice every hour. Unrelated to Apdex (at the moment), we have also generalized the barycentric system to a 3-dimensional visualization for network performance data. In this case, 4 performance metrics (multicast, broadcast, unicast, and idle) for 1000. Yes, Barry-4 displaying one thousand network segments in 3 sq. inches looks like this (click on the image to see Barry-4 tumbling in virtual 3-space): Since the the tetrahedron is a 3-dimensional geometrical figure (a 3-simplex) which requires to be rendered on a 2-dimensional screen, we need the ability to swivel the tetrahedral axes in real time. User-controlled mouse actions would be a convenient way to do that. Now, imagine being able to that while the cloud of points moves in time to reflect changes in the sampled values of the network metrics. Animation encodes a 5 coordinate, so we are really displaying (4 + 1)-dimensional data in the same small area we used to display the Adpex data (a 2-simplex). Yet another form of information compression. My Canadian colleague, Mario Jauvin, and I will be presenting the details of this work at CMG session 511: "Seeing It All at Once with Barry," next Monday afternoon, as well as the Apdex sub-conference on Wednesday afternoon.