tag:blogger.com,1999:blog-69777559593498470932024-03-05T10:02:12.818-08:00The Pith of PerformancePossibly pithy insights into computer performance analysis and capacity planning based on the Guerrilla
<a href="http://www.perfdynamics.com/books.html">series of books</a> and <a href="http://www.perfdynamics.com/Classes/schedule.html">training classes</a> provided by <a href="http://www.perfdynamics.com/">Performance Dynamics Company</a>.Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.comBlogger404125tag:blogger.com,1999:blog-6977755959349847093.post-89727461063598829822021-04-20T09:14:00.002-07:002021-04-20T09:20:43.368-07:00PDQ Online Workshop, May 17-21, 2021PDQ (Pretty Damn Quick) is a free, open source, performance analyzer available from the <a href="http://www.perfdynamics.com/Tools/PDQcode.html" target="_blank">Performance Dynamics web site</a>.
<p>
All modern computer systems, no matter how complex, can be thought of as a directed graph of individual buffers that hold requests until to be serviced at a shared computational resource, e.g., a CPU or disk. Since a buffer is just a queue, any computer infrastructure, from your laptop up to Facebook.com, can be represented as a directed graph of queues.
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxiuElSSV_VFo6KK9cHiyQqMVfpiseHuRIJIdprdQkseCPzMl27R-fE7EFoZPRekQJTXL6RNNBDkXjzp0bE1_k91M-ucqIlho4vtEPI4ecf2wAFOqeY8m4zaagAnQm7Kn8mce2OmnvL1M/s916/JackNet.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="400" data-original-height="200" data-original-width="916" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxiuElSSV_VFo6KK9cHiyQqMVfpiseHuRIJIdprdQkseCPzMl27R-fE7EFoZPRekQJTXL6RNNBDkXjzp0bE1_k91M-ucqIlho4vtEPI4ecf2wAFOqeY8m4zaagAnQm7Kn8mce2OmnvL1M/s400/JackNet.png"/></a></div>
The directed arcs or arrows in such a graph correspond to workflows between different queues. In the parlance of queueing theory, a directed graph of queues is called a <i>queueing network model</i>. PDQ is a tool for predicting performance metrics such as, waiting time, throughput, optimal user-load.
<p>
Two major benefits of using PDQ are:
<ol>
<li> confirmation that monitored performance metrics have their expected values
<li> predict performance for circumstances that lie beyond current measurements
</ol>
<a href="http://www.perfdynamics.com/Classes/Outlines/pdqw.html" target="_blank">Find out more</a> about the workshop and
<a href="http://www.perfdynamics.com/Classes/schedule.html" target="_blank">register today</a>.
Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com0tag:blogger.com,1999:blog-6977755959349847093.post-29807714173602098002020-11-26T05:21:00.000-08:002020-11-26T05:21:19.364-08:00PDQ 7.0 is Not a TurkeyGiving Thanks for the <a href="http://perfdynamics.com/Tools/PDQcode.html" target="_blank">release of PDQ 7.0</a>, after a 5-year drought, and just in time for the <a href="http://perfdynamics.com/Classes/schedule.html" target="_blank">PDQW workshop</a> next week.
<p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjF6_2dFKoYRrhDUEcwCqdhkZfoEz5qDUaUJLfNpytg31xdg2e7S7JPbdxHyqbiWK_CPwJOUZX-wP0t6WDW99Xuq5_KmOa1Ad7Adtz63jZq8ENv1nmnJ5JNfYusKtnlhGuUhwLb9ZKHbd0/s818/pdq7.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="320" data-original-height="459" data-original-width="818" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjF6_2dFKoYRrhDUEcwCqdhkZfoEz5qDUaUJLfNpytg31xdg2e7S7JPbdxHyqbiWK_CPwJOUZX-wP0t6WDW99Xuq5_KmOa1Ad7Adtz63jZq8ENv1nmnJ5JNfYusKtnlhGuUhwLb9ZKHbd0/s320/pdq7.png"/></a></div>
<p>
<h3>New Featues</h3>
<ol>
<li> The introduction of the STREAMING solution method for OPEN queueing networks. (cf. CANON, which can still be used).
<li> The <b>CreateMultiNode()</b> function is now defined for CLOSED queueing networks and distinguished via the MSC device type (cf. MSO for OPEN networks).
<li> The format of <b>Report()</b> has been modified to make the various types of queueing network parameters clearer.
<li> See the <b>R Help pages</b> in RStudio for details.
<li> Run the <b>demo(package="pdq")</b> command in the R console to review a variety of PDQ 7 models.
</ol>
<p>
<h3>Maintenance Changes</h3>
The migration of Python from 2 to 3 has introduced maintenance complications for PDQ. Python 3 may eventually be accommodated in future PDQ releases. Perl maintenance has ended with PDQ release 6.2, which to remain compatible with the Perl::PDQ book (2011).
Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com0tag:blogger.com,1999:blog-6977755959349847093.post-40913633748381136002020-11-08T10:07:00.001-08:002020-11-09T17:07:34.806-08:00PDQ Online Workshop, Nov 30-Dec 5, 2020 PDQ (Pretty Damn Quick) is a queueing graph performance analyzer that comes as:
<ol>
<li> free open source <a href="http://www.perfdynamics.com/Tools/PDQcode.html" target="_blank">software package</a>
<li> with an online <a href="http://www.perfdynamics.com/Tools/PDQman.html" target="_blank">user manual</a>
</ol>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBzpAP0tl2UsQPAwcpFTbvBF5dSmvrlwoFoxE4iJqCwe5LoeTmX3oaziO6L3eYcCdZed9BIvN8sOm8driGmD6OOA7H-ryF6lIq_ahrP5N7pFK2g7s3d_GIbKz7l-HVH7r8CKRbKpbWN9M/s551/PDQlBig.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="400" data-original-height="423" data-original-width="551" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBzpAP0tl2UsQPAwcpFTbvBF5dSmvrlwoFoxE4iJqCwe5LoeTmX3oaziO6L3eYcCdZed9BIvN8sOm8driGmD6OOA7H-ryF6lIq_ahrP5N7pFK2g7s3d_GIbKz7l-HVH7r8CKRbKpbWN9M/s400/PDQlBig.png"/></a></div>
<p>
As shown in the above diagram, any modern computer system can be thought of as a directed graph of individual buffers where requests wait to be serviced at some kind of shared computational resource, e.g., a CPU or disk. Since a buffer is just a queue, any computer infrastructure, <b>from a laptop to Facebook</b>, can be represented as a directed graph of queues. The directed arcs or arrows correspond to workflows between the different queues. In the parlance of queueing theory, a directed graph of queues is called a <b>queueing network model</b>. PDQ is a tool for calculating the performance metrics, e.g., waiting time, throughput, optimal load, of such network models.
<p>
Some example PQD applications include models of:
<ul>
<li> Cloud applications
<li> Packet networks
<li> HTTP + VM + DB apps
<li> PAXOS-type distributed apps
</ul>
</p>
Two major benefits of using PDQ are:
<ol>
<li> confirmation that monitored performance metrics have their expected values
<li> predict performance for circumstances that lie beyond current load-testing
</ol>
This particular PDQ workshop has been requested for the <b>Central European Time</b> zone.
<ul>
<li> All sessions are INTERACTIVE using Skype (not canned videos)
<li> Online sessions are usually 4 hours in duration
<li> A typical timespan is 2pm to 6pm CET each business day
<li> A nominal 5-10 minute bio break at 4pm CET
<li> Attendees are encouraged to bring there own PDQ projects or data to start one
</ul>
<ol>
<li> <a href="http://www.perfdynamics.com/Classes/Outlines/pdqw.html" target="_blank">Find out</a> more about the workshop.
<li> Here is the <a href="https://www.eventbrite.com/e/pdq-workshop-eu-online-edition-tickets-114810814236" target="_blank">REGISTRATION page</a>.
</ol>
Hope to see you at PDQW!
Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com0tag:blogger.com,1999:blog-6977755959349847093.post-20568923937541044092020-02-11T11:49:00.000-08:002020-02-11T12:45:30.778-08:00Converting Between Human Time and Absolute Time in the ShellThis is really more of a note-to-self but it may also be useful for other readers.
<p>
Converting between various time zones, including UTC time, is both a pain and error-prone.
A better solution is to use absolute time. Thankfully, Unix provides such a time: the so-called <a href="https://en.wikipedia.org/wiki/Unix_time">Epoch time</a>, which is the integer <b>count</b> of the number of seconds since January 1, 1970.
<p>
Timestamps are both very important and often overlooked: the moreso in the context of performance analysis,
where <a href="https://calendar.perfplanet.com/2018/time-the-zeroth-performance-metric/">time is the zeroth metric</a>.
In fact, my preferred title would have been, <i>Death to Time Zones</i>
but that choice would probably have made it harder to find the main point here, later on, viz., how to use the Unix epoch time.
<p>
Although there are <a href="http://www.unixtimestampconverter.com">web pages</a> that facilitate such time conversions, there are also shell commands that are often more convenient to use, but not so well known. With that in mind, here are some examples (for future reference) of how to convert between human-time and Unix time.
<h2>Examples</h2>
The following are the bash shell commands for both MacOS and Linux.
<p>
<h3>MacOS and BSD</h3>
Optionally see the human-readable date-time:
<pre class="source-code"><code>
[njg]~% date
Tue Feb 11 10:04:32 PST 2020
</code></pre>
Get the Unix integer time for the current date:
<pre class="source-code"><code>
[njg]~% date +%s
1581444272
</code></pre>
Resolve a Unix epoch integer to date-time:
<pre class="source-code"><code>
[njg]~% date -r 1581444272
Tue Feb 11 10:04:32 PST 2020
</code></pre>
<p>
<h3>Linux</h3>
Optionally see the human-readable date-time:
<pre class="source-code"><code>
[njg]~% date
Tue Feb 11 10:04:32 PST 2020
</code></pre>
Get the Unix integer time for the current date:
<pre class="source-code"><code>
[njg]~% date +%s
1581444272
</code></pre>
Resolve a Unix epoch integer to date-time:
<pre class="source-code"><code>
[njg]~% date -d @1581444272
Tue Feb 11 18:04:32 UTC 2020
</code></pre>
Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com0tag:blogger.com,1999:blog-6977755959349847093.post-82990441940926679302019-06-03T17:52:00.000-07:002019-12-16T20:36:35.988-08:00Book Review: How to Build a Performance Testing Stack From ScratchWriting a technical book is a difficult undertaking. As a technical author myself, I know that writing well is both arduous and tedious. There are no shortcuts. Over the last 40 years or so, computer-based tools have been developed to help authors write <b>printed</b> textbooks, monographs and technical articles. <a href="https://en.wikipedia.org/wiki/LaTeX">LaTeX</a> (pronounced <i>lar-tek</i>) reigns supreme in that realm because it's not just <i>word processing</i> software but, a full-blown digital <a href="https://en.wikipedia.org/wiki/Typesetting">typesetting</a> application that enables authors to produce a single <i>camera-ready</i> PDF file (bits) that is fit for direct printing. Not only does LaTeX correctly typeset characters and mathematical symbols but it can generate the Table of Contents and the corresponding Index of terms, together with correctly cross-referenced callouts for numbered chapters, sections, figures, tables and equations.
<p>
In the meantime, over the past 20 years, the nature of the book itself has become progressively more digital. The ability to render the book-block on <b>digital devices</b> as an "e-book" has made printed hardcopies optional. Although purely digital e-books make reading more ubiquitous, e-book file formats and display quality varies across devices, i.e., laptops, phones tablets, and various e-readers. Any reduction in display quality is offset by virtue of e-books being able to include user <i>interaction</i> and even <i>animation</I>: features entirely beyond the printed book. In that sense, there really has been a revolution in the publishing industry: books are no longer about books, they're now about <i>media</i>.
<p>
Recently, I became aware of an undergraduate calculus textbook that makes powerful use of animation and audio. The author is a academic mathematician and he published it on <b>YouTube</b>! Is that a book or a movie? Somehow, it's a hybrid of both and, indeed, I wish I'd been able to learn from technical "books" like that. Good visuals and animations can make difficult technical concepts much easier to comprehend. Progressive as all that is, when it comes to <b>technical</b> e-books, I'm not aware of any single authoring tool that can match the quality of LaTeX, let alone incorporate user-interaction and animation. If such a thing did exist, I would be all over it. And Markdown doesn't cut it. But, digital authoring tools are continually evolving.
<p>
Matt Fleming (<a href="https://twitter.com/fleming_matt">@fleming_matt</a> on Twitter), the author of <b><i>How to Build a Performance Testing Stack From Scratch</i></b>, opted to use a static e-book format—not because it produces the most readable result, but because it is the best way to reach a wider audience at lower cost than a more expensive print publisher.
The e-publisher in this case is (the relatively unknown)
<a href="https://www.ministryoftesting.com/about-us">Ministry of Testing Ltd</a> in Brighton, UK and is available on Amazon for the <a href="https://www.amazon.com/Build-Performance-Testing-Stack-Scratch-ebook/dp/B07GB8SW4P">Kindle reader</a>. I was also able to read it using iBooks on Mac OS X.
<p>
The range of topics covered is very extensive. I've included the Table of Contents here because it is not viewable on Amazon:
<blockquote>
Part 1
<ul style="list-style-type:none;">
<li> Step 1: Identify Stakeholders
<li> Step 2: Identify What to Measure
<li> Step 3: Test Design
<li> Step 4: Measuring Test Success and Failure
<li> Step 5: Sharing Results
</ul>
Part 2
<ul style="list-style-type:none;">
<li> Understanding Statistics
<li> Latency
<li> Throughput
<li> Statistical Significance
</ul>
Part 3
The Benchmark Hierarchy
Picking Tests
Validating Tests
Part 4
<ul style="list-style-type:none;">
<li> Use a Performance Framework
<li> Ensure the Test Duration is Consistent
<li> Order Tests by Duration
<li> Keep Reproduction Cases Small
<li> Setup The Environment Before Each Test
<li> Make Updating Tests Easy
<li> Errors Should Be Fatal
</ul>
Part 5
<ul style="list-style-type:none;">
<li> Format
<li> Use Individual Sample Data
<li> Detecting Results in the Noise
<li> Outliers
<li> Result Precision
<li> If All Else Fails Use Test Duration
<li> Delivering Results
</ul>
</blockquote>
The overall e-book presentation of "Performance Testing Stack" seems underdeveloped. Most topics could have been greatly expanded. But, as Matt informed me, this is probably due to the content amounting to a concatenation of previously written blog posts. As I said earlier, there are no shortcuts to writing well. The paucity of detail, however, is offset by the shear enthusiasm the Matt brings to his writing: an aspect that deserves separate acknowledgement because performance testing is a very complex subject which can otherwise appear dry and mind-boggling to the uninitiated. And it's the uninitiated that Matt wants to reach. He has written this book in order to encourage the uninitiated reader to seriously consider entering the field.
<p>
Some points that could have been developed further, include:
<ul>
<li> Plots and tables can be expanded for legibility by double clicking on them
<li> The term "benchmark" needs better explaination
<li> Section 2.1 discusses the Harmonic mean but there's no discussion of the Geometric mean.
<li> Section 3.3 on Distributions does not clearly distinguish between analytic (parametric) distributions and sample distributions (which usually have no analytic form).
<li> Section 5 (p.85) on Result Precision needs to discuss the difference between <b>accuracy</b>, <b>precision</b>, and <b>error</b>.
</ul>
<p>
Conversely, Matt's enthusiasm may have gone a bit overboard in his choice of title. The book promises:
<blockquote>
<em>This book will walk you through designing and building a performance testing stack from scratch; step by step from planning and running performance tests through to understanding and analysing the results. If you’re new to performance testing or looking to expand your understanding of this topic then this book is for you!</em>
</blockquote>
Unfortunately, this book doesn't provide enough details to actually build a test stack—which would've been very cool. Rather, it presents a comprehensive <b>overview</b> of all the major concepts that one needs to absorb in order to develop a running performance testing stack. But even with the more limited scope, this book is still important because, off hand, I don't know of any other source where one can be introduced to performance testing without drowning in a sea of terminology, procedures and architectures.
<p>
Ultimately, this e-book is a great starting point for <b>newbies</b>, as well was being a good reminder for <b>seasoned testers</b> about what <i>should be done</i> in good performance tests. Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com0tag:blogger.com,1999:blog-6977755959349847093.post-92069017695673748732019-01-02T17:55:00.000-08:002019-01-02T17:59:35.361-08:00DSConf 2019 Featured Talk"<a href="http://www.perfdynamics.com/Test/dsconf-abst.html">Applying The Universal Scalability Law to Distributed Systems</a>"
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKXliyk_26-mMAzRVrX8SJc9UFY564-iUb7btGzPtrTc-mCFoKKHIYL6C87Qu9oYy6gPiWF73Raf6msZAalbmbVzrmC32SM3Tbmf8_h8CKR9VJmDy3kU8lp9MxpOShXR2XQBB2eWIAJ_o/s1600/Dv4br6lUwAEcHcl.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKXliyk_26-mMAzRVrX8SJc9UFY564-iUb7btGzPtrTc-mCFoKKHIYL6C87Qu9oYy6gPiWF73Raf6msZAalbmbVzrmC32SM3Tbmf8_h8CKR9VJmDy3kU8lp9MxpOShXR2XQBB2eWIAJ_o/s400/Dv4br6lUwAEcHcl.jpg" width="400" height="209" data-original-width="1200" data-original-height="628" /></a></div>
<p>
<b>DSConf'19</b> - <a href="https://dsconf.in">Distributed Systems Conference</a> (scroll down)<br>
Pune, India<br>
11am IST<br>
February 16
<p>
I'm very much looking forward to this event and I thank <a href="https://twitter.com/ShripadAgashe">@ShrivedAgashe</a> for the invitation.
Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com0tag:blogger.com,1999:blog-6977755959349847093.post-73537354707550772962018-06-25T10:09:00.000-07:002018-07-03T15:32:21.512-07:00Guerrilla 2018 Classes Now OpenAll Guerrilla training classes are now <a href="http://www.perfdynamics.com/Classes/schedule.html" target="_blank"><b>open for registration</b></a>.
<ol>
<li> <a href="http://www.perfdynamics.com/Classes/Outlines/guerilla.html" target="_blank"><b>GCAP: Guerrilla Capacity and Performance</b></a> — From Counters to Containers and Clouds
<li> <a href="http://www.perfdynamics.com/Classes/Outlines/gdata.html" target="_blank"><b>GDAT: Guerrilla Data Analytics</b></a> — Everything from Linear Regression to Machine Learning
<li> <a href="http://www.perfdynamics.com/Classes/Outlines/pdqw.html" target="_blank"><b>PDQW: Pretty Damn Quick Workshop</b></a> — Personal tuition for performance and capacity mgmt
</ol>
<p>
The following highlights indicate the kind of thing you'll learn. Most especially, how to make better use of all that <b>monitoring</b> and <b>load-testing</b> data you keep collecting.
<ul>
<li> <a href="https://www.youtube.com/watch?v=HBYGR7ou_6o&t=1s" target="_blank" >How to save millions of dollars with a one-line performance model</a> (<b>video</b>)
<li> <a href="https://goo.gl/xxk68p" target="_blank" >How to minimize chargeback after you lift and shift to the cloud</a> (<b>video</b>)
<li> <a href="http://perfdynamics.blogspot.com/2016/10/crib-sheet-for-emulating-web-traffic.html" target="_blank">How to correctly emulate web traffic on a load-testing rig</a> (<b>PDF</b>)
</ul>
<p>
See what <a href="http://www.perfdynamics.com/Classes/comments.html" target="_blank"> Guerrilla grads are saying</a> about these classes. And how many instructors do you know that are available for you from 9am to 9pm (or later) each day of your class?
<p>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.perfdynamics.com/Classes/schedule.html" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8AMMZ6vu8Flw1voJdF-fisw1VaoPADl2HyAjSd9-AsFtfg3X0D_vYOPccBKHhO_slP9tUDB1KBq_ambrTs8WOXxzR0mKwnbjVGikgkxd2UXjrWhZUYsvEICzSyuNDFD3n71ixEd-f4_Y/s400/ghat-cloud.jpg" width="400" height="300" data-original-width="400" data-original-height="300" /></a></div>
<p>
Who should attend?
<ul>
<li> IT architects
<li> Application developers
<li> Performance engineers
<li> Sysadmins (Linux, Unix, Windows)
<li> System engineers
<li> Test engineers
<li> Mainframe sysops (IBM. Hitachi, Fujitsu, Unisys)
<li> Database admins
<li> Devops practitioners
<li> SRE engineers
<li> Anyone interested in getting beyond performance monitoring
</ul>
<p>
As usual, <a href="http://www.fourpointspleasanton.com">Sheraton Four Points</a> has bedrooms available at the Performance Dynamics discounted rate. The room-booking link is on the registration page.
<p>
Tell a colleague and see you in <b>September</b>!Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com0tag:blogger.com,1999:blog-6977755959349847093.post-82363476467691555922018-06-20T11:24:00.000-07:002018-07-24T19:34:38.875-07:00Chargeback in the Cloud - The MovieIf you were unable to attend the live presentation on <a href="http://perfdynamics.blogspot.com/2018/04/virtual-cloudxchange-2018-conference.html" target=" _blank">cost-effective defenses against chargeback in the cloud</a>, or simply can't get enough performance and capacity analysis for the AWS cloud (which is completely understandable), here's a <a href="https://www.cmg.org/2018/07/exposing-the-cost-of-performance-hidden-in-the-cloud/" target=" _blank">direct link</a> to the video recording on CMG's YouTube channel.
<p>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://www.cmg.org/2018/07/exposing-the-cost-of-performance-hidden-in-the-cloud/" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaeGxVTD1ZzavSSLnNWtZSnpu4763YcnF5R38KLK7VItJ5AaHdFgVgDv8K2MhlNS9vtBXodgExhX6CA_3p-6CdK9ZNb76WqIW624BkoIJnBqr26Nf2D0dv8sv7wQwYa73KJFCZVMFJ98U/s400/youtube-tweet.png" width="400" height="237" data-original-width="1440" data-original-height="852" /></a></div>
<p>
The details concerning how you can do this kind of cost-benefit analysis for your cloud applications will be discussed in the upcoming <a href="http://www.perfdynamics.com/Classes/Outlines/guerilla.html" target=" _blank">GCAP class</a> and the <a href="http://www.perfdynamics.com/Classes/Outlines/pdqw.html" target=" _blank">PDQW workshop</a>.
Check the corresponding <a href="http://www.perfdynamics.com/Classes/schedule.html" target=" _blank">class registration pages</a> for dates and pricing. Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com2tag:blogger.com,1999:blog-6977755959349847093.post-10503330191127704012018-05-20T19:45:00.005-07:002018-11-20T11:15:01.570-08:00USL Scalability Modeling with Three Parameters<b>NOTE: </b>Annoyingly, the remote <i>mathjax</i> server often takes it's sweet time rendering LaTex equations (like, maybe a minute!!!). I don't know if this is deliberate on the part of Google or a bug. It used to be faster. If anyone knows, I'd be interested to hear; especially if there is a way to speed it up. And no, I'm not planning to move to WordPress.
<p>
<b>Update of Oct 2018:</b> Wow! MathJax performance is back. Clearly, whinging is the most powerful performance optimizer. :)
<p>
<h3>
The 2-parameter USL model</h3>
The original USL model, presented in my <a href="http://www.perfdynamics.com/books.html">GCAP book</a> and updated in the blog post
<a href="http://www.perfdynamics.com/Manifesto/USLscalability.html">How to Quantify Scalability</a>, is defined in terms of fitting <b>two</b> parameters $\alpha$ (contention) and $\beta$ (coherency).
\begin{equation}
X(N) = \frac{N \, X(1)}{1 + \alpha \, (N - 1) + \beta \, N (N - 1)} \label{eqn: usl2}
\end{equation}
<p>
Fitting this nonlinear USL equational model to data requires several steps:
<p>
<ol>
<li> <i>normalizing</i> the <b>throughput</b> data, $X$, to determine <i>relative capacity</i>, $C(N)$.
</li>
<li> equation (\ref{eqn: usl2}) is equivalent to $X(N) = C(N) \, X(1)$.
</li>
<li> if the $X(1)$ measurement is missing or simply not available—as is often the case with data collected from production systems—the GCAP book describes an elaborate technique for <i>interpolating</i> the value.
</li>
</ol>
The motivation for a 2-parameter model arose out of a desire to meet the twin goals of:
<ol>
<li> providing each term of the USL with a proper physical meaning, i.e., not treat the USL like a conventional multivariate statistical model (statistics is not math)
</li>
<li> satisfying the <a href="http://perfdynamics.blogspot.com/2011/06/winking-pink-elephant.html">von Neumann criterion</a>: minimal number of modeling parameters
</li>
</ol>
Last year, I realized the 2-paramater constraint is actually overly severe. Introducing a third parameter would make the statistical fitting process even <i>more universal</i>, as well as simplify the overall procedure. For the USL particularly, the von Neumann criterion should not be taken too literally. It's really more of a guideline: fewer is generally better. <a name='more'></a> Additionally, Baron Schwarz told me that he'd had better luck fitting production <a href="https://www.youtube.com/watch?v=AryK1fqWlWU">RDBMS data in Excel</a> by substituting a third parameter into the numerator of the USL. As ever, the question remained: How could this actually work?
<p>
<h3>
The 3-parameter USL model</h3>
Going back to equation (\ref{eqn: usl2}), let's just consider the simplest case where scaling is <i>linear-rising</i>, as would be the case for ideal parallelism. In the linear region, where $\alpha = \beta = 0$, equation (\ref{eqn: usl2}) simplifies to
\begin{equation}
X(N) = N \, X(1) \label{eqn: usl1}
\end{equation}
<p>
In other words, the overall throughput $X(N)$ increases in simple proportion to $N$. The "single-user" throughput, $X(1)$, doesn't change and therefore acts like a <i>constant of proportionality</i>.
<p>
But what happens when we don't know the value of $X(1)$? That means the $X(1)$ factor in equations (\ref{eqn: usl2}) and (\ref{eqn: usl1}) is <b>undefined</b>. We might denote this situation by writing
<p>
\begin{equation}
X(N) = N \, ? \label{eqn: uslx}
\end{equation}
<p>
Of course, that makes no sense, mathematically speaking. As already mentioned, the conventional way out of this situation is to estimate the value of $X(1)$ using mathematical interpolation. But here's the <b>epiphany</b>.
<p>
Rather than using the more complicated interpolation procedure, we can simply appeal to <i>statistical regression</i>! Yes, that's right, we treat the USL equation as a conventional multivariate statistical model. After all, we're already using <i>nonlinear</i> statistical regression to determine the $\alpha$ and $\beta$ parameters. More importantly, since statistics is not math, we can replace equation ($\ref{eqn: uslx}$) with a statement about <i>correlation</i>, rather than strict equality. In statistical models, that's accomplished by introducing another parameter (I'll call it $\gamma$, since that's the third letter of the Greek alphabet) to replace the question mark in equation ($\ref{eqn: uslx}$), namely
<p>
\begin{equation}
X(N) = N \, \gamma \label{eqn: uslg}
\end{equation}
<p>
The new parameter $\gamma$ is just a constant of proportionality that represents the <b>slope</b> of the line associated with ideal parallel scaling. See the plots below.
<p>
And here's a little piece of magic. If we choose $N = 1$ in equation ($\ref{eqn: uslg}$), it becomes $X(1) = \gamma$. So, when the $\gamma$ parameter is determined by statistical regression, it also tells us the estimated value of $X(1)$, whether it was measured or missing. In other words, we don't need to do any explicit interpolation because the nonlinear regression procedure does it automatically by fitting the third parameter.
<p>
Equation (\ref{eqn: usl2}) is now replaced by a 3-parameter version of the USL model:
\begin{equation}
X(N) = \frac{N \, \gamma}{1 + \alpha \, (N - 1) + \beta N \, (N - 1)} \label{eqn: usl3}
\end{equation}
<p>
Unlike the 2-parameter USL, equation (\ref{eqn: usl3}) can be fitted directly to your throughput measurements without the need to do any data normalization or interpolation. The following examples show the results of fitting the 3-parameter USL model.
<p>
<h3>
Load-test data</h3>
These are load-test data and the "single-user" throughput was measured as $X(1) = 955.16$ per unit time. The 3-parameter USL fit is summarized in the following plot.
<p>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiStewKaxM96H8Muc85LX4yB8tq6cuAAgzCEWUQ-lSHzS7pMuOSSOw1r-pu3Up6r0CGAvmPCwRX-CB4XhETmSkDmlP-9sRB5yh05b420ZFV7iJ90iW-srt5A6Zr4Ib41b75yKY03yTFoDo/s1600/Rplot.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="448" data-original-width="629" height="285" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiStewKaxM96H8Muc85LX4yB8tq6cuAAgzCEWUQ-lSHzS7pMuOSSOw1r-pu3Up6r0CGAvmPCwRX-CB4XhETmSkDmlP-9sRB5yh05b420ZFV7iJ90iW-srt5A6Zr4Ib41b75yKY03yTFoDo/s400/Rplot.png" width="400" /></a></div>
<p>
The fitted value of $\gamma = 995.65$, which is the estimated value of $X(1)$. It can also be regarded as the slope of the linear-rising throughput indicated by the sloping red line on the left of the plot.
<p>
<h3>
Production data</h3>
These data are from a continuously running production system and thus, no $X(1)$ was ever produced.
<p>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgs6o9ZoRf5uArwbG8Es1M8VHtShRisWrerVdjSX7wLOoLtOQSCOL69Lhv31-ruHLIqZqM8koccCYuAuuW87M6qIxvu_ZsMf3Z3PbhQMlaErqxsgXkkVP9j-MaRwZIdyjcqj0TpEXtpL4k/s1600/Rplot-tomcat.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="448" data-original-width="629" height="285" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgs6o9ZoRf5uArwbG8Es1M8VHtShRisWrerVdjSX7wLOoLtOQSCOL69Lhv31-ruHLIqZqM8koccCYuAuuW87M6qIxvu_ZsMf3Z3PbhQMlaErqxsgXkkVP9j-MaRwZIdyjcqj0TpEXtpL4k/s400/Rplot-tomcat.png" width="400" /></a></div>
<p>
The fitted value of $\gamma = 3.22$ is also equivalent to the estimated value of $X(1)$. Similarly, it can be regarded as the slope of the linear-rising throughput on the left of the plot. Interestingly, in these data, $\alpha = 0$, while $beta$ is non-zero. That suggests there is no significant contention in the workload but there is some data exchange coherency at play.
<p>
One word of caution. Fitting the 3-parameter USL can be more sensitive to the actual data, especially with a large number of production data scatter points. I'll go into all this, and more, in the upcoming <a href="http://www.perfdynamics.com/Classes/schedule.html">Guerrilla training classes</a>. Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com9tag:blogger.com,1999:blog-6977755959349847093.post-73203999715148476642018-04-22T13:09:00.001-07:002018-06-26T07:55:31.009-07:00The Geometry of Latency... AKA <a href="https://en.wikipedia.org/wiki/Hyperbola">hyperbolae</a>.
<p>
Here's a mnemonic tabulation based on <i>dishes</i> and <i>bowls</i>:
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfJAmJQbiGHasvw6viX8Z8DkyUiaT2eL7AmoTO8Y7zlhPsMz0A1IZ2Iwt3HX7OC8FUMGKv5gGU367WApATX1tt_QK0WcYR_feZsJbjnBIxgv0OEZLX12FGjlpLmA7JdnU44KmNHwBDB6A/s1600/Screen+Shot+2018-04-22+at+Sun%252C+Apr+22%252C+2018+-+52.25+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfJAmJQbiGHasvw6viX8Z8DkyUiaT2eL7AmoTO8Y7zlhPsMz0A1IZ2Iwt3HX7OC8FUMGKv5gGU367WApATX1tt_QK0WcYR_feZsJbjnBIxgv0OEZLX12FGjlpLmA7JdnU44KmNHwBDB6A/s400/Screen+Shot+2018-04-22+at+Sun%252C+Apr+22%252C+2018+-+52.25+PM.png" width="400" height="201" data-original-width="778" data-original-height="390" /></a></div>
<p>
Hopefully this makes amends for the more complicated explanation I wrote for CMG back in 2009 entitled: <a href="https://www.cmg.org/publications/measureit/2009-2/mit62/measureit-issue-7-08-mind-your-knees-and-queues/">"Mind Your Knees and Queues: Responding to Hyperbole with Hyperbolæ"</a>, which I'm pretty sure almost nobody understood.
Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com1tag:blogger.com,1999:blog-6977755959349847093.post-14633564217992791602018-04-21T08:12:00.000-07:002018-07-25T18:20:42.344-07:00Virtual cloudXchange 2018 ConferenceOur abstract has been accepted for presentation at the FREE <a href="https://www.cmg.org/2018/07/exposing-the-cost-of-performance-hidden-in-the-cloud/" target=" _blank">cloudXchange online event</a> to be held by CMG on <b>June 19th at 10am Pacific</b> (5pm UTC).
[<a href="https://speakerdeck.com/drqz/exposing-the-cost-of-performance-hidden-in-the-cloud">Extended slides</a>]
<p>
<blockquote>
<center>
<h2>Exposing the Cost of Performance<br>Hidden in the Cloud</h2>
<br>
Neil Gunther<br>
<i>Performance Dynamics, Castro Valley, California</i>
<br><br>
Mohit Chawla<br>
<i>Independent Systems Engineer, Hamburg, Germany</i>
<p>
10am Pacific Time on June 19, 2018
</center>
<p>
Whilst offering lift-and-shift migration and versatile elastic capacity, the cloud also reintroduces an old mainframe concept—chargeback—which rejuvenates the need for performance analysis and capacity planning. Combining production JMX data with an appropriate performance model, we show how to assess fee-based EC2 configurations for a mobile-user application running on a Linux-hosted Tomcat cluster. The performance model also facilitates ongoing cost-benefit analysis of various EC2 Auto Scaling policies.
</blockquote>
<p>Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com0tag:blogger.com,1999:blog-6977755959349847093.post-18361813689782852322018-03-14T09:11:00.000-07:002018-06-26T07:55:49.208-07:00WTF is Modeling, Anyway? A conversation with performance and capacity management veteran Boris Zibitsker, on his <a href="https://www.youtube.com/watch?v=HBYGR7ou_6o&t=121s" target="_blank"><i>BEZnext</i> channel</a>, about how to save multiple millions of dollars with a <b>one-line</b> performance model (at 21:50 minutes into the video) that has less than 5% error. I wish my <a href="http://perfdynamics.blogspot.com/2013/04/adding-percentiles-to-pdq.html" target="_blank" >PDQ models</a> were that good. :/
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://www.youtube.com/watch?v=HBYGR7ou_6o&t=1s" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjm1j0q9Zg8g3BZOihLUhy20Bk3rWNQ4P0vtf0EwUTM2O2ZYNgCUyv_MW2sopCgeWxOFM1aa8ffvThuOcpQOE-hbgytXVGWttu6H63KyLtopPHjs4dtn9mPJj9qxCzBCZkHCHkTBs-lbZ4/s400/BEZnext.png" width="400" height="248" data-original-width="1438" data-original-height="891" /></a></div>
<p>
The strength of the model turns out to be its <b>explanatory</b> power, rather than prediction, per se. However, with the correct explanation of the performance problem in hand (which also proved that all other <b>guesses</b> were wrong), this model correctly predicted a 300% reduction in application response time for essentially no cost. Modeling doesn't get much better than this.
<p>
<h3>Footnotes</h3>
<ol>
<li> According to <a href="https://books.google.com/books?id=FxMQJqS3etcC&pg=PA1&lpg=PA1&dq=pricing+IBM+SP2&source=bl&ots=UYdWt0-vxt&sig=zZU1a-H02nGx_IA7LpAfqyXk9uM&hl=en&sa=X&ved=0ahUKEwim38ipvd3ZAhXri1QKHdeyDsw4ChDoAQgwMAM#v=onepage&q=pricing%20IBM%20SP2&f=false" target="_blank">Computer World in 1999</a>, a 32-node IBM SP2 cost $2 million to lease over 3 years. This SP2 cluster was about 6 times bigger.
<li> Because of my vain attempt to suppress details (in the interests of video length), Boris gets confused about the kind of files that are causing the performance problem (near 26:30 minutes). They're not regular data files and they're not executable files. The executable is already running but sometimes waits—for a long time. The question is, waits for what? They are, in fact, special font files that are requested by the X-windows application (the <i>client</i>, in X parlance). These <b>remote</b> files may also get cached, so it's complicated. In my <a href="http://www.perfdynamics.com/Classes/schedule.html">GCAP class</a>, I have more time to go into this level of detail. Despite all these potential complications, my 'log model' accurately predicts the mean application launch time.
<li> Log_2 assumes a binary tree organization of font files whereas, Log_10 assumes a denary tree.
<li> Question for the astute viewer. Since these geophysics applications were all developed in-house, how come the developers never saw the performance problems before they ever got into production? Here's <a href="https://twitter.com/DrQz/status/979117391807311872">a hint</a>.
<li> Some ppl have asked why there's no video of me. This was the first time Boris had recorded video of a Skype session and he pushed the wrong button (or something). It's prolly better this way. :P
</ol>Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com1tag:blogger.com,1999:blog-6977755959349847093.post-9429152204746973442018-02-21T00:08:00.000-08:002018-06-26T07:56:10.623-07:00CPU Idle Is Not Like White SpaceThis post seems like it ought to be too trite to write but, I see the following performance gotcha cropping up over and over again.
<p>
Under pressure to consolidate resources, usually driven by management and especially regarding processor capacity, there is often an urge to "use up" any idle processor cycles. Idle processor capacity tends to be viewed like it's <a href="https://www.thefreedictionary.com/Whitespace">whitespace</a> on a written page—just begging to be filled up.
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbP6gTC47K2jU_3xj2YAqZcmvIqtf7hyxmyHB5UauUBwGv9L4-x5G9QeTf_qmuci_LaI_0MwTIIvjpPnAUzDfXbnNgieXKTIXQJn8uwtyMSB1KswtGL2IUITNpe1dcLvqZGqnQBXrDU7s/s1600/whitespace1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbP6gTC47K2jU_3xj2YAqZcmvIqtf7hyxmyHB5UauUBwGv9L4-x5G9QeTf_qmuci_LaI_0MwTIIvjpPnAUzDfXbnNgieXKTIXQJn8uwtyMSB1KswtGL2IUITNpe1dcLvqZGqnQBXrDU7s/s400/whitespace1.png" width="309" height="400" data-original-width="1237" data-original-height="1600" /></a></div>
<p>
The logical equivalent of filling up the "whitespace" is absorbing idle processor capacity by migrating applications that are currently running on other servers and turning those excess servers off or using them for something else.
<a name='more'></a>
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhG-vLjWPeKsLivZeIjsZgPV2r5QIFtjRJU7MtWR_7KN7wfMw2n0TvyWuk-qpRUeojASmTuajlWh454do7rxeMumCOv6jcBp02FVWZ_9k113sEeJuOGn6ru_D7EgLdgxxA8nMhjlg_1s3I/s1600/whitespace2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhG-vLjWPeKsLivZeIjsZgPV2r5QIFtjRJU7MtWR_7KN7wfMw2n0TvyWuk-qpRUeojASmTuajlWh454do7rxeMumCOv6jcBp02FVWZ_9k113sEeJuOGn6ru_D7EgLdgxxA8nMhjlg_1s3I/s400/whitespace2.png" width="309" height="400" data-original-width="1237" data-original-height="1600" /></a></div>
<p>
The blindspot, however, is that idle processor capacity is not like whitespace and the rush to absorb it is likely to have unintended consequences. The reason is that performance metrics are neither isolated nor independent of one another. Most metrics are:
<ol>
<li> related to one another due to <b>interdependencies</b> between various computing subsystems
<li> related in a <b>nonlinear</b> way: a small change in one metric can cause a large change in another
</ol>
The first couple of days of my <a href="http://www.perfdynamics.com/Classes/schedule.html">Guerrilla Capacity Planning</a> course lays out a consistent framework that demonstrates how all the familiar performance metrics are related.
<p>
For example, application <a href="https://www.cmg.org/publications/measureit/2009-2/mit62/measureit-issue-7-08-mind-your-knees-and-queues/">response time depends nonlinearly on processor utilization</a>. In the above case, it may be have been forgotten that the processor utilization must be kept <i>low</i> in order for the application to meet its response time SLA. A lot of idle processor cycles can <i>appear</i> to be unused processor capacity only because there is no obvious <i>warning sign</i> that low CPU is a necessary condition for correct application performance.
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhX0H7T0I2koU3TlVKK0oOJe8uBUtGvPvOcUr3KmSwax9k9iDQ2tP2eIKWqVnyb2xWb65eKLy41P_jx2XHRLldL02iFdRupWDu1qS4oYRimcj8ttdG155NOcbhFOh96COO5T1AmD35i1pc/s1600/whitespace3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhX0H7T0I2koU3TlVKK0oOJe8uBUtGvPvOcUr3KmSwax9k9iDQ2tP2eIKWqVnyb2xWb65eKLy41P_jx2XHRLldL02iFdRupWDu1qS4oYRimcj8ttdG155NOcbhFOh96COO5T1AmD35i1pc/s400/whitespace3.png" width="309" height="400" data-original-width="1237" data-original-height="1600" /></a></div>
<p>
Even on a printed page, whitespace is not usually an invitation to start scribbling on it. Similarly, a notification equivalent to "<a href="http://www.this-page-intentionally-left-blank.org">This page intentionally left blank</a>" would be a useful reminder in the context of potential application migration.
<p>
Of course, any page that says "This page left blank" isn't blank, but that's a topic for a different discussion. :)Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com0tag:blogger.com,1999:blog-6977755959349847093.post-56253409874596394082017-08-04T11:15:00.000-07:002017-09-04T08:25:00.500-07:00Guerrilla Training September 2017This year offers a new 3-day class on <a href="http://www.perfdynamics.com/Tools/PDQ.html">applying PDQ</a> in your workplace. The classic Guerrilla classes, <a href="http://www.perfdynamics.com/Classes/Outlines/guerilla.html">GCAP</a> and <a href="http://www.perfdynamics.com/Classes/Outlines/gdata.html">GDAT</a>, are also being presented.
<p>
Who should attend?
<ul>
<li> architects
<li> application developers
<li> performance engineers
<li> sys admins
<li> test engineers
<li> mainframe sysops
<li> database admins
<li> webops
<li> anyone interested in getting beyond ad hoc performance analysis and capacity planning skills
</ul>
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEie3t7VCk2lDozykOCQVuL1fftRg2ySZb-UV2jtU6HX7dlHmduxU3ddL85j25TJdtn-qF9ndchGvwN8oQR0-uwn9rt607yrMSpGhJklOK0AsY24sae5rGUaiRCu985ITXz5w1mcdEatVwo/s1600/ghat-cloud.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEie3t7VCk2lDozykOCQVuL1fftRg2ySZb-UV2jtU6HX7dlHmduxU3ddL85j25TJdtn-qF9ndchGvwN8oQR0-uwn9rt607yrMSpGhJklOK0AsY24sae5rGUaiRCu985ITXz5w1mcdEatVwo/s400/ghat-cloud.jpg" /></a></div>
<p>
Some course highlights:
<ul>
<li> There are only 3 performance <b>metrics</b> you need to know
<li> How to <b>quantify scalability</b> with the <a href="http://www.perfdynamics.com/Manifesto/USLscalability.html">Universal Scalability Law</a>
<li> <a href="http://queue.acm.org/detail.cfm?id=2789974">Hadoop performance</a> and capacity management
<li> <b>Virtualization</b> Spectrum from hyper-threads to cloud services
<li> How to detect <b>bad data</b>
<li> Statistical <b>forecasting</b> techniques
<li> <b>Machine learning</b> algorithms applied to performance data
</ul>
<p>
<a href="http://www.perfdynamics.com/Classes/schedule.html">Register online</a>. Early-bird discounts run through the end of <b>August</b>.
<p>
As usual, <a href="http://www.fourpointspleasanton.com">Sheraton Four Points</a> has bedrooms available at the Performance Dynamics discounted rate. Link is on the registration page.
<p>
Also see what <a href="http://www.perfdynamics.com/Classes/comments.html">graduates are saying</a> about these classes.
<p>
Tell a colleague and see you in <b>September</b>!Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com1tag:blogger.com,1999:blog-6977755959349847093.post-31175084192355008392017-03-15T10:53:00.002-07:002020-08-19T10:30:51.583-07:00Morphing M/M/m: A New View of an Old QueueThe following abstract has been accepted for presentation at the 21st Conference of the International Federation of Operational Research Societies — <a href="http://ifors2017.ca" target="_blank">IFORS 2017, Quebec City, Canada</a>.
<p>
<ul>
<li> <b>Update</b> July 31, 2017: Here are my
<a href="https://speakerdeck.com/drqz/m-a-new-view-of-an-old-queue" target="_blank">IFORS slides</a>
<li> <b>Update</b> June 08, 2018: In response to an audience question at my IFORS 2017 session, I have now demonstrated that there is an upper bound for the error in the morphing approximation. See footnotes below.
<li> <b>Update</b> August 18, 2020: This <a href="https://arxiv.org/abs/2008.06823" target="_blank">new paper</a> has the complete exact solution method that I had been seeking all along.
</ul>
<p>
<blockquote>
This year is the centenary of A. K. Erlang's paper [1] on the determination of waiting times in an M/D/m queue with $m$ telephone lines.<sup>*</sup> Today, M/M/m queues are used to model such systems as, call centers [3], multicore computers [4,5] and the Internet [6,7]. Unfortunately, those who should be using M/M/m models often do not have sufficient background in applied probability theory. Our remedy defines a <i>morphing approximation</i><sup>†</sup> to the exact M/M/m queue [3] that is accurate to within 10% for typical applications<sup>‡</sup>. The morphing formula for the residence-time, $R(m,\rho)$, is both simpler and more intuitive than the exact solution involving the Erlang-C function. We have also developed an <a href="http://perfdynamics.blogspot.com/2016/07/erlang-redux-resolved-this-time-for-real.html" target="_blank" >animation of this morphing process</a>. An outstanding challenge, however, has been to elucidate the nature of the corrections that transform the approximate morphing solutions into the exact Erlang solutions. In this presentation, we show:
<ul>
<li> The morphing solutions correspond to the $m$-roots of unity in the complex $z$-plane.
<li> The exact solutions can be expressed as a rational function, $R(m,z)$.
<li> The poles of $R(m,z)$ lie inside the unit disk, $|z| < 1$, and converge around the Szego; curve [8] as $m$ is increased.
<li> The correction factor for the morphing model is defined by the deflated polynomial belonging to $R(m,z)$.
<li> The pattern of poles in the $z$-plane provides a convenient visualization of how the morphing solutions differ from the exact solutions.
</ul>
</blockquote>
<p style="font-size:85%;">
* Originally, Erlang assumed the call <i>holding time</i>, or mean service time $S$, was deterministic with unit period, $S=1$ [1,2]. The generalization to exponentially distributed service periods came later. Ironically, the exponential case is easier to solve than the apparently simpler deterministic case. That's why the M/D/1 queue is never the first example discussed in queueing theory textbooks.<br>
† The derivation of the <i>morphing model</i> is presented in Section 2.6.6 of the 2005 edition of [4], although the word "morphing" is not used there. The reason is, I didn't know how to produce the exact result from it, and emphasizing it would likely have drawn unwarranted attention from Springer-Verlag editors.
By the time I was writing the 2011 edition of [4], I was certain the approximate formula did reflect the morphing concept in its own right, even though I still didn't know how to connect it to the exact result. Hence, the verb "morphs" timidly makes its first and only appearance in the boxed text following equation 4.61.<br>
‡ The relative error peaks at 10% for $m \sim 20$ and $\rho \sim 90\%$, then peaks again at 20% for $m \sim 2000$ and the servers running 99.9% busy. However, the rate of increase in peak error attenuates such that the maximum error is less than 25%, even as $m \rightarrow \infty$ and $\rho \rightarrow 100\%$. A <a href="https://twitter.com/DrQz/status/1005112190733508608" target="_blank">plot of the corresponding curves</a> gives a clearer picture. This behavior is not at all obvious. Prior to this result, it could have been that the relative error climbed to 100% with increasing $m$ and $\rho$.
<p>
<h3>References</h3>
<ol>
<li> A. K. Erlang, "Solution of Some Problems in the Theory of Probabilities of Significance in Automatic Telephone Exchanges," Electroteknikeren, v. 13, p. 5, 1917. <br>
<li> A. K. Erlang, "The Theory of Probabilities and Telephone Conversations," Nyt Tidsskrift for Matematik B, vol 20, 1909. <br>
<li> E. Chromy, T. Misuth, M. Kavacky, "Erlang C Formula and Its Use In The Call Centers," Advances in Electrical and Electronic Engineering, Vol. 9, No. 1, March 2011.
<li> N. J. Gunther, <a href="http://www.perfdynamics.com/iBook/ppa_new.html" target="_blank" ><i>Analyzing Computer System Performance with Perl::PDQ</i></a>, Springer-Verlag, 2005 and 2011. <br>
<li> N. J. Gunther, S. Subramanyam, and S. Parvu, "A Methodology for Optimizing Multithreaded System Performance on Multicore Platforms," in <a href="https://www.amazon.com/dp/0470936908/" target="_blank" ><i>Programming Multicore and Many-core Computing Systems</i></a>, eds. S. Pllana and F. Xhafa,
Wiley Series on Parallel and Distributed Computing, February 2017.
<li> N. J. Gunther, "Numerical Investigations of Physical Power-law Models of Internet Traffic Using the Renormalization Group," IFORS 2005, Honolulu, Hawaii, July 11—15. <br>
<li> T. Bonald, J. W. Roberts, "Internet and the Erlang formula," ACM SIGCOMM Computer Communication Review, Volume 42, Number 1, January 2012.
<li> C. Diaz Mendoza and R. Orive, "The Szegő curve and Laguerre polynomials with large negative parameters," Journal of Mathematical Analysis and Applications, Volume 379, Issue 1, Pages 305—315, 1 July 2011.
</ol>Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com9tag:blogger.com,1999:blog-6977755959349847093.post-41644548831594282102017-01-17T19:26:00.000-08:002017-11-11T08:01:34.267-08:00GitHub Growth Appears Scale Free<b>Update of Thursday, August 17, 2017:</b>
It's looks like we can chalk up another one for the scale-free model (described below) as Github apparently surpasses 20 million users. Outgoing CEO Wanstrath mentioned this number in an emailed statement to
<a href="http://www.businessinsider.com/github-ceo-chris-wanstrath-to-step-down-become-executive-chairman-2017-8">Business Insider</a>.
<blockquote>
<i>"As GitHub approaches 700 employees, with more than $200M in ARR, accelerating growth, and more than 20 million registered users, I'm confident that this is the moment to find a new CEO to lead us into the next stage of growth. ....."</i>
</blockquote>
<h3>The Original Analysis</h3>
In 2013, a <a href="http://redmonk.com/dberkholz/2013/01/21/github-will-hit-5-million-users-within-a-year/" target="_blank"> Redmonk blogger claimed</a> that the growth of GitHub (GH) users follows a certain type of diffusion model
called <a href="https://en.wikipedia.org/wiki/Bass_diffusion_model">Bass diffusion</a>. Here, growth refers to the number of unique user IDs as a function of time, not the number project repositories, which can have a high degree of multiplicity.
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmaliUhCt38gxfUpcqNxUfcYc_i4wycsI2qVAi2Tuj5sPLskTgOhY9lU60PSdCUx5fBKRmnObyZmrIc0REi2zfp1QiEj5m-nQimYy4-VtOisyPrd8it6-QyHftj8QhsJ0b8EYDIpzefTE/s1600/tweet%252Bpic.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmaliUhCt38gxfUpcqNxUfcYc_i4wycsI2qVAi2Tuj5sPLskTgOhY9lU60PSdCUx5fBKRmnObyZmrIc0REi2zfp1QiEj5m-nQimYy4-VtOisyPrd8it6-QyHftj8QhsJ0b8EYDIpzefTE/s320/tweet%252Bpic.png" width="284" height="320" /></a></div>
<p>
In a response, I tweeted a plot that suggested GH growth might be following a power law, aka <i>scale free</i> growth. The tell-tale sign is the asymptotic linearity of the growth data on <b>double-log</b> axes, which the original blog post did not discuss. The periods on the x-axis correspond to years, with the first period representing calendar year 2008 and the fifth period being the year 2012. <a name='more'></a>
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFtvZ4oYA2DB6Q4h4aLN3g36QKQkWI2I8WFkUv44fLtb2buiTj8m-q06WoIjxTh1w6uc_2-UfEU9Az67ZgfxSSi_608zhLg7XzdMx79jlLeZo9ECS9LfMla083Yt3SzS7jQ2WBwbE1aUI/s1600/406378aa.2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFtvZ4oYA2DB6Q4h4aLN3g36QKQkWI2I8WFkUv44fLtb2buiTj8m-q06WoIjxTh1w6uc_2-UfEU9Az67ZgfxSSi_608zhLg7XzdMx79jlLeZo9ECS9LfMla083Yt3SzS7jQ2WBwbE1aUI/s320/406378aa.2.jpg" width="320" height="175" /></a></div>
<p>
<a href="https://en.wikipedia.org/wiki/Scale-free_network">Scale free networks</a> can arise from <i>preferential attachment</i> to super-nodes that have a higher vertex degree and therefore more connections to other nodes, i.e., a kind of rich-get-richer effect. Similarly for GH growth viewed as a particular kind of social network.
The interaction between software developers using GH can be thought of as involving super-nodes that correspond to influential users attracting prospective GH users to open a new account and contribute to their project.
<p>
On this basis, I predicted GH would reach 4 million users during October 2013 and 5 million users during April 2014 (yellow points in the <i>Linear axes</i> plot below). In fact, GH reached those values slightly earlier than predicted by the power law model, and slightly later than the dates predicted by the diffusion model (modulo unreported errors in the data).
<p>
Since 2013, new data has been reported so, I extended my previous analysis. Details of the respective models are contained in the R script at the end of this post.
In the <i>Linear axes</i> plot below, both the diffusion model and power model essentially form an envelope around the newer data: diffusive on the upper side (red curve) and power law on the lower side (blue curve). In thise sense, it could be argued that the jury is still out on which model offers the more reliable predictions.
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjsUzZyyGqAze3HdwqHkzbugYGmZNeNeq_TfmerwfHUgfBaWhvWRIQ8agC6V-64KY9042CjjzUGHv-6cn1nnSSa35zrG_0j4GGbQpgKIT9uT1CuapGT635bZb27O73vz4Q9KdvF4y0vvTM/s1600/rplot-linear.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjsUzZyyGqAze3HdwqHkzbugYGmZNeNeq_TfmerwfHUgfBaWhvWRIQ8agC6V-64KY9042CjjzUGHv-6cn1nnSSa35zrG_0j4GGbQpgKIT9uT1CuapGT635bZb27O73vz4Q9KdvF4y0vvTM/s400/rplot-linear.png" width="400" height="341" /></a></div>
<p>
However, there is an aspect of the diffusion model that was overlooked in 2013. It predicts that GH growth will eventually <b>plateau</b> at 20 million users in 2020 (the 12th period, not shown) because it is a type of
<a href="https://en.wikipedia.org/wiki/Logistic_function">logistic function</a> that has a characteristic <a href="https://en.wikipedia.org/wiki/Sigmoid_function" target="_blank">sigmoidal</a> or 'S' shape. The beginnings of this leveling off (the top of the 'S') is apparent in the 10th period (i.e., 2017).
By contrast, the power law model predicts that GH will reach 23.65 million users by the end of 2017 (yellow point). Whereas the two curves envelope the more recent data in periods 6–9, they start to diverge significantly in the 10th period.
<blockquote>
<em>"GitHub is not the only player in the market. Other companies like GitLab are doing a good job but GitHub has a huge head start and the advantage of the network effect around public repositories. Although GitHub’s network effect is weaker compared to the likes of Facebook/Twitter or Lyft/Uber, they are the default choice right now."</em> —<a href="https://medium.com/@moritzplassnig/github-is-doing-much-better-than-bloomberg-thinks-here-is-why-a4580b249044#.5wyb1l2qh" target="_blank">GitHub is Doing Much Better Than Bloomberg Thinks</a>
</blockquote>
Although there will inevitably be an equilibrium bound on the number of active GH users, it seems unlikely to be as small as 20 million, given the combination of GH's first-mover advantage and its current popularity. Presumably the private investors in GH also hope it will be a large number. This year will tell.
<p>
<pre class="source-code"><code>
# Data source ... https://classic.scraperwiki.com/scrapers/github_users_each_year/
#LINEAR axes plot
plot(df.gh3$index, df.gh3$users, xlab="Period (years)",
ylab="Users (million)", col="gray",
ylim=c(0, 3e7), xaxt="n", yaxt="n")
axis(side=1, tck=1, at=c(0, seq(12,120,12)), labels=0:10,
col.ticks="lightgray", lty="dotted")
axis(side=2, tck=1, at=c(0, 10e6, 20e6, 30e6), labels=c(0,10,20,30),
col.ticks="lightgray", lty="dotted")
# Simple exp model
curve(coef(gh.exp)[2] * exp(coef(gh.exp)[1] * (x/13)),
from=1, to=108, add=TRUE, col="red2", lty="dot dash")
# Super-exp model
curve(49100 * (x/13) * exp(0.54 * (x/13)),
from=1, to=120, add=TRUE, col="red", lty="dashed")
# Bass diffusion model
curve(21e6 * ( 1 - exp(-(0.003 + 0.83) * (x/13)) ) / ( 1 + (0.83 / 0.003) * exp(-(0.003 + 0.83) * (x/13)) ),
from=1, to=120, add=TRUE, col="red")
# Power law model
curve(10^coef(gh.fit)[2] * (x/13)^coef(gh.fit)[1], from=1, to=120, add=TRUE,
col="blue")
title(main="Linear axes: GitHub Growth 2008-2017")
legend("topleft",
legend=c("Original data", "New data", "Predictions", "Exponentital", "Super exp", "Bass diffusion", "Scale free"),
lty=c(NA,NA,NA,4,2,1,1), pch=c(1,19,21,NA,NA,NA,NA),
col=c("gray", "black", "yellow", "red", "red", "red", "blue"),
pt.bg = c(NA,NA,"yellow",NA,NA,NA,NA),
cex=0.75, inset=0.05)
</code></pre>Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com1tag:blogger.com,1999:blog-6977755959349847093.post-7623978816404452482016-10-08T14:55:00.010-07:002021-09-22T14:26:18.816-07:00Crib Sheet for Emulating Web TrafficOur paper entitled, <a href="https://arxiv.org/abs/1607.05356" target=" _blank"><em>How to Emulate Web Traffic Using Standard Load Testing Tools</em></a> (PDF) is now available online and will be presented at the upcoming <a href="https://www.cmg.org/event/performance-capacity-2016-conference/" target=" _blank">CMG conference in November</a>.
<p>
<blockquote>
<b>Presenter</b>: James Brady (co-author: Neil Gunther)<br>
<b>Session Number</b>: 436<br>
<b>Subject Area</b>: APM<br>
<b>Session Date</b>: Wed, November 9, 2016<br>
<b>Session Time</b>: 1:00 PM - 2:00 PM<br>
<b>Session Room</b>: PortofinoB
</blockquote>
<a name='more'></a>
<p>
The motivation for this work harks back to a <a href="https://groups.google.com/forum/#!msg/guerrilla-capacity-planning/muhF8eqoFVY/odqgXOFZvZUJ" target=" _blank">Guerrilla forum in 2014</a> that essentially centered on the same topic as the title of our paper. It was clear from that discussion that commenters were talking at cross purposes because of misunderstandings on many levels. I had already written a <a href="http://perfdynamics.blogspot.com/2010/05/emulating-internet-traffic-in-load.html">much earlier blog post</a> on the key queue-theoretic concept, viz., holding the $N/Z$ ratio constant as the load $N$ is increased, but I was incapable of describing how that concept should be implemented in a real load-testing environment.
<p>
On the other hand, I knew that Jim Brady had presented a similar implementation in his 2012 CMG paper, based on a statistical analysis of the load-generation traffic. There were a few details that I couldn't quite reconcile in Jim's paper but, at the CMG 2015 conference in San Antonia, I suggested that we should combine our separate approaches and aim at a definitive work on the subject. After nine months gestation (ugh!), this 30-page paper is the result.
<p>
Although our paper doesn't contain any new invention, per se, the novelty lies in how we needed to bring together so many disparate and subtle concepts in precisely the correct way to reach a complete and consistent methodology. The complexity of this task was far greater than either of us had imagined at the outset. The hyperlinked Glossary should help with the terminology, but because there are so many interrelated parts, I've put together the following crib notes in an effort to help performance engineers get through it (since they're the ones that most stand to benefit). The <a href="https://arxiv.org/abs/1607.05356" target=" _blank">key results of our paper</a> are indicated by <font color="red">◀</font>
<ol>
<li> Standard load-test tools only allow <b>finite</b> number of <i>virtual</i> users
<li> Web traffic is characterized by an <b>unlimited</b> number of <i>real</i> users
<li> Attention usually focused on <b>SUT</b> (system under test) performance
<li> We focus on the <b>GEN</b> (load generator) characteristics
<li> Measure <b>distribution</b> of web requests and their mean arrival rate
<li> Web traffic should be a <b>Poisson process</b> (cf. <a href="https://web.archive.org/web/20110719123704/http://oldwww.com.dtu.dk/teletraffic/erlangbook/pps131-137.pdf" target=" _blank">A.K. Erlang 1909</a>) <font color="red">◀</font>
<li> Requires statistically <b>independent</b> arrivals, i.e., no correlations
<li> Independent requests should arrive <b>asynchronously</b> into the SUT
<li> But virtual-user requests are <b>synchronized</b> (correlated) in SUT queues <font color="red">◀</font>
<li> De-correlate arrivals by shrinking SUT <b>queues</b> (<b>Principle A</b>): <font color="red">◀</font>
<ul>
<li> Shrink by increasing think delay $Z$ with user load as a fixed ratio
<li> Traffic rate $\lambda$ approaches a <b>constant</b> $\lambda = N/Z$ as SUT queues shrink
</ul>
<li> <b>Check</b> requests are Poisson by measuring the <i>coefficient of variation</i> ($CoV$)
<li> Require $CoV \approx 1$ for a Poisson process (<b>Principle B</b>) <font color="red">◀</font>
</ol>
Originally, I assumed the paper would be no more than a third it's current length. Wrong! My only defense is: it's all there, you just need to read it. (tl;dr doesn't apply) Apologies in advance, but hopefully, these crib notes will help you.
Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com4tag:blogger.com,1999:blog-6977755959349847093.post-48012112491704923562016-10-01T17:11:00.000-07:002018-12-22T11:36:15.742-08:00A Clue for Remembering Little's LawsDuring the <a href="http://www.perfdynamics.com/Classes/Outlines/gdata.html">Guerrilla Data Analysis class</a> last week, alumnus Jeff P. came up with this novel mnemonic device for remembering all <a href="http://perfdynamics.blogspot.com/2014/07/a-little-triplet.html">three forms of Little's law</a> that relate various queueing metrics to the mean arrival rate $\lambda$.
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQtv80HJqokaAPA86X1AEoFFDBuUeiCfgqcqH0CNeCYYB3mmp6P5sX36mUn0PKxqoJAX17ur4RFu6_t8_e63pmKDErGl6zxaIpBpAQ8VHQxYruj-LMcsKq-5Bm-eNkZh6_mL_7uvUmpZ0/s1600/little-qlu-annotated.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQtv80HJqokaAPA86X1AEoFFDBuUeiCfgqcqH0CNeCYYB3mmp6P5sX36mUn0PKxqoJAX17ur4RFu6_t8_e63pmKDErGl6zxaIpBpAQ8VHQxYruj-LMcsKq-5Bm-eNkZh6_mL_7uvUmpZ0/s320/little-qlu-annotated.png" width="320" height="166" /></a></div>
<p>
The right-hand side of each equation representing a version of Little's law is written vertically in the order $R, W, S$, which matches the expression $R=W+S$ for the mean residence time, viz., the sum of the mean waiting time ($W$) and the mean service time ($S$).
<p>
The letters on the left-hand side: $Q, L, U$ (reading vertically) respectively correspond to the queueing metrics: queue-length, waiting-line length, and utilization, which can be read as the word <i>clue</i>.
<p>
Incidentally, the middle formula is the version that appears in the title of John Little's <a href="http://www.cs.bilkent.edu.tr/~tugrul/CS518/Papers/little.pdf">original paper</a>:
<blockquote>
J. D. C. Little, ``A Proof for the Queuing Formula: L = λ W,'' <br><em>Operations Research</em>, 9 (3): 383—387 (1961)
</blockquote>Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com0tag:blogger.com,1999:blog-6977755959349847093.post-71973120375296504922016-08-03T17:54:00.001-07:002018-06-26T07:57:13.064-07:00PDQ as a Performance Periscope<!-- PDQ: Performance with Pedagogy -->
<blockquote>
<i>This is a guest post by performance modeling enthusiast, Mohit Chawla, who brought the following very interesting example to my attention. In contrast to many of the examples in my <a href="http://www.perfdynamics.com/iBook/ppa_new.html" TARGET="_blank">Perl::PDQ book</a>, these performance data come from a production web server, not a test rig. —NJG
</i></blockquote>
<p>
Performance analysts usually build performance models based on their understanding of the software application's behavior. However, this post describes how a PDQ model acted like a periscope and also turned out to be pedagogical by uncovering otherwise hidden details about the application's inner workings and performance characteristics.
<p>
<h3>Some Background</h3> <a name='more'></a>
A systems engineer needed some help analyzing the current performance of a web application server, as well as its capacity consumption. Time series data and plots provided a <i>qualitative</i> impression for this purpose; mostly to sanity-check the data. While the act of observing and interpreting information contained in these time-series data was helpful for forming an empirical understanding of traffic patterns and resource utilization of the application, it wasn't sufficient to make an accurate judgement about the expected performance of the web server upon changing the application configuration or system resources, i.e., real capacity planning.
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBbaqfSOllDJJL9K07WKHX7NR8z0EHk8Ae2jp9zxn768qsI3K0hxY03hfUcQdsr_MItufN56GVSbfaNYMeQTKmBRKlHLd0ppfa4Z1_eS0e5NqVs6apm6Pb39gavmP6vBgRMDo7nXbx0ko/s1600/tomcat-timeseries-R.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBbaqfSOllDJJL9K07WKHX7NR8z0EHk8Ae2jp9zxn768qsI3K0hxY03hfUcQdsr_MItufN56GVSbfaNYMeQTKmBRKlHLd0ppfa4Z1_eS0e5NqVs6apm6Pb39gavmP6vBgRMDo7nXbx0ko/s400/tomcat-timeseries-R.png" width="400" height="303" /></a></div>
<p>
In fact, what was needed was some kind of "periscope" to see above the "surface waves" of the time-series performance data. In addition, a more <i>quantitative</i> framework would be useful to go beyond the initial qualitative review of the time-series. All of this was needed pretty damn quick! ... <a href="http://www.perfdynamics.com/Tools/PDQcode.html" TARGET="_blank">Enter PDQ</a>.
<p>
<h3>Transforming the Data</h3>
Let's start by examining the general structure of the steady-state data before applying any queueing models to it.
In other words, the original time series data for measured response times, $R$ (see the above Figure) was transformed to show the relationship between $R$ and the corresponding load, $N$, on the web server during the same time buckets.
This transformation is achieved by applying <a href="http://perfdynamics.blogspot.com/2014/07/a-little-triplet.html" TARGET="_blank">Little's law</a>
\begin{equation}
N = X R,
\end{equation}
where $X$ is the measured throughput of the application.
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOSuUbJtBwFsv611m7O19qfnEULJbuENVTMAprm7lCCq5lDC-CCpb6DoBx60UXWKYhhq4IEikbHLzvUp8oAz70q4VNWzXL1dfW9RFmhm89wS0W92anihnMvMewzRT8VLwlgrP9YQ95q-s/s1600/hockey-stick.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOSuUbJtBwFsv611m7O19qfnEULJbuENVTMAprm7lCCq5lDC-CCpb6DoBx60UXWKYhhq4IEikbHLzvUp8oAz70q4VNWzXL1dfW9RFmhm89wS0W92anihnMvMewzRT8VLwlgrP9YQ95q-s/s400/hockey-stick.png" width="400" height="303" /></a></div>
<p>
It's now immediately apparent that these transformed data have the general shape of a statistical hockey stick as a function of the number of threads actively processing requests. The hockey stick shows that resource saturation sets in around $N = 250$ concurrent threads, after which the response times start climbing steadily up the hockey-stick handle.
<p>
<h3>The PDQ Model</h3>
What was known about the application is that each request is handled by a thread in a blocking fashion so, the number of threads in the system answering requests will be rate-limited due to their blocking characteristic. This makes it appropriate to construct a closed PDQ model for this application.
<p>
In the first attempt at constructing the PDQ model for these data, the predicted $R$ values were not close to the measured response times.
Subsequently, we added 250 "dummy queueing nodes" to account for missing response times. This technique is described in detail in Chapter 12 <i>"Web Application Analysis with PDQ"</i> of the <a href="http://www.perfdynamics.com/iBook/ppa_new.html" TARGET="_blank">Perl::PDQ book</a>.
The physical interpretation of these additional PDQ nodes was not understood initially.
Nevertheless, the PDQ model now accurately matched the response time data, as shown in this plot:
<!-- response-time graph image -->
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWyYdhks2UTTJ2PDvtLms-lYU_Ts-7vPyddNA369ofkMAu3fmbxesmVnIt0gmTKO8hVHguSjyx3iPtj5dn_MATTw2uZQklK-y7v7pUT21x9j2q3dyndl24z8cAtNGNJstY0wLfBbwuTZ0/s1600/tomcat-R.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWyYdhks2UTTJ2PDvtLms-lYU_Ts-7vPyddNA369ofkMAu3fmbxesmVnIt0gmTKO8hVHguSjyx3iPtj5dn_MATTw2uZQklK-y7v7pUT21x9j2q3dyndl24z8cAtNGNJstY0wLfBbwuTZ0/s400/tomcat-R.png" width="400" height="303" /></a></div>
<p>
Similarly, the mean throughput values predicted by PDQ also matched the measured throughput data:
<p>
<!-- throughput graph image -->
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPnDwAUevI2Y0jQMy9U-phuhkLh7iU9dUwUVRdhXhyx1W1e81QVMm-qJN59ppDSXMdEFTi1mCmdcWSUOTZpUFBEPPsWrvinZcFwNpToUPrP6wEf1r0cldeKd4EYveUqtvrYDHn35tej5U/s1600/tomcat-X.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPnDwAUevI2Y0jQMy9U-phuhkLh7iU9dUwUVRdhXhyx1W1e81QVMm-qJN59ppDSXMdEFTi1mCmdcWSUOTZpUFBEPPsWrvinZcFwNpToUPrP6wEf1r0cldeKd4EYveUqtvrYDHn35tej5U/s400/tomcat-X.png" width="400" height="303" /></a></div>
<p>
This is where the PDQ model also became pedagogic and exposed some previously hidden performance aspects of this web application. But before getting into that, a word about service times.
<p>
The performance data contained estimated service times for the various workloads based, once again, on Little's law:
\begin{equation}
\rho = X \, S,
\end{equation}
or
\begin{equation}
S = \frac{\rho}{X}, \label{eqn:stimes}
\end{equation}
where $\rho$ is the CPU utilization, $X$ is throughput and $S$ the service time.
The slope of the hockey stick handle corresponds to $S_{max}$, the bottleneck service time that limits the maximum possible throughput of the system according to, $X_{max} = 1 / S_{max}$. See Section 7.3 of the <a href="http://www.perfdynamics.com/iBook/ppa_new.html" TARGET="_blank">Perl::PDQ book</a>
<p>
Here are a few selected data values, where $U_{dat}$ refers to the CPU utilization of the server, $S_{est}$ is the estimated service time, $X_{dat}$ is the throughput and $N_{est}$ is the estimated concurrency:
<!-- table with three data points -->
<table align="center" border="0" cellpadding="5" cellspacing="5">
<tr>
<th>Nest</th> <th>Xdat</th> <th>Sest</th> <th>Udat</th>
</tr>
<td>133.833750</td> <td>416.460510</td> <td>0.000880</td> <td>0.366420</td>
</tr>
<tr>
<td>137.725453</td> <td>425.790466</td> <td>0.000864</td> <td>0.367760</td>
</tr>
<tr>
<td>138.430266</td> <td>413.350861</td> <td>0.000842</td> <td>0.348060</td>
</tr>
</table >
As you can see, the mean service times derived using equation (\ref{eqn:stimes}) are less than one millisecond each,
whereas the response times are clearly much higher, viz., in the hundreds of milliseconds range. This takes us back to the "hidden latencies" exposed by the PDQ model.
<p>
We added around 250 "dummy queues" each with a service time of about one millisecond per thread. In this way, the sum of the residence times across all 250 dummy queues matched the total response time seen by a user, yet none of the dummy queue service times exceed $S_{max} = 1.25$ milliseconds seen in the slope of the hockey-stick handle. The conjectured explanation was that the dummy queues represented some sort of fast lookup in cache memory, or possibly some kind of polling performed by each thread.
<p>
<h3>Confirmation</h3>
After discussing this conjecture with application developers, it turned out that each thread blocks until responses from various external services are received, and this processing happens inside a loop where each thread waits in the loop until it receives a response. That's exactly the kind of polling PDQ predicted. And so, we now knew what limited the application's performance and where it could be improved.
<p>
That PDQ periscope turned out to be pretty damn pedagogical!
<p>Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com0tag:blogger.com,1999:blog-6977755959349847093.post-55280139107957162012016-07-28T16:47:00.003-07:002017-03-16T15:10:45.055-07:00Erlang Redux Resolved! (This time for real)As I show in my <a href="http://www.perfdynamics.com/books.html" target="_blank">Perl::PDQ book</a>,
the residence time at an M/M/1 queue is trivial to derive and (unlike most queueing theory texts) does not require any probability theory arguments.
<a href="http://www.perfdynamics.com/Classes/schedule.html" target="_blank">Great for Guerrillas!</a>
However, by simply adding
another server (i.e., M/M/2), that same Guerrilla approach falls apart. This situation has always bothered me profoundly and on several occasions I thought I saw how to get to the exact formula—the
Erlang C formula—Guerrilla style.
But, on later review, I always found something wrong.
<p>
Although I've certainly had correct pieces of the puzzle, at various times, I could never get everything to fit in a completely consistent way.
No matter how <a href="http://www.perfdynamics.com/Test/erlang.html" target="_blank">creative I got</a>,
I always found a fly in the ointment.
The best I had been able to come up with is what I call the "morphing model" approximation where you start out with $m$ parallel queues at low loads and it morphs into a single $m$-times faster M/M/1 queue at high loads.
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9wiSqVacJarGGKXutfzQ0ipk2zCoXg7kdyPvDdHWXG8jj5uqrSgX2GMDa8n6tOpmmjFz9SyeJw1kKidSKqs0HMzFnoQnZtgYfrFH_JUz2mHUOW4smikEC3xFcoTC6No9ysapCrVB-6Mg/s1600/amorph.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9wiSqVacJarGGKXutfzQ0ipk2zCoXg7kdyPvDdHWXG8jj5uqrSgX2GMDa8n6tOpmmjFz9SyeJw1kKidSKqs0HMzFnoQnZtgYfrFH_JUz2mHUOW4smikEC3xFcoTC6No9ysapCrVB-6Mg/s400/amorph.gif" width="400" height="104" /></a></div>
<p>
That model is also exact for $m = 2$ servers—which is some kind of progress, but not much. Consequently, despite a few misplaced enthusiastic announcements in the past, I've never been able to publish the fully corrected morphing model. <a name='more'></a>
<p>
Previous misfires have included:
<ul>
<li> Falsely claimed in this <a href="http://perfdynamics.blogspot.com/2008/01/erlang-explained.html" target="_blank">2008 blog post</a>.
There, I show a Table of how complicated the Erlang B and C functions are when expressed as rational functions of
$\rho$ (the per-server utilization) for $m = 1, 2, \ldots, 6$ service facilities.
<li> As a side note, it's precisely those impenetrable polynomials with unfathomable coefficients that completely put me off the approach I am about to describe here.
<li> In the Comments section of that same post, Boris Solovyov asked in 2013: "Did you ever write this out in full?"
I sheepishly had to report that I hadn't had time to pursue it (which is mostly true). I hope he reads this post.
<li> A more intuitive explanation of the motivation for the morphing model was given in this <a href="http://perfdynamics.blogspot.com/2011/07/the-multiserver-numbers-game.html"target="_blank">2011 blog post</a>, but no advance on the long sought-after correction terms.
<li> Falsely claimed again in this <a href="http://perfdynamics.blogspot.com/2012/01/my-year-in-review-2011.html"target="_blank">2012 blog post</a>. There's a photo of one of my whiteboards, deliberately kept inscrutably small—a good choice, as it turned out.
</ul>
<p>
I think I know how Kepler must've felt. The difference between an M/M/1 queue and an M/M/m queue is like the difference
between a circle and an ellipse. Just by changing the width slightly, the calculation of the perimeter becomes enormously
complicated—in fact, it doesn't even have an analytic solution!
Now, however, I believe I finally have the correct approach for sure. Really!... Yep... Uh huh... No, seriously. Well, see for yourself
<p>
<h3>The Starting Point</h3>
Start with the <b>morphing</b> approximation for the M/M/m waiting time in my
<a href="http://www.perfdynamics.com/books.html" target="_blank">Perl::PDQ book</a>—Chap. 2 in the original 2004 edition or Chap. 4 in the 2nd 2011 edition.
\begin{equation}
\Phi_W^{approx} (m, \rho) \, = \, \frac{1}{\rho^{-m} \, \sum_{k=0}^{m-1} \rho^k} \label{eqn:morphfun}
\end{equation}
This definition has a denominator involving a truncated geometric series in $\rho$.
It is used to derive the Guerrilla approximation for the residence time at an M/M/m queue with mean service time $S$
\begin{equation*}
R^{approx} \, = \, \frac{S}{1 - \rho^m}
\end{equation*}
which is exact for $m = 1, 2$.
The exact general formula for the residence time is given by
\begin{equation*}
R^{exact} \, = \, S \; + \; \frac{C(m, \rho)}{m( 1 - \rho)} \, S
\end{equation*}
where $C(m, \rho)$ is the Erlang C function defined in (\ref{sec:realEC}).
<p>
<h3>Five Easy Pieces</h3>
The following 5 steps provide the corrections to the approximation in (\ref{eqn:morphfun}) and result in the exact Erlang C function without resorting to the usual probability arguments found in almost all queueing theory textbooks. Along the way,
and quite unexpectedly (for me), I derive the Erlang B function as an intermediate result. Originally, I thought I would have to introduce that function as a kind of axiomatic probability. I wasn't thinking of it as a major waypoint.
As far as I'm aware, this derivation has never been presented before because you have to be sufficiently perverse to have thought up the morphing approximation in the first place.
<ol>
<li>
Replace $\rho$ in (\ref{eqn:morphfun}) by the traffic intensity $a = m \rho$
\begin{equation}
\Phi_W^{approx} (m, a) \, = \, \frac{1}{a^{-m} \, \sum_{k=0}^{m-1} ~a^k} \label{eqn:morpha}
\end{equation}
<li>
Convert $\Phi_W^{approx}$ to a truncated exponential series by applying $a^n \mapsto a^n / n!$
\begin{equation}
\frac{1}{m! a^{-m} \, \sum_{k=0}^{m-1}~\frac{a^k}{k!}}
\end{equation}
<li>
Extend the summation over all $m$ servers to yield the famous
<a href="http://en.wikipedia.org/wiki/Erlang_%28unit%29#Erlang_B_formula" target="_blank">Erlang B function</a> for an M/M/m/m queue
\begin{equation}
B(m, a) \, = \, \frac{\frac{a^m}{m!}}{\sum_{k=0}^{m}~\frac{a^k}{k!}} \label{eqn:EB}
\end{equation}
Historically, $B(m, a)$ has been associated with the probability that an incoming telephone call is blocked and lost from the system, e.g., gets an engaged signal. I can't remember the last time I heard an engaged signal. I think they've been replaced by voice-mail and Muzak.
<li>
Using the more compact notation: $A_m = a^m / m!$ and $\Sigma_k$ for the reduced sum over $(m - 1)$ terms, rewrite (\ref{eqn:EB}) as
\begin{equation}
B(m, a) \, = \, \frac{A_m}{A_m + \Sigma_k}
\end{equation}
<li>
Scale $A_m$ by $(1 - \rho)^{-1}$ to introduce the infinite possible wait states and arrive at
\begin{equation}
\Phi_W^{exact} (m, a) \, = \, \frac{1}{1 \, + \, (1 - \rho) \, m! \, a^{-m} \, \sum_{k=0}^{m-1}~\frac{a^k}{k!}} \label{eqn:myEC}
\end{equation}
which is the fully corrected version of (\ref{eqn:morphfun}).     <b><i>Q.E.D.</i></b>
</OL>
Furthermore, it can be shown that (\ref{eqn:myEC}) is identical to the famous
<a href="https://en.wikipedia.org/wiki/Erlang_%28unit%29#Erlang_C_formula" target="_blank">Erlang C function</a>
\begin{equation}
C(m, a) \, = \, \frac{ \frac{a^m}{m!} \, \big(\frac{m}{m-a}\big) }{1 \, + \, \frac{a}{1!} \, + \, \frac{a^2}{2!} \, + \, \cdots \, + \, \frac{a^{m-1}}{(m-1)!} \, + \, \frac{a^m}{m!} \, \big(\frac{m}{m-a}\big) } \label{sec:realEC}
\end{equation}
Historically, $C(m, a)$ has been associated with the probability that an incoming telephone call must wait to get a connection (aka "call waiting"), rather than being dropped, as it is in B(m, a). However, since $C(m, a)$ determines the mean waiting time in any M/M/m queue, it applies to any multi-server system, e.g., modern multi-threaded applications.
<p>
Equation (\ref{sec:realEC}) was first published by <a href="https://en.wikipedia.org/wiki/Agner_Krarup_Erlang" target="_blank">A. K. Erlang</a> in 1917 (he of the Copenhagen Telephone Company), along with (\ref{eqn:EB}). Thus, next year will be its centennial. Nice timing on my part (although that was never the plan—there never being <i>any</i> plan).
<p>
It's worth emphasizing that the morphing approximation (\ref{eqn:morphfun}) accounts for about 90% of what is going on with $R^{exact}$ in the M/M/m queue. The remaining 10% contains the minutiae regarding how the waiting line actually forms. But, as you can see from the above transformations, it's a rather subtle 10%.
<p>
<h3>Next Steps</h3>
This is just a sketch proof. I've suppressed a lot of details because there are many and I have a 100 pages of typeset notes to back that up.
I know a lot about what <i>doesn't</i> work. (Sigh!)
Now that I have the correct mathematical logic sketched out, I'm quite confident that it can also be supplemented with a more visual representation of how the
corrections to the morphing function (\ref{eqn:morphfun}) arise.
<p>
<h3>Postscript (Thu Mar 16 15:10:09 PDT 2017)</h3>
The <a href="http://perfdynamics.blogspot.com/2017/03/morphing-mmm-new-view-of-old-queue.html" target="_blank" >culmination of this work</a> will be presented at <a href="http://ifors2017.ca" target="_blank">IFORS 2017</a>.Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com0tag:blogger.com,1999:blog-6977755959349847093.post-84426255724071246512016-06-08T10:18:00.000-07:002016-06-08T20:55:19.064-07:002016 Guerrilla Training Schedule 2016After a six month hiatus working on a major consulting gig, <a href="http://www.perfdynamics.com/Classes/schedule.html">Guerrilla training classes</a> are back in business with the three classic courses: <a href="http://www.perfdynamics.com/Classes/Outlines/gboot.html">Guerrilla Bootcamp</a> (<b>GBOOT</b>), <a href="http://www.perfdynamics.com/Classes/Outlines/guerilla.html">Guerrilla Capacity Planning</a> (<b>GCAP</b>) and <a href="http://www.perfdynamics.com/Classes/Outlines/gdata.html">Guerrilla Data Analysis Techniques</a> (<b>GDAT</b>).
<p>
See what <a href="http://www.perfdynamics.com/Classes/comments.html">graduates are saying</a> about these courses.
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEie3t7VCk2lDozykOCQVuL1fftRg2ySZb-UV2jtU6HX7dlHmduxU3ddL85j25TJdtn-qF9ndchGvwN8oQR0-uwn9rt607yrMSpGhJklOK0AsY24sae5rGUaiRCu985ITXz5w1mcdEatVwo/s1600/ghat-cloud.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEie3t7VCk2lDozykOCQVuL1fftRg2ySZb-UV2jtU6HX7dlHmduxU3ddL85j25TJdtn-qF9ndchGvwN8oQR0-uwn9rt607yrMSpGhJklOK0AsY24sae5rGUaiRCu985ITXz5w1mcdEatVwo/s400/ghat-cloud.jpg" /></a></div>
<p>
Some course highlights:
<ul>
<li> There are only 3 performance <b>metrics</b> you need to know
<li> How to <b>quantify scalability</b> with the <a href="http://www.perfdynamics.com/Manifesto/USLscalability.html">Universal Scalability Law</a>
<li> <a href="http://queue.acm.org/detail.cfm?id=2789974">Hadoop performance</a> and capacity management
<li> <b>Virtualization</b> Spectrum from hyper-threads to cloud services
<li> How to detect <b>bad data</b>
<li> Statistical <b>forecasting</b> techniques
<li> <b>Machine learning</b> algorithms applied to performance data
</ul>
<p>
<a href="http://www.perfdynamics.com/Classes/schedule.html">Register online</a>. Early-bird discounts run through the end of <b>July</b>.
<p>
As usual, <a href="http://www.fourpointspleasanton.com">Sheraton Four Points</a> has bedrooms available at the Performance Dynamics discounted rate.
<p>
Tell a friend and see you in <b>September</b>!Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com0tag:blogger.com,1999:blog-6977755959349847093.post-36296083649531382732016-05-14T13:44:00.000-07:002016-05-14T21:50:52.429-07:00PDQ 7.0 Dev is UnderwayThe primary goal for this release is to make <a href="http://www.perfdynamics.com/Tools/PDQ.html">PDQ</a> acceptable for uploading to <a href="https://cran.r-project.org">CRAN</a>. This is a non-trivial exercise because there is some legacy C code in the PDQ library that needs to be reorganized while, at the same time, keeping it consistent for programmatically porting to other languages besides R—chiefly Perl (for the <a href="http://www.perfdynamics.com/iBook/ppa_new.html">book</a>) and Python.
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5CPFzSy4LpDvdDhVai_3TBL0dcdUUEyfuG-4_lnC78w27q0c2khNDoifWotYNkDPilKQhmcvSxxjRxRPMSnGx6ut0yfmx3nn3yC7toJxY39FlMbHTKr5fuq9UO3NE7eTrqyHbBc0CnjY/s1600/Screen+Shot+2016-05-07+at+10.19.42+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5CPFzSy4LpDvdDhVai_3TBL0dcdUUEyfuG-4_lnC78w27q0c2khNDoifWotYNkDPilKQhmcvSxxjRxRPMSnGx6ut0yfmx3nn3yC7toJxY39FlMbHTKr5fuq9UO3NE7eTrqyHbBc0CnjY/s400/Screen+Shot+2016-05-07+at+10.19.42+PM.png" /></a></div>
<p>
To get there, the following steps have been identified:
<ol type="A" style="font-weight: bold;">
<li> <h3>High Priority</h3>
<ol>
<li>Migrate from <a href="https://sourceforge.net/projects/pdq-qnm-pkg/">SourceForge</a> to <a href="https://github.com">GitHub</a>.
<li> Change the return type for these functions from int to void:
<ul>
<li> PDQ_CreateOpen()
<li> PDQ_CreateClosed()
<li> PDQ_CreateNode()
<li> PDQ_CreateMultiNode()
</ul>
Using the returned int as a counter was deprecated in version 6.1.1.
<li>Convert <a href="http://www.perfdynamics.com/Tools/PDQ-R.html">PDQ-R</a> to <a href="http://www.rcpp.org">Rcpp</a> interface.
<li>Clean out the Examples directory and other contributed code directories leaving only Examples that actually use the PDQ C library.
<li>Add unit tests for PDQ C library, as well as the Perl, Python, and R languages.
<li>Get interface accepted on <a href="https://cran.r-project.org">CRAN</a>
<li> Add the ability to solve multi-server queueing nodes servicing an arbitrary number of workloads.
</ol>
<p>
<p>
<li> <h3>Low Priority</h3>
<ol>
<li>Get interface accepted on <a href="http://www.cpan.org">CPAN</a> and <a href="https://pypi.python.org/pypi">PyPI</a>.
<li>Convert to build system from makefiles to <a href="https://en.wikipedia.org/wiki/CMake">Cmake</a>.
</ol>
</ol>
<p>
Stay tuned!
<p>
—njg and pjp
Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com2tag:blogger.com,1999:blog-6977755959349847093.post-65198786524396231632016-05-13T22:27:00.000-07:002016-05-13T22:29:25.913-07:00How to Emulate Web Traffic Using Standard Load Testing ToolsThe following abstract has been submitted to <a href="https://www.cmg.org/event/performance-capacity-2016-conference/">CMG 2016</a>:
<p>
<blockquote>
<b>How to Emulate Web Traffic Using Standard Load Testing Tools</b>
<p>
<i>James Brady (State of Nevada) and Neil Gunther (Performance Dynamics)</i>
<p>
Conventional load-testing tools are based on a fifty year old time-share computer paradigm where a finite number of users submit requests and respond in a synchronized fashion. Conversely, modern web traffic is essentially asynchronous and driven by an unknown number of users. This difference presents a conundrum for testing the performance of modern web applications. Even when the difference is recognized, performance engineers often introduce virtual-user script modifications based on hearsay; much of which leads to wrong results. We present a coherent methodology for emulating web traffic that can be applied to existing test tools.
</blockquote>
<p>
Keywords: load testing, workload simulation, web applications,
software performance engineering, performance modeling
<p>
Related blog posts:
<OL>
<li> E<a href="http://perfdynamics.blogspot.com/2010/05/emulating-internet-traffic-in-load.html">mulating Web Traffic in Load Tests</a>
<li> <a href="http://perfdynamics.blogspot.com/2010/04/mapping-virtual-users-to-real-users.html">Mapping Virtual Users to Real Users</a>
<li> <a href="http://perfdynamics.blogspot.com/2007/05/how-to-extend-load-tests-with-pdq.html">How to Extend Load Tests with PDQ</a>
</OL>
Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com0tag:blogger.com,1999:blog-6977755959349847093.post-1031425471691540712015-09-29T11:02:00.000-07:002015-09-29T11:11:28.671-07:00Remember the Alamo at CMG 2015The Alamo is a reference to an episode in Texan history about <a href="http://history.howstuffworks.com/history-vs-myth/remember-the-alamo.htm/printable">defeat and revenge</a>. But, there's nothing defeatist or mythical about the sessions I'll be giving at <a href="http://www.cmg.org/conferences/performance-capacity-2015/">CMG in San Antonio</a> this year.
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg0kcJiYf8c1E1FcBYW9Oky1SjzYZLn8p13Uro-NxoZSI3HYeHDnWLDAH4DpnFZpoX6UjjgwbsIgp_Tgdptuq6lgc7KVsfMEWdqLwrh0jb4P8p5H8L907Ft9PsXfGuR5Ougy1-K1ekIrXc/s1600/sh6HnnAk9hTa3osejnRov9D8.jpeg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg0kcJiYf8c1E1FcBYW9Oky1SjzYZLn8p13Uro-NxoZSI3HYeHDnWLDAH4DpnFZpoX6UjjgwbsIgp_Tgdptuq6lgc7KVsfMEWdqLwrh0jb4P8p5H8L907Ft9PsXfGuR5Ougy1-K1ekIrXc/s400/sh6HnnAk9hTa3osejnRov9D8.jpeg" /></a></div>
<p>
<h3>Workshop: <i>How to Do Performance Analytics with R</i>, Mon Nov 2, 8-12am</h3>
You've collected cubic light-years of performance monitoring data, now
whaddya gonna do? Raw performance data is not the same thing as
information, and the typical time-series representation is almost the
worst way to glean information. Neither your brain nor that of your
audience is built for that (blame it on Darwin). To extract pertinent
information, you need to transform your data and that's what the R
statistical computing environment can help you do, including
automatically.
<p>
Topics covered will include:
<ul>
<li> Introduction to R using RStudio
<li> Descriptive statistics
<li> Performance visualization
<li> Data reduction techniques
<li> Multivariate analysis
<li> Machine learning techniques
<li> Forecasting with R
<li> Scalability analysis
</ul>
<p>
<h3>Invited talk: <i>Hadoop Super Scaling</i>, Wed Nov 4, 5-6pm</h3>
The Hadoop framework is designed to facilitate parallel-processing massive
amounts of unstructured data. Originally intended to be the basis of Yahoo's
search-engine, it is now open sourced at Apache. Since Hadoop has a broad range
of corporate users, a number of companies offer commercial implementations or
support for Hadoop.
<p>
However, certain aspects of Hadoop performance---especially scalability---are
not well understood. One such anomaly is the claimed flat scalability benefit
for developing Hadoop applications. Another is that it's possible to achieve
faster than parallel processing. In this talk I will explain the source of these
anomalies by presenting a consistent method for analyzing Hadoop application
scalability.
<p>
<h3>CMG-T: <i>Capacity and Performance for Newbs and Nerds</i>, Thur Nov 5, 9-11am</h3>
In this tutorial I will bust some entrenched myths and develop basic
capacity and performance concepts from the ground up. In fact, any
performance metric can be boiled down to one of just three metrics. Even
if you already know metrics like, throughput and utilization, that's not
the most important thing: it's the relationship *between* those metrics
that's vital! For example, there are at least three different definitions
of utilization. Can you state them? This level of understanding can make a
big difference when it comes to solving performance problems or presenting
capacity planning results.
<p>
Other myths that will get busted along the way include:
<ul>
<li> There is no response-time knee.
<li> Throughput is not the same as execution rate.
<li> Throughput and latency are not independent metrics.
<li> There is no parallel computing.
<li> All performance measurements are wrong by definition.
</ul>
<p>
No particular knowledge about capacity and performance management is assumed.
<p>
See you in <a href="http://www.cmg.org/conferences/performance-capacity-2015/registration/">San Antonio</a>! Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com2tag:blogger.com,1999:blog-6977755959349847093.post-16246608683341499012015-08-24T15:31:00.000-07:002015-09-29T14:34:32.287-07:00PDQ Version 6.2.0 ReleasedPDQ (Pretty Damn Quick) is a FOSS <a href="http://www.perfdynamics.com/Tools/PDQ.html">performance analysis tool</a> based on the paradigm of queueing models that can be programmed natively in
<ul>
<li> <a href="http://www.perfdynamics.com/Tools/PDQ-R.html">R</a>
<li> <a href="http://www.perfdynamics.com/Tools/PDQpython.html">Python</a>
<li> <a href="http://www.perfdynamics.com/Tools/PDQperl.html">Perl</a>
<li> C and several other languages.
</ul>
<p>
This minor release is now available for <a href="http://www.perfdynamics.com/Tools/PDQcode.html">download</a>.
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTAofYPeVklmVkK7FNaJNUW6GfVehfWeIdbVRNyk_ZqwODAUAJ5Fontip3LwExKvPOtHc5oK9N6k9LK6YWWogXZy17hLZsIhf65AlJlEmLGL6rWxgkpB_Mtc_-WP-xntBpNMMVijV3mp8/s1600/PDQ-R-Screen.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTAofYPeVklmVkK7FNaJNUW6GfVehfWeIdbVRNyk_ZqwODAUAJ5Fontip3LwExKvPOtHc5oK9N6k9LK6YWWogXZy17hLZsIhf65AlJlEmLGL6rWxgkpB_Mtc_-WP-xntBpNMMVijV3mp8/s400/PDQ-R-Screen.png" /></a></div>
<p> <a name='more'></a>
If you're new to PDQ, here's a simple queueing model written R that you can paste directly into an <a href="https://www.rstudio.com/products/RStudio/#Desktop">RStudio</a> console or script window:
<pre class="source-code"><code>
# A simple M/M/1 queueing model in R-PDQ.
require(pdq)
# input parameters
arrivalRate <- 0.75
serviceRate <- 1.0
## Build and solve the PDQ model
Init("Single queue model") # Initialize PDQ
CreateOpen("Work", arrivalRate) # Create workload
CreateNode("Server", CEN, FCFS) # Def single server
SetDemand("Server", "Work", 1/serviceRate) # Def service time
Solve(CANON) # Solve the model
Report() # Formatted output
</code></pre>
Also, check out the relevant <a href="http://www.perfdynamics.com/books.html">books</a> and <a href="http://www.perfdynamics.com/Classes/schedule.html">training classes</a>. Neil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.com0