This topic goes back to my M.Sc. thesis, when my supervisor, Prof. Christie Eliezer (2nd from left in the photo), who was one of Dirac's very few research students, told me about the Dirac-Sommerfeld paradox. He thought it might be pertinent to my thesis.
Being a physics "genius", I didn't pursue it because I assumed it had been resolved by more modern developments like, Quantum Field Theory — a subject that neither Eliezer nor I knew much about. I was wrong. Very wrong!
While reading it I had some thoughts that turned out to be too long for tweeting a response. So, here I am revisiting my traditional blog, mostly for the formatted space it provides, but also to remind myself that it still lives!
Quite apart from my subsequent remarks below, the inferences about internal disk structure drawn from the microbenched timing-data are quite remarkable. And the beautiful plots generated from those data (e.g., mapping thousands of sector defects) are something to behold. Understandably, HDD manufacturers are not keen to include such plots in their marketing collateral.
The author states the following as his prime motivations for developing the extended microbenchmark algorithms.
"I had initially set out to detect the number of platters in a disk without opening up a disk, but in modern disks this simple-sounding task requires measuring several other properties before inferring the count of recording surfaces."Although a truly laudible effort, that is largely driven by scientific "curiosity", I can't help but wonder if the author is straining at a gnat."Many of the prior works had practical applications in mind (as opposed to pure curiosity), and were willing to trade measurement speed for a small loss in accuracy. Although [my microbenchmark] algorithms are not fundamentally different, prior work likely avoided exploring these because these measurements can take many hours and there is little practical benefit for applications such as performance modelling or improving data layouts in filesystems."
With a bit of perseverance, I could find several corresponding drive specs online. And, although manufactuers never lie (smirk) the search terms can vary widely.
Platters = disks = discs... Surfaces = heads = actuators...Applying the author's terminology:
The author infers the same number of WD S25 used surfaces (3) based on his benching data which, in itself, is quite remarkable. On the other hand, this same information is generally included in the disk manufacturer specs (or similar docs). Worst case, you could actually contact the manufacturer and they would very likely tell you; especially if they thought they could sell you some disks.
Computer manufacturers of all stripes are in business to make a profit and to avoid being eaten by the competition. They will do whatever it takes (good, bad or ugly), whenever it's deemed necessary, to maintain or improve their market position. An unfortunate casuality of this ongoing commercial warfare is that any painstakingly acquired scientific analysis can become an historical footnote overnight.
In that vein, it's a fool's errand to try and model individual modern disks. In principle, you would need to construct a simulation model to handle all the potential transient behaviors. But that, in turn, requires deciphering the embedded controller algorithms, which are not only highly complex but proprietary. You'll never figure them out and no manufacturer will ever reveal them. So, a performance simulation is actually a non-starter.
On a more positive note, there is a surprisingly counterintuitive technique for constructing accurate performance models of collective disks, like SANs, using the PDQ (Pretty Damn Quick) performance analyzer. That's something I demonstrate in my Guerrilla Capacity and Performance (GCAP) classes.
All modern computer systems, no matter how complex, can be thought of as a directed graph of individual buffers that hold requests until to be serviced at a shared computational resource, e.g., a CPU or disk. Since a buffer is just a queue, any computer infrastructure, from your laptop up to Facebook.com, can be represented as a directed graph of queues.
Two major benefits of using PDQ are:
As shown in the above diagram, any modern computer system can be thought of as a directed graph of individual buffers where requests wait to be serviced at some kind of shared computational resource, e.g., a CPU or disk. Since a buffer is just a queue, any computer infrastructure, from a laptop to Facebook, can be represented as a directed graph of queues. The directed arcs or arrows correspond to workflows between the different queues. In the parlance of queueing theory, a directed graph of queues is called a queueing network model. PDQ is a tool for calculating the performance metrics, e.g., waiting time, throughput, optimal load, of such network models.
Some example PQD applications include models of:
Converting between various time zones, including UTC time, is both a pain and error-prone. A better solution is to use absolute time. Thankfully, Unix provides such a time: the so-called Epoch time, which is the integer count of the number of seconds since January 1, 1970.
Timestamps are both very important and often overlooked: the moreso in the context of performance analysis, where time is the zeroth metric. In fact, my preferred title would have been, Death to Time Zones but that choice would probably have made it harder to find the main point here, later on, viz., how to use the Unix epoch time.
Although there are web pages that facilitate such time conversions, there are also shell commands that are often more convenient to use, but not so well known. With that in mind, here are some examples (for future reference) of how to convert between human-time and Unix time.
[njg]~% date
Tue Feb 11 10:04:32 PST 2020
[njg]~% date +%s
1581444272
[njg]~% date -r 1581444272
Tue Feb 11 10:04:32 PST 2020
[njg]~% date
Tue Feb 11 10:04:32 PST 2020
[njg]~% date +%s
1581444272
[njg]~% date -d @1581444272
Tue Feb 11 18:04:32 UTC 2020
In the meantime, over the past 20 years, the nature of the book itself has become progressively more digital. The ability to render the book-block on digital devices as an "e-book" has made printed hardcopies optional. Although purely digital e-books make reading more ubiquitous, e-book file formats and display quality varies across devices, i.e., laptops, phones tablets, and various e-readers. Any reduction in display quality is offset by virtue of e-books being able to include user interaction and even animation: features entirely beyond the printed book. In that sense, there really has been a revolution in the publishing industry: books are no longer about books, they're now about media.
Recently, I became aware of an undergraduate calculus textbook that makes powerful use of animation and audio. The author is a academic mathematician and he published it on YouTube! Is that a book or a movie? Somehow, it's a hybrid of both and, indeed, I wish I'd been able to learn from technical "books" like that. Good visuals and animations can make difficult technical concepts much easier to comprehend. Progressive as all that is, when it comes to technical e-books, I'm not aware of any single authoring tool that can match the quality of LaTeX, let alone incorporate user-interaction and animation. If such a thing did exist, I would be all over it. And Markdown doesn't cut it. But, digital authoring tools are continually evolving.
Matt Fleming (@fleming_matt on Twitter), the author of How to Build a Performance Testing Stack From Scratch, opted to use a static e-book format—not because it produces the most readable result, but because it is the best way to reach a wider audience at lower cost than a more expensive print publisher. The e-publisher in this case is (the relatively unknown) Ministry of Testing Ltd in Brighton, UK and is available on Amazon for the Kindle reader. I was also able to read it using iBooks on Mac OS X.
The range of topics covered is very extensive. I've included the Table of Contents here because it is not viewable on Amazon:
Part 1The overall e-book presentation of "Performance Testing Stack" seems underdeveloped. Most topics could have been greatly expanded. But, as Matt informed me, this is probably due to the content amounting to a concatenation of previously written blog posts. As I said earlier, there are no shortcuts to writing well. The paucity of detail, however, is offset by the shear enthusiasm the Matt brings to his writing: an aspect that deserves separate acknowledgement because performance testing is a very complex subject which can otherwise appear dry and mind-boggling to the uninitiated. And it's the uninitiated that Matt wants to reach. He has written this book in order to encourage the uninitiated reader to seriously consider entering the field.Part 2
- Step 1: Identify Stakeholders
- Step 2: Identify What to Measure
- Step 3: Test Design
- Step 4: Measuring Test Success and Failure
- Step 5: Sharing Results
Part 3 The Benchmark Hierarchy Picking Tests Validating Tests Part 4
- Understanding Statistics
- Latency
- Throughput
- Statistical Significance
Part 5
- Use a Performance Framework
- Ensure the Test Duration is Consistent
- Order Tests by Duration
- Keep Reproduction Cases Small
- Setup The Environment Before Each Test
- Make Updating Tests Easy
- Errors Should Be Fatal
- Format
- Use Individual Sample Data
- Detecting Results in the Noise
- Outliers
- Result Precision
- If All Else Fails Use Test Duration
- Delivering Results
Some points that could have been developed further, include:
Conversely, Matt's enthusiasm may have gone a bit overboard in his choice of title. The book promises:
This book will walk you through designing and building a performance testing stack from scratch; step by step from planning and running performance tests through to understanding and analysing the results. If you’re new to performance testing or looking to expand your understanding of this topic then this book is for you!Unfortunately, this book doesn't provide enough details to actually build a test stack—which would've been very cool. Rather, it presents a comprehensive overview of all the major concepts that one needs to absorb in order to develop a running performance testing stack. But even with the more limited scope, this book is still important because, off hand, I don't know of any other source where one can be introduced to performance testing without drowning in a sea of terminology, procedures and architectures.
Ultimately, this e-book is a great starting point for newbies, as well was being a good reminder for seasoned testers about what should be done in good performance tests.