The Valley Ain't What It Used To Be

Important interview in the SF Chron with hi-tech maven, Judy Estrin, where she warns about lagging innovation in the Si Valley and the USA. She is quoted as saying:

"That all sorts of forces, from Wall Street's quarterly profit pressures to worsening government deficits, have undermined support for the forward-looking research on which future products will depend."

I've certainly been complaining for a long time about the excessive influence of Wall St. on shortening capacity planning horizons. Estrin also claims that the environment for innovation in Silicon Valley and the United States has already changed for the worse. Whether you agree everything she says or not, I think her comments are important because there seem to be precious few figureheads who are prepared to stand up and tell as it is.

Updated on Thu Aug 28, 2008: Case in point from Slashdot regarding Alcatel-Lucent (née Bell Labs) yanking the financial plug on basic science "like yer startin' a mower!" (to quote Elaine Benes). As I've said before , the Edison model is dead.

Applying the USL Model to Multivalued Data

A reader was having trouble applying the USL model to his data and asked me to take a look at it. I can't discuss his data because it's private, but I can show you what I did using some simulated data that has the same properties. The data was collected off a production system not a controlled load-test platform, and this brought to the surface two aspects of applying the USL which I had not faced previously:

It's the PLANNING, Stupid!

The phrase "It's the ECONOMY, stupid!" helped boost Bill Clinton's election prospects in 1992. In 1999, when eBay.com was having its "CNN moments" (as we called them, back then), I declared that capacity planning seemed to be an oxymoron for many pre-bubble-bursting web sites. They were prepared to throw any amount of money at lots of iron, which presumably meant they understood the "capacity" part (capital expenditure). It's the "planning" part they didn't grok. Planning means thinking ahead. Planning requires investment in the future. If the Financial Dept. can do it, why can't IT?

GCaP Book Availability

Guerrilla graduate, Greg Rogers, alerted me to the fact that Amazon.com is showing my GCaP book as being "Temporarily out of stock". Sideways, this turns out to be good news. I know some of you will be thinking ahead to Xmas gifts, so don't be dissuaded. :-) I contacted my editor at Springer.de and here's what's going on.

Back in April, I was busily directing the attention of the GCaP course attendees to the Guerrilla Manual booklet inserted in the back cover when, to their surprise and my chagrin, they discovered that their copies did not have the booklet. Mine did, but it was an older copy. What the ...!? Turns out, it was a production error in the latest printing, which is now in the process of being corrected. The misprint versions have possibly been "recalled" and that's why it is temporarily out of stock. I'm sure Amazon will still be happy to take your order. Thanks, Greg.

Scalability Theorems Paper Now Available

As promised, my paper containing the proofs that the universal scalability law is equivalent to the synchronous throughput bound of a machine repairman model of a multiprocessor with state-dependent service rate, is now available from the arXiv preprint server under the title "A General Theory of Computational Scalability Based on Rational Functions," and covers the following topics:

How to Recover the Missing X(1) for the USL Scalability Model

When it comes to assessing application scalability, controlled measurements of the type that can be obtained with tools like Grinder or LoadRunner, are very useful because they provide a direct measurement of the throughput, X(N), as a function of the vuser/generator load, N. These data can be input easily into my universal scalability model (USL). To apply USL, however, you need to normalize all the X(N) data to the X(1) value. It often happens that the X(1) value may not have been measured by your QA group or it simply may not be measurable easily. What do you do in that case? You estimate it (carefully).

Mr. Amdahl Calls the Repairman

Back in January, I reported that my Universal Scalability Law (USL) had been proven to be equivalent to the synchronous throughput of the load-dependent machine repairman queueing model. Although I wasn't lying, what I really had was what we call a "sketch" for a proof. The details needed to be filled in, but that was just a question of finding the time to do it. In May, I found time to write the first draft and thought I would be finished shortly thereafter. That attempt ground to a halt for two reasons (you might want to refer to Appendix A in my GCaP book for the background):

1. Simulations that supported the theorem involved a barrier synchronizer, and for this to work continuously, my colleague Jim Holtman, discovered that the distribution of parallel execution times (with mean 'think' time Z) could only be deterministic (D). That's a special case of the repairman: D/M/1//N, whereas the equations in my proof show quite clearly that it must work for M/M/1//N.
   
2. There's a theorem in queueing theory (Bunday and Scraton, 1980) which states that what holds for the repairman (M/M/1//N) also holds for a more general source of requests viz., G/M/1//N. Since G is a superset of the distributions D and M, this theorem suggests that our simulations probably shouldn't be restricted to D-distributed think/execution times as observed in (1). Or should they?