tag:blogger.com,1999:blog-6977755959349847093.post5327312516285387277..comments2024-03-01T21:53:15.921-08:00Comments on The Pith of Performance: PostgreSQL Scalability Analysis DeconstructedNeil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.comBlogger9125tag:blogger.com,1999:blog-6977755959349847093.post-44180380107129121232012-04-30T09:43:03.865-07:002012-04-30T09:43:03.865-07:00Matteo,
Indeed, it is possible to talk about res...Matteo, <br /><br />Indeed, it is possible to talk about response-time (R) scalability and from the USL standpoint, we can derive R from the throughput data (X). I explain how to do all this in my upcoming <a href="http://www.perfdynamics.com/Classes/schedule.html" rel="nofollow">Guerrilla classes</a>.<br /><br />If we take the Postgres data (above) as the example, then what we can calculate from the USL model is the <b>mean</b> R: as in, statistical mean (average) or 1st moment. The median is p50 and is not a moment of the underlying statistical dsn; which we generally don't know. If I were to plot mean-R for the Postgres data, I already know that it should have the classic "hockey stick" shape. To the degree that it doesn't, we have to explain why not.<br /><br />With regard to your point about identifying queueing models, most queueing models compute measures that are characterized by the statistical mean. We may also be able to calculate higher moments from the assumed dsn in certain cases.<br /><br />So, on the one hand, we would prefer to have sample moments (average, variance, etc.) from the data to compare with any queueing models. Percentiles (e.g., p50, p90, p95), on the other hand, are merely a way of producing a ranked categorization of the sampled data.Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-79931001411629924402012-04-30T05:26:17.342-07:002012-04-30T05:26:17.342-07:00Hi Dr. Gunther,
I have a question that is maybe mo...Hi Dr. Gunther,<br />I have a question that is maybe more linked to response time analysis than scaling (if you allow such a distinction). Many performance tools and collectors return service time metrics in a synthetic way, such as: avg time, median, 90th, 95th, 99th percentiles. Is it possible from these numbers to understand which dsn they belong to (exponential, power law, normal, etc)? I'm wondering if this would help as an indication of the correct queueing model to be used (I'm re-reading 2.11 paragraph "Generalized servers" of your great "Analyzing Computer System Performance with Perl::PDQ" book).<br />Thanks<br />MatteoPMatteohttps://www.blogger.com/profile/04543214611852575232noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-57474646346976729102012-04-19T03:03:22.588-07:002012-04-19T03:03:22.588-07:00I believe a list of basic reading resources could ...I believe a list of basic reading resources could help any technical lead who meaasures performance. Apart from your blog I think these well known links helped me.But I had to search hard. These are the basics though.<br /><br />http://www.itl.nist.gov/div898/handbook/index.htm<br /><br />The desk reference of statistical quality methods<br /><br />What is recommendation for techniques to fit data to distributions ?<br /><br />Is something like "Goodness-of-fit techniques" by Ralph B. D'Agostino, Michael A. Stephens is recommended ?<br /><br />Also http://cran.r-project.org/doc/contrib/Ricci-distributions-en.pdf<br /><br /><br />Thanks.<br /> for the enlightenmentMohan Radhakrishnanhttps://www.blogger.com/profile/08457140016320542845noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-79970896429982682002012-04-13T10:51:13.178-07:002012-04-13T10:51:13.178-07:00The reason I ignored the points where N>32 is b...The reason I ignored the points where N>32 is because the data were collected on a system with 32 cores. So there are multiple factors limiting scalability here. Where clients <= cores, we have one set of bottlenecks, principally due to lock contention within PostgreSQL but perhaps also partly due to operating system or hardware effects. However, once the number of clients exceeds the number of cores, we're bound to hit a wall: if all the available CPU resources are already in use, adding more clients can't really continue to improve throughput. What I want to measure is whether it's possible to add throughput by adding more hardware resources (cores), NOT whether or not throughput will flatten out when we run out of cores. The answer to the latter question seems pretty self-evident: if we're efficiently using every core, then the best we can hope for when we run out of cores is that throughput will remain stable. In reality, of course, it will drop off slightly, because task-switching is not perfectly efficient.Robert Haashttps://www.blogger.com/profile/08393677427643988650noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-78768855229241387842012-04-11T20:24:59.472-07:002012-04-11T20:24:59.472-07:00Laks,
You are quite correct but someone pointed o...Laks,<br /><br />You are quite correct but someone pointed out the same typo via email earlier today. Now corrected and thank you for pointing it out.Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-16140408259228870562012-04-11T20:12:43.589-07:002012-04-11T20:12:43.589-07:00Should the first Normalized Capacity/Throughput eq...Should the first Normalized Capacity/Throughput equation be otherway around i.e<br /> Normalized capacity at N i.e CN = XN /X1 <br /><br />Please ignore if you had spotted it already.<br /><br />Cheers<br />LaksLAKSHMINARAYANAN SESHADRIhttps://www.blogger.com/profile/04134848453971331207noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-57258906801648661302012-04-11T12:59:09.059-07:002012-04-11T12:59:09.059-07:00More about MJ on FF:
* Browser Compatibility
* MJ ...More about MJ on FF:<br />* <a href="http://www.mathjax.org/resources/browser-compatibility/" rel="nofollow">Browser Compatibility</a><br />* <a href="http://www.mathjax.org/2012/03/02/news/mathjax-2-0-and-the-default-rendering-in-firefox/" rel="nofollow">MJ 2.0 and default rendering in Firefox</a><br /><br />Welcome to the web. :/Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-80778514647209723512012-04-11T12:34:39.465-07:002012-04-11T12:34:39.465-07:00Looks great in Safari. :)Looks great in Safari. :)Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-62334540751792447122012-04-11T12:29:05.441-07:002012-04-11T12:29:05.441-07:00the equation didn't display correctly on Firef...the equation didn't display correctly on Firefox 11.metasofthttps://www.blogger.com/profile/17149213781391733478noreply@blogger.com