tag:blogger.com,1999:blog-6977755959349847093.comments2024-03-01T21:53:15.921-08:00The Pith of PerformanceNeil Guntherhttp://www.blogger.com/profile/11441377418482735926noreply@blogger.comBlogger472125tag:blogger.com,1999:blog-6977755959349847093.post-5376289622947244942020-11-24T09:52:02.547-08:002020-11-24T09:52:02.547-08:00I don't know why I ever paid any attention to ...I don't know why I ever paid any attention to von Neumann's maxim: ""With four parameters I can fit an elephant. With five I can make his trunk wiggle." Old school !!!<br />http://perfdynamics.blogspot.com/2011/06/winking-pink-elephant.html<br /><br />The GPT-3 natural language model has a whopping 175 billion parameters. <br />https://venturebeat.com/2020/05/29/openai-debuts-gigantic-gpt-3-language-model-with-175-billion-parameters/<br />Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-83639839326225598242018-10-28T14:57:46.560-07:002018-10-28T14:57:46.560-07:00Also LOVING the speedier MathJax performance. Thx ...Also LOVING the speedier MathJax performance. Thx Google or Blogger or whoever. :)Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-75530355398262189252018-10-28T14:51:49.593-07:002018-10-28T14:51:49.593-07:00As a result of teaching yet another GCAP class (at...As a result of teaching yet another GCAP class (at a corporate site), I discovered that you can use the Excel Solver to compute all three USL parameters simultaneously. Although I would never trust any serious performance analysis to Excel, I'm mildly impressed that the Solver can be applied to the 3-parameter USL model. Woo hoo!Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-62116914472728177812018-10-07T18:40:43.131-07:002018-10-07T18:40:43.131-07:00Looks like my bitching about the performance of M...Looks like my bitching about the performance of MathJax on blogger has paid off, royally. <br /><br />Paraphrasing Mel Brooks, <a href="https://www.youtube.com/watch?v=7bF38rT7et0" rel="nofollow">it's good to be a bitch</a>.Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-26208187330837057472018-07-17T21:22:14.368-07:002018-07-17T21:22:14.368-07:00Updated the video link to our presentation because...Updated the video link to our presentation because CMG edited down the original combined YouTube recorded sessions into separate video segments and posted them on their web site.Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-89626567425857882322018-07-12T15:25:17.032-07:002018-07-12T15:25:17.032-07:00Hello to you,
I have implemented a Universal Sca...Hello to you,<br /><br /><br />I have implemented a Universal Scalability Law for Delphi and FreePascal, you can take a look at it here:<br /><br />https://sites.google.com/site/scalable68/universal-scalability-law-for-delphi-and-freepascal<br /><br /><br />Also i have just invented a scalable reference counting with efficient support for weak references, you can take a look at it here:<br /><br />https://sites.google.com/site/scalable68/universal-scalability-law-for-delphi-and-freepascal<br /><br /><br />I have also invented many other scalable algorithms that you will find on my website here:<br /><br />https://sites.google.com/site/scalable68/<br /><br /><br />Thank you,<br />Amine Moulay Ramdane.<br /><br /><br />Cyber10https://www.blogger.com/profile/00399764022497122745noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-45926551729209623862018-07-07T20:53:35.855-07:002018-07-07T20:53:35.855-07:00Added Microsoft capacity planning spreadsheet for ...Added Microsoft capacity planning spreadsheet for Azure to the extended cloudXchange <a href="https://speakerdeck.com/drqz/exposing-the-cost-of-performance-hidden-in-the-cloud" rel="nofollow">presentation slides</a>Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-74416531910714095842018-06-29T10:07:26.452-07:002018-06-29T10:07:26.452-07:00I just read that the letter 'c' was common...I just read that the letter 'c' was commonly used to represent the natural log base 2.7182, even after Euler had introduced the letter 'e' for it almost a century before Laplace wrote the 1816 supplement to his Théorie Analytique des Probabilities (the source of the derivations in the above post). I thought it might've just been a typo in the English translation but, that now seems unlikely. So, I've edited the original post to reflect that historical fact.Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-39478017773575808502018-06-19T08:30:07.681-07:002018-06-19T08:30:07.681-07:00We produce USL models using data from our producti...We produce USL models using data from our production systems using this method to estimate X(1). We have thousands of "real" load generators from which we feed the model ~30 days of hourly data points accross a range of load levels. This helps us 1.) get a feel for how much throughput capacity is remaining and 2.) observe how scaling changes between software releases (as defined quantitatively via the contention and coherency parameters). We're also toying with modeling impacts to response time by leveraging the predicted throughput from the USL models. Anonymoushttps://www.blogger.com/profile/08374463586201008123noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-64441722846907052382018-06-15T18:19:02.207-07:002018-06-15T18:19:02.207-07:00Consistent with my remarks (at the end of this vid...Consistent with my remarks (at the end of this <a href="https://www.youtube.com/watch?v=HBYGR7ou_6o&t=1s" rel="nofollow">video conversation</a> with Boris) about not really knowing how the plethora of machine learning model parameters are determined: "[Ai] Researchers, <a href="http://www.sciencemag.org/news/2018/05/ai-researchers-allege-machine-learning-alchemy" rel="nofollow">Rahimi said</a>, do not know why some algorithms work and others don't, nor do they have rigorous criteria for choosing one AI architecture over another."Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-62495857217295555232018-05-31T10:24:01.791-07:002018-05-31T10:24:01.791-07:00"One parameter is always enough" (See Fi...<a href="https://goo.gl/LAobHU" rel="nofollow">"One parameter is always enough"</a> (See Fig. 1a) Doh!<br /><br />There goes my <a href="https://goo.gl/nBpjQJ" rel="nofollow">Winking Pink Elephant</a> :(Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-24902065939441307992018-05-30T19:23:17.499-07:002018-05-30T19:23:17.499-07:00This comment has been removed by the author.Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-90463402300783883302018-05-21T07:21:19.913-07:002018-05-21T07:21:19.913-07:00I switched from MathJax to KaTeX and got much bett...I switched from MathJax to KaTeX and got much better performance.<br />https://www.xaprb.com/blog/switching-mathjax-katex/Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-919104921368017372018-04-22T13:48:49.087-07:002018-04-22T13:48:49.087-07:00Hi Neil,
Gotta say, this picture made zero sense ...Hi Neil,<br /><br />Gotta say, this picture made zero sense to me, so I went and read your original article, which made way more sense. Thank you, it was a very interesting read even though I can't say I understood all of it (curves - yes, proportional equations - no).testinglass aka Viktoriiahttps://www.blogger.com/profile/14181711385755927932noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-8507035693421501492017-12-19T21:06:04.219-08:002017-12-19T21:06:04.219-08:00Looking forward to it :-)Looking forward to it :-)AlexGilgurhttps://www.blogger.com/profile/01961676712058514905noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-64705688361006493142017-12-19T17:00:08.693-08:002017-12-19T17:00:08.693-08:00Thank you. This year, I presented at the
IFORS co...Thank you. This year, I presented at the <br /><a href="http://perfdynamics.blogspot.com/2017/03/morphing-mmm-new-view-of-old-queue.html" rel="nofollow">IFORS conference</a> in Quebec City, Canada. <br /><br />Maybe see you at CMG 2018. :)Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-5421834510854693892017-12-19T06:48:46.900-08:002017-12-19T06:48:46.900-08:00Nice overview! Thanks for the reminder :-). Missed...Nice overview! Thanks for the reminder :-). Missed you at #CMGimPACtAlexGilgurhttps://www.blogger.com/profile/01961676712058514905noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-28164847829318000422017-10-18T12:18:27.365-07:002017-10-18T12:18:27.365-07:00Very nice postVery nice postEnthusiasm Quoteshttps://www.blogger.com/profile/13788878293976552512noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-34417867802014285882017-08-18T10:27:50.526-07:002017-08-18T10:27:50.526-07:00Looks like we might have some new evidence in supp...Looks like we might have some new evidence in support of the scale-free model of Github growth. See Update upstairs.Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-39685165996090248102017-08-16T07:50:55.253-07:002017-08-16T07:50:55.253-07:00Cool but you really should come to my GCAP or PDQ ...Cool but you really should come to my GCAP or PDQ <a href="http://www.perfdynamics.com/Classes/schedule.html" rel="nofollow">Guerrilla class</a> and learn in <b>DAYS</b> what will otherwise take you months (or longer).Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-66612131710094179812017-08-16T07:08:09.789-07:002017-08-16T07:08:09.789-07:00Thanks very much! I am diving in to these and the ...Thanks very much! I am diving in to these and the other great resources you have including your books to learn more.<br />kshttps://www.blogger.com/profile/16640185679910305713noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-47414395071107922192017-08-14T18:20:53.538-07:002017-08-14T18:20:53.538-07:00You raise a lot of sub-topics here.
The above po...You raise a lot of sub-topics here. <br /><br />The above post is intended to underscore the idea that all performance metrics can be broken down into these 3 fundamental metric types (time, rate, or number). Conversely, other derived metrics can be built up from them. <br /><br />To address the questions about what to measure in an IoT application, I'd suggest drawing a functional block diagram, or diagrams, along the longs suggested in this <a href="http://perfdynamics.blogspot.com/2010/08/where-to-start-with-pdq.html" rel="nofollow">post on PDQ</a>. The goal is to fill in all the boxes you draw with times (either known or unknown). If there are boxes that cannot be assigned a time then, either the box is wrong or something needs to be measured that isn't. Educated guesses are also completely fine. Once you have the boxes all filled in it's a fairly straightforward matter to generate the corresponding <a href="http://www.perfdynamics.com/Tools/PDQ.html" rel="nofollow">PDQ code</a>. The PDQ model will soon let you know if your proposed processing times are consistent or not—that's one of its main values. And when I say "consistent' that includes a validation of the measurement instrumentation not being broken or misleading.<br /><br />In cases were the time cannot be directly measured, it may be inferred or derived from Little Law or other metric relationships.Neil Guntherhttps://www.blogger.com/profile/11441377418482735926noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-73615559418168838392017-08-14T17:04:38.312-07:002017-08-14T17:04:38.312-07:00Hi Neil,
In http://perfdynamics.blogspot.com/2016...Hi Neil,<br /><br />In http://perfdynamics.blogspot.com/2016/10/crib-sheet-for-emulating-web-traffic.html I asked a question which I think is more appropriate here.<br /><br />If you cannot capture the arrival and completion time for every message flowing through a complex system like IoT solution; what are the essential metrics that you have to capture?<br /><br />For example, say that millions of devices are writing to one IoT Hub (or Kafka cluster), and then 20 VMs are simultaneously processing messages from 20 partitions (devices are partitioned by device id hash). In this situation, IoT Hub (or Kafka cluster) is W(aiting) time, and Message Processor code is S(ervicing) time (http://perfdynamics.blogspot.com/2014/07/a-little-triplet.html).<br /><br />In the Message Processor running on each VM, what should one capture gor time interval T (say 15 seconds)?<br />1. arrival messages <br />2. average latency of message in IoT Hub (i.e. average Waiting time): This can be calculated from timestamp in message when retrieved.<br />3. average latency to "process"/"complete" in Message Processor (i.e. average Service time)<br /><br />Each processor on each VM would then send this telemetry to a central server.<br /><br />Is there anything else should capture? As suggested by https://www.percona.com/blog/2011/04/27/the-four-fundamental-performance-metrics/? <br />kshttps://www.blogger.com/profile/16640185679910305713noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-15183001915995544902017-08-14T17:04:18.735-07:002017-08-14T17:04:18.735-07:00Hi Neil,
In http://perfdynamics.blogspot.com/2016...Hi Neil,<br /><br />In http://perfdynamics.blogspot.com/2016/10/crib-sheet-for-emulating-web-traffic.html I asked a question which I think is more appropriate here.<br /><br />If you cannot capture the arrival and completion time for every message flowing through a complex system like IoT solution; what are the essential metrics that you have to capture?<br /><br />For example, say that millions of devices are writing to one IoT Hub (or Kafka cluster), and then 20 VMs are simultaneously processing messages from 20 partitions (devices are partitioned by device id hash). In this situation, IoT Hub (or Kafka cluster) is W(aiting) time, and Message Processor code is S(ervicing) time (http://perfdynamics.blogspot.com/2014/07/a-little-triplet.html).<br /><br />In the Message Processor running on each VM, what should one capture gor time interval T (say 15 seconds)?<br />1. arrival messages <br />2. average latency of message in IoT Hub (i.e. average Waiting time): This can be calculated from timestamp in message when retrieved.<br />3. average latency to "process"/"complete" in Message Processor (i.e. average Service time)<br /><br />Each processor on each VM would then send this telemetry to a central server.<br /><br />Is there anything else should capture? As suggested by https://www.percona.com/blog/2011/04/27/the-four-fundamental-performance-metrics/? <br />kshttps://www.blogger.com/profile/16640185679910305713noreply@blogger.comtag:blogger.com,1999:blog-6977755959349847093.post-28736251531952390412017-08-06T07:31:01.006-07:002017-08-06T07:31:01.006-07:00Thanks for the article.
Indeed, I have also seen ...Thanks for the article.<br /><br />Indeed, I have also seen articles and papers regarding trying to improve how benchmarking and performance testing for IoT. Some others that may be of interest:<br /><br /> • IoTAbench: an Internet of Things Analytics benchmark: http://www.hpl.hp.com/techreports/2014/HPL-2014-75.pdf<br /> • RIoTBench: A Real-time IoT Benchmark for Distributed Stream Processing Platforms: https://arxiv.org/pdf/1701.08530.pdf<br /> • A Model to Evaluate the Performance of IoT Applications http://www.iaeng.org/publication/IMECS2017/IMECS2017_pp147-150.pdf<br /> • IoT TCP: http://www.tpc.org/tpc_documents_current_versions/pdf/tpcx-iot_v1.5.x.pdf<br /><br />For emulating IoT traffic, most I believe are using non-standard tooling to emulate that devices. For example, a cluster to run containers that have device logic in them. So, they may indeed test up to the actual number of emulated devices vs. simulating with a combination of virtual users and think time. So this may be a difference in approach.<br /><br />I think a potential large different between IoT solutions and Web Apps s is that they:<br /> 1. Are messaging based: data flow of message sources and sinks<br /> 2. Implement hot, warm, and cold paths<br /> 3. Leverage, Real-time analytics (CEP, Stream processing), In-memory computing (Spark, etc.), Indexed Storage (Solr, etc.), MapReduce (Spark, Hadoop)<br /> 4. Can think of IoT as data flow of sources and sink<br /><br />(1) Event Production -> (2) Event Queueing & Stream Ingestion -> (3) Stream Analytics -> (4) Storage & Batch Analysis -> (5) Presentation and Action<br /><br />And technologies below:<br /><br />1) Devices and Gateways<br />2) Azure Event Hubs, IoT Hub ; or Kafka<br />3) Azure Stream Analytics; or Spark Streaming<br />4) Azure Data Lake, CosmosDB, SQL Database, SQL Data Warehouse; or Spark, Hadoop [*]<br />5) Microsoft Power BI; or Tableau<br /><br />[*] http://perfdynamics.blogspot.com/2015/03/hadoop-scalability-challenges.html. Big Data is usually part of IoT solution.<br /><br />With this type of complexity, it seems critical to have telemetry needed to do performance and scalability analysis. <br /><br /> 1. Ideally you would capture telemetry for each message with timestamps at key points in data flow, including correlation id for end to end visibility<br /> 2. Or if that is not possible (because development has not been done, or throughput is too high) just capture metrics that are needed for performance and scalability analysis:<br />At important points along message data flow capture: time, count, rate. What metrics would you recommend capturing in a messaging system like this? For example in (#2) in stream ingestion code, this may be cluster of N VMs mapped 1:1 to a partition in IoT Hub, each processing messaging in batches of X messages. Then sending for further processing to (#3) Stream Analytics or for storage, analytics to (#4) Data kshttps://www.blogger.com/profile/16640185679910305713noreply@blogger.com