Load Average in FreeBSD

In my Perl PDQ book there is a chapter, entitled Linux Load Average, where I dissect how the load average metric (or metrics, since there are three reported numbers) is computed in shell commands like uptime, viz.,

[njg]~/Desktop% uptime
16:18  up 9 days, 15 mins, 4 users, load averages: 2.11 1.99 1.99

For the book, I used Linux 2.6 source because it was accessible on the web with convenient hyperlinks to navigate the code. Somewhere in the kernel scheduler, the following C code appeared:

#define FSHIFT    11          /* nr of bits of precision */ 
#define FIXED_1   (1<<FSHIFT) /* 1.0 as fixed-point */ 
#define LOAD_FREQ (5*HZ)      /* 5 sec intervals */ 
#define EXP_1     1884        /* 1/exp(5sec/1min) fixed-pt */ 
#define EXP_5     2014        /* 1/exp(5sec/5min) */ 
#define EXP_15    2037        /* 1/exp(5sec/15min) */ 

#define CALC_LOAD(load,exp,n) \
 load *= exp; \
 load += n*(FIXED_1-exp); \
 load >>= FSHIFT;

where the C macro CALC_LOAD computes the following equation

Instrumentierung – die Chance für Linux?

My latest article for the German publication Linux Technical Review appears in Volume 8 on "Performance und Tuning" and discusses a possible roadmap for future Linux instrumentation and performance management capabilities. The Abstract reads:

• German: Linux könnte seine Position im Servermarkt ausbauen, wenn es dem Vorbild der Mainframes folgte und deren raffiniertes Performance-Management übernähme.
• English: Linux could be in a position to expand its presence in the server market by looking to mainframe computer performance management as a role model and adapting its instrumentation accordingly.

Topics discussed include a comparison of time-share scheduling (TSS) with fair-share scheduling (FSS) and the Linux Completely fair scheduler (CFS), how to achieve a more uniform interface to performance and capacity planning measurements, and the kind of advanced system management capabilities available on IBM System Z for both their mainframes and clusters.

Cisco Systems: "It's the switch, stupid!"

Today, Cisco Systems (San Jose, California) announced its mother of all switching platforms, the Nexus 7000 Series, aimed at what it calls Data Center 3.0 (analogous to Web 2.0, I presume. I missed Data Center 2.0). Cisco is essentially trying to eliminate the need for separate storage networks, server networks, routing, switching and virtualization, by combining them all into a single unified fabric and managing it through Cisco's new proprietary NX-OS ("nex-os", get it?) operating system.

The Nexus 7000 will deliver up to 15 Tbps of switching capacity in a single chassis, with 512 ports for 10 Gbps ethernet, and eventually it is slated to be delivered with 40 Gbps and 100 Gbps ports. Some of the claimed performance speeds-and-feeds appear rather breathtaking:

• Copy the entire Wikipedia database in 10 milliseconds.
  
• Copy the entire searchable Internet in 7.5 Minutes.
  
• Download all 90,000 Netflix movies in 38.4 seconds.
  
• Send a high-resolution 2 megapixel photo to everyone on earth in 28 minutes.
  
• Add a Web server in 9 seconds rather than 90–180 Days.
  
• Transmit the data in all U.S. academic research libraries (estimated at more than 2,000 TB) in 1.07 seconds.
  

If nothing else does it, the 3 significant digits in the last claim tells you this is marketing-speak (read: calculated using max bandwidth assumptions), so a liberal dusting of sodium chloride is recommended.

The concept of a "data center" is currently undergoing a serious transformation and it will be interesting to see how this kind of mega-switch stacks up against alternative approaches, such Google's Data Center in a Box.

Linux Instrumentation: Is Linus Part of the Problem?

I was interested to read in a LinuxWorld article entitled "File System, Power and Instrumentation: Can Linux Close Its Technical Gaps?", that Linus Torvalds believes the current kernel instrumentation sufficiently addresses real-world performance problems.

This statement would be laughable, if he weren't serious. Consider the following:

• How can current Linux instrumentation be sufficient when older UNIX performance instrumentation is still not sufficient?
  
• UNIX instrumentation was not introduced to solve real-world performance problems. It was a hack by and for kernel developers to monitor the performance impact of code changes in a light-weight O/S. We're still living with that legacy. It might've been necessary, but that doesn't make it sufficient.
  
• The level of instrumentation in Linux (and UNIX-es) is not greatly different from what it was 30 years ago. As I discuss in Chap. 4 of my Perl::PDQ book, the idea of instrumenting an O/S goes back (at least) to c.1965 at MIT.
  
• Last time I checked, this was the 21st century. By now, I would have expected (foolishly, it seems) to have at my fingertips, a common set of useful performance metrics, together with a common means for accessing them across all variants of UNIX and Linux.
  
• Several attempts have been made to standardize UNIX performance instrumentation. One was called the Universal Measurement Architecture (UMA), and another was presented at CMG in 1999.
  
• The UMA spec arrived DOA because the UNIX vendors, although they helped to design it, didn't see any ROI where there was no apparent demand from users/analysts. Analysts, on the other hand, didn't demand what they had not conceived was missing. Such a Mexican standoff was cheaper for the UNIX vendors. (How conveeeeenient!) This remains a potential opportunity for Linux, in my view.
  
• Rich Pettit wrote a very thoughtful paper entitled "Formalizing Performance Metrics in Linux", which was resoundingly ignored by Linux developers, as far as I know.
  

Elsewhere, I've read that Linus would like to keep Linux "lean and mean". This reflects a naive PC mentality. When it comes to mid-range SMP servers, it is not possible to symmetrize the kernel without making the code paths longer. Longer code paths are necessary for control and scalability. That's a good thing. And improved performance instrumentation is needed a fortiori for the layered software architectures that support virtual machines. Since Linus is the primary gate-keeper of Linux, I can't help wondering if he is part of the solution or part of the problem.