57 lines
4.1 KiB
HTML
57 lines
4.1 KiB
HTML
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
|
|
<HTML>
|
|
<HEAD>
|
|
<META NAME="GENERATOR" CONTENT="SGML-Tools 1.0.9">
|
|
<TITLE>Linux Benchmarking HOWTO: Introduction </TITLE>
|
|
<LINK HREF="Benchmarking-HOWTO-2.html" REL=next>
|
|
|
|
<LINK HREF="Benchmarking-HOWTO.html#toc1" REL=contents>
|
|
</HEAD>
|
|
<BODY>
|
|
<A HREF="Benchmarking-HOWTO-2.html">Next</A>
|
|
Previous
|
|
<A HREF="Benchmarking-HOWTO.html#toc1">Contents</A>
|
|
<HR>
|
|
<H2><A NAME="s1">1. Introduction </A></H2>
|
|
|
|
<P>
|
|
<P><EM>"What we cannot speak about we must pass over in silence."</EM>
|
|
<BLOCKQUOTE>
|
|
<EM>Ludwig Wittgenstein (1889-1951), Austrian philosopher </EM>
|
|
</BLOCKQUOTE>
|
|
<P>Benchmarking means <B>measuring</B> the speed with which a computer system will execute a computing task, in a way that will allow comparison between different hard/software combinations. It <B>does not</B> involve user-friendliness, aesthetic or ergonomic considerations or any other subjective judgment.
|
|
<P>Benchmarking is a tedious, repetitive task, and takes attention to details. Very often the results are not what one would expect, and subject to interpretation (which actually may be the most important part of a benchmarking procedure).
|
|
<P>Finally, benchmarking deals with facts and figures, not opinion or approximation.
|
|
<H2><A NAME="ss1.1">1.1 Why is benchmarking so important ? </A>
|
|
</H2>
|
|
|
|
<P>
|
|
<P>Apart from the reasons pointed out in the BogoMips Mini-HOWTO (section 7, paragraph 2), one occasionally is confronted with a limited budget and/or minimum performance requirements while putting together a Linux box. In other words, when confronted with the following questions:
|
|
<UL>
|
|
<LI>How do I maximize performance within a given budget ? </LI>
|
|
<LI>How do I minimize costs for a required minimum performance level ? </LI>
|
|
<LI>How do I obtain the best performance/cost ratio (within a given budget or given performance requirements)? </LI>
|
|
</UL>
|
|
<P>one will have to examine, compare and/or produce benchmarks. Minimizing costs with no performance requirements usually involves putting together a machine with leftover parts (that old 386SX-16 box lying around in the garage will do fine) and does not require benchmarks, and maximizing performance with no cost ceiling is not a realistic situation (unless one is willing to put a Cray box in his/her living room - the leather-covered power supplies around it look nice, don't they ?).
|
|
<P>Benchmarking per se is senseless, a waste of time and money; it is only meaningful as part of a decision process, i.e. if one has to make a choice between two or more alternatives.
|
|
<P>Usually another parameter in the decision process is <B>cost</B>, but it could be availability, service, reliability, strategic considerations or any other rational, measurable characteristic of a computer system. When comparing the performance of different Linux kernel versions, for example, <B>stability</B> is almost always more important than speed.
|
|
<H2><A NAME="ss1.2">1.2 Invalid benchmarking considerations</A>
|
|
</H2>
|
|
|
|
<P>
|
|
<P>Very often read in newsgroups and mailing lists, unfortunately:
|
|
<OL>
|
|
<LI>Reputation of manufacturer (unmeasurable and meaningless). </LI>
|
|
<LI>Market share of manufacturer (meaningless and irrelevant). </LI>
|
|
<LI>Irrational parameters (for example, superstition or prejudice: would you buy a processor labeled 131313ZAP and painted pink ?) </LI>
|
|
<LI>Perceived value (meaningless, unmeasurable and irrational). </LI>
|
|
<LI>Amount of marketing hype: this one is the worst, I guess. I personally am fed up with the "XXX inside" or "kkkkkws compatible" logos (now the "aaaaaPowered" has joined the band - what next ?). IMHO, the billions of dollars spent on such campaigns would be better used by research teams on the design of new, faster, (cheaper :-) bug-free processors. No amount of marketing hype will remove a floating-point bug in the FPU of the brand-new processor you just plugged in your motherboard, but an exchange against a redesigned processor will. </LI>
|
|
<LI>"You get what you pay for" opinions are just that: opinions. Give me the facts, please. </LI>
|
|
</OL>
|
|
<HR>
|
|
<A HREF="Benchmarking-HOWTO-2.html">Next</A>
|
|
Previous
|
|
<A HREF="Benchmarking-HOWTO.html#toc1">Contents</A>
|
|
</BODY>
|
|
</HTML>
|