Aucbvax.2725 fa.works utzoo!decvax!ucbvax!works Tue Aug 18 02:56:03 1981 WorkS Digest V1 #7 >From DUFFEY@MIT-ML Mon Aug 17 22:14:30 1981 WorkS Digest Sat, 15 Aug 1981 Volume 1 : Issue 7 Today's Topics: Micro Benchmarks ---------------------------------------------------------------------- Date: 14 Aug 1981 15:28 EDT From: Marshall.WBST at PARC-MAXC Subject: Micro Benchmarks A salesman from Intel was here this week and said that EDN has retracted its benchmarks and will run another set. He said that the new numbers show the 8086 only 10-15% slower than the 68000. He said the retraction would be in the next issue. Sidney Marshall - Rochester N.Y. ------------------------------ Date: 14 Aug 1981 11:36 PDT From: Kosower at PARC-MAXC Subject: Re: Benchmarking new Micros Worse yet, benchmarks can be doubly deceiving, since they may induce you to choose a micro for all the wrong reasons. Unless you have a very specific application in mind, or unless you plan to design and build your own operating system from scratch, software development facilities, system quality, and especially user interface quality are extremely important. A Dorado, for example, may run faster than greased lightning, but it would be about as useful as an F-15 powered by turboprop engines if it had IBM-quality software running on it. After all, most of us do not want to write reams and reams of code in some J-random processor's assembly language (more so if it's microcodable); so quality of the high-level language available and quality of the compiler ARE important. Furthermore, most applications change, and even with changes, their lifetime is limited, so that other development facilities (editor, debugger, etc.) are ALSO important. If it takes you 10 times longer to write a program that will take 10 times longer to debug and will eventually run 10 times slower, on processor A whose raw speed is 10 times greater than that of processor B, which one would you choose? Choosing B will save you time, to say nothing of frustration, even though it is a "slower" processor. These are not idle thoughts: an IBM 370/168 has tremendous raw speed, but some of the cruftiest software ever written makes it seem slower than the US Postal Service. Admittedly, almost all of the software for micros such as the 8086 and 68000 is pretty awful, but I still think it's worth keeping the above considerations in mind. As Allan points out, just because something is easy to measure does not mean it's useful or even meaningful. David A. Kosower ------------------------------ Date: 14 August 1981 1832-EDT (Friday) From: David.Lamb at CMU-10A Subject: EDN benchmarks Allan Schiffman may be right that the EDN benchmarks are "about as useless as benchmarks usually are," but the MCF (Military Computer Family) test specifications on which they were supposed to be based *can* be used in a sensible fashion to evaluate a computer architecture. The original experiment design was set up at CMU with the co-operation of one of the members of out Statistics department, who designed the experiment to separate variance on the tests into differences based on programmers, particular tests, and the architecture itself. The notion was to see how good the *architecture* (instruction set, visible registers, etc.) was, rather than any particular implementation of the architecture. Several different measurement scales were set up. One measured the number of bytes need to encode the programs, another measured the memory accesses needed in an idealized implementation, and the third measured the amount of data transferred between registers in the idealized implementation. I'm a little disappointed that the tests are being used now for a different purpose; it's not clear to me that you want the same kinds of tests to do a comparison of particular implementations of the architecture, as was the case in the EDN tests. ------------------------------ End of WorkS Digest ******************* ----------------------------------------------------------------- gopher://quux.org/ conversion by John Goerzen of http://communication.ucsd.edu/A-News/ This Usenet Oldnews Archive article may be copied and distributed freely, provided: 1. There is no money collected for the text(s) of the articles. 2. The following notice remains appended to each copy: The Usenet Oldnews Archive: Compilation Copyright (C) 1981, 1996 Bruce Jones, Henry Spencer, David Wiseman.