|We have 7 guests and 0 members online|
You are an anonymous user. You can register for free by clicking here
6.2 - Distro Day (Measuring the benefits of the Gentoo approach)(1083 total words in this text)
Contributed by Indranath Neogy
A gallery of photos from the day can be found ..
First of all, a big thankyou to Evolution
Xtreme for the loan of the hardware and to Scott Middleton of Linux IT for all his help
arranging the test.
Scott gets the credit for thinking of this test. He said
"We're constantly hearing how the source based nature of the Gentoo
distro makes better use of your hardware, but no-one seems to have
really tested it. What kind of gains are involved over distros which
use binary packaging?" He arranged with Evolution Xtreme for the loan of
3 identical machines to test with.
Creating the Test
Obviously, the most direct way to test the compile time optimisations
of Gentoo is to compile 3 Gentoo systems with different settings and
then compare them. However, this really misses some of the story. Each
distro has it's own attitude to the kernel and whilst it may be i386
or i586 compiled, it will have had some adjustments made to it. In
reality, few people will be choosing to install Gentoo with less than
recommended optimisations for their system. They are interested in a
tradeoff between optimization and convenience. Thus, we aimed to
compare Gentoo with an i386 based distro and an i586 based
distro. With the assistance of some PLUG members we decided on Debian as
the i386 candidate and Mandrake as the i586, in part as those were the
options where people were available to do the install.
The following tests were outlined:
Time to open a large sheet in Gnumeric.
Time to perform a kernel compile.
Time to perform "Duplicate Image" in Gimp.
Time to perform a heavy "Unsharp Mask" in Gimp.
Time to start OpenOffice "from scratch".
Time to reload OpenOffice.
User experience to be assessed by all present on the day, using
Galeon, Evolution, OpenOffice.
To make it easier to standardize for these tests we picked Gnome 2 as
the Desktop Environment. This necessitated the use of the "Testing"
flavour of Debian.
The boxes from Evolution
Xtreme had the following configuration:
Celeron 2 GHz Processor
256 MB DDR RAM
SAMSUNG - SP4002H 40G HD
MSI 6533E main board
All SIS chipset
00:00.0 Host bridge: Silicon Integrated Systems [SiS] SiS651 Host (rev02)
00:01.0 PCI bridge: Silicon Integrated Systems [SiS] SiS 530 Virtual PCI-to-PCI bridge (AGP)
00:02.0 ISA bridge: Silicon Integrated Systems [SiS] 85C503/5513 (rev25)
00:02.5 IDE interface: Silicon Integrated Systems [SiS] 5513 [IDE]
00:02.7 Multimedia audio controller: Silicon Integrated Systems [SiS]
SiS7012 PCI Audio Accelerator (rev a0)
00:03.0 USB Controller: Silicon Integrated Systems [SiS] SiS7001 USB Controller (rev 0f)
00:03.1 USB Controller: Silicon Integrated Systems [SiS] SiS7001 USB Controller (rev 0f)
00:03.3 USB Controller: Silicon Integrated Systems [SiS] SiS7002 USB 2.0
00:0f.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
RTL-8139/8139C/8139C+ (rev 10)
The 8139 NIC used the 8139too driver in all installations.
The Debian box was installed by Garry Buckle with aid from Matt
Kemner. A standard Debian Testing setup was installed, but X was not
persuaded to start with the stock SIS driver. As the stock kernel did
not contain framebuffer support a new one (v2.4.21) was compiled to
get video working. Upon testing with hdparm, it was apparent that this
machine was having troubles setting above udma2. Eventually this
problem was traced to the HD cable, a salutary lesson in the
variability of identical hardware setups.
The Gentoo setup by Bill Kenworthy was compiled using the "stock"
kernel source and the "-march=pentium3 -pipe -O3" compile
flags. hdparm was needed to get dma on the ide running, despite it
being in the kernel, but "xfree --configure" worked for Bill using the
stock SIS driver. (Apparently the first time the command has worked
for him!) The Gentoo install suffered a couple of false starts due to
a typo using grub and OpenOffice was still being compiled the night
before the test. 11 hours later the OpenOffice compile was still going
and we thus had to regretfully abandon that portion of the test.
Garry's friend Joris (visiting from Belgium) put in the hard work on
the Mandrake install, clicking "Yes" and "Next" like a pro to complete
an impressively easy install. It defaulted to vesa framebuffer
display, similar to the Debian install.
Test 1 : Opening a 32,000 line sheet in Gnumeric.
We began this test with the mindset of testing the default installs
provided by each distribution.
Round 1 Results :
This was clearly bizarre. Debian and Gentoo both installed version
1.0.13 of Gnumeric, vs. Mandrake's choice of 1.0.12. We redid the test
for Debian and Gentoo using Gnumeric 1.0.12.
Round 2 Results :
The first lesson of this test is that minor version numbers can
apparently make a large difference. If you're using Gnumeric 1.0.13
with large sheets and waiting a while for them to open, it's probably
worth investigating version 1.0.12.
Test 2 : The Gimp
Each installation had the same version of the Gimp, 1.2.3. The tile
cache size was set to 96Mb on each machine.
Rounds 1 and 2 involved a non-standard filter replicator-gimp12.scm, doing a 2x2 replication in
the first round and a 4x4 in the second.
Round 3 was an Unsharp Mask with settings of Radius 25, Amount 5 and Threshold 255.
Test 3 : Kernel Compile
The same 2.4.21 source was copied to all machines and compiled using
the same options. However, it should be noted that the Debian system
used gcc 3.3.1 whilst the Mandrake and Gentoo installations used gcc 3.3.2 .
Test 4 : User Impressions
Due to the pressures of time, the user impression portion of the test
was not fully explored. The consensus of interaction with Gnome
Terminal, Gnumeric and the Gimp suggested that there was little
difference between the machines. The Gentoo machine seemed to draw a
little slower than the others which perhaps indicates that the vesa
framebuffer is a better choice than the SIS driver.
A quite unexpected set of results. We fully expected much more
significant variance between these Linux distros. We certainly
expected Gentoo to lead the tests, which has not been the case.
Suggestions for the disparity between the expected and actual results
have included different Gentoo compile options, in particular -O2
rather than -O3 flags, however many people recommend -O3 for this
kind of Celeron, so further experimentation may be in order. Likewise,
the stock gentoo-sources kernel includes optimisations for interactive
desktop usage but in our (limited) user impressions this benefit did
not show through. A further test is proposed for future months
involving P4 hardware and more time to compile and test which should
provide further information.
However, so far none of the distros appears to have a comprehensive
performance advantage, in the end it seems Linux is Linux and you
should pick the flavour that provides the most convenience for you.
Thanks again to Scott, Matt, Bill, Garry, Joris and Evolution Xtreme.
First | Second | Third | Fourth | Fifth | Sixth | Seventh | Eighth | Ninth | Tenth |