home | user blogs | submit | FAQ | search | top stories | user account
Interview: Con Kolivas Submitted by Jeremy on Wednesday, October 16, 2002 - 15:40
LinuxCon Kolivas, a practicing doctor in Australia, has written a benchmarking tool called ConTest which has proven to be tremendously useful to kernel developers, having been designed to compare the performance of different versions of the Linux kernel. He was kind enough to speak with us, explaining how he got started on this project, what makes his benchmark unique, and how to interpret the resulting output. Comparing the 2.5 development kernel to the 2.4 stable kernel, Con says, "a good 2.5 kernel (and that's not all of them) feels faster than 2.4 in most ways and this bodes well for 2.6." The interesting results from his frequent benchmarks back up this statement.

Con also describes his high performance patchset for the 2.4 stable kernel, currently at version 2.4.19-ck9. This patchset adds a number of performance boosting patches ideal for a desktop environment, such as the O(1) scheduler, kernel preemption, low latency and compressed caching. Read on for the full interview...

JA: Please share a little about yourself and your background...

Con Kolivas: I'm 32 years old, live and grew up in Melbourne Australia, am very happily married and have a 9 month old son. I'm a little embarrassed people get me confused for a kernel hacker, as my real profession is very remote from IT. I'm a doctor; a specialist in anaesthesia.

JA: How and when did you get started with Linux?

Con Kolivas: I grew up with computers (geek) but did not work or study them in any formal manner. While studying medicine and then specialising I went a long time without computers at all (the post amiga days). When I finally got back to computers in about 97 I was incredibly frustrated with the microsoft based machines I only had to work with after being so happy with the performance and flexibility of much lower spec amiga machines. A friend introduced me to linux in 1997 but being too far removed from computers at the time I found it difficult to get started with it. In 1999 I decided to try again and got quite addicted (bordering on the obsessive) in a very short timespace. 6 months later I gave up on other OSs as I noticed linux had a momentum that would make it unstoppable, even if it definitely wasn't (and still isn't) the best tool for all tasks. I've used numerous distributions in the past but Mandrake gets me up and running with more things working with less fuss so I tend to stick with it.

JA: When did you first start reading kernel code?

Con Kolivas: 2.4.18 when I started trying to merge O1, preempt, low latency and compressed cache. After applying each patch I had to sort out the problems with each merge and found that looking at the code it made a lot of sense to me and I could sort out the problems - mind you I can't program in c at all. Look at the code for long enough and you start understanding what it is doing.

JA: You've recently been doing quite a lot of work on a benchmarking tool called Contest. How is this tool different than other benchmarks?

Con Kolivas: Long story to explain this. When I started merging interesting kernel patches for 2.4.18 that were known for improving system response initially people just gave me small amounts of positive feedback. When I posted that I had merged the patches for 2.4.19 for some reason it attracted a lot more feedback. This time I had people repeatedly asking me if I had benchmarked these patches; could I substantiate my claims that they made the system more responsive. I used the excellent resources of the open source development lab http://www.osdl.org to benchmark my kernels and got the results I expected - virtually unchanged from a vanilla kernel. At about the same time Rik van Riel had been defending his -rmap patches repeatedly on lkml and the #kernelnewbies channel about the fact that although benchmarks didn't show any improvement in performance, users had found that it made a difference. Many lkml threads followed about how one thing benchmarked after the other was not a real measure of system responsiveness. None of the standard benchmarks available at the time would tell you that. I was quoted as saying "a good anecdote is worth a thousand benchmarks". We all know that if you start a cpu intensive process in the background, linux won't bat an eyelid with no noticeable slowdown in system response. Do a big file write or untar a file and try to do anything and be prepared to go make yourself a coffee while waiting. Rik encouraged me and others to "do something" about this on IRC. Repeatedly on lkml Linus was quoted as saying "if we can't measure it it doesn't exist" and Rik said "If we don't measure it our method of development will ensure it won't exist". Even though my c programming skills are shall we say bordering on the /dev/zero I had been thinking about this very thing and knew I could do it with a simple script.

Contest (pun and name courtesy of Rik van Riel) takes an easily reproducible thing to do - compile a kernel - which represents a whole swag of things a user may do and notice a slowdown on the machine; use heavy cpu, file IO etc and times it in different settings. It is run on as fresh a machine as possible in single user mode to eliminate the influence of other activity on the results. Then it flushes the memory and all the swap so the benchmark is always starting up "cold". Then it times a kernel compilation by itself, and in the presence of a number of different loads - a heavy context switching load (process_load) a heavy file write (IO_load), file read (read_load) memory grabbing (mem_load), extracting a tar (xtar_load), creating a tar (ctar_load) and a huge directory listing (list_load). The idea is that by doing it for the duration of the kernel compile it will increase the signal to noise ratio of the test and pick up slowdowns that we may momentarily notice when trying to do things on our machines. This was quite a departure from the "throughput" approach to benchmarking, and appears to more realistically represent what happens in the real world.

JA: For what sorts of benchmarking is Contest best suited?

Con Kolivas: Contest is a very specific tool for kernel comparisons. Because the tools (c compiler etc) and hardware do not change between benchmarks it can only note a difference between kernels. As I was watching kernel development for 2.5 head toward the heavy iron I, and many others, were concerned that the desktop was taking a back seat and that it would suffer and be worse by 2.6. A few threads on lkml went so far as saying this [change or that] would benefit NumaQ machines and be only a small detriment to ordinary machines. Contest is good at picking up changes that would cause real slowdowns on ordinary machines under stress. In fact, the pickup on these machines with contest is greater than big machines which tend to have hardware that compensate in one way or another. As it turns out, though, these slowdowns that affect smaller machines, if corrected, benefit across the board.

JA: For what sorts of benchmarking is Contest not well suited?

Con Kolivas: For just about everything else. Unfortunately although it's an easy to use script for any user, the results from a users point of view are not useful - kernel development was the intended audience.

Comparisons between hardware - even with minor changes - are meaningless. It's almost impossible to pickup what has caused the difference. A faster hard disk for example can speed up the benchmark by taking some of the load off the machine OR it can slow it down by being so busy writing the cpu chokes and doesn't get a chance to do anything else.

The traditional benchmark measures of throughput, iterations, data processed etc are not measured to any great extent. The background load is the only thing measured as lets say you start writing a file in the background - you want the machine to remain responsive, but you also want the file to be written as fast as possible. Contest gives a measure of the responsiveness in the kernel compilation time, and a measure of the file write as the number of "loads" in the result. Ideally time will be low first and loads high second but the balance can swing, and contest will demonstrate this.

JA: Have you received much feedback on your benchmarking tool and the results you regularly post to the Linux kernel mailing list?

Con Kolivas: I've received a lot of feedback. The most reassuring thing is that I've been getting feedback from the people who actually have the ability to act on the information themselves. This has prompted me to spend an unbelievable amount of time developing contest, as I didn't really have the skills to create it in the first place - just the idea - and their feedback has helped develop it. Andrew Morton has given me probably the best comments. Many of his ideas for loads have been incorporated into contest and he has the ability and knowledge of the workings to best interpret the data. Some of the recent feedback has suggested that people _with_ programming skills (unlike me) are developing other benchmarks based on the ideas I used in contest. This is a good thing. Although I am clear of most of the limitations of contest, I have to learn the skills to work around them. While it's good fun, I don't want biased or inaccurate results swaying kernel development the wrong way. I suspect it won't be long before someone realizes I'm a fraud and displaces me.

JA: What are some benchmarks that are beginning to use ideas originated with contest, and what aspects are being used?

Con Kolivas: Bill Davidsen, who's sig reads "Doing interesting things with little computers since 1979" is developing a responsiveness benchmark and just started posting his results to lkml which he is calling "resp1 trivial response time test". In his words "The benchmark runs a noload case, then forks a load process (or processes) in the background, pauses long enough to simulate user interaction (and get swapped out if the system is memory stressed), then reports the time it took to complete, including the ratio of loaded to noload time." The concept being used is that of trying to perform tasks in the presence of different loads much like contest does. It seems to be a very interesting benchmark and if he develops it I think contest will already have a successor.

JA: Based on your benchmarking tests, what observations can you make about the 2.5 development kernel, especially compared to the 2.4 stable kernel.

Con Kolivas: As most of the contest results show changes in scheduling, IO and VM, I can't really comment on any other area. It surprised me when I first started looking at the VM code just how restrained some of the 2.4 stuff was and how many changes have gone into 2.5. Despite this, there are heaps of great ideas for the VM that just won't make it into 2.6, and may not even be ready for 2.8 because of how stepwise the development needs to be. The results I've obtained show that sometimes the most minor changes can have enormous implications for one part of contest or the other. There is no doubt that file writing is a _lot_ faster in 2.5 and is so fast it can all but kill the machine. Secondly the kernel has become very swap happy. This has upset a few people and AKPM is heavily working on a vm_swappiness feature that still needs quite a lot of tuning to not choke the machine in either direction. However a good 2.5 kernel (and that's not all of them) feels faster than 2.4 in most ways and this bodes well for 2.6. What I have noticed with contest is that great code untested in one way or another is not necessarily a speedup unless it's tuned. Hopefully contest and any successor benchmarks can help that.

JA: What are some recent changes in 2.5 that have had significant impact on kernel performance?

Con Kolivas: When rmap was being integrated into 2.5 the results were less than great. At the same time Andrew Morton was heavily putting together his mm patch series which showed substantial improvements with contest (and other benchmarks) and his patches have gradually been incorporated into 2.5. His (and those contributing to his) vm patches have made enormous measurable gains. The deadline scheduler introduced significantly prevented kernel choking. His vm swappiness feature which still needs some tuning can be either very good or very bad for different contest results depending on whether it is set to low or high - he is modifying and tuning it for this to be tamed and the results are already improving dramatically. I came in to 2.5 too late to know exactly where the gains prior to this occurred, but almost across the board 2.5 outperforms 2.4 in contest.

JA: What do you mean when you say that file writing is so fast in 2.5 it can "all but kill the machine"?

Con Kolivas: When the IO scheduler was updated in 2.5 it was proudly announced that 2.5 can keep 60 spindles saturated with data. Well this may have been true, but what happened during this heavy file writing is nicely demonstrated in contest - the kernel could not do anything else at the same time. IO load would show enormous amounts of background load being performed, but a simple kernel compile could take an hour during this load instead of one minute!

JA: What improvements do you still have planned for contest?

Con Kolivas: I would like to find a way to measure the load cpu% during each kernel compilation more accurately. At the moment the load starts 5 seconds before each kernel compilation and finishes a variable length of time after the compile. The load cpu% is simply that returned by the time command at the end of the load running - this overestimates the load. Also the method used to estimate the number of loads is not perfect.

Other people have been sending me patches to clean up what c code is in there.

I'm greatly at the mercy of the kernel hackers themselves as to the direction it will take from here, as they have been suggesting the loads to add. At the moment I have no other loads planned that are reasonable to include. Beyond this my plans are for it to be deprecated in favour of a better benchmark.

JA: When you post your contest results to the lkml, they are divided between several types of loads. What follows, as an example, is the output of the 'io_load' section taken from a recent test you've posted. Can you explain what the individual columns mean, and how I can interpret these numbers?
Kernel [runs] Time CPU% Loads LCPU% Ratio
2.4.19 [3] 492.6 14 38 10 7.33
2.4.19-ck7 [2] 174.6 41 8 8 2.60
2.4.19-ck9 [2] 140.6 49 5 5 2.09
Con Kolivas: The first column shows the name of the kernel and in [] parentheses how many times this particular test was conducted for that kernel. The Time column shows how long it took to perform a simple kernel compilation in the presence of (in this case) io load. CPU% shows the average cpu% kernel compilation was using when this load was running. Loads is an internal number showing the amount of work the background load performed during this test - the absolute number is not meaningful in any other context. LCPU% shows the average cpu% the load was using during the duration the load was running. Finally ratio is a simple ratio of Time compared to the reference (in this case 2.4.19 with no load).

What we want is the following, in this order of importance:
Time needs to be as low as possible (and therefore ratio as close to 1)
CPU% needs to be as high as possible
Loads needs to be as high as possible
LCPU% needs to be as high as possible.

Using your cropped example, you can see that -ck9 manages to take the least time to compile a kernel, and in that time the loads and load cpu% are lower than the others, but given that time is the most important, the other numbers take a back seat. If for example time was the same between kernels but one had a higher load, then the higher load one was performing better, and so on. This is a good example because it shows that when the kernel is busy writing large files (IO_load), the cpu is not being used to it's fullest as the cpu% + lcpu% is not even close to 100% - ie the cpu is idle waiting for the kernel to give it something to do. You can see interpreting this data can be difficult if many factors change at once.

JA: I've been happily using your -ck7 patchset since it was released. Are you still maintaining these patchsets?

Con Kolivas: My idea for the -ck patchsets was to put together great patches that make a real difference to performance into a stable package without fluff. My original idea was to merge O(1), preempt, low latency and the compressed cache patches. ck7 doesn't have the compressed cache, but does have either the aa vm or rmap vm as well, and the feedback I've received so far shows it to be as stable as vanilla 2.4.19. Which is the way I'd like to keep it. I have had requests for one feature or another, and I've sent out some custom patches for people when they've asked but have not made them part of the default ck patchset. I actually do plan to release another -ck for 2.4.19 though, and this is to add compressed caching. I've been battling with it for some time and found a problem when mixing this with the -aa vm (it is virtually incompatible with rmap at this time). After contesting it I've decided that to release ck9 (as it will be) I'll have to back out the aa vm or rmap option (ck7 will still be available with these). All my testing has shown this combination to be as stable as ck7 but perform better. This is what I use on my home machine and I thought I'd make sure it was stable before releasing it. I am interested in feedback though ;). I'll continue with ck for 2.4 wherever possible until 2.6 comes out as I don't think any of these patches will make it into mainstream 2.4

JA: What happened to -ck8?

Con Kolivas: Should have guessed you'd ask that. A ck7-ck8 diff was posted on my homepage but not recommended to be used. This added compressed caching to ck7. When I used it with the added vm changes from the -aa kernels the machine would experience strange prolonged pauses. Backing out the -aa changes fixed it. Others have described this problem with these patches even without the compressed caching but I had not seen it. So I eventually gave in and removed the -aa addons and released it as ck9, as it outperformed just those addons. Plus I had many requests for ALSA and XFS so I added those too.

JA: Do you intend to merge any additional patches into your -ck patchset?

Con Kolivas: Not unless I get specific requests ;-) I'm not aware of any other major performance boosts that are stable. A few small patches around that suggest they may be useful I've contested and the results have not been significant. I intend to bring out a separated version of ck9 so people can add just the patches they want. I also plan to bring out a version of ck9 (with xfs and alsa) suitable for smp; ie with -aa vm instead of -cc.

JA: How stable do you feel the compressed caching code is?

Con Kolivas: Very. The author (R.De Castro) gives no guarantee to their stability (he calls it v0.24pre4) but I've not had any lockups on the machines I've run it on (longest has been 3 weeks). Mind you, compressed caching offers nothing to machines with heaps of ram, and is not compatible with SMP, but my target audience was originally the desktop.

JA: What other projects are you currently working on?

Con Kolivas: I hover from one little thing to the next. I do have a full and very busy lifestyle and I already spend way too much time on linux (just ask my wife)

JA: Can you describe your development environment, including the hardware and software tools you typically use?

Con Kolivas: Not being a software developer myself I never learned to use real tools. I use kate (kde advanced text editor), patch and diff and that's about it. I guess pico at the console for the least fuss. I do most of my work an a modest laptop at the moment (see my benchmarks results for specs).

JA: Have you worked with any other open source kernels?

Con Kolivas: I don't have the time or background in computer systems for this to be possible.

JA: What do you enjoy doing in your non-Linux time?

Con Kolivas: Spending time with my fabulous family. I also enjoy high performance motor vehicles, classical music, science fiction novels, audiophile hifi and playing nintendo (gamecube - now there's a platform we can learn something from - turn it on and it just works tm.)

JA: Is there anything else you'd like to add?

Con Kolivas: Yes, after having looked at some of the kernel code I feel IMHO that something must happen. Magic numbers must die. There are heaps of occasions where a value must be put to a variable, and that value will be chosen depending on benchmarking. These numbers are always a compromise. I submitted an idea for an autoregulated number for vm_swappiness to AKPM. I had no intention of the code actually being used (I only put it through some brief testing), but I was trying to demonstrate what these numbers need. Magic numbers should be autoregulated by the kernel. They should be variable and work from some sort of feedback loop. My experience with human physiology has shown me that there are an almost infinite number of autoregulated feedback loop type control systems in the human body that give it incredible flexibility to cope under all sorts of situations. I believe, and hope, that this approach could add to the flexibility of the linux kernel.

JA: What sorts of feedback did you receive from Andrew Morton regarding your vm_swappiness patch?

Con Kolivas: He said it could work but that he didn't want to get too complex in there. I assume he meant he didn't want an algorithm included in vmscan (even though it's very basic). He also wanted to offer people a swappiness dial that had simple to understand effects they could manipulate, meaning they could set vm_swappiness to the number they wanted rather than have the machine choose it. I suggested letting them set the maximum vm_swappiness instead but the discussion thread trailed off at that point. He is still putting in a lot of work in other areas of vm_swappiness. Currently the default is set at 60 (range 0-100).

JA: Thanks for taking the time to answer my questions! Your -ck patchsets are greatly appreciated daily on my desktop system, and I'm certain your contest benchmarks are ultimately going to result in a better 2.6 kernel.

Con Kolivas: You're most welcome. I'm a verbose kind of person so I didn't need much prompting. I'm glad you enjoy ck as much as I did putting it together. I greatly look forward to 2.6 myself as well and hope I'm helping in whatever small way I can.

Related links:
[ add new comment ]

Control panel

Comment viewing options:

Select your prefered way to display the comments and click 'Update settings' to active your changes.

Subject: I'm very pleased Jeremy
Author:David Nielsen
Date:Wednesday, 10/16/2002 - 18:34
This is IMHO the most useful and informative interview Kerneltrap has done to date. I was very surprised to discover that Con was a doctor of trait and not a code geek - that certainly gave me the courage to fire up KVim and mangle some kernel code of my own, as the good doctor told us, it's really not that scary.

I'm glad that Con and Rik stepped to the plate and made the tools avail. to benchmark the kernel properly, so that we can get 2.5 tuned using some real life figures and not just "magic" numbers, guess work and estimates.

Hell I blame Con for me running 2.5 kernels as my default kernel, all those pretty figures made me crave the speed for my desktop.

But what I really wanted to know, would there be any interest(or sense) in setting up a database of contest results, I imagine that this would require quite some information to stored, like kernel configuration, hardware and of course the all important contest results.

btw, am I the only one who's a bit concerned about a _doctor_ who uses the nickname conman ? :)
[ reply to this comment ]

Subject: Database
Author:Con Kolivas
Date:Thursday, 10/17/2002 - 03:52
Hey that edit made me sit up and take notice of your comment. It's con the man, I'm not a conman...(g) Since I'm doing most of the benchmarking pretty much all the data to date I post with each benchmark and the hardware stays static. With time of course the number of benchmark results will probably not make it feasible to post all the results. I will be putting them in a place they can all be seen, but noone else is doing any benchmarks with contest at the moment so a complex database is not required.
[ reply to this comment ]

Subject: thanks
Date:Saturday, 10/19/2002 - 08:28
I'll soon start benchmarking some of the kernels on my site. I've been interested in tracking kernel performance changes for awhile. I plan on starting with a vanilla 2.4 and then moving through the 2.5 series as it progresses to 2.6 (or 3.0). I had planned on using lmbench as well as a real-world kernel-compile to test. I may just add Con's tests to the set.
So thanks.

One of these days, at:
[ reply to this comment ]

Subject: Magic Numbers
Date:Wednesday, 10/23/2002 - 20:36
Am intrigued by your comment that magic numbers should be autoregulated by the kernel - your reference to human physiology seems to be intuitively correct.
A thought struck me while I was reading this: surely if vm_swappiness can be autoregulated then so can any magic number - and if this is so - then can the same autoregulating code be used to perform this task? In essence then, your code could form the heart of an autoregulating system. Just a thought, as i said - IANAKH (I am not a kernel hacker).
Would love if you could expand on this topic.

[ reply to this comment ]

Subject: Great idea...
Author:David Nielsen
Date:Thursday, 10/24/2002 - 01:28
I _strongly_ agree with Con on the subject of magic numbers, I would like it if the self regulating system at least was merged into a seperate branch for testing - if it performs well then we can consider merging into the mainstream kernel.

I'm against the whole dial options, it's sub optimal, using dials you'll have to know a bit more about your system and the way fx. the VM works on your system - and for that to happen the author of patch would have to provide lots of information for kernel configuration. In order for us to even to use the dial, let alone use it correctly.

Dials and magicnumbers might be less complex and "easier" to code/use - but I feel that we at least should give the idea of a self tuning system a decent shot.
[ reply to this comment ]

Subject: Autoregulation
Author:Con Kolivas
Date:Thursday, 10/24/2002 - 10:30
2.5 is a lot better of than 2.4 was in this respect. Most of the developers are concerned with this idea as turning the kernel into a never ending series of complicated loops and feedback that would be difficult to track the effects of. Nevertheless I believe that the numbers that do remain (and there are quit a few) still stand to benefit by autoregulation. However, for such a thing to be implemented, the people who set the magic numbers in the first place are the best positioned to know what values should feed back into the loop to tell it when to change the value of the magic number. AKPM I understand is already doing such a thing with his vm_swappiness feature, even though he doesnt call it this. There are still heaps of other settings, though, that may benefit from such action. Even something like the Hz value that can now be set (see other talkback) could be regulated now that changing the value doesnt affect other values.

However, this is not just a matter of me or someone writing a autoregulation.c and other routines sending it values. A better approach is a rethink of how to set values in the kernel globally. This would require a paradigm shift in the way people program for the kernel. The changes to do such a thing are actually not really big in terms of programming - but the values to feed back and how they affect the magic number - knowing these is the key.
[ reply to this comment ]

Subject: This is problematic
Date:Friday, 10/25/2002 - 01:28
The problem with this of course is that not people want to tune
all the "knobs" the same way. It depends entirely on your workload
and the kind of service you want.

Some may want to have ultimate response time to user input.

Some might want best lowest interrupt delay (for smooth audio

Others may want to degrade response time for more processing
througput (think cluster machines).

An auto-tuning system at a minimum would need to know what kind
of workload, performance, response time, etc, the user is looking

Our bodies have this information encoded in our genes. It's hard
coded for human beings. We're setup based on the climate we've
been living in, the kind of work we do, etc. Other species, even
closely related to humans (great apes), have different hard coding.
But your autotuning would essentially be a try to use the same
tuning for different species. Or something kinda like that.

I'm not saying this is a bad thing, or isn't worth trying. I
think it is. However, it's an extremely difficult task and no
matter what you do, it's going to have disadvantages for one
class of task (species) or another.

[ reply to this comment ]

Subject: Evolution
Author:Con Kolivas
Date:Friday, 10/25/2002 - 17:49
I dont doubt that poorly done it would not be worth attempting. I dont envision the kernel will suddenly become a species of it's own overnight. However I do believe the ability to autoregulate based on the workloads, conditions, hardware etc is what will make it more flexible in the long run. This is something that will require it to break free of it's current limitations and is not going to happen any time soon. Evolution takes time, and small, safe, high gain low risk steps are what should be done at a time. As it is I can already see it being done by some of the newer code without me to say they should do it. I just hoped the concept was understood by those that weren't already thinking of it.
[ reply to this comment ]

Subject: my 2 cents (IANAKH)
Date:Saturday, 10/26/2002 - 03:06
The problem with a feedback system is that it has a big performance hit. Every feedback operation requires additional computations, and not all will be computed in constant time. This will result in higher latency and more expensive operations.

This is an interesting idea, and I might be curious to see an implementation, but I'm not sure this is a good thing to have in a real-world use.
[ reply to this comment ]

Subject: Simplicity
Author:Con Kolivas
Date:Saturday, 10/26/2002 - 05:08
Your point is well taken. The key for it to work would be for it to be as _simple_ an algorithm as possible. A simple addition or subtraction to the magic number based on input from another number would not be a performance hit. Latency would not be affected by such a thing, especially if the magic number "rests" on the midpoint where it would have been had it not been regulated.
[ reply to this comment ]

Subject: Not problematic, but difficult
Date:Tuesday, 10/29/2002 - 18:24
Autotuning is one of the main caracteristics of human beings compared to other species. Some scientists call this adaptation.

Of course we have hard-coded things in our genes, but we have also a brain that adapt our comportment to our environment. The same way, the operating system kernel should adapt to the tasks it is used for.

Autoregulation proposed by Con is a first step in adding artificial intelligence to the kernel.

Olivier Mengué
[ reply to this comment ]

Subject: optimal auto-tuning
Date:Saturday, 11/09/2002 - 10:21
IAKH (another one :P) but I must say I find Con's idea at least intriguing and the proposal of a generalised autotune functionset even hotter.
Here are some considerations:
As it has already been pointed out, different macchines and users try to "focus" on specific strong points in the system performance. Therefore a regulatory function should have to consider specific "presets" pretty much invalidating its reason to existence (other than a user-friendly value setting method). To get the best out of an "autotuner" it should auto-regulate the magic numbers according to what's currently happening in the machine. Am I pushing hard on devs? (like streaming audio) What about my disk IOs? etc... So, the autotuner should have presets that only define "preferential" paths of fixing what becomes problematic on the machine at the moment. It now becomes obvious that by using a linear feedback design, our algorithm will "lag behind" eventual troubles only to fix them after they occur. This is not optimal. To keep a track with Con's physiology example, having such a "lag" in our movement evaluation, we would probably take a fall anytime we tried to climb a stair. What really happens is that our cerebellum estimates with minimal anticipation what our position in space will be and adjusts before the events. So how do we build a kernel "cerebellum"? By calculating the trends of each "load variable" and adjusting magic numbers before the loads hit the ceiling. In that way we don't have to define soft limits under the critical values to avoid issues before they occur and thus we can exploit the machine at 100% on any preferential performance. This might sound terribly complex but in fact it isn't. Note that the complexity of such an algorithm does not rise lineary but exponentialy by increasing the monitored variables (degrees of freedom of the system) and the variables in our case are just a few (don't even try to compare them with the human body moving in space :P). Yes, I do realise that this will inevitably introduce an overhead but this is our bet: give most possible at the lowest cost. Such a design would only damage a cluster node designed to perform a single thing or an anyhow "dedicated" machine - be it fileserving, calculation or whatever. But this is not our target. Our target is Joe Random using his linux box that might fire up a midi sequencer to write some music but still wants a fast compile and a responsive menu on his favorite DE. Right now, the only solution for such "multiversed" machines is to make a compromise and avoid pushing settings of this or that to hard. It's not a flexible solution. If the autotuner manages to cost Joe only a slight overhead in his ever-growing powerful CPU and RAM while guaranteeing fluid operation even at limits I'd call that a success. To make an example: I work a lot with computer music. To know that I can only load reliably 30 tracks instead of 32 in my sequencer is no big deal. What makes me sweat is the fear that my box will overload just when I'm recording because popped up a file manager to get another sample and the sound went cranky - I'll have to do it again!
Ehh, that was rather lengthy... thanks for your patience!
[ reply to this comment ]

Subject: Nice perspective can I also add...
Author:Con Kolivas
Date:Sunday, 11/10/2002 - 09:29
The human heart is probably the best example people can understand. It takes but one beat to get the next setting right. I think people are getting carried away with what sort of performance hit this would be to the kernel. The code should be extremely simple. Addition/subtraction or byte shifting one variable in an if{} statement would not be measurable. Furthermore, the autoregulation can happen before the actual effect of a changed variable will be used.

I have included my first autoregulation patch in ck13 - it uses longer timeslices for scheduling compared to the stock kernel and shortens the timeslices based on the number of running or uninterruptible tasks _before_ the next timeslice is given out. Instead of timeslices being fixed (10 milliseconds for nice 20 and 300 milliseconds for nice -20 in O(1)) it now varies. While it's only 1 in thousands of variables in the kernel it serves as a very simple example of it working. The effect is only measurable in absolute latency in milliseconds rather than by contest - this fact alone shows how rapid the regulation is/can be.
[ reply to this comment ]

Subject: I agree...
Date:Sunday, 11/10/2002 - 11:59
As I said, even if we make a "central" autoregulator that controls quite some kernel variables, I'm ready to bet that the gain/cost ratio in performance and resources will be significantly above 1.

Actually I will be compiling tonight the WOLK-3.7.1 release that includes the timeslice regulator from your -ck13 patchset and, YES, I am going to give this kernel as hard times as it gets. I also invite other readers here to give it a run: what proves a good design and gives hints for further development is the Real Life environment.

PS: I was quite surprised to find a "proposal" coded and ready to test in such sort period of time. Way to go, Con!
[ reply to this comment ]

Subject: Good job Con!
Date:Wednesday, 10/16/2002 - 23:36
I will be testing this week/weekend with a ck9 patch on my box. Just happy to see all your hard work is getting the recognizion (never won a spelling bee) it deserves.

Your buddy, Rick
[ reply to this comment ]

Subject: 2.5
Date:Thursday, 10/17/2002 - 03:20
For those interested, I've tried a few 2.5 kernels on my system recently (39, 40, 41) and I have noticed pretty big differences in performance on my desktop system.

Kde3 feels a lot more responsive for sure.

I don't have VM issues with a gig of ram so I can't give anyone my verdict on 2.5's vm stuff.
[ reply to this comment ]

Subject: excellent interview
Date:Thursday, 10/17/2002 - 18:58
Thanks Con. Great article, a non-programmer doctor in Australia gets involved with open source community, creates a useful benchmark tool that will help produce a better kernel for all machines, especially typical low end user machines.
This is an excellent example which explains the rapid development and improvement of Linux.
I'm now inspired to start following LKML more closely and look at some of the code. Perhaps this non-programmer will be able to contribute in some way as well.
[ reply to this comment ]

Linux Daily News Network

Log in


Remember me

» Register
» New password

- Submit story
- Contact Us
- Features (Oct 29, 06:08 AM)
- FreeBSD (Oct 18, 08:36 AM)
- Hurd (Nov 09, 06:08 AM)
- Linux (Nov 16, 05:18 AM)
- NetBSD (Oct 24, 05:56 AM)
- OpenBSD (Nov 01, 05:48 PM)
- Tools (Oct 16, 04:38 AM)

Who's online
There are currently 5 users and 342 guests online.

Top stories
Today's top stories:
- Linux: Andrew Morton on 2.5's performance improvements
- Linux: gconf - a gtk kernel configurator
- Feature: New Kernel Configuration System
- Interview: Andrew Morton
- Linux: 2.5.47 released
- Linux: qconf Screenshots
- Linux: 2.6 vs. 3.0; What's In A Name?
- Hurd: Release of GNU/Hurd 1.0 Delayed
- Linux: 2.5.45 released

All time top stories:
- Linux: Native POSIX Threading Library (NPTL)
- Linux: qconf Screenshots
- Linux: Andrew Morton on 2.5's performance improvements
- Linux: 2.6 vs. 3.0; What's In A Name?
- Linux: What's Left To Merge By 2.6
- Linux: Panicking In Morse Code
- Feature: New Kernel Configuration System
- Linux: The Future Of The 2.0 Kernel

home | user blogs | submit | FAQ | search | top stories | user account

Except where otherwise stated original content is (c)2002 KernelTrap.
Trademarks are the property of their respective owners. Comments are the property and responsibility of their authors.
KernelTrap is powered by Drupal.