Older blog entries for DV (starting at number 178)

Graphical boot

Spent most of last week doing some cleaning and fixes on rhgb the Red Hat/Fedora graphical boot, it's kind of crazy but fun, starting an X server while you just have / mounted read-only, I think I learnt quite a bit in the process.

Gamin

I also made a new release of gamin today, seems the inotify in-kernel replacement for dnotify starts to get some momentum, this is cool. At the moment gamin is really Linux specific, but if you want to make it work on Solaris (or something else) and know how to do this, I take patches, see #153105.

Grinding the Gnome

Okay maybe I should have done this before the 2.8 release but I didn't really had time. I just valgrinded a Gnome session, I had to stop once the gnome session got ready the VM exploded (I used the full --tool=memcheck a lighter tool would allow to do further checking). The result aren't too bad, it is a bit difficult to separate what might be false positive though, it would be great if valgrind could use the -debuginfo packages to provide better logs too.

The Seven Samurais

I got the DVD of Kurosawa masterpiece. In general I have zero interest for war movies like Hollywood tend to produce, but fifty years later even if in black and white Kurosawa's film is still an impressive piece, and the final message about the decline and fate of warlords in a world dominated by the lower casts is still worth thinking about. It used to be the sign of a society rising to better standard that the lower class were getting wealthy, if you don't see this trend, your society is on the decline.

Crazy week

A really busy week, culminating Wednesday with a release of gamin, the deadline for Fedora Core Test2 freeze, my birthday, getting my new laptop and cooking for 10 persons in the evening.

Releases

Killed the last know obvious gamin bugs and made the 0.0.9 release so that I don't get thousand of bug reports when people start installing FC3 test2 where gamin deprecates the original fam implementation. I also pushed a release of libxml2 and libxslt fixing a few bugs in the previous releases. Kasimier is busy cleaning up the Schemas stuff against the W3C (NIST/Sun/Microsoft) based testsuite. Apparently the whole set of tests are not totally clean but apparently there is an update from W3C coming "soon".

Birthay

Sep 1st was really crazy. At some point I was cooking, building the gamin release and following the W3C XML Core teleconf at the same time. Luckily I didn't burnt any food and everything went well. Main dishes were a fish curry and ducks roasted with peaches, mangoes and a bit of honey. Lot of punch and wine disapeared too. I got a book, a DVD and an orchid... The orchid is very nice apparently a breeding of Brassia and Miltonia. I'm quite fond of orchids but so far I have resisted growing some because of the work needed, I cleaned up a small green house left by my mother and put the orchid and a few other plants in it, it's now sitting in my office, it's nice maybe I will start collecting orchids again.

Laptop

I also finished installing the Dell P800 laptop, a bit heavy, doesn't feel very solid, but the 1920x1200 screen is amazing, looking at pictures on it is baffling. I got most everything working, except the wireless card as the centrino 2200 chipset is not yet really ready, so I'm just using my old wireless card which works out of the box. Also there is no manual brightness control, so how to you lower brightness to preserve batteries ? I could not find any key to do it, and the printed doc is ... useless. A nice box overall !

Rich and HTML validation

Try HTML Tidy the original, and if you have a linux or Solaris box around "xmllint --noout --html http://.....html", if it is XHTML then "xmllint --noout --valid ...." instead. Note that xmllint does not do SGML pure validation so the first case is not complete. You should generate XHTML1 anyway if you're concerned about validity really.

Gamin

Released 0.0.7 yesterday, this fixes monitoring of files initially missing. It also enable both kernel (dnotify) and polling monitoring of resource. It serves 2 purpose at the moment, one is checking when monitored resources do not exist (dnotify forces to open[1]() the file) the second one is to minimize the monitoring costs on bysy resources, in that case monitoring is switched to polling untill the resource comes back to a quiet status. An example of this is that watching an active directory (say you're running make in a source tree you're watching) should not eat all the CPU anymore due to the constant flow of kernel dnotify events, FAM events, and nautilus redisplay.

[1] why on earth is advogato post replacing open followed by () with concat () in the articles, it did hit me once and I though I made the mistake myself but I just experienced that again, the engine replaced the string.

libxml2 and libxslt

I made new releases last week-end, libxml2-2.6.12 looks fine, fixes a lot of bugs and Kasimier really improved the W3X XML Schemas support. On the other hand libxslt-1.1.9 has a C file with a character \0 in it in a comment, gcc doesn't see a problem there, but Solaris compiler really dislike it, the patch is in CVS but I may roll a new release to get this fixed. Otherwise seems the integration in Safari is progressing nicely.

blogging

There is a lot of thing I realize I wanted to talk about but I just can't remember what it was. My attention span is getting short (old age effect ? ;) and opening the Advogato page to post is not convenient. As a result, the more busy I am the less I blog, while it should really be the way around. I also realize I miss blogging in French, there are things I don't feel can be appropriately carried in english. But I don't want to drop advogato either, as a result I don't blog much, maybe it's a good thing actually.

Gamin

With the help of John McCutchan, the code base is being cleanup up seriously. There is still a big problem to work around involving polling when dnotify doesn't work (temporary storage, NFS or non-existing resources) and a mysterious bug.

libxml2 and libxslt

So Apple is now using libxml2 for XML parsing in Safari and working toward implementing XSLT client side with libxslt. Rather than trying to work out the transformation in a KHtml DOM they plan to use libxml2/libxslt, reserialize and reparse, it will be interesting to see the performances of such an approach, after all a from a user point of view this won't be slower than XSLT (or PHP or both) on the server (except when caching). A simpler DOM not intended to be a base for rendering is likely to prove more effective even if it means a reparsing. Well at least this will provide some feedback for libxslt on the browser. Maybe Mozilla will be able to drop Transformix after all ;-P !

While I'm focusing on gamin, William Brack has been working continuously on cleaning up the libxml2 bugs which accumulated lately. Kasimier Buchcik also provided lot of improvement to XML Schemas support lately. I shall do a new release within a few week to bring those to the masses.

64 bits

I finally ordered a 64 bit CPU, it just arrived with a motherboard, I have been toying with the idea of getting a 64bit boxes for testing for ages (for examples the old Multia Alpha), but never made the investment. Found a cheapo mobo for an Athlon64 3000+, which makes an affordable deal, I will finally be able to try Fedora Core x86_64 and run regression tests there easilly.

One of the thing I wanted to talk to AMD representative at OLS but never managed to do: get valgrind on the x86_64. If they want to make it the developper platform of choice then that's one of the key tools we want. It can be done in 2 ways:

• valgrinding 64bits apps: i.e. the common usage.
• valgrinding 32bits apps: use the extra set of registers, dedicate them to the 32bits CPU emulation, the main problem of valgrind is the execution cost, and such an approach could seriously improve the emulation speed.

This might be a bit more expensive than cloning 500 CDRoms to give away to developpers, but far more effective !

Back from OLS

Finally back from Ottawa, so exhausted by travel I slept 12 hours in a row last night. Good conference overall, it's a bit scary to listen about Linux powering 512 CPU single image systems as well as digital camera, and see support for hot plug CPU add and removal being added while on the other hand the embedded system guys trick it to boot in less than a second. The system is becoming a completely all-purpose stable OS, and even though there is some need for more specialized features (Desktop/inotify, virtualization, QoS, etc...) they are now mostly small incremental changes that are expected to be pushed directly on the main branch. This reminds me on the way I handle libxml2 and libxslt, no more API changes, everything goes to head, no backporting, and people get fixes on latest releases. This really minimize the updates/fix deliveries delay, maximixing the bandwidth between producers of the features/fixes and the consumers, to me this is mostly a sign of having reached a high stability level, next step is to just burn it in silicon and forget about it.

Laptop

The laptop is now declared officially dead, I should hopefully get a replacement soon, Alan apparently want the corpse to increase his collection of dead hardware :-)

Linsys

Following the wireless experiment done at LesSuites hotel for OLS, I bought the Linksys WRT54GS we had in our room, it's part of a crazy plan to not rely on a normal PC for my network infrastructure and provide wireless in the other side of the appartment. I will be playing with it and OpenWRT once I have a bit of time.

Gamin

Among the first thing done today while processing mail was integration of a couple of patches for gamin, one fixing FAM disconnection API and the other one providing support for the inotify experimental kernel API replacing dnotify. I hope this goes in the kernel, dnotify signal based interface plus the requirement to open the monitored file is really bad and inotify should fix a number of problems this raises.

Fedora

We also had an interesting BOF at OLS, good to meet the people though lot of them were Red Hat employees. Checking the agent logs of Speakeasy new rpmfind.net box, it's impressive to see that nearly half of the requests for the last 6 days seems generated by Yum, the fedora update agent (Python-urllib/1.15 would be yum or up2date I think). There is a lot of metadata work to be done around linux distros and packaging, for distros, mirror, repositories, packages and projects. With a bit of luck all this is gonna fit together and provide a standard and stable set of data upon which new set of maintenance tools can be build, with better integration, better UI, faster updates while still maintaining a secure framework.

IBM laptop

Seems I managed to extract one hour of use without freeze from the damn thing by removing the hard disk and smashing the damn thing on the floor repeatedly out of frustration. When sadism and computing meets ...

OLS and networking

The Linksys box we installed in the suite works pretty well. It's serving a lot of users in the building. This afternoon gb tried desperately to get other boxes to relay the servive for rooms without an ADSL connection. They are having much fun, but of course this isn't working :-) . Ssh'ed in the box, it's pretty cool to see a minimal Linux-2.4 system running. One of the speeches I went to today was about speeding up the boot process, but while I usually don't care, the constant reboots of the laptop have made me realize that it is really slow to boot. Even with a lot of services disabled, it's still horribly slow. Some of the recipes used by the embedded guys should be usable as-is for normal desktop systems, even if we don't reboot the Linux boxes often in normal conditions, speeding up that process would be part of the polishing we should put up for desktop offerings IMHO.

OLS and Ottawa

In Ottawa, currently at the Desktop meeting that Chris blogged about already. Not very technical so far, but nonetheless interesting, including a couple of nice demos.

F....g laptop

My (or more precisely Red Hat's) laptop decided to start dying yesterday. The damn thing reboots randomly and usually I can't use it for more than 20mn before it freezes. It's really old (IBM 600 E), so I can't really blame the solidity of the beast. Anyway this mean I will be mostly out of reach during this week at the conference, even though it's usually well connected. The damn thing rebooted twice during while writing this entry :-(

gamin

Some cleanup, some fixes, removed threading in the server, and made the client code restartable (i.e. if the server vanishes, the client has enough information to reconnect to the previous state), which seems to work with the gnome-panel and nautilus. I think it's time to test it seriously, so it will show up in Fedora Core 3 (and Rawhide) future versions in the meantime people interested can find it on gnome.org.

Off to OLS

and to the Dektop meeting in Ottawa. I'm all for reducing travels but OLS is a must IMHO. I glanced though the program, looks extremely good as usual, and I'm really looking forward meeting with friends. Time to pack for a week or so ...

World domination

PHP5 using libxml2/libxslt for XML processing is now released, MacOS X will now ship with libxslt (will Safari ever use it for XSLT support ? That would be fun !). Of course the transition to PHP5 will take time but it seems a very large fraction of Apache deployments use PHP. It's good to see the code used in a large scale.

Rpmfind(s)

The saga of restoring rpmfind services goes on, of course a couple of hours after I wrote about rpmfind.net being back in shape one of the drives died, apparently it makes a cyclic click audible though a closed door :-\ , it was hosting the packages pages of course, Murthy in action. I will swap a new drive in probably today. In the meantime the speakeasy people contacted me, they built a new box, first surprize is that it's an Opteron box :-) , they initially installed the x86 RHEL version which was working surizingly well, but they reinstalled with the AMD64 version, I don't know for desktop work but for servers x86_64 rocks. Rebuilding the services takes a while still, even at 2MBytes/s it takes an awful long time to resynch hundreds of gigabytes of mirrored stuff.

gamin

Struggling quite a bit with some bizarre behaviour I want to fix before pushing it as the replacement for fam in Fedora Core. At least it builds, but debugging threaded stuff is a pain, I absolutely need to remove the threading. I also think I will not enable Suspend/Resume operations that FAM provides, they mean the server has to stack operations if the client asks for them and this really doesn't sound like a proper client/server model for operations. Beside that I doubt it's used anywhere on the desktop, well pushing it for test2 of Fedora Core 3 should help find if it's too strict or not.

GCC aliasing and runtime profiling

Seems other people are interested at shaving their code size by 5-10% following the work done with Arjan on gcc aliasing for library symbols and using gcc runtime profiling. fcrozat also activated it on the Cooker build and gained 7% on size apparently. Seems gtk and some parts of OpenOffice are also potential candidates for those optimizations.

libxml2 and libxslt

Despite the number of changes made in the last release they seems stable, ncm might a bit optimistic (or ironic ;-) to call them "stellar". William and Kasimier chased down a problem in Schemas debugging showing up only in 64bits. William had the machine with 64bits and Kasimier had the knowledge of the code. This resulted in a long IRC session of cut and past gdb and discussion. I think a GDB <-> IRC connector would rock, just so often people manage to get into a problem, but reproducing it outside of their environment is nearly impossible (code/data/platform issues are not always shareable), If we had a standalone python (or perl :-\) script which could create a gdb (or other DB) session and gate it as a user on a public IRC channel, there is tons of obscure bugs that could be debugged in a matter of minutes instead of standing unreproduced in a bugzilla entry. Googling a bit for this I could not find any such tool, it should not be too hard too do, but IMHO would be extremely valuable.

Releases

Last releases for libxml2 and libxslt were 6 weeks ago, so 2.6.11 and 1.1.8 are out, they include a lot of bug fixes especially for libxml2. Also there is now a person working on fixing XML Schemas (Kasimier Buchcik) who apparently can make his way though the spec and my code, he started doing regression testing against W3C testsuite. Interestingly he was so far a Windows only user/developper, but as he got involved he's getting into Linux.

GCC improvements

With Arjan van der Ven we played with various options to reduce code size and local library calls costs, we managed to get 5-10% improvement using gcc aliasing for internal libraries call (see elfgcchack.h for the scary but XSLT generated header), the RPM on fedora also uses gcc runtime profiling to try to optimize the resulting code, but this requires gcc-3.4 or later.

But anyway the biggest improvement seems to simply use gcc-3.4, code generation seems way better:

   text    data     bss     dec     hex filename
 937377   32352   35124 1004853   f5535 /usr/lib/libxml2.so.2.6.11
 881072   31608   35828  948508   e791c /usr/lib/libxml2.so.2.6.11

the first one is on RHEL AS 3 with gcc version 3.2.3, the second is on Fedora Core 2 with gcc version 3.4.0, nice work from the GCC hackers, probably the SSA improvements.

Update: I was told 3.4 doesn't have SSA, so expect even better code for 3.5 :-)