jd:/dev/blog: 2010

I finally took some time to finish my color contrast corrector.

It's now able to compare two colors and to tell if they are readable when used as foreground and background color for text rendering. If they are too close, the code corrects both colors so to they'll become distant enough to be readable.

To do that, it uses color coordinates in the CIE L*a*b* colorspace. This allows to determine the luminance difference between 2 colors very easily by comparing the L component of the coordinates. The default threshold used to determine readability based on luminance difference is 40 (on 100), which seems to give pretty good results so far.

Then it uses the CIE Delta E 2000 formula to obtain the distance between colors. A distance of 6 is considered to be enough for the colors to be distinctive in our case, but that can be adjusted anyway. That depends on reader's eyes.

If both the color and luminance distances are big enough, the color pair is considered readable when used upon each other.

If these criteria are not satisfied, the code simply tries to correct the color by adjusting the L (luminance) component of the colors so their difference is 40. Optionally, the background color can be fixed so only the foreground color would be adjusted; this is especially handy when the color background is not provided by any external style, but it the screen one (like the Emacs frame background in my case).

Here is an example result generated over 10 pairs of random colors. Left colors are randomly generated, and right colors are the corrected one.

bg: DarkSeaGreen4 fg: gray67 ->                             bg: #4a6b4b fg: #cccccc
bg: SlateGray4 fg: forest green ->                          bg: #9faec0 fg: #005700
bg: grey13 fg: grey36 ->                                    bg: #131313 fg: #6c6c6c
bg: MediumPurple2 fg: honeydew ->                           bg: #9e78ed fg: #f0fff0
bg: grey43 fg: chartreuse3 ->                               bg: #5e5e5e fg: #79de25
bg: linen fg: DeepPink2 ->                                  bg: linen fg: DeepPink2
bg: CadetBlue4 fg: blue1 ->                                 bg: #6c9fa4 fg: #0000e1
bg: gray33 fg: NavajoWhite3 ->                              bg: #525252 fg: #cfb58c
bg: chartreuse1 fg: RosyBrown3 ->                           bg: #9cff38 fg: #b28282
bg: medium violet red fg: DeepPink1 ->                      bg: #9c0060 fg: #ff55b9

All this has been written in Emacs Lisp. The code is now available in Gnus (and therefore in Emacs 24) in the packages color-lab and shr-color.

A future work would be to add support for colour blindness.

As a side note, several people pointed me at the WCAG formulas to determine luminance and contrast ratio. These are probably good criteria to choose your color when designing a user interface. However, they are not enough to determine if displayed color will be readable. This means you can use them if you are a designer, but IMHO they are pretty weak for detecting and correcting colors you did not choose.

Last week, I spent some time implementing various color manipulation routines. The ultimate goal was to find a way to determine if a text in a certain color was readable on a background with a different color.

Something I failed to do so far, despite my research in the area.

However, since I think my code could be useful for other people, I've set up a tiny git repository with the routines I wrote.

The funniest one to implement was CIEDE2000. I verified my code with the data given in the specifications and can assure it's correct. :-)

Org has a nice option which allows you to show week-end days in a different color in your agenda. That means that Saturday and Sunday (when I do not work) are fontified with `org-agenda-date-weekend'.

But there are other days I do not work, like my vacations or holidays. Therefore, I've wrote a patch to add `org-agenda-day-face-function' which is optionally called to determine what should be the face used to fontify a day. This allows me to use the same face for holidays and for week-end days, like for last Thursday which was an holiday in France.

That patch has been merged in Org last week.

Last week, I worked on my Google Maps for Emacs extension. I've introduced a new format handling for locations which include the longitude and latitude. The initial format was just a string describing the location, which was obviously too limited.

It now prints coordinates of the different elements when the mouse is over the map, with other information.

It also center the map on M-x google-maps and set a default zoom level. This is something which was not set because it's not a good idea to set center coordinates in order to see all points on the map automatically. But you can still remove the centering by pressing "C". On the other hand, setting it automatically allows to move the map easily, and I think that what most users want to do.

I've also added a "place my home on the map" feature, accessible by pressing "h" on any map. That adds a marker according to the location set in Emacs using the calendar- variables.

This feature is also available under Org by pressing C-u C-c M-l, which shows the location of your appointment with your home on the map too.

Finally, you also get caching so it does not request images you already seen, which makes the moving nicer and faster to use, and prompt history.

My latest patch for Org mode has been accepted by Carsten today. It adds support for custom category icons in all views, like agenda or todo.

You just need to configure org-agenda-category-icon-alist and it will work out of the box.

Last week, I wrote a couple of patches to add support for transparency when Emacs is displaying GIF images. Until now, it was displaying the color used to define transparency in the file data. Now it displays the image correctly by using the frame color as the transparency color, like it's done for other image formats.

The patches have not been merged yet, but will probably be soon.

We all know the good old Emacs mode-line you got under every window. Since the beginning (a long time ago), it starts and ends with dashes. I've proposed a patch to remove them.

Before:

After:

This has been merged in Emacs 24. You won't see any more ugly dashes in graphical mode.

Recent Emacs versions have a wonderful capacity to hide the mouse pointer as soon as you type and insert characters in a buffer. This is controlled by the `make-pointer-invisible' variable, which is set to t by default.

However that does not hide the pointer when simply moving the cursor on screen. Therefore, I've started to use `mouse-avoidance-mode', which make the mouse pointer jump if your cursor hits it.

Unfortunately, if your cursor hits the invisible mouse pointer, `mouse-avoidance-mode' makes it jump too, because it does not know it is invisible.

Well, it did not know. Now it does, thanks to my patches which have been merged in Emacs 24. Using the new function `frame-pointer-invisible-p', one can know if the mouse pointer has been hidden by Emacs. Therefore I enhanced `mouse-avoidance-mode' to use it, and everything is alright now. :-)

I've recently got a mail from one of my faithful reader, asking why not considering notmuch.

Actually, I think notmuch already exists in a better way, and that's called IMAP and Sieve.

What notmuch does, is tagging your mail with tags (obviously), based on filtering rules you write. The big downside is that you have to tag all your mails on your computer.

And if you use several computers, you'll have to tag several times your mails. And you'll have to find a way to maintain your rules to be identical on all your computers. That does not scale.

Using Sieve for mail filtering, one can do already that actually, and much more.

A notmuch rule like:

notmuch tag +intel from:intel.com and not tag:intel

Can be written as a Sieve rule like:

if address :all :contains "From" "intel.com" {
        addflag "intel";
}

The flags extension for Sieve is explained in RFC5232.

The Sieve based solution has the advantage of being treated server side, and therefore not subject to multiple or different MUA usages. It's also fast, if you use a good IMAP server like Dovecot, which has indexing, etc.

Furthermore, Sieve can obviously do a lot more than tagging, like splitting into different mailboxes, filtering with regexp usage, vacation, etc.

And if you want to fetch your mail locally, you can synchronize the IMAP box entirely with any software able to (like OfflineIMAP).

Now, what's probably missing, is a correct support for IMAP flags on various MUA around. But that's not something notmuch helps to solve either. :-)

This last couple of days I've been dedicated making Gnus… fresher.

I've decided to give a whirl on Gravatar support. I already tried the gravatar.el lying on the Interweb, but well, it was crap: it used wget to fetch pictures, therefore was totally synchronous. Reading each mail was slower. The cache did not even have TTL, as far as I recall.

So, I've now wrote gravatar.el implementing the Gravatar API. Asynchronously of course. With cache, TTL, etc. Perfect. :-)

Then I've composed gnus-gravatar.el, implementing a washing function adding Gravatar for From field and/or Cc/To fields, like done for picons.

As I was expecting, the patch was badly received by the GNU guys, which start talking about thinks like external resources, privacy, non-free software, etc. Boring.

Fortunately, Lars allowed me to push the patch in git so everybody can give it a try. I'm now waiting for feedbacks in order to know if I will have to maintain this patch outside Gnus, or not.

Here's the mandatory screen-shot.

As I already wrote too many times, I've started to use Gnus 6 months ago, and never looked back.

At that time, I joined the ding mailing list in order to ask some dumb questions and, once, send a patch. There were very low activity on that list.

Until Lars, the original Gnus author, came back.

Three weeks ago, he started to wrote a new wash function to render HTML mails properly, with pictures. It's named `gnus-html', and is (for now) based on w3m (but not on emacs-w3m, which is not part of Emacs).

Last week, I've sent a set of patches to replace the usage of curl by the standard `url-retrieve' function to fetch images, plus various enhancement. It seems that my work was good enough that Lars offered me write access to the git repository. I can therefore mess up the Gnus entirely. Hurrah!

I've continued to work on `gnus-html' and recently merged a set of patches improving image retrieval (which is now done in parallel) and starting to use `url-cache' to cache image for a defined period of time. Of course, I found a bunch of tiny bug and special case while reading RSS feeds and various HTML mails, and fixed them all along.

Lars added a libxml binding for Emacs 24, providing the `html-parse-string' function. His future plan seems to be the abandon of w3m in favor of a native parsing via libxml to render HTML, and therefore, HTML mails.

I should also mention the new `nnimap' back-end; Gnus has been designed to read NNTP newsgroups, and not mails. Consequently, it had a very poor behaviour when used with a back-end such has IMAP.

Lars took a week to rewrite entirely our dear `nnimap' back-end, and make it act in a more expected way. There's still a bunch of bug and code to write, but it is at least usable and seems faster than the old code.

Last thing I did was to rewrite the icon support in the group buffer. When I started to use Gnus, I was curious and tried to configure this. I never managed to make it work, and now know and understand why it was broken. So I ended rewriting entirely, and now it works. I never though I would understand, fix, and commit this code when reading the Gnus documentation this winter, but hell yeah, I did.

Now I've still several little project to improve things in all sort of area. We'll see what I'll do next. :-)

Another week, another Emacs extension!

I had (once again) a wonderful idea: what if I could have the weather forecasts in my Org agenda? Wouldn't that be wonderful?

My quest started by looking for a service offering a good weather forecast API. I found nothing simple as the hidden Google Weather API, which is nice, but… not documented. Not at all. Not a single line. Nah.

Then, I wrote a google-weather extension, implementing a basic Emacs Lisp API to retrieve data from the Google service:

ELISP> (google-weather-data->forecast (google-weather-get-data "Paris"))
(((9 8 2010)
  (low "53")
  (high "63")
  (icon "http://www.google.com/ig/images/weather/rain.gif")
  (condition "Rain"))
 ((9 9 2010)
  (low "53")
  (high "69")
  (icon "http://www.google.com/ig/images/weather/chance_of_rain.gif")
  (condition "Scattered Showers"))
 ((9 10 2010)
  (low "54")
  (high "72")
  (icon "http://www.google.com/ig/images/weather/partly_cloudy.gif")
  (condition "Partly Cloudy"))
 ((9 11 2010)
  (low "55")
  (high "75")
  (icon "http://www.google.com/ig/images/weather/partly_cloudy.gif")
  (condition "Partly Cloudy")))

My API even implements data caching, which is nice to speed up the agenda display.

By the way, I think my next job will be to hack on the url-cache feature of Emacs, which is utterly buggy and has probably never be used. But that's another topic.

Finally, I just had to write another module on top of that to export the forecasts to Org. A screen shot is probably better than a long and boring explanation, so here's the result.

My only regret is that the icons provided by Google are ugly squares, so I did not want to use them. On the other hand, I did not found any icon set that would have all the icons Google provides (around 20). So I felt back on the icon naming specification to map the Google images to standard images. Any better idea would be welcome, of course.

All the information can be found on the Google Weather for Emacs extension homepage.

I recently decided to use OfflineIMAP to synchronize my mails on my laptop. It's a great piece of software, and allows me to read my mail while I'm offline.

I use it with Gnus, of course. But I lacked a proper way to integrate OfflineIMAP with it, so I decided to write a little Emacs extension to run and monitor OfflineIMAP directly from Emacs.

Here comes offlineimap.el, an Emacs extension to run OfflineIMAP directly within Emacs. It will display OfflineIMAP output in a buffer, and optionally shows the current OfflineIMAP operation in the mode line.

By default the status is in the mode line only if you are in the Gnus group buffer. But that's customizable, of course, since this is Emacs!

If you are using el-get, there's already a recipe to install it!

Today's fun idea was to put all my contacts stored into BBDB on a Google Maps' map, using my Google Maps extension for Emacs.

With the help of a few lines of Lisp glue:

(google-maps-static-show
 :markers
 (mapcar
  (lambda (address-entry)
    `((,(concat
         (mapconcat
          'identity
          (elt address-entry 1) ", ") ", "
          (elt address-entry 2) ", "
          (elt address-entry 3) ", "
          (elt address-entry 4) ", "
          (elt address-entry 5)))))
  (mapcan
   (lambda (record)
     ;; We need to copy the returned list, because mapcan will modify it later
     (copy-list (bbdb-record-addresses record)))
   (bbdb-records))))

we can make that:

It's really simplistic, but I did not need more to have fun. :-) This could be extended to set a specific marker and/or color for each contact, with a legend. I'll let that as an exercise for my readers.

The Debian package of Emacs Muse was maintained by Michael W. Wolson, who is also the upstream author of that software.

He announced months ago that Muse needed a new upstream maintainer. That's not something I'm willing to do, since I really think Muse has been superseded by Org mode nowadays.

However, I'm still using Muse to maintain this blog with my muse-blog extension, since Org still lacks some infrastructure to maintain and publish a blog.

Therefore, I adopted the Emacs Muse Debien package and updated it to the latest version!

rainbow-mode had a big success and good feedbacks when I released it for the first time a couple of months ago.

Several users asked to me request its inclusion into Emacs. Therefore, some days ago, I proposed to merge it inside Emacs trunk. My request has been denied, but the mode has been added to the Emacs 24 package repository.

In the mean time, I've added support for hsl() and hsla() support, and added CSS 3/SVG color names.

Even if I recently stated I lost some of my faith in XCB, I still sometimes hack things to add support for it.

These last days, I've worked on a D-Bus port from Xlib to XCB. The port was quite straight forward, since there's only a little piece of D-Bus using X, which is dbus-launch.

I though D-Bus was a good candidate, since it's part of the Freedesktop initiative. Therefore, I was expecting a warm welcome and some enthusiasm from a fellow project.

My contribution got one useful review, and a cold reply from Thiago Macieira (a KDE/Qt/Nokia developer):

No, sorry, I don't agree.. I've just checked and my Solaris machine doesn't have XCB. Please do not remove the X11 code. You may add the XCB code, but you cannot remove the X11 code.

This is not really the kind of answer I expected, actually. I then reworked the code to please Thiago, and added some #ifdef to add XCB support to D-Bus, with a fallback to libx11 where XCB would not be available.

Havoc Pennington replied:

Given that libX11 now uses xcb as backend, I don't understand the value of porting to use libxcb directly when there isn't an issue of round trips or other stuff. It will just make #ifdef hell, while the X11 API is an API that works on both xcb and non-xcb platforms. Maybe people should be thinking about porting xcb to non-Linux platforms? The X protocol should be the same on other UNiX, so xcb in theory ought to work fine if you just compiled it on Solaris/BSD, same as GTK or dbus or Qt would work fine.

The last part "Maybe people should be thinking about porting xcb to non-Linux platforms?" is still unclear to me, even though I asked Havoc to explain what he meant.

Finally, Thiago refused to merge the patch:

[…] thanks for the patch, but like Havoc I am unsure of the value. We can't drop the X11 codepaths now because too many systems exist without XCB. Adding the XCB codepaths only made it more complex, even though you did a good job.

I can't disagree with that conclusion: using both XCB and X11 make the code unreadable for little gain. That's why I did replace libx11 by XCB directly in the first version of the patch. On the other hand, D-Bus people does not seems to really care about making their software evolve in the right direction, even if that requires users to upgrade their systems.

I think D-Bus using and depending on XCB would have been a good point to push adoption of XCB. Unfortunately, it seems you can't even rely of projects of the same initiative (i.e. Freedesktop) to work together to make things a little bit better.

After 5 years of existence, XCB is still not so obvious to people, and making it adopt is going to be a challenge for the next years. The upside is that new X.org 7.6 will bring XCB with it, as part of the katamari.

Since I have started to use Org-mode, I though it was missing something to have appointment locations on a map. Of course, it's easy to get a LOCATION property from an entry, and then browse-url on Google Maps.

But it is too easy for me, so once again I said: challenge accepted! I will bring Google Maps into Emacs!

After several hours of work, the google-maps-el project shows a map!

It fully implements the Google Static Maps API and the Google Maps Geocoding API.

You can type M-x google-maps and type some place to see it marked on map. Of course you can do much more, as seen in the screen shot above.

I've also completed all of this with a small org-location-google-maps which simply show a Google Maps' map for the location of an event in Org mode by pressing C-c M-l in an Org buffer or in the Org agenda.

While customizing Emacs this last weeks, I had the need to customize also the color theme.

Color themes are always a pain in the ass to edit, because you're supposed to read color strings like #aabbcc and guess what colors they represent.

This is why I wrote rainbow-mode, a minor mode for Emacs that will highlight strings that represents color, using the color they represent.

This support hexadecimal syntax, HTML color name, X color names and rgb() CSS syntax.

This last weeks, I've worked on implementing the Desktop Notification Specification into Emacs.

It allows sending desktop notification in a very simple way.

(notifications-notify
    :title "You've got mail!"
    :body "There's 34 mails unread"
    :app-icon "~/.emacs.d/icons/mail.png"
    :urgency 'low)

It supports the protocol signals (NotificationClosed and ActionInvoked) and the two main methods (Notify and CloseNotification).

The methods specification are implemented entirely (hints, replaces, actions, icon, etc).

The signals are supported via callbacks function provided on the notification creation.

It have been merged into Emacs trunk today.

2010-06-09  Julien Danjou  <julien@danjou.info>

        * net/notifications.el: New file.

This also allowed me to discover, raise and fix a bug in the D-Bus binding of Emacs, which will be probably fixed in trunk soon.

A couple of months ago, I've started using ERC to hang out on IRC. I've read all the pages on EmacsWiki about it, just to see how far I could customize it.

I must admit that I was not disappointed, even if I expected to be. It's quite a nice software, and once well configured it's more convenient that my old irssi setup.

While browsing EmacsWiki, I read an interesting idea about channel scoring/temperature on the erc-track page. The idea is to see if it's worth jumping to an IRC channel to see what people are talking about.

Challenge accepted!

I sat up and started to dig though ERC source code to find the information I needed about variables and functions.

I finally did write something nice, which I called erc-track-score. And yet another piece of software I wrote for my lovely Emacs!

How does it work? Ha-ha, I was sure you would ask. You're so predictable, dude! Read the following, and you'll know everything you ever wanted to know about it since the moment you read the title of that blog entry.

Which probably turned you on.

Nasty you.

First of all, the score of a channel starts at zero. Zero means "seriously, don't bother, nothing is happening here".

Upon each new message arrival, the score is incremented by 1. If a new message contains a keyword, your nickname or is sent by a pal, the score is increased by configurable values, by default between 2 and 20 points, depending on the match type. On the other hand, when a message is send by some fool, the score is decreased by 1 by default.

Obviously, if the score is going negative, you really should not jump to the channel.

Finally, the score is permanently and slowly brought back to 0. By default, the score is decreased by 1 point every 10 seconds.

Overall, reading the score should gives you a good idea of the channel temperature.

I'm still not sure what is the best formula to compute the score, but so far the default values seem quite good. We'll see.

Two years ago, while working on awesome, I joined the Freedesktop initiative to work on XCB. I had to learn the arcane of the X11 protocol and all the mysterious and old world that goes with it.

Now that I've swum all this months in this mud, I just feel like I need to share my thoughts about what become a mess over the decades.

When I was unborn…

…the Toto band were releasing their song "Africa" and some smart guys were working on a windowing system: the X Window System (this is its full name) which therefore has a (too) long history. The latest version of its protocol, the 11th one, has been designed in the 80's. You can learn more about the history in the Wikipedia article about X.

In 2010, we still listen disco music and we still use various protocols designed in the 80's and even before X. Music have evolved, protocols have evolved, and so did X11.

The problem is that X11 did not evolve that well. The guys at MIT-and-some other-places-with-very-smart-people-in-it created X version 1 in 1984, and updated it until X version 11 (the one we're still using) in 1987. Eleven version in 3 years, that was following the "release early, release often" model. But I don't know why, it just stopped to happen for the last 23 years¹.

I don't know what changes have been made in the first 11 major versions of the X protocol, but I'm rather sure we should have deserve a couple of major version updates this last 2 decades.

In my humble opinion, X11 was not designed to live 23 years. But hey, I'm not blaming anyone here: I was 4 years old and playing Lego ® when they released this latest version of the X protocol, so there is little chance I'd have done something better.

1. That's not totally true: they added (and then deprecated) many extensions.

We won't fix. We'll work-around.

That is probably one of the guideline of the X protocol for the last years. And don't misread me: I'm not bashing anyone thereafter.

Since the X11 protocol was aging, the X guys started to add extensions. They added tons of them over the years. This, in application of one of the early principles of X:

It is as important to decide what a system is not as to decide what it is. Do not serve all the world's needs; rather, make the system extensible so that additional needs can be met in an upwardly compatible fashion.

All of them with no exception were added because, bad luck, the X11 protocol did not anticipated the things that happened in the last 23 years, like video, OpenGL, multiple monitors, or the pleasure to draw oval windows. Some of this extensions are still in use, while some of them have been dropped.

While this is not a bad thing to extends the protocol, it seems like a bad thing to try to fix the protocol with for example the XFixes extension, even with all the good intentions Keith Packard might have in his greatness.

Actually it's even worst than you think

The X11 protocol (without extensions) defines about 120 types of requests: create a window, move a window, etc.

Nowadays, there's at least 25 % of them which are useless: usage of server-side font, or the drawing of squares and polygon, are unused by any modern application or toolkit. All of this is superseded by requests from extensions, like the XRender one.

The handling of multiple monitors displays has totally been screwed up. X11 has been designed to work in Zaphod mode (independent monitors). But Xinerama, and nowadays XRandR have replaced it up: recent X servers (released after ~2007) does not support Zaphod mode anymore, even if it's a core piece of the X11 protocol.

Worst: on many requests, there's limitation or design flaws, like described in this document: Why X Is Not Our Ideal Window System by DEC researchers.

We'll add more broken standard on top of that

Following its early principle, X does not define policies but only mechanisms, which seems like a good thing,

Consequently, people started writing specifications to determine a number of stuff and dogmas: ICCCM. That was 22 years ago in 1988. It's useless to add that many things in this specification are now obsolete, useless, or that it misses many modern stuff.

I was not the only one to think that. The people from what will be the major desktop environments, KDE and GNOME, saw that too in the 90's while I was learning to count. So they wrote EWMH, another standard that comes on top of ICCCM and extends it with nifty features like maximization, full screen mode, etc.

The problem is that this standard has also been written by narrow-minded people who at that time, were working on GNOME or KDE (and maybe others). This desktop environments were having and still have some strong concepts of how should work a desktop: "it should have work-spaces", "a window is only on one workspace", "we only see a workspace at a time", "you do not have multiple screens", etc.

Dude, we don't care: we have toolkits!

This vision of how the desktop should work have now been written in marble in all applications and libraries implementing EWMH, like GTK+ or Qt.

Nowadays, everybody forgot about all of this standards. Toolkits have implemented this, circumvented the X11 protocol limitation and flaws, and nobody wants to look back.

Like all standards, obviously some people implemented them badly. This had some side effects, like OpenOffice acting like a pager.

We don't look back? Worst, we forgot where we came from!

With all these layers of bad designed standards, the desktop continued to evolve for more than a decade. They continued to add more standard, the more recent ones being based on D-Bus like the Desktop Notification Specification or the latest Status Notifier Specification developed by KDE.

The Status Notifier is a new implementation of the good old system tray based on XEmbed, using D-Bus instead of the X11 mechanisms, and adding the possibility to show the system tray with something else than icons.

This specification draft saw an important issue and design flaw raised by Wolfgang Draxinger in this thread on the XDG mailing-list. What Wolfgang points out, is that X is network-oriented, and D-Bus is not. Therefore, making the Status Notifier specification to use D-Bus to pass system tray messages around is a bad idea, since running a X application from host A on host B will draw the system tray on the wrong host!

Apparently, reading the thread, this does not fear some of the KDE people:

of course this is a bizarre corner case not worth much thought. at least that's what you'll think until you actually run into it yourself (be it because you are testing something or because you are setting up some weird kiosk environment).

What Oswald describes as a corner case is an actual common use case for many of us. Of course, YMMV.

From my point of view, this is a step back in the wrong direction. But we can conclude that the network part of X is now worthless, to at least KDE.

I used to believe in XCB

When I joined Freedesktop, it was to work on XCB, the X C Binding. XCB is a nice, clean, 21st century technology based API to play with the X11 protocol. Its code is auto generated based on XML file describing the protocol.

In comparison, Xlib is 80's obfuscated code with almost no comments and hard-coded things. Only a few people can understand some of its corner like its i18n or XKB implementations. And all its code is synchronous.

For people not knowing it yet, X is a network protocol where you send request (like a GET in HTTP) and then get a response. Xlib forces the application to wait for the reply to its request, so the application is blocked until the X server sends the reply to the request. XCB on the other hand does not block and allows the application to send a batch of requests, do some other stuff in the mean time, and then gets the replies.

It's like your Web browser would send one request at a time to a Web server, and would wait until you downloaded all the images one by one to display the page.

In cases where X and all its clients are on the same host, the latency is small and not really visible, therefore the gain for XCB to be asynchronous is small. On slow network however, the gain can be huge, as proved in the rewrite of xlsclients with XCB by Peter Harris.

One of the long standing goal of the XCB folks is to kick-out Xlib, to increase speed and hides latency in X11 applications. That requires to port many libraries, because almost none of them (Cairo being an exception) supports XCB.

From where I stand, I don't really see if the work is worth it now. The desktop world is trusted by GNOME and KDE, meaning GTK+ and Qt. It seems none of this toolkits are interested to work on XCB, neither on the X protocol. They probably put hard effort in bypassing X limitation and flaws, and they now sit on top of crap of workarounds and broken-by-design-standard implementation. It seems to me they don't want to go back in the layers and improves things.

They're too high to go back down and they don't see what the gain would be.

Enlightenment with its EFL was the first toolkit to have an XCB back-end with the work of Vincent Torri. Unfortunately, the back-end is not maintained and nobody cares about it. Last time I tried it, it did not compile at all.

X12 ?

There's a page called X12 on the Freedesktop wiki, listing all the things that should be fixed some days. Unfortunately, the list continues to grow up an no one talks about working on X12.

On the other hand, there's a handset of people trying to work when they will have time on XKB2, the second version of the "let's-try-to-fix-up-the keyboard-part-of-the-protocol-we-wrote-23-years-ago-a-second-time" extension.

To me, it does not seem X12 will happen in the next decade neither.

Alternative ?

Do we got alternative to X ? There's Wayland, but it's far from being usable. There's DirectFB, but that's not very portable. None seems candidate to replace X some days to me.

Anyhow, none of the main toolkits around support this alternative. GTK+ once supported DirectFB, but as far as I know, it is not supported nor works nowadays, as stated by Josselin Mouette. This is why recent versions of the Debian installer have migrated to X for the graphic part, thanks to Cyril Brulebois work.

Conclusion

XCB has been around for more than half-a-decade, and very few people showed interested in it. As far as I can see, nobody is interested to use the X protocol and everybody tries to encapsulate it in some higher-level API as soon as possible to stop seeing it. This leads to poorly written application and toolkits, with a lot of ugly hack.

All of that also means that starting to write applications and graphical toolkits based on XCB would be a very interesting project, but that would lead to spend too much time learning to circumvent the X protocol flaws, things that have been done in years by predecessors like Qt and GTK+.

Major toolkits implementations have almost nothing to win in going back in the dark water of X. I guess most of their folks prefer to work on shiny 3D effects based on your GPS location, rather than redefining better basis for everyone.

The manpower available in the X world is very small. Debian lacking of X maintainers is just the summit of that. There is very smart and very competent and skilled guys in the X world, as you can see by simply reading blog posts on Planet Freedesktop for example (me excluded). Unfortunately, there's not enough of them to cover all the things involved in X: input devices, graphics devices, new protocol extension specification and so on. The X server is really late, and it seems most of the developers prefers to work on the server itself than on the protocol behalf. Which is understandable.

I'm curious to see where all of that will lead in the upcoming years. I've been walking in the X world hallways for about 3 years now, and I feel desktop alternatives to KDE and GNOME will all die sooner or later. The time were you could choose between a dozen "modern" window managers has passed away.

After all, maybe that is simply Darwinism applied to computer software.

I'm trying to work on XCB this week. And today I've started to accomplish the second step of a long term goal: making an X11 only library using XCB as its primary interface instead of Xlib.

Last year, I had extended the API of startup-notification to support XCB as a back-end. This had been made possible by factorizing some code, duplicating the X11 code and translating it into equivalent XCB.

Today, I've accomplished the second step, being dropping the Xlib code inside startup-notification to keep only the XCB one.

For this, I used the x11-xcb library, which is available when Xlib is compiled with XCB as its transport, which is nowadays the standard.

This library provides the function XGetXCBConnection, which can convert a Display pointer to a xcb_connection_t pointer. Consequently, it's now possible to write and execute XCB based code and being compatible with Xlib.

I've made some benchmark of my work for the occasion, in order to measure what the gain is.

The first table described 1000 launches of a fake application (a modified version of the startup-notification test suite actually). The X server is local (the latency is very minimal then). The gain is computed between the same back-end type for the total time. Full XCB is the "version" I'm working on.

Version - Back-end	User time (seconds)	Kernel time (seconds)	Total time (seconds)	Gain
0.10 - libx11	3.20	7.42	12.989	-
0.10 - libxcb	2.76	7.36	12.414	-
Full XCB - libx11	2.74	7.50	12.380	4.6 %
Full XCB - libxcb	2.72	7.16	12.037	3.0 %

The user time and kernel time are provided but are not really interesting. XCB does not offers a big gain in CPU execution time, but is more about latency. Anyhow, there's always a gain with XCB.

This second table describe the same test but running only 100 times over a slow network.

Version - Back-end	Total time (seconds)	Gain
0.10 - libx11	76	-
0.10 - libxcb	35	-
Full XCB - libx11	72	5.2 %
Full XCB - libxcb	33	5.7%

The gain is relatively small, about 5 %. But anyhow, there's still a gain. Note that the difference between the execution time of the same test written in XCB and Xlib is just huge. I've tried to optimize the Xlib test, but I did not manage to win more seconds.

In conclusion, considering that startup-notification is only used when an application launches another application, the perceivable gain might be even smaller. But anyhow, I think it's worth it.

Digging into the fabulous world of Emacs and Lisp, I wanted to use it to build my personal Web site and my blog. I already moved from ikiwiki to Emacs Muse for my HTML pages some weeks ago.

Muse provides an extension to maintain a journal, called muse-journal. Unfortunately, it was far to fulfill all my needs, and I decided that it would be a good exercise to write a better extension.

Consequently, I started to wrote my own extension, which I named muse-blog.

And this is now what is used to build this blog. :-)

In February 2009, my friend dim tried to force me using Emacs. I know a couple of people using it and Gnus for reading their mail, and it always made me curious.

At that time, more than a year ago, Emacs 22 and Gnus did not seem usable from my point of view.

But around mid February, with the help of dim, I tried again to start using Emacs.

Actually, this was not something new for me. I (very basically) used Emacs between 2000 and 2006. In 2006, when I finished the university and started working at Easter-eggs, I met a couple of vim enthusiasts. They taught me how to use it in various ways, and I started to know more about vim than Emacs, so I switched.

This time, I started by configuring it, but reading the manual and also learning a bit of Lisp. It took me several weeks, but step by step I learned many, many things. And I must admit, I liked it.

I've configured and starting to use some very important mode, like Gnus, Muse, the famous Org mode or even ERC.

I'll probably talk about various Emacs related things in the near future, since I already wrote more than a thousand lines of Lisp in the last 2 months.

Anyhow, I'd just conclude by asserting that my new Emacs/Gnus/Org/ERC setup beats my old vim/mutt/nothing/irssi to the death with a baseball bat. :-)

Two years ago, I wrote about my switch from my beloved xmms to audacious.

During this 2 years with Audacious, I suffered a bit. It was working quite fine, but I saw no big progress around it. Life happened, and I had to use a network system to play music. I started to use PulseAudio over TCP, but it does not work well, and does not work at all with Audacious (and even if the plugin is provided by upstream). So I decided to dump it.

And some days ago I discovered Sonata, a MPD client. I never liked MPD so far because all clients I found were lame.

But I really like Sonata. It allows me to listen music the way I still want: load everything in one playlist, listen everything randomly or type a song/artist to jump to it directly in the current playlist. It even has some nice feature (lyrics, so I'll be able to song out loud, covers, tag editing…) and is written in Python and GTK+ (some days I may even hack it!).

You can rest in peace x11amp :-p