Older blog entries for freetype (starting at number 26)

16 Jan 2006 (updated 16 Jan 2006 at 12:03 UTC) »
titus: configure scripts are not as portable as you believe. A script generated on Linux won't always work correctly as-is on FreeBSD, Cygwin or Mac OS X, the same is true for scripts generated on these platforms.

Moreover, they're huge (often > 100KB) ugly beasts that can change considerably between autoconf invokations, especially when your developers are on distinct platforms. Why would you want to put that kind of poop into your CVS repository ?

you'd better rant that developers should document which versions of autoconf/automake/libtool is needed instead.

bi: the exact French word is "gouvernement" and describes exactly what I said in my previous post (i.e. the executive office designated by the President, and nothing more).

Other funny factoïds:

  • since he's the head of the "gouvernment", our "Premier Ministre" is not elected (unlike the one in the UK).

  • a "cabinet" refers in France to the specific office of a given "ministre". So you can say "le cabinet du ministre de la Culture", which is only a tiny part of the "gouvernement".

French Politics for Dummies

It seems there is confusion about what happened yesterday regarding the "DADVSI code", so here's a quick intro on how French law-making works:

First, the actors

  • The government is a set of people chosen by the President to rule the country's affair. These people are not elected and do not represent the people of France in any way; they're just chosen by the President, and can be swapped/fired at will (which happens pretty often ;-)

  • The parliament is a set of locally-elected deputies. Each one has won a local election to be a representative for a specific region/district of France. Anybody can be a deputy, but you'll need to win the election, where being affiliated to a strong political party helps a lot (especially when you're a pedantic snob that never shook hands with the populace, like someone I know who became a deputy :-)

  • The senate is a set of old-bums that are elected by the Deputies every nine-years. It is used as a sort of geriatric facility for old politicians who can't do anything else. It's really a strange oddity in our political system because most of them are aged over 70 and don't have an idea of the country's current affairs.

Then, the Rules

  • One of the government's role is to propose new laws, or amendments to existing ones. All these proposals are debated at the Parliament by the deputies.

  • The deputies can reject a proposal, accept it pro forma, or more generally request specific changes and amendments before accepting them. Each proposal must be debated for at least two sessions, except under exceptional conditions.

  • When a proposal if accepted, even in modified form, it is sent to the Senate. The senators must also debate the proposal and have the power to reject it or accept it. I don't believe they can change it however. There is only one debate there.

  • When a law has been accepted by both the Parliament and the Senate, the Government must officially publish a special document, named "Décret d'application" which states exactly when the law shall start to be applied, wether it is retro-active, etc...

    It's important to understand that if the decret is not published, the law is pending and shall not be applied by judges, even if they want to :-) Interestingly, there exist a lot of "pending" laws in France that will likely never be applied.

So, what happened here

  • The government, and more specifically the "Ministre de la Culture" (yes, we have that here) proposed a sort of super-DMCA law to "protect" the authors of copyrighted works by making DRM tempering illegal (bye bye VLC's DVD playing capabilities), providing information about DRM systems illegal, and explicitely refusing fair uses practices for the purchasers of DRM-ed products.

  • The government has invoked exceptionnal reasons to impose a single debate at the parliament, instead of two; it also planned a very late hour for the debate just before Christmas. This is usually done to "ease" the acceptance of proposals, since less deputies in the debate generally mean less debates, change requests, refusals, etc...

  • The parliament has nonetheless imposed several amendments, some of them that would explicitely make non-commercial P2P sharing a recognized "fair use", as long as a sort of global tax be paid by users on their ISP's bill.

What does it mean

The law isn't official, so P2P isn't legal in France. Not yet. That also means that we can still watch DVDs though. There is nearly 0% chance that the Senate will accept the proposal, or that the Government will publish the corresponding Decret.

Not also that the "global tax" system isn't in place yet. Even if the law was applied NOW, P2P sharing would still be illegal.

Hope this helps.

Exhaustion

I still don't sleep enough,

Books

To follow La Mode, I remember reading and appreciating these in 2005:

  • "Le Samouraï Virtuel", a.k.a. "Snow Crash" (N. Stephenson)
  • "Le dernier théorème de Fermat", a.k.a. "Fermat's last theorem" (S. Singh)
  • "Histoire des Codes Secrets", a.k.a. "The Code Book" (S. Singh)
  • "Dernier inventaire avant liquidation" (F. Beigbeder)

FreeType

Couldn't find a lot of free time to work on it; however, I could study ZLib's latest revisions. It turns out that the authors have done great things: cleaning up the sources (no more "one-letter-identifiers-only" coding style), improving performance and reducing memory footprint. Great !!

Web 2.0

I love this, that's sooooo true :o)

21 Oct 2005 (updated 21 Oct 2005 at 09:08 UTC) »
FreeType

I'm still trying to get rid of big memory eaters in FreeType. I have found the following offenders to date:

  • the LZW decompressor, used to support pcf.Z and bdf.Z font files on legacy systems (i.e. SunOS) allocates a 400 KB memory block before reading anything from a file.

  • the PCF driver copies large portions of font files into memory, when it should really re-load them on demand.

  • the BDF driver allocates a 64 Kb heap block before reading anything. It also duplicates a lot of things in memory when it should re-read them from the file.

I've started working on the first one. It seems the code we use came from the BSD "uncompress" program. Its source code is cryptic, suffers from bad design and slow read performance. I rewrote it in a few hours into something much more elegant. Try to compare this and this.

The good news is that testing with timR24.pcf.Z shows that opening is now 27% faster, glyph loading is now 63% faster, and that the new code saves 330KB of heap memory with this font. not bad for such a small change :-)

However, this is for a deprecated format that nearly nobody uses. I'll probably work on the PCF reader later. Its source code comes from the XFree86 distribution, and probably needs a good rewrite as well.

I'm also wondering if there isn't a way to optimize the zlib source code used for .gz support. Last time I checked, it was horrible as well, but it seems they've done very good thing for version 1.2.

21 Oct 2005 (updated 21 Oct 2005 at 08:40 UTC) »
Pango, Text Layout and UI Performance

Kudos to Federico for starting to optimize Pango. I know fom experience that even a small performance difference in text layout alone (i.e. without touching glyph rendering) has a huge impact on user's perceived interface speed. That's because toolkits and applications tend to draw a _lot_ of text, even when they could trivially avoid it.

When I had to add Arabic to my company's products, the text layout operation timing initially doubled due to the support of the Unicode BIDI algorithm. It was a custom solution since we couldn't use either ICU or FriBidi. I didn't think it would be a big issue since, in the graphics architecture we were using back then, the time to transfer text glyphs to the screen was very important. My own tests didn't show any perceivable difference anyway when the whole application was running.

Well, I was wrong. It turned out that the application could switch dynamically between English and Arabic presentation mode, and the client complained that scrolling lists in Arabic was slower than in English. How much slower ? Well, a really _tinsy_ bit slower, you had to scroll several pages on the list to really show the difference. But they were horrified by this and considered this to be a top-priority issue.

After a quick analysis, it turned out that:

  • the list component redraw the whole page everytime you went from one item to the next, even though it only needed to update two lines

  • due to the component's design, this required a rewrite, and there was no time to do that (different team, different planning, etc..)

I ended up rewriting the BIDI implementation and the Arabic shaper, with all kinds of tricks. It's probably a bit slower than ICU's algorithm (which is insanely clever, by the way), but at least I knew it would be manageable by someone other than me at the company ;-)

The end result was that laying out Arabic text is now only 10% slower than English in our product, mainly because we don't use OpenType fonts !!.
Problem solved. Except we're probably never going to fix the list component. Ah ah...

Life

Alice is slowly getting better. She doesn't scream as much as she used too, and only wakes up once in the middle of the night. That's a huge improvement, and we're just beginning to recover though my brain is still terribly dizzy at the moment :-)

FreeType

FreeType 2.1.10 is out, this fixes a bunch of bugs, but the memory savings I have mentioned in a previous diary entry are disabled by default. This is because this breaks several libraries on Unix that unfortunately use internal FT2 structures which changed during the optimizations.

The next version will be called 2.2.0, use a major library version number of 7 (instead of 6), and will _not_ install the internal/private headers (this was our original mistake which created this situation). Instead, they'll be replaced by files generating #errors at compile time with a message explaining the situation.

What it means is that an undecided amount of code is going to break when compiled against this version of the library, which is why I've begun writing patches for Pango, FontConfig and a few other libraries. I'm currently working on SDL_ttf and Qt.

It turns out to be a lot simpler than what I expected; my problem is that I don't know the exact list of libraries and applications that directly depend on FreeType internal structures and functions. If you happen to work at at Linux distribution company, or simply have a lot of disk space, huge bandwidth and plenty of free time (which really isn't my case at the moment), I'd be glad if you could grep for some of the lines below and report which packages contain them:

grep "FT_INTERNAL_" sources
grep "freetype/internal/" sources

Of course, 2.2.0 isn't out until we get a sufficient number of patches and make them available on our web site. I don't expect to see that much problem however once the popular libs are out.

And yes, I've used Koders Search (www.koders.com) and it seems absolutely useless for this purpose.

Laptop

I've installed Ubuntu Hoary on my laptop, dual booting with XP, and I'm very impressed with it. However, trying to use it on an on-going basis is _very_ frustrating, because it lacks many of the niceties I'm used to on Windows. I'm not talking about Windows-specific items, rather about all the free cool-addons, most of them free software, that I regularly use.

Does anyone know an equivalent for the following on Gnome, or even, sigh..., on KDE:

  • TortoiseCVS
  • TortoiseSVN
  • Putty Pageant (I mean a Gnome or KDE applet)

And I can't get the damn thing to suspend properly, not even hibernate. I normally don't use the feature, because XP boots _very_ quickly on this machine, but startup is so slow on Linux that it's painful. For the record, I use this machine on the train twice a day, each trip being 30 minutes, it's not like a nearly 2 minutes boot isn't a problem for me.

Grosse Fatigue

I'm a father again. Alice was born on April 3rd this year. She's a magnificent baby, and more importantly, she's absolutely normal and doesn't have any of her sister's problems. Life is good, life is good. And Lucie doesn't even seem to be jealous at the moment.

On the other hand, Alice's still absolutely unable to sleep at night, and my wife and I have been seriously lacking any kind of decent sleep for more than a month now. I'm more lucky since I don't have to breast-feed her, but "a deep rest" is now a distant concept, at least for the next couple of months :-).

Language Wars

rmathew, you might want to take a look at the D Language: from my point of view, this is the best designed language currently available. The pros are:

  • All the niceties of C++ (OO, exceptions, templates, compiled, easy linking to C libraries) without the hurdles (atrocious syntax, atrocious compile times)
  • ultra fast compilation time, excellent execution times (easily beats C++ at many tasks)
  • dynamic arrays, strings, dictionaries and complex numbers as builtins, not ugly library/template hacks
  • the language is garbage-collected: the killer productivity enhancement. (Can be disabled if you want it).
  • built-in support for versioning, design-by-contract and much more.
  • much simpler and cleaner syntax.
  • a GCC Frontend is even available and works on Unix, Windows, MacOSX

There are still some cons at the moment though:

  • first of all, the language is still in beta, which means it's still slightly evolving.
  • the frontend isn't part of the standard GCC distribution, making it more difficult to distribute widely
  • GDB doesn't understand D name mangling yet, which makes debugging quite challenging sometimes. But you won't need it that much in my experience
  • Documentation exists but is scattered around several web pages.
  • It doesn't have the huge runtime libraries that come with Java or C# (though it can interface to C easily)

However, all these points can be solved in a short time frame. I predict this will happen once the language freezes. As for me, I've completely ditched C++ for my personal projects, it simply isn't worth the effort for what I do.

Hope this helps

1 Apr 2005 (updated 1 Apr 2005 at 12:37 UTC) »
Shared Libraries and Memory Usage on Linux

I've written a small program to measure the 'memory payload' of using shared libraries on Linux. What it shows is the amount of private process memory that is taken by the simple fact of _linking_ a program to a shared object. This corresponds roughly to the following data:

  • the specific position-independent-code tables, like the GOT, PLT and DLT. These are automatically generated by the dynamic loader. The more functions the library imports/exports, the bigger these tables are. They also explain why _any_ shared library uses at least 4 KB/process.

  • any writable static data used by the library itself.

  • finally, any absolute pointer constant found in the library, since these cannot be placed within PIC code. Any C++ shared library is going to have all of its vtables in the .data section for this reason.

Note that this does not include the memory used by the libraries when they're used (e.g. when allocating from the heap with 'malloc').

The code is available here. Python 2.1 or above required.

Results are interesting. On my work system (RedHat 7.3), the biggest "eaters" are the following:

 process     KB    KB per    library
   count  total   process
--------------------------------------------------------
       8   1312       164    /usr/lib/qt-2.3.1/lib/libqt.so.2.3.1
      59   1180        20    /lib/i686/libc-2.2.4.so
       8    768        96    /usr/lib/libkdeui.so.3.0.0
      18    576        32    /lib/i686/libpthread-0.9.so
       5    440        88    /local/dturner/opt/lib/libstdc++.so.5.0.5
      19    304        16    /usr/X11R6/lib/libX11.so.6.2
       8    256        32    /usr/lib/libkdecore.so.3.0.0
      59    236         4    /lib/ld-2.2.4.so
       8    224        28    /usr/lib/libstdc++-3-libc6-2-2-2.10.0.so
       4    192        48    /lib/libcrypto.so.0.9.6b
     ...
total payload = 10044 KB

This seems pretty reasonnable, though there are some surprises here (it's an XFCE desktop with only two KDE apps opened, no KDM !). I'd better run the script on more recent distributions to see how they compare however.

Dell Laptops

In October 2002, I was joking at the fact that Dell sells the same computer hardware with a slightly different casing for a lot more dollars. Well, it turns out that this is even more true than I imagined.

I discovered this because I wanted to buy a docking station for my Inspiron 8200, but couldn't find one at an affordable price. While at the same time, many companies on the web sell refurbished Latitude hardware for totally ridiculous amounts (and as you might guess, the Inspiron refuses to work when mounted on a Latitude station).

And then, I found this page which explains how to flash my laptop's BIOS to make it believe it is a Latitude C840. Following the steps detailed here, I could dock my laptop into a C/Port II that I bought for the shocking sum of 17€, shipping included (I'm not kidding) !!.

But this is not all. For some reason, the graphics card in my laptop had previously melted. Very fortunately, I could find someone to sell me a used graphics card for the laptop. It happens that this new card if a Quatro, instead of a GeForce. Because of this, and because I have more than 512 Mb of RAM, the new BIOS believes the machine is now a Precision M50, the "ultra-high-end" "version" of this computer when they were released.

Apparently, these things are known facts, but it's always pleasant to perform a little magic on this kind of hardware. This laptop is now used as a desktop computer, except that it's a lot more silent than the latest bunch of PCs :-)

17 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!