Older blog entries for AlanHorkan (starting at number 372)

OpenRaster Paths (or Vectors)

Summary: plugin updated to allow round-trip of paths.

The MyPaint team are doing great work, making progress towards MyPaint 1.2, I encourage you to give it a try, build it from source or check out the nightly builds. (Recent windows build Note: the filename mypaint-1.1.1a.7z may stay the same but the date of build does change.)
The Vector Layers feature in MyPaint is particularly interesting. One downside though is that the resulting OpenRaster files with vector layers are incompatible with most existing programs. MyPaint 1.0 was one of the few programs that managed to open the file at all, presenting an error message only for the layer it was not able to import. The other programs I tested, failed to import the file at all. It would be great if OpenRaster could be extended to include vector layers and more features but it will take some careful thought and planning.

It can be challenging enough to create a new and useful feature, planning ahead or trying to keep backwards compatibility makes matters even more complicated. With that in mind I wanted to add some support for vectors to the OpenRaster plugin. Similar to my previous work to round-trip metadata in OpenRaster I found a way to round-trip Paths/Vectors that is "good enough" and that I hope will benefit users. The GNU Image Manipulation Program already allows paths to be exported in Scalable Vector Graphics (SVG) format. All paths are exported to a single file, paths.svg and are imported back from that same file. It is not ideal, but it is simple and it works.

Users can get the updated plugin immediately from the OpenRaster plugin gitorious project page. There is lots more that could be done behind the scenes, but for ordinary users I do expect any changes as noticeable as these for a while.

Back to the code. I considered (and implemented) a more complicated approach that included changes to stack.xml, where raster layers were stored as one group, and
paths (vectors layers) as another group. This approach was better for exporting information that was compatible with MyPaint but as previously mentioned, the files were not compatible with any other existing programs.

To ensure OpenRaster files that are back compatibility it might be better to always include a PNG file as the source for every layer, and to find another way to link to other types of content, such as text or vectors, or at some distant point in the future even video. A more complicated fallback system might be useful in the long run. For example the EPUB format reuses the Open Packaging Framework (OPF) standard, any pages can be stored in multiple formats, so long as it includes a fallback to another format, ending with a fallback to a few standard baseline formats (i.e. XHTML). The OpenRaster standard has an elegant simplicity, but there is so much more it could do.

Syndicated 2015-04-06 22:00:30 from Alan Horkan

OpenRaster Metadata

Summary: plugin updated to allow round-trip of metadata.

OpenRaster does not yet make any suggestions on how to store metadata. My preference is for OpenRaster to continue to borrow from OpenDocument and use the same format meta.xml file, but that can be complicated. Rather than taking the time to write a whole lot of code and waiting do metadata the best way, I found another way that is good enough, and expedient. I think ordinary users will find it useful -- which is the most important thing -- to be able to round-trip metadata in the OpenRaster format, so despite my reservations about creating code that might discourage developers (myself included) from doing things a better way in future I am choosing the easy option. (In my previous post I mentioned my concern about maintainability, this is what I was alluding to.)

A lot of work has been done over the years to make the The GNU Image Manupilation Program (GIMP) work with existing standards. One of those standards is XMP, the eXtensible Metadata Platform originally created by Adobe Systems, which used the existing Dublin Core metadata standard to create XML packets that can be inserted inside (or alongside) an image file. The existing code creates an XMP packet, let's call it packet.xmp and include it in the OpenRaster file. There's a little more code to the load the information back in and users should be able to go to menu File, Properties and in Properties dialog go to the tab labelled Advanced to view (or set) metadata.

This approach may not be particularly useful to users who want to get their information out into other applications such as MyPaint or Krita (or Drawpile or Lazpaint) but it at least allows them not to lose metadata information when they use OpenRaster. (In the long run other programs will probably want to implement code to read XMP anyway, so I think this is a reasonable compromise, even though I want OpenRaster to stay close to OpenDocument and benefit from being part of that very large community.)

You can get the updated plugin immediately from the OpenRaster plugin gitorious project page.

If you are a developer and want to modify or reuse the code, it is published under the ISC License.

Syndicated 2015-04-06 20:36:09 from Alan Horkan

OpenRaster Python Plugin

Thanks to developers Martin Renold and Jon Nordby who generously agreed to relicense the OpenRaster plugin under the Internet Software Consortium (ISC) license (it is a permissive license, it is the license preferred by the OpenBSD project, and also the license used by brushlib from MyPaint). Hopefully other applications will be encouraged to take another look at implementing OpenRaster.

The code has been tidied to conform to the PEP8 style guide, with only 4 warnings remaining, and they are all concerning long lines of more than 80 characters (E501).

The OpenRaster files are also far tidier. For some bizarre reason the Python developers choose to make things ugly by default, and neglected to include any line breaks in the XML. Thanks to Fredrik Lundh and Effbot.org for the very helpful pretty-printing code. The code has also been changed so that many optional tags are included if and only if they are needed, so if you ever do need to read the raw XML it should be a lot easier.

There isn't much for normal users unfortunately. The currently selected layer is marked to the OpenRaster file, and also if a layer is edit locked. If you are sending files to MyPaint it will correctly select the active layer, and recognize which layers were locked. (No import back yet though.) Unfortunately edit locking (or "Lock pixels") does require version 2.8 so if there is anyone out there stuck on version 2.6 or earlier I'd be interested to learn more and I will try to adjust the code if I get any feedback.
I've a few other changes that are almost ready but I'm concerned about compatibility and maintainability so I'm going to take a bit more time before releasing those changes.

The latest code is available from the OpenRaster plugin gitorious project page.

Syndicated 2015-03-23 18:35:16 from Alan Horkan

Krita 2.9

Congratulations to Krita on releasing version 2.9 and a very positive write-up for Krita by Bruce Byfield writing for Linux Pro Magazine.

I'm amused by his comment comparing Krita to "the cockpit of a fighter jet" and although there are some things I'd like to see done differently* I think Krita is remarkably clear for a program as complex as it is and does a good job of balancing depth and breadth. (* As just one example: I'm never going to use "File, Mail..." so it's just there waiting for me to hit it accidentally, but far as I know I cannot disable or hide it.)

Unfortunately Byfield writes about Krita "versus" other software. I do not accept that premise. Different software does different things, users can mix and match (and if they can't that is a different and bigger problem). Krita is another weapon in the arsenal. Enjoy Krita 2.9.

Syndicated 2015-02-25 23:42:50 from Alan Horkan

OpenRaster Python Plugin

OpenRaster Python Plugin

Early in 2014, version 0.0.2 of the OpenRaster specification added a requirement that each file should include a full size pre-rendered image (mergedimage.png) so that other programs could more easily view OpenRaster files. [Developers: if your program can open a zip file and show a PNG you could add support for viewing OpenRaster files.*]

The GNU Image Manipulation Program includes a python plugin for OpenRaster support, but it did not yet include mergedimage.png so I made the changes myself. You do not need to wait for the next release, or for your distribution to eventually package that release you can benefit from this change immediately. If you are using the GNU Image Manipulation Program version 2.6 you will need to make sure you have support for python plugins included in your version (if you are using Windows you wont), and if you are using version 2.8 it should already be included. (If the link no longer works, see instead https://gitorious.org/openraster/gimp-plugin-file-ora/ as I hope the change will be merged there soon.)

It was only a small change but working with Python and not having to wait for code to compile make it so much easier.

* Although it would probably be best if viewer support was added at the toolkit level, so that many applications could benefit.

Syndicated 2015-02-18 19:14:34 from Alan Horkan

OpenRaster and OpenDocument: Metadata

OpenRaster is a file format for the exchange of layered images, and is loosely based on the OpenDocument standard. I previously wrote about how a little extra XML can make a file that is both OpenRaster and OpenDocument compatible. The OpenRaster specification is small and relatively simple, but it does not do everything, so what happens if a developer wants to do something not covered by the standard? What if you want to include metadata?

How about doing it the same way as OpenDocument, it does not have to be complicated. OpenDocument already cleverly reused the existing Dublin Core (dc) standard for metadata, and includes a file called meta.xml in the zip container. A good idea worth borrowing, a simplified example file follows:

Sample OpenDocument Metadata[Pastebin]

(if you can't see the XML here directly, see the link to Pastebin instead.)

I extended the OpenRaster code in Pinta to support metadata in this way. This is the easy part, it gets more complicated if you want to do more than import and export within the same program. As before the resulting file can renamed from .ora to .odg and be opened using OpenOffice* allowing you to view the image and the metadata too. The code is Pinta OraFormat.cs is freely available on GitHub under the same license (MIT X11) as Pinta. The relevant sections of are "ReadMeta" and "GetMeta". A Properties dialog and other code was also added, and I've edited a screenshot of Pinta to show both the menu and the dialog at the same time:

[* OpenOffice 3 is quite generous, and opens the file without complaint. LibreOffice 4 is far less forgiving and gives an error unless I specifically choose "ODF Drawing (.odg)" as the file type in the Open dialog]

Syndicated 2015-02-13 00:38:44 from Alan Horkan

OpenRaster and OpenDocument

OpenRaster is a file format for layered images. The OpenRaster specification is small and relatively easy to understand, essentially each layer is represented by a PNG image, and other information is contained written in XML and it is all contained in a Zip Archive. OpenRaster is inspired by OpenDocument.
OpenDocument is a group of different file formats, including word processing, spreadsheets, and vector drawings. The specification is huge and continues to grow. It cleverly reuses many existing standards, avoiding repeating old mistakes, and building on existing knowledge.

OpenRaster can and should reuse more from OpenDocument.

It is easy to say but putting it into practice is harder. OpenDocument is a huge standard so where to begin? I am not even talking about the OpenDocument Graphics (.odg) specifically but more generally than that. It is best that show it with an example. So I created an example OpenRaster image with some fractal designs. You can unzip this file and see that like a standard OpenRaster file it contains:

 ├ mimetype
 ├ stack.xml
 ├ data/
 │  ├ layer0.png
 │  ├ layer1.png
 │  ├ layer2.png
 │  ├ layer3.png
 │  ├ layer4.png
 │  └ layer5.png
 ├ Thumbnails/
 │  └ thumbnail.png
 └ mergedimage.png

It also unusually contains two other files manifest.xml content.xml. Despite the fact that OpenDocument is a huge standard the minimum requirements for a valid OpenDocument file comes down to just a few files. The manifest is a list of all the files contained in the archive, and content.xml is the main body of the file, and does some of the things that stack.xml does in OpenRaster (for the purposes of this example, it does many other things too). The result of these two extra files, a few kilobytes of extra XML, is that the image is both OpenRaster AND OpenDocument "compatible" too. Admittedly it is an extremely small tiny subset of OpenDocument but it allows a small intersection between the two formats. You can test it for yourself, rename the file from .ora .odg and LibreOffice can open the image.

To better demonstrate the point, I wanted to "show it with code!" I decided to modify Pinta (a Paint program written in GTK and C#) and my changes are on GitHub. The relevant file is Pinta/Pinta.Core/ImageFormats/OraFormat.cs which is the OpenRaster importer and exporter.

This is a proof of concept, it is limited and not useful to ordinary users. The point is only to show that OpenRaster could borrow more from OpenDocument. It is a small bit of compatibility that is not important by itself but being part of the larger group could be useful.

Syndicated 2014-12-12 04:03:51 from Alan Horkan

Developing MComix. Part 1: A rose by any other name...

Comix is a comic and image viewer program written using Python and GTK. I was interested in making a few changes to the project. I am getting back into programming, learning a more about PyGTK and hopefully making some improvements the program will fit better on a small screen netbook. Unfortunately Comix is unmaintained, the developer of the project seems to be unavailable and has not made any updates since 2009. (You can discover this for yourself by checking the Comix website particularly by checking the Sourceforge page for Comix and taking a look at the last date on the Changelog, and various reports in the bug tracker. It may still be possible to get in contact with him but the project is not active.)

I was pleased to discover MComix a fork of Comix, continuing on from Comix version 4.0.4. It isn't clear what the M in MComix stands for but from the picture in the about dialog I think "Monkey Comix" is a fairly safe guess.
(Later I also discovered another fork of Comix by HellHover which incorporates many of the patches submitted to Comix. It is more like a spork than a fork, as it keeps close to the original.)

The attitude and prompt reponse from the MComix developers was encouraging so I got to work putting together a few changes and sending in a patch (after a discussion clarifying that MComix is licensed under the GNU GPL version 2).
Freedom to change the source code and even fork the project is a great power to have but it comes with responsibility. Getting others to help is not easy, you want people to submit code, help with translations, or get a project packaged nicely for different operating systems, and generally help with the work of maintaining a project, so even though forking is possible it is not something you want to do unless absolutely necessary. Knowing it would take some time and effort to make changes it is a great relief to know there is a good chance others will accept the change help maintain it. Putting a lot of work into a patch and having it go to waste, is a big disappointment and severly discourages anyone from contributing to a project.

The patches took less time to write and test than the email explaining the rationale for the changes. Developers often like to do things their own way, and a without a proper explanation, patches might not be accepted. As a fork of Comix I was optimistic the developers would be more accepting of change but I wasn't going to take any chances.

The first - and to my mind most important - change was relatively simple but something I would strongly recommend to any program, especially a fork: separate out the name of the program. I set a constant and called it "APPNAME" and replaced the word MComix, so that if the name ever needs to be changed again or even changed back to Comix it can be done with only a few small changes.
The long and winding history of Mosiac, Netscape, Mozilla, m/b, Phoenix, Firebird, Firefox, Iceweasel, is a particularly extreme example of the many name changes a project can go through. If someone doesn't like the name the can easily change it without needing to fork the project. Making it a little bit easier for others to customize and fork the code gives them one less reason why they would need to, and they can continue to pool their efforts on the things they do agree on.

The patch also contained changes to help with internationalisation (i18n) and localisation (l10n).
There was also a small change to the command line arguments.

When the patch was accepted I was very satisfied and confident the developers were the kind of people I would enjoy working with further. Answers to a few other questions in my email confirmed our interests were different but mutually beneficial. Straight away I was thinking about the next changes I might make.

Syndicated 2011-04-23 23:49:11 from Alan Horkan

Peazip: Convert CBR to ZIP

I had a tshirt with the slogan "More people have read this shirt than have read your blog" and I am always a little surprised when someone does read my writings on the web. I was pleased to see that someone thought my batch file to repack from CBR to CBZ might be useful for people who want to read comics on mobile devices.

I was thinking about rewriting the batch file to work with programs other than 7-zip and I rediscovered Peazip a program I had not tried in years. The user interface is still a bit rough in places but it includes a very useful tool for Convert archives from different formats. It is a convenient way to convert files from CBR to ZIP which I think some users will prefer to using a DOS batch file. There does not seem any way to automatically rename the ZIP files to CBZ, so for I prefer to keep using the batch file. (The Convert tool in Peazip will unpack all the files and then repack all files, which can take up a lot of disk space with temporary files, so be careful if you try to convert many files at once.)

The biggest advantage of the batch file is that it does more than just repack from CBR to CBZ. I continue to modify it in other ways to better clean up the Comic files as I repack them, such as removing signature or banner images added by scanners. It is tricky to come up with a pattern that works all the time so I only included a simple example before, but the following patterns catch a lot of the tags and banner images:

MOVE z*.* ..\  
MOVE xx*.* ..\  
MOVE *tag*.* ..\  
MOVE *team*.* ..\  

The banner images can take up quite a bit of space, more than 1MB in rare cases, so removing them often saves more space than anything else such as special compression tricks. Instead of adding a large image file the batch file could be changed to add a small text file containing comments by adding the following command before recompressing:
echo "Repacked by 7-zip" > Comment.txt 

Tidying up file names is another thing that could be added to the batch file.

As I said earlier I am glad at someone found the batch script useful, hopefully Peazip will be useful too.

Syndicated 2011-04-06 00:52:03 from Alan Horkan

Comics: Convert CBR to CBZ

Comics are great. I'd gotten out of the habit but I still enjoy picking up a good Batman story now and again, but other than that it is hard to know what to pick. The painted comics of Alex Ross (Kingdom Come) are beautiful, the stories of Warren Ellis (Transmetropolitan) are compelling.

Can you judge a book by it's cover? If it is a graphic design book, or art book I think you can and with comics I often do choose on that basis (and all too often get caught out when the interior art is entirely different from the cover). In the long run though it is stories and the writers that keep me coming back and if I particularly like the style of the artist that is a bonus.

Chatting with the local equivalent of Comic Book Guy is one way to get ideas of what to try but picking a book off a shelf and having a look is usually the best way. The internet has provided us with an even bigger bookshelf to flick through, and there are groups who scan the latest comic books and package them up for easy sampling, usually a series of JPEG image files packaged together as a single convenient archive. Very often this archive they are packaged in use RAR compression CBR (Comic Book RAR archive), but other times the container uses Zip compression CBZ (Comic Book Zip archive). The essential difference is that Zip is free and RAR is proprietary.

I wanted to convert these CBR files to CBZ. I could do that by unpacking and repacking the files on a case by case basis but that is repetative and uninteresting. Instead I decided to make things more complicated and come up with a more general automated solution. That solution might be of use to others, especially those who do not know how to write MSDOS batch files, so I'm sharing it. On the internet someone else there is almost always someone else who has the same idea as you, and I was able to find something very close: Andrew Armstrong had already written a Mass Zip, RAR to 7Zip Recompression Batch File and all I needed to do was make some small adjustments so that it would to convert RAR files and CBR files into CBZ files. If you want details of what exactly is happening you should read his detailed description. It took me far less time to make my changes than it to write them up and explain them, the hard work was already done.

The script requires the excellent free compression program 7-zip and you must have it installed. To keep things simple we assume 7-zip is installed in the default location C:\Program Files\7-zip\7z and we do not try to check. If for example you were using a German version of Microsoft Windows 7-zip might be installed in a program called "Programme" rather than "Program Files" or if you wanted to use a portable flash drive and use Portable 7-zip you would also need to change the program before it would work. (We could try to avoid the first problem by checking to see if %ProgramFiles% is C:\Program Files as expected.

Another known issue the sometimes causes the script to fail is when file path names are too long. Microsoft Windows limits file names to 260 characters in length. The script creates a temporary folder based on the name of the CBR and if there is a subfolder inside the CBR the filename can sometimes be too long.

As this script is intended to be used to unpack not just any generic RAR file but a Comic Book Archive there are other changes we can include. Creators of Comic Book archives compress a whole folder instead of picking out just the picture files they want, and all sorts of other things get included by mistake. Microsoft Windows includes a hidden file called Thumbs.db which stores thumbnail images of your files. We can safely delete this, Windows will recreate it if the comic book archive is ever unpacked and it is needed. To do this I added the line DEL /F /S /Q Thumbs.db (which is short for DELete by Force, in all Subdirectories, and do it Quietly without reporting any errors).

The scanners who create Comic Book archives often like to include a banner image saying who they are and often with a message such as reminding readers that if they like the comic they should buy it too. These files are often given a file name starting with 'Z' so that it will appear last in the sequence so I added another line to delete any file that starts with the letter 'Z' DEL /F /S /Q z*.*. If the banner image has a different name this won't work but it is an example of what you can do to fit the generic scripts to better fit that task.

One change I have made might seem surprising. I have set the Zip compression to level zero, which is no compression at all (-mx=0). I could save a small amount of space by using the maximum amount of compression but this would take longer to pack the files, and more importantly it would make it slower each and every time you want to unpack and read the files. The Zip archive convenient container to make things easier to organize, and JPEG files are already quite heavily compressed.

If you want to try it out for your self, copy the commands below into a text file, change the name to cbr2cbz.bat or similar, put it in a folder with some CBR or RAR files and away you go.

REM Based on 
REM Mass Zip, RAR to 7Zip Recompression Batch File by Andrew Armstrong
REM http://aarmstrong.org/tutorials/mass-zip-rar-to-7zip-recompression-batch-file
REM Find all .cbr and .rar files in the current directory, recompress them into .cbz and .zip files - but do not delete the originals.

ECHO Searching for CBR and RAR files to recompress into CBZ (Comic Book Zip) archives.

for %%F in (*.cbr *.rar) do (
ECHO Found %%F to recompress...
REM Extract...
"C:\Program Files\7-zip\7z" x "%%F" -o"%%F contents"
REM Does the directory exist? has 7zip created it correctly?
IF EXIST "%%F contents" (
REM Change directory, create zip of contents of directory...
CD "%%F contents"
REM remove banners 
DEL /F /S /Q z*.*
REM Delete Thumbnail files. 
DEL /F /S /Q Thumbs.db
REM compression level is 0, store only. 
"C:\Program Files\7-zip\7z" a -tzip "../%%~nF.cbz" * -mx=0
CD ..
REM Delete the temporary extraction folder
RMDIR "%%F contents" /S /Q
ECHO Recompressed %%F to %%~nF.zip
ECHO Search ended.


Syndicated 2011-02-01 01:57:35 from Alan Horkan

363 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!