UPDATE: Because of the wonderful reception across the internet, I’ve put together an instruction page on how to get this set up on your own server.
I don’t really like database driven photo management software, and prefer instead to manage my photos in a good old no-nonsense directory structure. For this reason, I was particularly attracted to Zenphoto as a means of getting my photos online, as it works on directory structures. Unfortunately, Zenphoto is horrible; it’s riddled with bugs, inconstant, a cluttered architecture, and most of all, it’s extremely slow. Every time it runs, it re-scans directories and makes a bazillion SQL calls. The viewer interface is also outdated and clunky, having a different html page for each photo. So I went back to the drawing board and considered how to make things better.
Introducing PhotoFloat. The idea is this — instead of scanning and caching metadata and thumbnails during page load time, everything is to be done prior. It’s a bit of an old school mentality. There is a script that generates static json files of metadata and album structures and static thumbnails of images, so that all the content can be served directly by Apache. Why? Because I only need to generate new thumbnails and data files when I upload new images (or alternatively, on a cron job). So that’s what I did; I wrote a simple python script that walks a directory structure looking for new or changed images and albums. It’s smart too — to be super zippy, it does file modification time comparisons. It also cleans up after itself, deleting stale files.
So I have all my original images on my webserver, because I have Dreamhost’s unlimited hosting. I also have another directory that I populate with symlinks to the directories I actually want online. Every time there are new images, my python script fires up, and updated json data files and thumbnail files are generated.
Great, but where does this leave us? What can we do with json files? This is where things become wonderful. Since all the data for the gallery is AJAX fetchable, there is a single html page and a single javascript file that takes care of the whole gallery. That’s right — all of the display of views is done client side, and in one page load.
To keep track of pages and for swapping around links, each different album and different image has it’s own hash url, like, for example: #!/new_hampshire_in_snow_3.15.11-3.17.11/img_1919.jpg. It’s all lower case with naughty characters stripped out to keep up with the patterns of wordpress and other web apps. These function as permalinks.
The albums have extensive support for EXIF metadata, which can be loaded by clicking ‘show metadata’, and a transparent box slides up over the photo. There’s also the ability to download the original photos.
Each album gets a randomized thumbnail which assigns probabilities to each image in the album based on the number of images in each album and the depth of subalbums. The randomization algorithm is all done at client side.
Images are preloaded. Album data is prefetched. Everything is cached sanely. JSON files are gzipped. There are animations between views and smooth scrolling. The right and left arrow keys work. Clicking on the photo advances it, like on Facebook. Finally, I do include one dynamic script — a simple php script that takes old Zenphoto URLs and translates them into the new ones, so that people with old links can still access the same photos.
Essentially, there are a lot of little details that had to be done right, and to my knowledge, no web gallery that works on directory structures has done it well, making an ajaxy and speedy gallery. So now you have PhotoFloat. I’ve just finished writing it, and the code is a bit of a mess, but let me know if you have any suggestions or find any bugs.
You can browse the code in the git repository or try it out live on my photo site. If you make any modifications of my code or use it on your own site, please inform me and send any modifications back to me. Remember to run make on the web directory to minify the css and javascript, and also, be sure to change the google analytics tracking ID in web/js/999-googletracker.js.
Comments? Suggestions?
Update 2 for KDEers: It looks like some people from kipi-plugins and kphotoalbum are interested in building integration for this in.
Update 3: Following a suggestion in the comments below, URLs now use #!, which google translates to a special query string, and I’ve written a serverside component that executes the JavaScript and displays static content for googlebot. This allows the metadata to be crawled.
Niiice!
Bookmarked! Definitely to be revisited and attempted for my own photos to be put online.
So without Javascript, your gallery degrades to a blank page
Yes. JavaScript is part of the interwebs and is here to stay.
Nice gallery software. It definitely fills a void. Also, great photos on your site!
So, any thoughts on possibly segregating the frontend and turning it into an export plugin using kipi-plugins? Or is this something that’s deceptively more complicated than it seems?
This sure beats the pants off of what it currently uses for gallery export
This very much would be possible. I’m already in contact with the fella from KPhotoAlbum about his ideas. I’d love to get this kipi-integrated. Frontend and backend are already separated. The frontend works by simply looking for json files and thumbnails in a certain directory. Nothing more than that. So it’d be easy to make many different generators of those json files and thumbnails to integrate. My python scanner script is only one such possibility.
Very cool. I don’t know if kipi-plugins has any sort of restrictions on how complex or detailed the configuration UI for the plugin can be.
Particularly in something like this, there are a plethora of options that could be offered; fonts, layout, image compression, offer “download original,” etc etc.
I like most of the design choices you’ve made for the frontend, but there are some things I’d like to be able to change. Don’t know how I feel about the italicized font in the overview, and I would also like to play with horizontal padding between albums.
A thought that occurred to me while writing that; different users’ preferences for visual design could probably be placated by offering 3 or 4 default packaged themes for the gallery.
On that note, how easily themeable is this? Is the layout code mostly in javascript or CSS?
great job, btw, way to scratch that itch
The vast majority of the layout is done in CSS, so it would be fairly trivial to just supply additional CSS files.
I’m really no web designer and I haven’t much of a clue what I’m doing w.r.t. web design, so if you have any suggestions about the current theme, do let me know. Spacing, colors, fonts… I REALLY need some feedback in this department.
I imagine things could be configurable with a config.json, but this is an extra asset that has to be patched. Maybe the best way to offer a feature like “download original” would be to in the javascript check whether the html element exists, and if it does, supply the link, and if it doesn’t, don’t. That way, to disable it, all you’d have to do is remove the relevant part from the html. What do you think?
I’m really no designer either, I just know what I personally prefer in looks — but dare I say, this might qualify for a Season of KDE project? ,:) http://toscalix.blogspot.com/2011/05/season-of-kde-we-need-you-kde.html
Regarding “download original,” i’ll have to skim the javascript to see how you have the code laid out before answering that, and I’m on a phone right now, not a proper computer. */bookmarks for later tonight/*
Session of KDE would in fact be pretty cool. Well, stay in touch.
Use google hash-bang urls so that it’ll be able to index the meta-data of the image
Forgot to mention the url
http://code.google.com/web/ajaxcrawling/
What a wonderful idea. I will indeed add this.
I’ll have to add some HtmlUnit integration into the current rewrite.php aspect. This is possible. Standby.
Implemented. Let me know if you have any suggestions.
Blog post about it: http://blog.zx2c4.com/589
That’s a great application ! Is there a way to directly draw the original image instead of the thumbnail when its dimensions are less than the precomputed images ?
Yes… I had noticed this before. I can, in fact, do this. I’ll have it done soon.
[...] not quite. A comment in my blog post pointed me toward Google’s AJAX Crawl specification, which is incredible. Basically, sites [...]
I just downloaded an tried your software.
So far I am positively impressed: the concepts/reasoning behind this project match what I expect and what I did not find in other ‘gallery’ tools.
Some remarks (I guess most of these remarks are caused by the youth of the project):
- error handling: I got an error on one of my images while converting ” File “/usr/lib/python2.6/dist-packages/PIL/PngImagePlugin.py”, line 337, in load_prepare
raise IOError(“cannot read interlaced PNG files”)” This is clearly a problem of PIL, but handling the exception gracefully (maybe log it and instead of the image put a warning) would be ideal.
- sorting: it would be great if the user could control the sorting of directory and files. E.g. in my case sort the directories DESCENDING and files ASCENDING (maybe everything needed is a two line change in PhotoAlbum.py, replacing the sort() calls with sort(reverse=True)
About error handling: I had fixed this last night for somebody else, I think, who had a fairly similar problem with this commit http://git.zx2c4.com/PhotoFloat/commit/?id=6d75a3ed642529a99227d578f74ec14bd7e03d42 . Be sure you have the latest. If this doesn’t fix it, would you send me a full stack trace of the error and the PNG you used to trigger it?
Sorting — yes indeed, it needs some more customization. I think I’m going to move to model where you can specify a config.json with various values to give to the python sorter, which can include separate settings for different directories. Additionally, each directory will be able to have it’s own config.json for overrides. This will be combined with an equally as configurable system for the javascript application (such as disabling ‘download original’). Stand by.
I downloaded the newest version from git and there is a bug:
File “/home/lombardo/t/PhotoFloat/scanner/PhotoAlbum.py”, line 189, in _metadata
self._attributes["meteringMode"] = self._metadata.metering_list[exif["MeteringMode"]]
IndexError: list index out of range
Notice the different upper/lower/cameling of MeteringMode.
Actually it’s not a problem with camel casing. You are, evidently, using a JPEG that has strange EXIF data. I based my information off Canon’s spec. Would you email me the image that caused the error?
In any case, I’ve added some guarding against screwy files:
http://git.zx2c4.com/PhotoFloat/commit/?id=fb01d666b8646161552fdeb838f83ab07b63dc7a
So now you shouldn’t get the error any more.
This looks to be at an awesome start! A feature that’s very important to me in a gallery is the ability to easily order prints of the originals from a service such as Shutterfly or Snapfish. Would it be possible to integrate that in a future version?
It would be trivially possible.
I have no interest in doing it.
PhotoFloat aims to be clutter free.
hi Jason,
another great soft
I was wondering: are you still satisfied with dreamhost? would you recommend it?
matthias
I very much am. Their support is steller and you get unlimited space. If you sign up for them, use my link:
http://www.dreamhost.com/r.cgi?423703
Por favor
Hey, you’re a beast. Very inspiration coding.
I’m developing an exciting social networking site called Vines and we are testing your software out. One question:
What do you think the best way to add user comments to the photos?
Wonderful to hear!
Because comments are dynamic and potentially lenghty, you can’t really load them statically with the rest of the photos. Instead, I think it’s best to preload all the image data, like PhotoFloat does, but then make a separate AJAX request when each photo shows. Usually someone looks at the photo for a few moments before deciding to read/write comments anyway. This way, the data is always fresh, even if they’ve left open the window for a while and flip around the photos later in the day.
So, you do things at first like PhotoFloat does, but then on each photo load, another AJAX request is made to fetch the latest comments and insert them in a div at the bottom/top/side of the photo. This is, as far as I can tell, what Facebook does in their latest photo viewer iteration.
Soon after found this software and sow all your projects I can say you’re becoming my idol man
This is the best ever photo album I ever wanted. It’s actually something I have been looking for form months. I admire your work. I am usually not sharing my photos to any social network sites so this is what I need for the blog of my little baby and I am going to post his phptos soon using PhotoFloat.
I have one suggestion for a feature that would be nice to have. Actually I am willing to start learning python so if you are not going to implement it I may try by myself, but the problem I have lack of free time.
You know that on modern job sized photo cameras there is usually possibility for recording a video as well. Those days I am keeping my short video clips together with photos and locally I manage them with Shotwell application. It would be nice I think if PhotoFloat is also supporting the cool video tags in html5 and capable of generating simple thumbnails and putting videos just among the photos. Whoever want’s to play them just click on play button otherwise they should pass just like photo on the row. It’s not necessary to have flash player fallback in case browser is not html5 capable I think. Sooner or later there would be no more such browsers. The video transcoding again can be done automatically by the script using ffmpg etc. there must be som python module suitable for this.
Let me know what you think .. I guess it would be wonderful feature … everyone is taking short video clips from time to time it’s fun.
All the best!
Hi,
I got this error when I ran main.py to update the cache:
./main.py ../web/albums ../web/cache
2012-06-22T15:05:22.771311 [walking] albums
Traceback (most recent call last):
File “./main.py”, line 18, in
main()
File “./main.py”, line 12, in main
TreeWalker(argv[1], argv[2])
File “/home/martian/mafiainc.net/Дневника на Камен/PhotoFloat – Kamache/scanner/TreeWalker.py”, line 16, in __init__
self.walk(self.album_path)
File “/home/martian/mafiainc.net/Дневника на Камен/PhotoFloat – Kamache/scanner/TreeWalker.py”, line 53, in walk
entry = os.path.join(path, entry)
File “/usr/lib/python2.6/posixpath.py”, line 70, in join
path += ‘/’ + b
UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xd0 in position 27: ordinal not in range(128)
Thanks for reporting. Fixed here:
http://git.zx2c4.com/PhotoFloat/commit/?id=ba72d7c8b79b69fde376baa45f2f932934f83942
Hi !
Great work, this is exactly what I was looking for.
However, my cron job was unable to run the program without a python exception on file names encoding during tree scanning. Almost the same as previous post but this time in files being scanned, not initial paths. And my special characters are french accentuated letters, not cyrillic.
Everyting ran smoothly when run with my user as my LANG variable was set to en_US.UTF-8 but couldn’t solve the issue when setting this value in the cron job script… I couldn’t figure out why…
Anyway, I fixed it by adding this to main.py :
import sys
reload(sys)
sys.setdefaultencoding(‘utf-8′)
This is an ugly ugly horrible hack, but I applied it regardless.
http://git.zx2c4.com/PhotoFloat/commit/?id=0cf87e26ded73a4e76eb37494e34832905c834d2
Hello, your gallery is really great – fast, slick, and dead easy to setup. The only problem I have is that the JS code apparently breaks on IE7 – a friend reports me she can’t see my gallery from her oldish browser at work (she cannot upgrade it, of course). I’ll give it a go at fixing it as soon as I find an IE7 to test with – in the meanwhile, thanks for the great work.
Hello again,
solved the IE7 problem: the Javascript engine does not allow to access strings as arrays. Here is a working patch:
http://pastebin.com/CwJzXsYP
Great work, really!
Thanks a lot for the patch. I’ve applied it here and given you git author credit:
http://git.zx2c4.com/PhotoFloat/commit/?id=598fef93fd7aafd6845b777c344b22b971fe4526
Hi, is there any possibility to add security features? I don’t want to all people see my private library. Thanks?
It works well man. Maybe you wish to move the code to github? So other people could easily participate in improving it?
It would be nice to have possibility to restrict the library for special users. Any suggestion? (apart of simple restriction by apache/nginx configuration). Thanks!