SummerOfCode2009ideas

GSoC 2009 Project Ideas

IMPORTANT: Summer of Code is about Coding! All projects do require coding skills, preferably in the C programming language (read: if you don't have them yet, you will have to get them during the project if you want to be successful)

  1. GSoC 2009 Project Ideas
    1. PDF Export Plug-in
    2. Advanced GUI for brush dynamics
    3. More Flexible Abyss Policies (including Nearest Neighbour, a.k.a. clamp) and Consistent Image Size for GEGL Resamplers
    4. Fast Adaptive Resampler Tailored For Transformations Which Mostly Downsample (GEGL)
    5. Better nonlinear resampler with built-in antialiasing (GEGL)
    6. OpenGL GPU resampling in GEGL
    7. GEGL-based file loaders for GIMP
    8. Improve Foreground Selection Tool
    9. Use GEGL for transformations in GIMP
    10. Fractal Resizing
    11. Skin (Wavelets-based) Smoothen/Sharpen tool
    12. Highlights/Shadow compression
    13. GEGL brush core concept idea
    14. IWarp as interactive tool
    15. Implement perspective transformations in GEGL
    16. Develop Batch Image GIMP Plug-in

Please note that, although you are welcome to look at what is here, this is a workspace for mentors to write down and modify their ideas, and suggestions here should not be taken as necessarily viable projects until they have been finalized. Also, the fact that something appears here does not necessarily mean that anybody has volunteered to mentor it.

Note to people who add stuff here: Please try to add information about a proposal's overall complexity and experience that could be helpful. E.G. experience with GTK+, image manipulation algorithms, web application development, ...

When suggesting a project, please include information about yourself (your name and the IRC nick you're known by in #gimp). If someone is interested in mentoring a project, please add this information as well. Note: mentors HAVE TO BE present in #gimp.

SummerOfCode2008ideas might be interesting, but please keep in mind that stuff there might have become obsolete.

A note to authors maintainers of third-party plug-ins

There are great and large plug-ins out there which do add important functionality to GIMP - for example UFRaw, the GAP plug-in collection, the new GMIC, David's Batch Processor...

If you think that there is this special new interesting (both for the developer and the users) feature that would make your plug-in even more useful, but you're lacking the time to bring it out in a timely fashion, you are welcome to suggest and mentor this project yourself. The preconditions are that you subscribe to our gimp-developer mailing list and join our #gimp channel on irc.gimp.org.

PDF Export Plug-in

PDF support in GIMP is limited to loading PDF files via libpoppler (each page does become one layer in the image) and printing to PDF via the print dialog (reducing the image to one single layer).

It should not be too complex to create a plug-in that does offer a bit more options than this - the most basic approach could save each layer to a page in the resulting PDF file. But... why not keep text layers as text? And maybe paths or vector layers as vectors in the PDF file as well? And if we have this, how do we tell the plug-in what exactly should be one one page, and what on the next one?

It could be possible that the current image structure in GIMP does make anything beyond the basic approach - one layer, one page - impractical. A plug-in working on a GEGL tree - with pages composed of subtrees, and thus a more coherent definition of what should be one a single page - might be needed for the more advanced features.

Advanced GUI for brush dynamics

The goal of the project would be creating a curves based intuitive GUI for adjusting and mixing brush dynamics along with options to save and load preset brush configurations.

The student will be working with our UI designer based on feature spec I will be writing to create powerful and yet intuitive user interface to use dynamic brush controls. Current solution is use of check grid and single scale factor per driving dynamic. This setup is limited and not suitable for large number of possible driving and driven factors.

This project is suitable for a student interested in UI development and does not require deep knowledge of image processing. Generic coding skills and experience with GTK are of greater benefit.

More Flexible Abyss Policies (including Nearest Neighbour, a.k.a. clamp) and Consistent Image Size for GEGL Resamplers

Currently, GEGL has only one abyss policy: "transparent black." Consequently, resamplers "fade" near the boundary. This makes the fact that resamplers are typically programmed in a way which is not symmetrical top-down and left-right visibly obvious; worse yet, the abyss "creeps in" in a jarring manner when an entire image is rotated, even by a small angle. Many non-geometrical transformations---e.g., gaussian blur---are negatively affected as well.

Goal: Implement alternate (better) abyss policies, allow users/methods to choose, and make resampled image dimensions independent of the resampler footprint.

Part 1: Implement "user settable flat colour/transparency" abyss policy. (Comment from Nicolas: I am actually far from certain that this is a good idea. "Fading to a constant color" at the boundary is not really something which should be done by a resampler. So, I suggest that there be a discussion about this on gegl-developer first. Really, I think that Part 2 is what the one and only way an abyss policy should be for the GIMP.)

Part 2: Implement "nearest neighbour" abyss policy. This abyss policy maintains the monotonicity property for resamplers which have it (nearest neighbour, bilinear and nohalo), and is first order (meaning that any resampler which maintains flat colour areas---like nearest neighbour, bilinear, Catmull-Rom, YAFR and nohalo---will do so near and through the boundary when used with the nearest neighbour abyss policy).

Part 3: Make the dimension of the resampled image independent of the sampler footprint. That is, fix [WWW] http://bugzilla.gnome.org/show_bug.cgi?id=549999. A choice between the center (as in current GIMP) and corner (probably better) image size conventions should be made, or it should be user settable.

Needed skills: Solid C programming + GObjects + very basic image manipulation + the usual (svn etc) should be enough for this project.

Fast Adaptive Resampler Tailored For Transformations Which Mostly Downsample (GEGL)

General purpose resamplers generally do not give good looking results for transformations with strong downsampling (for example, thumbnail creation). (Exact area) box filtering works fairly well but usually introduces too much blur. On the other hand, interpolatory methods (like nearest neighbour, bilinear, bicubic or lanczos) all behave more or less like nearest neighbour, hence produce unsatisfactory results, when the downsampling ratio is high.

Goal:

Implement a "less blurry and faster box filtering" general resampler which is meant for situations in which (locally) downsampling is more typical than upsampling.

Technical approach:

Use exact area box filtering with a box with dimensions which depend on (an approximation of) the Jacobian matrix of the transformation. That is: When a gradient is large (directional upsampling), the box, in that direction, has diameter 1, but when the gradient is small (directional downsampling), the box, in that direction, has a diameter which increases but more slowly than for an exact area method. The choice which makes the "seam" between downsampling and upsampling smoothest is such that for "infinite" downsampling the diameter of the box is 1/e of what it would be for a typical approximate exact area box filtering resampler. Because, at heavy downsampling rates, less pixels are used to compute the averaged output value that with full-blown box filtering, there should be a substantial speed gain compared to standard box filtering---which generally is only implemented for resizing, not general resampling. Also, the downsampled image should look less blurry.

Technical issue:

To compute an "output pixel value," the resampler needs geometric information about how nearby "input" pixels are mapped by the transformation. In mathematical terms: The resampler needs to create or have access to (an approximation of) the transformation's Jacobian matrix (or transformation gradients). This can be obtained by applying the transformation to nearby pixel locations to get the approximation using finite differences. However, for most often used transformations, gradient information is easily obtained from the parameters of the transformation. Consequently, a workaround would be to have this resampler default to bilinear unless the calling method communicates gradient information to it (easily computed for affine transformations---this include image resizing, the most important case). At this point, I (Nicolas) favor the workaround, for the following reason: GEGL's current "pure" demand-driven structure means that resamplers have no information whatsoever about what other nearby locations are being resampled, and consequently there would need to be major changes to make this info available for arbitrary transformations (with, most likely, serious speed impact).

A higher quality alternative, suitable for both downsampling and upsampling (but still tuned for mostly downsampling), would involve gluing a (simplified?) variant of Craig DeForest's PDL::Transform method to the piecewise linear reconstructor part of nohalo level 1 in order to obtain a high quality exact area method.

[WWW] http://pdl.sourceforge.net/PDLdocs/Transform.html

Needed skills: Solid C programming + GObjects + some familiarity with image manipulation and/or variational grid generation + the usual (svn etc) should be enough for this project.

Better nonlinear resampler with built-in antialiasing (GEGL)

Currently, the first level of the simplest version of a novel family of resamplers, nohalo, has been programmed for GEGL (note that there is a further simplification to the scheme for speed which has not been implemented). Within gegl, it is currently named gegl-sampler-sharp.c. Nohalo is fast and looks good (Nicolas is very biased!), but if the enlargement ratio is more than about 2-3, jaggies start appearing if the image is not smooth. Also, it does not do so well with "text like" images. Adding more "NEDI-like" subdivision levels will make the result look both smoother and slightly sharper. In addition, there is a custom smoother which makes the overall scheme have the property that an image which locally consists of a 1 pixel wide diagonal colour on a constant colour background, when resampled, is also locally constant on diagonals parallel to the original one after one subdivision (if you think about what this means, you'll realize that this is some pretty strong antialiasing). The amount of this special smoothing being applied is easily understood as an "antialiasing" parameter, which should be user-settable. In addition, with the pre-smoother, the method should be appropriate for mild downsampling. And then, GEGL would have a "sharp" resampler with an antialiasing "slider."

Goal: Add more levels to nohalo, add the pre-smoother (nohalo + pre-smoother = snohalo), add the antialiasing parameter.

Note: Because nohalo and snohalo are brand new schemes---and only nohalo level 1 is available through GEGL (VIPS svn trunk also has snohalo level 1)---I have put some enlargements on the web (of downsampled images provided by Craig DeForest). The images given for nohalo are almost exactly the result from nohalo level 5, but the snohalo ones are more approximate representations of the true snohalo level 5, which will be slightly more antialiasing and blurry. If you know of anything which is obviously, or even arguably, better for upsampling for both test images, and is compatible with a pixel based demand-driven environment (without being painfully slow) please let me (Nicolas Robidoux) know. Nohalo level 1 is almost as fast as bilinear, so we're hoping that level 3 will not be much slower than bicubic. All images are in [WWW] http://web.cs.laurentian.ca/robidoux/misc/. See the [WWW] http://web.cs.laurentian.ca/robidoux/misc/README for descriptions.

(Nicolas: There is a student who is already involved in this project.)

Needed skills: Solid C programming + the usual (svn etc) + familiarity with image filtering algorithms should be enough for this project.

OpenGL GPU resampling in GEGL

This is a very challenging project: It requires software engineering (contribution to formulating how GPUs should interact with GEGL) and tricky programming interfacing multiple environments (GEGL and OpenGL) without much guidance from a mentor (implementation of the back end interface to GPUs).

About the lack of mentor: Nicolas and a close collaborator can mentor the nohalo/snohalo parts, but not with the GPU backend for GeglBuffer, which is the hard part.

The nohalo (gegl-sampler-sharp.c) resampler has been successfully programmed in HLSL/DirectX. For DVD to HDTV upsampling, it gets a performance comparable to hardware bilinear (between 12% and 93%). As a baby step toward GEGL rendering on the GPU, it may be a good thing to move nohalo to OpenGL and then interfacing the GPU version with GEGL. The "higher" versions of nohalo are currently being programmed in HLSL, so these could be moved to OpenGL + GEGL as well. Yet another variant, snohalo, particularly suitable for "text like" CG graphics, is also easily implemented.

(Nicolas: I don't have any idea how to interface GPU-based resampling with GEGL. This project is likely to require a very resourceful student and/or the input of more "senior" GEGL developers.)

Here is some feedback from Øyvind Kolås (by way of the gegl-developer list):

(I (Nicolas) am not qualified to guide a student through the key part of this project. I recommend that only a student willing to do the heavy lifting on his/her own should take it. Also, the student may as well simply implement plain vanilla hardware bilinear resampling: the backend is the key, what it's used with, initially, does not really matter.)

Figuring out how to properly structure the GEGL GPU back end should involve a lot of interaction with primary GEGL developers through the gegl-developer mailing list, in order to get their help, feedback, and more importantly, so they can vet it, which will make it more likely that it stay within GEGL as developed within the GSoC.

I (Nicolas) suggest that applicants to this project provide as much detail as possible about how to plan to implement the back end in their application, with details and justification, and alternative approaches if any. This is especially important because this will be mostly a self-directed project, and because it consists of putting together a back end essentially from scratch.

Needed skills: OpenGL + GPU programming + C programming + GObjects + whatever is needed to build the GEGL/GPU interface (no clue what this would be: could be a lot) + the usual (svn etc)

UPDATE ON THIS IDEA:

There is a good chance that this project, as it was initially described, was ill-conceived (by me, Nicolas; I apologize for this).Here is a more reasonable general direction, courtesy of Øyvind Kolås by way of the gegl-developer mailing list:

Implementing the framework for letting GEGL do processing on the GPU is an interesting topic. Last year GeglBuffer hadn't matured enough to advise someone to start the task; now it is. Here is a plan containing my thoughts on what is needs to be done from a GEGL perspective.

GPU based GeglOperations would be needed to implement and test GPU based resamplers for GEGL. Doing only one GeglOperation in a graph on the GPU will most likely be slower than doing all operations on the CPU; we need multiple operations to offset the performance cost of the CPU->GPU GPU->CPU transfers. Hopefully it is feasible to use shaders on tiles (or compose and split up larger buffers on the GPU), this would make it possible to keep track of where the valid tiles for a GeglBuffer reside; CPU or GPU side (using two revision counts). I hope this would make us able to use the same GeglBuffer handle for both CPU and GPU access in the same process minimizing the changes needed to the rest of GEGL.

The type of GeglOperation that is most important to get going is GeglOperationPointComposer operations or as a starting point the op gegl:over which is the normal layer mode in GIMP, all other such ops behave the same with regards to input and output buffers but use a different formula.

GeglOperationPointFilter is even simpler and would accommodate for the needs of operations like brightness contrast, desaturation and more. None of these would be using resamplers but they are the bulk of operations that needs to be performed.

The operations that are likely to need a resampler (GPU built in or custom) are the more complex ones that can have varying input and output buffer sizes. The most needed one is the transformation class of operations. (Currently only affine transformations are in GEGL, but we need perspective as well to avoid a feature regression from GIMP which does this.)

GEGL-based file loaders for GIMP

Since GIMP is making good progress towards supporting higher bit depths than 8 bits per channel there is a need for loading image data of file formats that supports higher bit depths, such as PNG. TIFF and OpenEXR. The exact approach at this is still to be decided but will only require a small set of discussions and agreements between core developers. Once the details are sorted out this be a fun and important GSoC project.

Improve Foreground Selection Tool

Currently SIOX, the algorithm underlying GIMP's foreground selection, is doing a hard segmentation. This means, a pixel is either foreground or background. It would be better if the foreground selection could implement a soft segmentation (also known as alpha-matting). This would improve the performance of the tool for areas where background and foregound overlap, such as in fine-grained, highly-textured, or semi-transparent objects like hair, clouds, or iced objects.

The papers on SIOX present a solution to this, that includes the so-called detail refinement brush (DRB). Examples can be seen at [WWW] http://www.siox.org/preview.html

Goal: Improve foreground extraction by introducing "soft segmentation".

Part 1: Implement the detail refinement brush as described on [WWW] http://www.siox.org

Part 2 (optional): Experiment with different ideas to reduce the user interaction needed for the detail refinement brush.

Technical approach:

A Java reference implementation of the detail refinement brush and conceptual documentation is available at: [WWW] http://www.siox.org/details.html

The project either requires translation of some Java code to C or re-implementation from the conceptual descriptions followed by an adaptation of it to the SIOX implementation in GIMP. The implementation of the DRB in GIMP as brush might require some GIMP interface (both API and GUI) adjustments.

Experience requirements: GNU C, GObjects, GTK, UI building, basic image processing skills

Nice to have but not required: scientific interest in image processing (for part 2).

Use GEGL for transformations in GIMP

With GEGL getting more and better scaling and transform algorithms, it is about time that GIMP starts to use these, This project is about changing GIMP to make use of GEGL operations for scaling and other transformations (rotate, shear, ...).

Comment from Nicolas: IMNSHO, it is necessary to address the abyss policies and "shifting corner" issues before this can be done. (See above "...Abyss...")

Fractal Resizing

Currently, GIMP implements a few resizing methods which has limitations. One way to give GIMP "production-ready" resizing features is to implement Fractal Resizing method, as described here : [WWW] http://ecommons.uwinnipeg.ca/archive/00000026/ and implemented here : [WWW] http://search.cpan.org/~rkobes/GD-Image-CopyIFS-0.25/CopyIFS.pm The GD implementation can be a starting point, and the function can be used for various resize operations within GIMP / plugins.

Skin (Wavelets-based) Smoothen/Sharpen tool

Give the user the possibility to quickly use this method to smooth/sharpen textures like skins Smooth : [WWW] http://registry.gimp.org/node/11742 Sharpen : [WWW] http://registry.gimp.org/node/9836 It can be done as a GEGL operator implementing on-the-fly wavelets decompose / blur one order of decomposition / recompose.

Part 1 : Port Wavelet decompose as a GEGL operator

Part 2 : Implement the smooth tool by blurring only one order of decomposition and recompose

Part 3 : Implement the sharpen tool

Part 4 : Integrate them in UI as an option for "Convolve" tool ([WWW] http://docs.gimp.org/en/gimp-tool-convolve.html)

Highlights/Shadow compression

To improve poorly exposed or high contrast images in a better way the existing Retinex function in GIMP, it is possible to implement such an algorithm, very well described in this paper : "A Locally Tuned Nonlinear Technique for Color Image Enhancement" [WWW] http://www.wseas.us/e-library/transactions/signal/2008/2007-295.pdf

It can consist in a GEGL operator. An interesting part would be to try out different blurring techniques to control the halo.

GEGL brush core concept idea

This is a brilliant idea from Alexia Death which could be a bit heavy to be done within the scope of GSoC, but who knows ...

[WWW] http://lists.xcf.berkeley.edu/lists/gimp-developer/2008-January/019496.html

A concept for brush core structure from events viewpoint.

* Events are recived from X

* Events are evaluated for:

Events are handed to paint core(if a paint tool is active).

(SEE the original message for more details)

One of the biggest benefits is that one core can handle many very different paint tools.

Paint tools can become plug-ins if desired, they only need to provide to core 3 functions, evaluate-event, re-evaluate-event, render-stamp. This makes creating new paint tools extreemly simple and lets people make tools that are free in setting their dynamics to whatever they need enabling very interactive and lifelike brushes. One tool I see is offspring of current vector brush that lets one save all dynamics curves it desires mapping them to any stamp parameter.

Support for current brushes can be enabled through always available default plug-ins.

This paint core does not handle any interaction with the canvas, only the shape and path of the tool. For physical media simulation a canvas that supports and can render different "paints" is needed. The canvas interaction is something that happens in the different types of canvas.

It goes along with the idea of non-destructive editing with GEGL, and further brush path editing :

by storing the paths separately from the canvas modification itself, one could change brushes on an existing path, apply plugins on them, refine the paths, etc ...

IWarp as interactive tool

Currently interactive transformations are available in form of a filter called [WWW] IWarp which has a tiny preview area only. Atomic undo/redo is not possible too. Having a true interactive tool based on this filter would make it much more useful and convenient to use.

There already is a preliminary patch from Tor Lillquist to start from: [WWW] http://lists.xcf.berkeley.edu/lists/gimp-developer/2008-February/019651.html

The discussion linked to above will give a good insight into further steps to be taken. The student will have to work with usability team in order to meet quality requirements.

Implement perspective transformations in GEGL

This is present in GIMP, but missing from GEGL.

Develop Batch Image GIMP Plug-in

I plan to create (or help create) an easy-to-use, gui-based GIMP batch image processor. The processor will allow users to chain together actions to be performed on a series of images matching a wildcard sequence. I'll start by creating a simple script to apply scaling and cropping to multiple images. Once I have a grasp on the code base and the challenges I'll face, I will create a more sophisticated plug-in that will use the undo history of the current image to alter multiple images identically. I plan to program in Python, possibly using the GIMP Python library [WWW] Gimp-Python.

The batch image processor will contain three main parts: the user interface, the internal interface, and the image retriever. All three parts will be loosely coupled for easy reuse and replace. If feasible, the user interface will dynamically populate itself with suitable tools. The internal interface will accept operations to be performed on a set of images, package them up in a Command object, and hand them off to the image retriever. The retriever will walk through a directory structure, using its command on every suitable image it finds. The directory-scanning image retriever will be able to vary independently of the internal interface, so that others can replace it with their own. I anticipate that structure will yield a flexible, extensible, and reusable program.

last edited 2009-04-03 13:22:21 by EvanKroske