The Source for Java Technology Collaboration
User: Password:

Chris Campbell

Chris Campbell's Blog

Easy 2D/3D Mixing in Swing

Posted by campbell on October 12, 2006 at 12:47 PM | Comments (14)

When You're A Jet...

A couple weeks back, I had the privilege of speaking to the Java Programming Club at Monta Vista High School in Cupertino, CA. I'm still amazed at the turnout: about 25 students showed up, all with varying levels of experience with Java, and all apparently willing to endure my so-so demos on their lunch hour. The thing that grabs me though is, if they have a Java club, does that imply that they have other similarly specialized programming clubs too? Do the Ruby, Haskell, Java, and Lisp clubs all confront each other just outside school grounds when they have a score to settle, รก la Sharks v. Jets?

Anyway, I'd like to thank their president, Anshul Bhagi, for inviting me. It was fun! I promised to post the (rather scattershot) slides from my talk, which mainly revolved around JDK 6 (the new SplashScreen API, 2D/3D mixing in Swing apps, how to get involved, etc). I asked the students if anyone had tried out JDK 6 snapshots, and was surprised to find that none of them had. All the more reason to pose a challenge to the students... By the end of this school year, I'd like to see at least two of them contribute fixes to the JDK 7 project. I know we've had a number of contributions for JDK 6 from the university level, but none that I'm aware of at the high school level, so I think that would be pretty cool.

It's A Bit Of A Pain

To get the students interested in doing some basic 2D and 3D graphics programming, I showed a couple of existing 2D/3D mixing demos, like Aerith. The great thing about demos like Aerith and its precursor Twinkle is that they're polished and they even behave like "real apps" (bells and whistles included), so ooh/aah factor is never an issue. But of course, we all know that the road to real-app-hood often involves thousands of lines of code, so if you're trying to learn from their source code, it's often difficult to locate the essence of the demo. Twinkle's cool and all, but the source code can be daunting to a newcomer (no fault of Romain's).

Quite frequently I hear from developers who are familiar with Swing and Java2D, but haven't the foggiest idea where to begin when it comes to using JOGL in their application. It's especially difficult for developers who have no 3D graphics programming experience to be thrown into the world of OpenGL so abruptly. This time around, I decided to develop my own demo from scratch (with only a few, sparse, documented classes) to help show how to use some complementary technologies in the same application:

Hans thinks we're incapable of writing a 2D/3D demo that goes outside the realm of photo viewers, and you know what, he's probably right (at least for this week). So, mea culpa, I wrote yet another photo-centric demo, but this time I stripped out all that pesky application logic. All that's left is a minimal demo that I hope is easy to understand for beginners. Here's a screenshot of "PhotoCube" (click to enlarge):


Or, why not run it yourself (the app itself is unsigned and requires JDK 6):

With OpenGL-based Java2D pipeline enabled
(should be faster than the other option)
With OpenGL-based Java2D pipeline disabled
(useful for comparing performance between the two,
or if you have trouble with the first link)

And finally, here's the source code (NetBeans project included).

The core functionality can be found in the DemoPanel class. The timingframework-specific code is isolated to the DemoPanel.initTimers() method. You'll note that I've demonstrated a couple different ways to use the timingframework API to animate various rendering elements, some simple and some complex. It's unfortunate that the code is so verbose; it's certainly better than doing it by hand, but there's still plenty of room for simplifying the API. Chet and others are looking into ways to reduce the amount of code needed to write common animations, and some tools support is possible, so I would expect this to become much simpler in the near future.

(As an aside, one thing I'd like to see is something like a RepaintingObjectModifier, because so often in Swing applications all you want to say is, for example, "animate the location property of this JComponent from here to there, and call repaint() upon each timing event". Right now you have to override ObjectModifier or TimingTarget to get this common behavior, which just adds to the verbosity.)

The 2D/3D mixing code is also in that class, look for the render2DBackground(), render3DScene, and render2DForeground() methods. This code is illustrating how you can override those 3 or 4 methods provided by the CompositeGLJPanel class (also included in the source code) to easily include both Java2D and JOGL rendering elements in your user interface. I'd really like to see something like CompositeGLJPanel included as a convenience class in one of the JOGL utility packages; GLJPanel itself can be really scary beast to someone just learning this stuff for the first time. CompositeGLJPanel lowers the barrier a bit and handles a lot of the common boilerplate code needed by simple applications. Let me know if you have any suggestions or ideas for improvement.

Finally, check out the PhotoCube class, where I demonstrate the nifty TextureIO utility classes. (They're not part of the official JSR-231 spec, but rather are part of the com.sun.opengl.* packages that ship with the JOGL 1.0.0 distribution.) These classes work just like my other favorite API, Image I/O, and really make it easy to load an image from disk (or any other location) into an OpenGL texture. From there, you can use the Texture class, a simple wrapper object for OpenGL textures, which again provides an easier and slightly more object-oriented approach to dealing with OpenGL (a decidedly non-OO library). Enjoy...

Okay, I swear, I'm really done now, once again proving that I'm incapable of short-and-sweet...

In my ears: Faust, "IV"
In my eyes: Kenzaburo Oe, "The Silent Cry"

Comments are listed in date ascending order (oldest first) | Post Comment

  • Really exciting stuff, especially watching it run so smoothly with my CPU usage at 0%!!! Still, I have one concern which is that it doesnt seem to like it when I resize the window. Does this affect all OGL accelerated windows? Is it possible to get windows that resize smoothly?

    Posted by: benloud on October 12, 2006 at 08:31 PM

  • Hey Chris--really beautiful demo! One question: is it possible to have Swing components render on "top of" JOGL components? Wondering if you could do something like have a JTable on one of the faces of the spinning cube ("*sigh*...they're never satisfied..."). Second--suggest you bundle jogl.jar and timingframework.jar with the sources, I had to pull those separately. Nice work! Patrick

    Posted by: pdoubleya on October 13, 2006 at 12:19 AM

  • Works great, thanks for the demo. By the way, its a bit silly having two web start links for the same program, but I know its necessary since pipelines can't be selected programmatically. I've talked with you & Dmitri about it before, but maybe this example of the problem will raise some discussion about how it can be solved...?

    Posted by: commanderkeith on October 13, 2006 at 07:47 AM

  • @benloud: What OS are you using? Are you referring to the noticeable lag that occurs upon resize? If so, then it's due to the way that I'm reloading the photo textures from disk after resize. There's a "REMIND" comment about it in the code, which I've purposely left as an exercise for the reader :) It's not something that should affect all OGL windows, e.g. regular Swing apps that use the OGL-based Java2D pipeline should resize quite smoothly.

    @pdoubleya: Actually those two jars are already in the lib folder of the source bundle. If you used NetBeans to open the project, it may just be that it got confused about the location of those jars.

    Re: your first question... You can slap (a snapshot of) a JTable or similar on a 3D surface using Swing and JOGL, but it's quite a hack at the moment, and making it so that it's interactive and not just a snapshot would take quite some work. See XTrans on the JOGL Demos page for a proof of concept. It would be cool if this were more feasible, perhaps somebody out there could explore this further as another part of the whole JOGL/Java2D/Swing interop story. (The LookingGlass folks are exploring ways to make Swing components work in 3D environments, which is yet another approach.)

    @commanderkeith: I could've bet someone $100 that you would be mentioning that issue here :) As we've said before, we know it's currently less-than-convenient, but we're thinking of ways to allow for selecting pipelines at runtime. It's a hard problem, but believe me, we're looking into it. It won't be in 6, but maybe in 7.

    Thanks, Chris.

    Posted by: campbell on October 13, 2006 at 09:21 AM

  • I'm such a bore, I know :) Nice slides by the way, I didn't know how nifty the splashscreen API would be, woo-hoo!

    Posted by: commanderkeith on October 13, 2006 at 05:59 PM

  • Yep Chris you were right ofcourse. I completed the little exercise you left in the code comments and loaded the texture data on application start up, and sure enough the resizing became nice and smooth. Great stuff!

    Posted by: benloud on October 14, 2006 at 01:18 AM

  • Chris--thanks for your response. Can I ask a simple, probably stupid, question? Why is it that Swing, being a lightweight framework using Java2D to render all components, can't just have a different canvas to render to, which happens to be a 3D canvas which translates graphics coordinates into 3D space? Eg. speaking at a high level, why is it that I can't provide a UI delegate that draws onto a 3D canvas instead of a 2D canvas an "be done with it?" Thanks, Patrick

    Posted by: pdoubleya on October 14, 2006 at 01:18 AM

  • I think its difficult because interaction with the components requires the transformation of the mouse click coordinates as well.
    You may find it useful to read how this feller did a 3D UI in swing:

    Posted by: commanderkeith on October 14, 2006 at 07:30 AM

  • Patrick: The Looking Glass did just that. You can render Swing UIs (interactive UIs) on 3D objects.

    Posted by: gfx on October 14, 2006 at 09:56 AM

  • Well, putting mouse interactions aside for a minute... (just a warning, I know absolutely nothing about OpenGL, although these sorts of demos are inspiring me to learn) The demo is great, but the images being transformed in 3D space are static images. I wonder if it's possible to use Java2D to render directly to an OpenGL texture. That would be the first step in having Swing component in 3D space. I know I can create a TextureData from a BufferedImage, but changing the BufferedImage wont affect the texture right, so a new TextureData would have to be created for every frame. I'm thinking of something like texture.getGraphics2D(). Is this feasible?

    Posted by: benloud on October 15, 2006 at 09:39 AM

  • @benloud; well most GUIs are static too. You only need to update the texture if the swing component is dirty. I am even calculating mipmaps - and the 3D swing frame doesnt feel slower. I am not a Java2D expert but if the JOGL Texture could provide Graphics2D it would be much cleaner (and we would save one evil "new" in the rendering loop ;) ). Try to post your suggestion into the JOGL forums at regards, bienator

    Posted by: bienator on October 15, 2006 at 10:39 AM

  • @benloud, @bienator: I don't think we would want to put a getGraphics() method directly on TextureData, because that class can encapsulate other types of data (e.g. raw NIO Buffer data) that aren't Java2D related. However, there is already a convenient Texture.updateImage() method that allows you to update a texture, and there's no need to create a new TextureData or BufferedImage each time. For example:
        BufferedImage img = new BufferedImage(...);
        TextureData texdata = TextureIO.newTextureData(img, false);
        Texture tex = TextureIO.newTexture(texdata);
        while (!done) {
            Graphics2D g2d = img.createGraphics();

    There is also a Texture.updateSubImage() method that allows you to update a texture at a given (x,y), but I noticed that there isn't one to update an entire (x,y,w,h) subregion. It would be great if someone could file an RFE on JOGL, that way it would be easier to use the kind of code I showed above to update only the dirty regions of the texture (in conjunction with Swing's RepaintManager).

    Thanks, Chris.

    Posted by: campbell on October 16, 2006 at 10:46 AM

  • Hi Chris, when I run the demo, NetBeans outputs: "Could not enable OpenGL pipeline for default config on screen 0" What does it mean?

    Posted by: ylzhao on October 19, 2006 at 12:23 AM

  • @ylzhao: That message means that the OpenGL-based Java2D pipeline is unavailable on your system. It could be caused by one of a number of reasons, e.g. are you using an unsupported graphics board, like Intel? Or maybe you don't have the latest drivers installed? You can get more details by setting the following environment variable before running an app, for example:
    % set J2D_TRACE_LEVEL=4
    % [jdk1.6.0]/bin/java -Dsun.java2d.opengl=True -jar SwingSet2.jar

    Thanks, Chris.

    Posted by: campbell on October 19, 2006 at 12:05 PM

Only logged in users may post comments. Login Here.

Powered by
Movable Type 3.01D