Notes on the future of Open RAW formats, and a look at DNG (by Stuart Nixon)

[Legal preamble - these are my own opinions only. I welcome and will happily publish valid corrections.]

Good morning.

There seems to be some confusion about DNG. Speaking both as a software developer and a photographer (and having had a lot to do with patent issues and image standards like JPEG 2000), I'd like to add my $0.02 to the debate.

DNG IS NOT THE ANSWER

Let me first make one thing clear: DNG is not an open standard for defining and storing all needed RAW camera information.

DNG makes the RAW format problem worse, not better.

DNG is not an open standard in that it does not document all the essential information contained in current RAW format files like NEF and CR2 (which also don't document this information).
In many ways, DNG can be viewed as simply yet another RAW format with undocumented information - except that DNG has the added risk that information can be lost during conversion to/from DNG and other RAW formats.
From a software developer point of view, DNG is a step backwards. From a camera manufacture's perspective, DNG does not address the missing elements in EXIF.
From a photographers perspective, DNG is dangerous because people believe they are storing for the future with the format, when nothing could be further from the truth.

THE SITUATION TODAY

A quick recap of the current state of the RAW format market is in order:

  • Aldus defined the TIFF 6.0 specification in 1992. Aldus was taken over by Adobe, who now control the format specification.
  • TIFF uses an extendable structure called IFD, which stands for Image File Directory. This makes it possible to add new content to TIFF files, and allows other formats like JPEG to be embedded into TIFF. The TIFF IFD structure can also be embedded into other formats like JPEG.
  • There are many, many sub-variations of TIFF, to the point that no one product (including Adobe's own products) reads and writes all the TIFF variations and permutations. And this is just for official TIFF format files - no product comes even close to reading all the different extended versions.
  • TIFF has many technical problems including being hard to implement, no standard support UNICODE support and limited to 2GB file size. Various hacks exist to address some of these issues, making technical consistency even worse.
  • Over the years, sub-variations, standards and extensions to TIFF were created by different groups. Some of the more important ones are:

IMPORTANT FORMATS AND STANDARDS FOR PHOTOGRAPHERS

  • IFD
    Deserves a special mention because it is the way TIFF can be extended.
  • IRB
    Various Image Resource Block (IRB) extensions to TIFF and other formats used by Adobe. Of particular interest are the IRBs used to store thumbnails and IPTC data.
  • IPTC-NAA
    A standard - predating TIFF - for storing meta data, used by the media industries. Adobe defined an IRB to store meta data in the IPTC-NAA format. This is the most common way meta data like copyright is stored in TIFF files, despite a slow move to XML.
  • EXIF
    A vitally important standard by Japanese companies, which started life as a standard way to store camera information, using the TIFF IFD structure, in JPEG. Quickly morphed into a general way to store camera information into TIFF files as well, by simply dropping the EXIF structure into TIFF files. A great deal of useful camera information is encoded into EXIF, with one glaring exception: because the EXIF standard was designed to encode meta data for JPEG files, it did NOT describe a way store/encode RAW data (this is the main issue that DNG attempts to address).
  • EXIF MakerNotes
    The curse of the photo industry. Because EXIF did not contain all camera information, it included a way for manufacturers to extend EXIF with manufacturer specific data - the "MakerNotes" section. This quickly grew in undocumented and incompatible ways, as manufacturers used MakerNotes to store information like the RAW data. 99% OF RAW FORMATS ARE SIMPLY TIFF + EXIF + MAKERNOTES. THE PROBLEM IS THAT THE MAKERNOTES ARE NON-STANDARD AND NOT-DOCUMENTED.
  • TIFF/EP
    An ISO standard for "Photography - Electronic still picture imaging - removable memory - Image data format" to quote the standard. A credible ISO attempt to standardise the blizzard of TIFF information. However, it is incompatible with EXIF in several crucial areas, and uses a totally different way to store thumbnail IFDs, making TIFF/EP problematic. Most products read TIFF/EP but use EXIF in preference.
  • .CRW, .CR2 and .TIF Canon RAW photo formats
    There are three main formats used by Canon for their RAW photo format. The oldest format is the .CRW format, which is known as the Canon CIFF format. This is used by the D30, D60 and so on. Canon then swapped to a new RAW format, with a .TIF extension, used by the 1Ds. Probably because of confusion between RAW TIFF files and normal TIF files, Canon then swapped to the .CR2 extension with the 1D MkII. Canon .CR2 and RAW .TIF files are essentially the same format. Canon’s RAW format is internally well structured and consistent between different cameras. The latest format draws heavily from the TIFF IFD format. Probably the purest and best of the various RAW formats in existence (albeit still undocumented by Canon). Uses a clever, lossless, JPEG encoding technique for RAW data.
  • .NEF
    Undocumented Nikon RAW format (actually two formats; one for scanners and one for DSLR's). Essentially EXIF format but inside TIFF rather than JPEG headers, and again with undocumented extensions. Unlike other manufactures, Nikon keeps tweaking NEF to the point that each camera and each Nikon editor product generates different, sometimes incompatible, NEF files. With Nikon NEF the best approach is the write-protect the original NEF file and never modify it to ensure maximum compatibility. Uses two fairly clumsy techniques for compressing RAW data; e.g lossy or lossless. Where as most manufacturers don't document their RAW format, Nikon went one step further and started encrypting crucial data in NEF files. In a curious coincidence, Mikon started encryption at the time newer Nikon cameras were released with very poor high ISO noise problems which must be removed in post processing.
  • DNG
    Yet another TIFF variation, this one pushed by Adobe.

    Has a number of advantages and disadvantages:

    • Good:
      • Defines a standard way to encode raw sensor data
      • Defines a standard way to transform colour
    • Bad:
      • Takes the proprietary RAW format problem and makes it much worse, with MakerNotes et al now being moved about.
      • Offers the option to include the entire original raw file - but unless everyone actually uses this (thus doubling file size) just gives a false sense of security.
      • Makes no attempt to define a standard way to store all the data currently stored in undocumented ways in MakerNotes.
      • [Section removed on clarification of Adobe RGB profile status]
      • Takes MakerNotes and moves them into another format, both perpetuating the problem and making it worse (as decoders often rely on absolution location information in Makernotes)
      • Makes no attempt to extend EXIF or TIFF/EP in a coherent fashion.
      • Controlled by a single manufacturer - Adobe - who given don't have a good history of managing the TIFF standard or allowing software developer of Adobe controlled software profiles or standards.

In addition to the above, other format issues deserve special mention:

  • Thumbnails
    There is a special place reserved in programmer hell for the authors of the various ways that thumbnails can be stored in TIFF related formats. Thumbnails are vital (obviously). Yet there are many ways to store them. Sometimes a photo will contain 4 or 5 thumbnails - all of them different and some of them not even matching the image any more. Because thumbnails are so important, software vendors have ended up creating their own catalogs for thumbnails. So each product has its own way of storing things. Thumbnails are currently stored in at least the following ways: TIFF thumbnails. A Adobe sub-IFD for thumbnails. TIFF/EP thumbnails. A different (and reverse to the above) standard. EXIF thumbnails. Yet another way to store them. NEF/CRW thumbnails. More ways to store thumbnails. MakerNotes thumbnails. Often also stored in makernotes. Photoshop IRB thumbnails. Another common thumbnail storage method.
  • JPEG 2000
    Despite a name similar to JPEG, the new JPEG 2000 is a totally new image format, using wavelet image encoding/compression instead of DCT encoding as used by JPEG.

    Very unlikely to be used for on-camera encoding for a long time, for a number of reasons:

    • Wavelet space encoding style not ideal for on-camera sensor encoding
    • Very complex to implement in fast / low power hardware
    • Patent risks associated with any new format.

    The core JPEG 2000 standard unfortunately does not define a standard Meta format, although it does define a way to add new information to a JPEG 2000 file. One extension to JPEG 2000 (so-called because extensions are not part of the core format and thus don’t have to be supported by JPEG 2000 applications) is the “JPX” file format, which is a superset of the "JP2" JPEG 2000 file format. JPX files enable definition of an ICM within a JPX file, and also have a detailed definition of storing Meta data using XML.

    In addition, it is possible that other formats (EXIF, IPTC et al) will simply be wrapped and stored in JPEG 2000 files. Given the many Meta data standards, and the lack of a core standard Meta definition within JPEG 2000, unified and generic support for meta data within JPEG 2000 is likely to be some time away. However (and despite its complexity), the JPEG 2000 Meta data stored in XML hold promise for the future.

  • XML
    Another way that Meta data can be stored is using XML. This can be embedded in the photo file itself for some formats (for example JPEG 2000), or stored as a "sidecar" file associated with the photo. The Adobe XMP format is one of several specific meta data definitions in XML. In theory XML offers considerable promise. In practice XML is currently implemented with many different “styles” (different ways of recording information), and a full XML implementation for meta data can be both quite slow and large. Not withstanding these limitations, XML is probably going to be the main standard in the long term for storing meta data.

WHAT DOES EVERYONE WANT?

We all want standards for photos, but for different reasons:

  • Photographers:
    • Want a way to store photos that will be around in 50 years.
    • Want to be able to use any product to edit any photo.
  • Large software vendors:
    • Want to have the standard under their control
    • Want a legal hold over potential competitors
    • Want to limit use of photos on competitive OSs/products
    • Want hidden "submarine" patent/legal/copyright control over standards
  • Small software vendors:
    • Want a common and uniform standard
    • Want protection from legal actions from large software vendors.
  • Enlightened camera companies:
    • Want and will use open standards like EXIF
    • Must be able to extend the standard as technology evolves
    • Don't want to have class actions against them in future years by people losing access to their photos.
  • Other camera companies
    • See formats as a competitive edge
    • Want to make money out of their own software and compete with software vendors

REAL WORLD ISSUES

There are some realities to consider:

  • On camera hardware:
    • Lots of effort goes into the on-camera hardware.
    • Hardware level support for IFD structure and JPEG encoding
    • Impractical to force a standard that does not do these on to camera companies.
    • Cameras evolving at a tremendous rate - can not be held up by standards
    • Likely to have major changes that will break standards
  • On standards:
    • XML is nice, but not going to make it onto the camera for a while.
    • JPEG 2000 is years away from common camera level use.
    • The TIFF IFD structure is horrible, but given it is widely used at the hardware level it is really the only way to go for now.

RECOMMENDATIONS

  1. Extend EXIF to standardise storing of RAW sensor data
  2. Extend EXIF to store common information currently stored in MakerNotes.
  3. Allow/encourage small software vendors (who don't have a particular axe to grind) to be involved
  4. Include a standard color profile/matrix for raw-to-calibrated conversion.
  5. Require all involved parties to waive legal/copyright control over widely used profiles and formats.
  6. Taking a sympathetic approach to the difficulties that camera manufacturers have with standards given rapid technology evolution and hardware design constraints.

SOLUTION

An open consortium to define a standard (EXIF 3.0 perhaps?) to store common camera information, by extending the EXIF standard.

It should include:

  • Compatiblity with EXIF 2.x
  • Ways to encode RAW data using JPEG lossy/lossless encoding that is easy for camera manufacturers to migrate to.
  • Additional tags to store all the stuff hidden in MakerNotes like:
    • Focus points
    • Sensitivity curves
    • White balance values
  • Standard transformation matrix conversions
  • Spectral sensitity curves
  • Recommendation of a non-manufacturer specific RGB profile wide colorspace to move away from Adobe RGB. Ditto for CMYK profiles et al.

I trust this (rather long) post is of interest.

Kind regards,

Stuart

Stuart Nixon – Mon, 2006/04/03 – 10:00am

Stuart Nixon published this

Juergen Specht – Mon, 2006/04/03 – 9:03pm

What does everyone want? I

What does everyone want?

I think Stuart needs to expand his scope of "everyone" beyond photographers, software vendors, and camera makers. A large part of the ecosystem, which seems to have been omitted by accident, ignorance, or design-- are the people who buy, manage, and produce photographic imagery.

This group includes archivists, image banks, designers, art directors, publishers, pre-press, and printers -- just to name a few. As they buy the commercial product photographers produce, they also have an interest, preferably cost-effective, in requesting the manner in which RAW files are submitted and managed.

What is it this group wants?

I can't speak for everyone, particularly as this is an evolving issue. However, from my chair, working as an art director/designer across the globe for over ten years, the last six in publishing, and currently pioneering RAW workflows, I have some knowledge of wants. First let me indicate some of the reasons why RAW files are preferable to bitmapped images such as TIFF or JPEG.

--convenient transformation into any colo(u)r managed workflow (16 or 8 bit)
--more flexibility up to production stage
-i.e. can be better managed for output, to match
a design, ink, paper, or press
--economy of smaller assets for storage, processing, and transmission
--recyclable assets which can be further enhanced as software improves

The real world issues for this group, can be summed up as "digital asset management". For this they will require:

--a fully portable raw container
- one that can be re-interpreted or translated across all manner of aging environments
--a fully extensible raw container
- preferably using XML
--compatability with a workflow revolution (analog to digital) they've already emerged from.
- aka: it's a small but "adobe-friendly" world

While initiatives such as full and open documentation of raw formats are encouraged, as are competitive formats that meet the above specifications--neither appear to be anywhere on the horizon. By default, DNG is becoming the only answer--thereby attracting a critical mass of interest. The hour is late, and if standards do not harmonize or evolve to the point where there is a consistant and cost effective way to manage these assets--the one solution becomes the only solution. The industry is already over burdened with incompatible file formats and tools to process them.

What happens when the DAM breaks?

As soon as there is sufficient infrastructure in place to manage digital assets, a submission spec will be created. It's easy to conceive of a situation where RAW submissions that don't meet industry wide templates could be rejected or penalized. It's my feeling that if the DAM is allowed to burst, the industry ecosystem will rush to the only calvary they know-- the DNG corp of engineers.

the born 2 design
design guy

nunatak – Wed, 2006/04/05 – 1:54am

Why DNG is the Answer (for

Why DNG is the Answer (for me)

Stuart,
I have enjoyed your posts to OpenRAW in the past, and think you are very knowledgeable about many of these issues. I would present the information slightly differently, however, and for me, DNG does look like the answer.

The first issue, one that you do not address at all, is the issue of predictable rendering across multiple applications, or even a single application over time. Simply put, there is no other available or pending solution that will allow users to fix a rendering of a RAW file, and embed it into a RAW container with third-party (non-manufacturer) RAW file converters. ( You can currently do this in a crippled fashion with Nikon Capture, not sure about Canon).

The implications of this for working pros, as well as institutions and the serious hobbiest are huge. Unless you use DNG, the work you do to fix and adjust RAW files in your RAW file conversion software is going to be invisible to any other application viewing or converting the RAW file. It's not a problem if you can do all your work in a single environment, like Aperture, C1 Pro, or Adobe Bridge, but it will be a huge problem if you want that work to be seen by others outside of that program, such as by DAM software. And it will be a large shock as photographers consider changing platforms or applications in the future, and realize that they will have to either shed all their previous adjustment work or re-do all of it. (In this context, DNG allows users a penalty-free exit from Adobe software).

The lack of a fixed rendering is also likely to come back and bite you even if you stay within a single application. Will files opened with CaptureNK look identical to the previous renderings, or will your entire collection need to be adjusted again? Nikon has a history of dropping support from Capture and toasting adjustments made to previous images.

This is going to be a pressing issue for many people as they start experimenting with multiple RAW file converters. We'll likely see a revision of the DNG spec that will provide for the saving of multiple renderings from different converters, as well as multiple renderings made from the same converter. Other cool changes are on the near horizon as well.

As to your point about AdobeRGB, I did some digging and here's what I found out. Adobe never intended for AdobeRGB to become the standard for image profiles, it just worked out that way. Because it is their intellectual property - and includes their name - they need to craft an EULA that protects that. As a lawyer, you understand that. At this point, they have never refused to let anyone include it in their software for free: all you have to do is ask.

This of course, is no guarantee that they will continue to do so. What would be ideal, you might ask? Well, the OpenRAW ethic would suggest that Adobe should publish the spec of the profile, and that is exactly what they have done. The only thing they are really controlling is the use of the Adobe NAME, not the profile. Anyone is free to make a version of the profile and call it StewartRGB, for instance.

Adobe has a rich (and I mean rich) history of doing very well with open standards, once they get to a point that they are mature. TIFF and AdobeRGB are just two examples. Adobe's corporate history shows that they are very comfortable competing in an environment of open architecture, and, in fact, they thrive in it. Contrast this history to any of the other major players who might be likely to spearhead such an effort (which is not cheap to accomplish).

I would summarize your argument by saying that Adobe is the worst company to do this work, and DNG is the worst format to do this in, except for all the others. The EXIF issues you point out are ones that illustrate why the camera manufacturers are unlikely to come together to create a universal standard.

As to the standard being owned by Adobe, I checked into that as well. The response I got was that the standards bodies don't want it until it's done. If you know someone else out there who is willing to fund and manage the effort, I'm sure the folks at Adobe would be interested in a discussion. At this point nobody is interested in doing that work.

I would suggest that DNG provides a current solution for real-world problems, with a high likelihood of being the best future solution as well. (Consider that Apple already supports DNG ON A SYSTEM LEVEL, and that Vista will as well.) Once photographers and other interested parties realize the issues associated with predictability of rendering, we will see a real spike in the number of people using the format.

You correctly point to some deficiencies with the TIFF format, and to some problems with the DNG format as currently implemented. I personally have a high level of confidence that the DNG problems will be addressed - I know for a fact that most of them are being currently addressed.

I would invite you to help iron out the problems with DNG, if you see fit.

Peter Krogh
Author, The DAM Book, Digital Asset Management for Photographers

peterkrogh – Thu, 2006/04/06 – 12:30am

As the above comments show

As the above comments show there is quite a difference of opinion regarding the value of DNG. Some see it as THE solution that can evolve with the market being offered freely by a benevolent Adobe. Others see DNG as the product of a commercial company using brilliant marketing to create a defacto standard controlled by them. The truth, and hopefully the bulk of opinions on the issue are somewhere in between.

DNG is not open and does nothing to solve the problem of undocumented formats. To a what I suspect is a large majority of digital photo creators and consumers that distinction is buried because of needing to get work done. Using DNG is the only solution available for those struggling to handle the flood of digital images. In that respect it has been a great suceess, My fear is that it's very success makes it harder to get rid of "ClosedRAW".

OpenRAW vs. ClosedRAW is not really about tiff specs or storage of meta-data. The technical details are just the symptoms. Dng treats the symptoms and offers relief of many of them. But it is not the cure.

Larry Strunk – Thu, 2006/04/06 – 1:32pm

Let's remember what DNG is.

Let's remember what DNG is. It is a SPECIFICATION. It is not a tool or a product. (Some statements that purport to be about DNG are really about Adobe's DNG Converter, or about the use of reverse-engineering of raw file formats. Those are VERY different topics).

Think of 3 parallel universes, and consider whether they can achieve DIPTOD. (Digital Image Preservation Through Open Documentation).

1. In the first universe, raw formats proliferate as now, but they are all openly documented. Will DIPTOD be achieved?

There is no evidence that it will. For example, when cameras use DNG, (an openly documented format), as their native raw format, there is no rush by software developers to support those cameras simply because of that documentation. Decisions are based on the business case - the return on investment. Eric Hyman of Bibble said recently "We currently add support for camera based on popularity, and so far leica hasnt moved up enough on our list". There is better published documentation of Sigma's X3F format than of other formats, but far worse support.

Think what this will be like for new software products in 10 years time, if raw-capable cameras continue to be launched at a rate of 50 or more per year. Products will support the raw formats that have maximum take-up (return), with minimum change from other formats they already support (investment).

2. In the second universe, all cameras have always used an openly documented common raw format. Will DIPTOD be achieved?

Yes. OpenRAW isn't needed.

3. In the third universe, raw formats proliferate as now, but there are "perfect" converters to an openly documented common raw format. (Here, "perfect" means that all the data is converted, and the RESULT is fully documented). Will DIPTOD be achieved?

Yes, if those converters are sufficiently exploited, and if all software products support that common raw format. (They may support the native raws as well - that it not the point here). Precursors to the common raw format needn't matter - it is the quality and take-up of the results of the conversion that matter.

The future in OUR universe isn't pure "2", of course. But turning it into "1" won't solve the problem. We need to turn it into a combination of "2" and "3". We need future cameras to use an openly documented common raw format, and we need "perfect" converters to an openly documented common raw format to cater for earlier cameras. And, of course, we need all software to support that common raw format, whatever else they do.

How can we achieve those "perfect" converters? The current imperfections in Adobe's DNG Converter are largely caused by lack of knowledge of the raw formats being converted. Perhaps the DNG Converter misses something because Adobe don't know about it. Or, even when it copies data across, we still don't know what has been copied across because we don't know what the data was to start with. Documentation of the original raw formats will help with both of those.

Because of the consequences of "1", I believe that one of the main achievements if OpenRAW succeeds will be the best possible DNG Converters, whether developed by Adobe or someone else. OpenRAW should be seen as a FACILITATOR for better support for DNG.

Barry Pearson – Thu, 2006/04/06 – 6:01pm

Did anyone actually read the

Did anyone actually read the Adobe profiles license that Stuart Nixon linked to?

http://www.adobe.com/support/downloads/iccprofiles/icc_eula_unix_dist.html

Stuart is quoting a section of the license completely out of context. EXHIBIT B is a sub-license that would apply to end user's of his software who get one of these Adobe created profiles when they receive his software. This prevents his *users* from further bundling the profiles in yet more software without downloading the profiles themselves (and thus agreeing to the bundling license).

A quote from the correct section that actually applies is to software creators who download the profiles is:

"2. LICENSE. Subject to the terms of this Agreement, Adobe hereby grants you the worldwide, nonexclusive, nontransferable, royalty-free license to use, reproduce, and publicly display the Software. Adobe also grants you the rights to distribute the Software:

(a) on a standalone basis, (b) as embedded within digital image files. (c) as embedded within hardware products that author digital images, where there is no End User access to the Software, and (d) as bundled with your own application software, provided that you comply with all the distribution requirements in Section 3 below."

Section 3 contains stuff to keep Adobe from being sued by users of software that bundles these profiles that Adobe is providing for free.

You can read on for all the details, but if you read it carefully, you will see that the license allows the profiles to be used in applications that Stuart incorrectly claimed that Adobe forbids.

As for DNG itself (the profile license Stuart Nikon made a fuss about has absolutely nothing to do with DNG anyway), the license is at:

http://adobe.com/products/dng/license.html

I suggest that you read it for yourself, rather than blindly accepting Stuart's false (in Adobe's case) list of the motivations of "Large Software vendors".

Thomas Knoll

Thomas Knoll – Thu, 2006/04/06 – 9:50pm

Profile clarification -

Profile clarification - thankyou

Thomas,

Thank you for your clarification on the profile question. I'm sending an email to ask that that portion of my email/article is edited to reflect the correct position. My apologies for my misunderstanding of the profile situation. I know a number of software vendors had confusion over this matter, so it is good to have this cleared up.

Turning now to the question of an open RAW format specification and DNG's position with respect to this, I'd be interested in hearing Adobe's views on the comments I raised in my original email.

Of particular interest is how to open up the hidden information held within MakerNotes, in a way that encourages camera manufacturers to use DNG as a native format.

To take a specific example, consider focus point information. This is by way of example only; the same problem exists for a wide range of other meta data held in proprietary RAW formats.

The current Exif specification does not contain a way to define details such as where the focus points are and which ones are active.

This information is also not defined in the DNG specification as it stands.

So, the only way to get extract focus point information is to extract the data from MakerNotes. So within DNG, the problem remains - the information is not documented.

A few things would help DNG to become considered as the Open RAW format:

- Add public fields into DNG for common camera values that are currently hidden in MakerNotes

- Work with manufacturers to decode and copy as many legacy MakerNotes as possible into these open fields.

- Make DNG follow the Exif standard and structure closely, to make it easier for camera manufacturers to use.

- Directly support the compression techniques used by different manufacturers, so they don't have to reimplement on-camera silicon.

- Have a constructive, open, dialog with other software vendors to find out why they are unhappy with DNG as it stands, and what could be done to address those concerns. Include smaller vendors in discussions.

- Resist marketing DNG as an open standard until all information is really is open in the format (e.g. not hidden in MakerNotes).

- Set up a vender neutral group to manage and guide DNG; this will eliminate concerns over Adobe's control of the format.

We all want to see a single, open, RAW format. Perhaps with revision, DNG could be that format (although not, in my opinion, with the current version).

Certainly I'd like to get the discussion back to looking at the issue of an Open RAW format. Adobe has at least tried to address the problem, which is a significant step forward.

Regards,

Stuart

Stuart Nixon – Fri, 2006/04/07 – 12:37am

Stuart, your retraction

Stuart, your retraction might also be appropriately reflected under your proposed solutions.

the born 2 design
design guy

nunatak – Sat, 2006/04/08 – 2:11am

Larry, Let me make sure I am

Larry,
Let me make sure I am being clear here:

>As the above comments show there is quite a difference of opinion regarding the value
>of DNG. Some see it as THE solution that can evolve with the market being offered
>freely by a benevolent Adobe.

I'm not saying that Adobe is benevolent, necessarily. What I am saying is that they are comfortable competing in an open architecture environment to a much larger degree than any other player on the horizon. I see it more as a confluence of interests, at least temporarily.

>In that respect it has been a great suceess, My fear is that it's very success makes it
>harder to get rid of "ClosedRAW".

Unfortunately, I agree with that. Although I son't think that DNG will be the biggest impediment to the success of OpenRAW. That will be taken care of by the refusal of camera makers to openly document the formats.

> OpenRAW vs. ClosedRAW is not really about tiff specs or storage of meta-data.

I actually think that the metadata people create once they start working with images will be of significantly more value that the relatively small amount of information hidden in the private maker notes.

You will have a lot more vested in your ACR, (or Aperture, or C1 or Bibble, or Silkypix, RSP, Lightroom) settings than you will in knowing how the camera remapped dark pixels.

On one hand you have a small amount of metadata that is contained in Private Makernotes. If this stuff was absolutely necessary for a great (or even good) conversion, then C1 Pro would not be able to beat manufacturer's software.

On the other hand, we have all the information we create about our images: how they are to be interpreted by RAW file converter(s), what they are about, how much you like them, and how they have been used. This information is extremely important to the photographer as well as other interested parties. In my own case, I consider this information to be far more important than, for instance, being able to read which shutter actuation made the photo.

This is not to say that I do not support OpenRAW. I was an early supporter and continue to support the effort. I just happen to think it is less likely to come to pass, unfortunately. I need to have, at the least, what DNG offers.

Peter

peterkrogh – Thu, 2006/04/06 – 10:08pm

There is much that I agree

There is much that I agree with in your reply. Image meta-data, including post capture is very valuble and needs to be carefully managed. That being said, there are significant areas where we disagree. The meta-data people create post capture can be managed in various ways by the owner of the image. Certainly there is a large need to educate image users/creators to the necessity to control and manage that meta-data, which is a need you are in part filling.

The meta-data created at image capture however is under very little control of the photographer. Currently the accesibility of that data is controlled by the camera maker. The relative value of that information should not be determined by anyone other than the creator or owner of the image. What is important to you may be of no significance to me, your worthless data may be anothers critical information.
Who gets to decide what data is important and what isn't? The fact that high quality images can be rendered without the data does not mean that it has no value. What would be possible if the unkown were known? Can anyone really answer that question without knowing that which is unkown?

To restate my earlier point, I do not think DNG is withour merit or value. It goes a long way towards relieving many of the symptoms of the RAW problem, especially in the field of digital image asset management. DNG has made life much simpler for the consumers of digital images.

DNG is not the solution to the core issue of the RAW problem.
DNG is a solution that solves many of the problems with RAW image use today.
Until RAW formats are openly documented, ClosedRAW will continue to cause problems for photographers and RAW file users. We may feel better for a while as each new 'treatment' overcomes individual 'symptoms' but the problem of ClosedRAW will only truly be cured when ClosedRAW file formats are Opened through their public documentation.

Larry Strunk – Mon, 2006/04/10 – 11:54am

Open Documentation -

Open Documentation - discussion of what is needed.

What is not being properly addressed by OpenRAW is "what has to be documented in order to preserve the digital image?" The assumption is that open (and presumably comprehensive) documentation of the raw format is both necessary and sufficient. I question both of those.

Imagine a software company in decades to come with some raw files from a 2006 camera, and open documentation of the raw format. They can access all the image data, but can they exploit it? Can they render it as a high quality image?

When the software designer asks "what was the strength of the anti-alias filter in the camera?", would that be documented in the raw format? Where? And there may be another 10 or 20 such questions, each of which is really about the design of the camera rather than the format of the raw file. Raw converters don't simply need to know how to get at the information in the raw file, they also need information about technology influences on the image data before it got to the raw file.

OpenRAW can HOPE that if camera manufacturers document their raw formats, they will ALSO provide sufficient documentation about the design of their camera models. Or OpenRAW could specify the necessary camera-model parameters that must be available in some form. But what are these parameters?

Ideally, instead of those design parameters simply being in documents, they would be held in openly documented "camera-profile" files in some common format. Instead of that future software having to cater individually for each camera-model, it could simply read the parameters from the appropriate file. If it is handling a D200, it would read in the freely-available D200 parameter file, and use that to influence rendering of the data from the raw file.

Now go one stage further. If those parameter files are not too big, instead of keeping them as separate files, why not incorpotrate them into EVERY raw file? So each raw file from a D200 would contain those D200 design parameters parameters, perhaps 20 or 30 of them, in an openly documented format.

Obviously, I have just described DNG. Those camera-model parameters are already documented - in the DNG specification.

The point I am making here is that DNG is not a side-issue or distraction or threat as far as OpenRAW is concerned. DNG was designed from the start to achieve "Digital Image Preservation", and for the essential image data its policy has always been "Open Documentation". DNG was designed to address every OpenRAW issue, except one - "secret sauce", or "undocumented Makernotes".

Here, OpenRAW needs to beware of possible absurdities in demanding open documentation of Makernotes before a raw format can be judged as OpenRAW-compliant. Example 1: Suppose a camera has a fully-documented raw format. So it is OpenRAW-compliant. Now suppose a firmware-upgrade adds an extra (undocumented) parameter. It ceases to be OpenRAW-compliant. Even if (say) that parameter was a diagnostic for the battery-condition! Example 2: Suppose a camera has a fully documented raw format except for the undocumented Makernote, but that is not needed for image rendering. Now suppose the camera has a menu-option "discard Makernote". Using that option - (throwing away data!) - would make the camera OpenRAW-compliant!

OpenRAW is about "Digital Image Preservation". The discussion about DNG gets confused by data which is not needed to preserve the "digital image", whatever else it is for. It is more important to ask "have we got everything we need?" than to ask "have we lost something we can manage without?"

And, if/when OpenRAW succeeds in obtaining open documentation of raw formats, including Makernotes, its criticisms of DNG will cease! Once such data is documented in the native raw, it will automatically be documented in the resultant DNG file. DNG (or some other common raw format) is the TARGET for "Digital Image Preservation". OpenRAW may help it get to the target.

Barry Pearson – Mon, 2006/04/10 – 5:10pm

Let's avoid a holy war! -

Let's avoid a holy war! -

I'm confident that virtually every advocate of DNG on the planet is in favour of open documentation of raw file formats. How could an advocate of DNG NOT be in favour of this? It would be a contradiction!

I'm less confident that virtually every OpenRAW supporter on the planet is in favour of the evolution of a common raw format. More precisely, I know some of them are NOT. For example, Iliah Borg said "It makes little sense to force a standard file format on companies, which will instead stifle their innovation".

This illustrates a gulf that needs to be bridged. Virtually all DNG-advocates are comfortable with OpenRAW's objectives. Some/many OpenRAW supporters are opposed to DNG. Yet surely ALL of us want to achieve "Digital Image Preservation"!

I believe that these are some elements of this gulf:

1. Some OpenRAW supporters think DNG is simply about reverse-engineering proprietary raw formats and converting these into DNG format using an Adobe DNG Converter.

(Anyone who believes this is beyond rational influence, and should seek expert help before they cause harm to themselves and others! I won't waste my time refuting this).

2. Some OpenRAW supporters think Adobe is trying to control the world of raw shooting, from what camera manufacturers can innovate, to what other software companies can develop and supply.

(This is so stupid that it defies belief! Adobe has greater resources to exploit camera manufacturers' innovations than any other company. The greater the innovation, the more likely it is that Adobe will stand out as one of the companies supporting it. What proportion of software products support Sigma/Foveon sensors? What proportion of companies support Fujifilm offset sensors? Or Sony 4-colour sensors? Adobe has lots to gain, and little to lose, from such innovations).

3. Some OpenRAW supporters think that DNG is a specification that has been frozen in time, and can't evolve.

(I've spent decades of my career involved in the evolution of computer-systems interface specifications as requirements change, and I don't see why digital image processing defies universal experience that evolution can be managed. DNG was designed for evolution. DNG has a useful version-management scheme built into it).

4. Some OpenRAW supporters think that if DNG has significant take-up, it will eliminate the need for the open documentation of existing raw file formats.

(Adobe, and Thomas Knoll, are forever requesting information about the raw file formats of new cameras. Often this involves requests for people in the Adobe forums to supply raw files! ADOBE NEED DATA about raw file formats, and make no secret of the fact!)

So:

If OpenRAW fails, people will have to turn to DNG as their best choice for "Digital Image Preservation". If OpenRAW succeeds, companies will develop better DNG converters, and the resultant documentation will encourage photographers to adopt DNG-based workflows. It is pointless to debate whether DNG will continue to make progress!

OpenRAW needs to position itself in an environment where DNG continues to make systematic progress. It needs to study the question: "in an environment where DNG is making systematic progress, how can OpenRAW best achieve its OWN objectives?"

Surely the answer is obvious?

Barry Pearson – Fri, 2006/04/07 – 4:01am

A Response from Adobe: This

A Response from Adobe:

This has definitely been an interesting discussion thread, and it’s helpful to see the range of views on the topic. Hopefully, you’ll all find it helpful to get the Adobe perspective directly.

Fundamentally, DNG is a pragmatic solution to a thorny problem. It seems that Stuart Nixon’s biggest problem with the DNG format is that it allows for the continuing use of MakerNotes. I can understand his desire to stick to the ideal of a fully transparent file format, but I believe that his proposed ideal is unlikely to ever win support from camera manufacturers, and the additional benefits it might bring are open to debate.

In order to be successful, camera manufacturers need to differentiate their cameras in the market. Each aspect of the camera that is “locked down” via standardization represents one less degree of freedom in differentiation.

MakerNotes provide a way for manufacturers to use a standard, compliant format, yet still add extra features for the purpose of differentiation. The presence of MakerNotes hasn’t been a problem for JPEG files. Despite the presence of undocumented MakerNotes, JPEG files from any camera are still universally acceptable. The content of the MakerNotes is not critical to the understanding of the files, and they are simply ignored by most software other than what’s provided by the camera manufacturer. It’s true that it could be helpful to have all of the additional information in MakerNotes publicly documented, but that doesn’t stand in the way of reading the files.

The goal of OpenRaw—and, presumably, of Stuart Nixon—is to have all raw formats fully and publicly documented. This is seen as being more important than having a single, common raw format. In contrast, we at Adobe believe that having a common raw format is the most important goal. As long as there is a baseline of publicly documented information present in the file to allow any software or hardware to process a high quality image, we think it’s acceptable—though not necessarily desirable—to have additional information within the file that’s not publicly documented.

Others in this thread have already pointed out the flaws with allowing for a plethora of different file formats, regardless of whether they’re publicly documented or not. Today, well over 100 cameras are supported by Adobe Camera Raw, and each of those has to be maintained. How long will it be before there are 1000 different formats supported? How long will it take to maintain all of those?

A common format is the only solution to this problem. It ensures that all cameras—even niche models—can be supported, now and in the future.

Clearly, Adobe is motivated to see that a common format takes hold, but not for any of the nefarious reasons Stuart Nixon implies. Having a common format makes it easier and less expensive for us to develop software that keeps our customers happy, and keeping our customers happy is what keeps us in business.

Adobe is not a camera company, so we can’t use our own hardware as a method of getting the format into the market. We need to make it as easy as possible for camera manufacturers to support it. That’s why we’ve based it on TIFF/EP, which is a format many of them already use as the basis for their proprietary formats. That’s why we support MakerNotes, so they can extend the format as necessary. And that’s why we publish the specification freely.

More importantly, we fully expect that, as momentum builds behind DNG and more manufacturers decide to support it, we will get requests for more changes to the format. That’s why we’ve built a versioning system into the format, to allow it to evolve while retaining backwards compatibility. We’ve certainly done our best to create a versatile format that can serve multiple camera manufacturers, but they know their needs more than we do.

We are also very open to the idea of releasing DNG to a standards body. This did not appear to us to be a good way to kick off the format, because there simply wasn’t enough interest within the standards bodies to pursue the issue at that time. We believed we could generate more momentum by releasing something to the public, where the pain of the problem was being felt most acutely. If the public enthusiasm leads to greater interest within a standards body, we’re happy to work with them.

We’re even open to accepting an alternative to DNG as a common raw format. Of course, we’ve put a stake in the ground and told customers that they can archive in DNG, so that means we’re committed to supporting it for the long term, or at the very least providing a smooth transition to a new as-yet-unknown standard. Nevertheless, we still think DNG is a good solution, and until someone can point to a concrete alternative, I think it would be much more constructive to publish articles that suggest modifications to DNG than to publish ones that proclaim that “DNG is not the answer.”

Kevin Connor
Senior Director of Product Management
Adobe Systems Incorporated

Kevin Connor – Mon, 2006/04/10 – 5:17am

I'd like to thank Kevin for

I'd like to thank Kevin for giving Adobe's public response to my DNG concerns.

Adobe has put a great deal of effort into resolving the closed RAW format problem with their DNG solution. DNG is the best hope for the future if certain problems can be addressed - and I think this is doable.

My concern is that DNG needs a bit more done to it before it can be promulgated as the solution to the RAW format problem. I am not alone with my concerns; once DNG addresses camera manufacturers and software vendors concerns then there will be a greater update of DNG.

There are a couple of key points that, once addressed, would help DNG's widespread acceptance. Some of these include:

1. Unified MakerNotes
Part of the problem with MakerNotes at present is they are difficult to propagate to other formats like DNG; there is no clearly defined structure for MakerNotes. If manufacturers really *have* to store undocumented information, then lets at least ensure a way to do this that enables the information to be copied with surety.

Recommendation: A "MakerNotes" class of IFD be defined, specifically for storing proprietary undocumented manufacturer (or software vendor) information. All we need to do is to specify that the information be stored using the usual IFD structure, with the MakerNotes IFD itself having a unique ID for each manufacturer (so that, for example, there can be several manufacturer notes within the DNG file). Although other products might not be able make use of the undocumented information, at least the data itself can be safely copied around. The current DNGPrivateData field does not go far enough; it really needs to document a sub-IFD method for MakerNotes.

2. Added EXIF style fields for common camera information
There are about 50 different fields that are needed to represent information presently held in MakerNotes. It is worth noting that, almost without exception, manufacturers already put data in to the EXIF fields when a field exists. So MakerNotes are really used as a way to store information where no standard already exists.

Recommendation: Adobe has defined a range of fields in DNG, this needs to be significantly extended to define *all* commonly used values of data. Most software vendors have lists of fields that they extract and add to the EXIF list; this list needs to be ratified and made available to camera manufacturers so they can use these instead of MakerNotes. I believe it is likely that manufacturers will use these fields if they exist.

3. Added formal XMP entries for the above
In the long term, XMP is the way to go. In the short term, it is unrealistic to expect camera manufacturers to use XML to encode data for their RAW files, and we all understand this. So ideally we want a migration path, where there is a 1:1 mapping of the enhanced EXIF data discussed in (2) above and XMP based data. This way, the manufacturer can write data using the EXIF / IFD style, and software vendors can "lift" that data up to the XMP level.

Recommendation:
XMP mirroring of the enhanced EXIF data, and a format specification on how to replicate between XMP and EXIF. Even better, an SDK that does this to reduce inter-vendor format errors.

4. Support of multiple CFA encoding methods
Manufacturers have silicon based CFA encoders. It is difficult for them to change these.

Recommendation:
Support for the 3 or so different encoding techniques (which are all JPEG based anyway) added to DNG, so that camera manufacturers can directly support DNG on-camera with minimal hassle.

5. Enhanced DNG converter to convert MakerNotes into the public fields
Copying current MakerNotes around has a lot of risk; there is a good chance that most products will not be able to correctly read the MakerNotes. Also, we want to encourage all the existing "legacy" RAW files to be converted with the hidden information copied over to the public fields.

Right now, we have the reverse situation - each software vendor has put so much effort into their own legacy RAW IO libraries, that this is a competitive advantage and so there is a disinclination to share information. I'd say that 99% of existing legacy RAW formats have been fully decoded; just not all by one vendor.

Recommendations:
(1) An open-source SDK is developed to convert from legacy RAW formats into DNG (and also useful as a SDK for DNG itself). Given various software vendors will have license and NDA restrictions that will prohibit their ability to release parts of their code, this needs to be an industry initiative. Once we have an open SDK where any software vendor can read legacy RAW formats into DNG, and then work in the DNG space, a lot of the software vendor resistance will melt away.

(2) A public bittorrent repository of legacy RAW format photos from common cameras, so vendors can share and test conversion logic. Again, right now each vendor has their own private stash of different RAW photos, that are not shared.

6. Clarity on IP, copyright and patent issues
This is not an Adobe specific issue. In fact, Adobe is better than most vendors in seeking to address these problems.

And make no mistake, these are the biggest concerns faced by manufacturers and software vendors. JPEG 2000 would have been killed as a standard if my company had not spent millions of dollars defending the standard from a patent-grab. Some manufacturers now encode conversion profiles into their RAW formats. Is another vendor allowed to use those profiles?

Photo related software vendors and camera manufacturers are particularly at risk, because of the value of their work. All a patent-litigator has to do is prove that some method infringes their patent, and then claim say $1 for every photo taken used their format. Hundred million dollar payments are not out of the question here, and this makes manufacturers and vendors very nervous.

Recommendations:
(1) An IP tag way of encoding ownership of key IP, indicating if it may be reused and if so how.
(2) Very clear indication on use on allowed usage for popular profiles as well as on matrix tranformations present in a DNG file that were created by a DNG converter.

7. Merging of the EXIF and DNG standards
As Kevin Connor noted, manufacturer formats are mostly based on TIFF/EP and EXIF. The problem is those standards did not contemplate RAW format. DNG can address these needs. If there was a merging of DNG and EXIF, into DNG/EXIF3 or whatever, then this would enable manufacturers to comfortably migrate. EXIF is a Japanese camera manufacturer driven standard. DNG is a American software vendor driven standard. The two need to come together.

Recommendation:
Once DNG is expanded to cover the holes in EXIF, a format approach is made to the EXIF group to merge the two standards.

Adobe clearly expresses a willingness to listen and work towards addressing issues.

I've detailed some points that in my view could help make DNG the open RAW standard for photographs. However, I'd like to take the discussion with Adobe and others off-list, and hopefully we can report back in the near future on the positive results of those discussions.

Thanks for your time,

Stuart

Stuart Nixon – Mon, 2006/04/10 – 3:43pm

Your TIFF/EP comment -

Your TIFF/EP comment -

Stuart, you say "As Kevin Connor noted, manufacturer formats are mostly based on TIFF/EP and EXIF. The problem is those standards did not contemplate RAW format".

That is a curious statement, because surely TIFF/EP has key parameters specifically to handle raw image data? It has "PhotometricInterpretation = CFA", and adds CFARepeatPatternDim and CFAPattern parameters. It allows "SamplesPerPixel = 1" for a CFA. In other words, it has parameters that describe the configuration of a Color Filter Array, so that raw image data for such a sensor can be stored.

TIFF/EP is out of date and not now fit for purpose, (if it ever was). Isn't DNG approximately what TIFF/EP would now be if TC42 could react fast enough?

Barry Pearson – Mon, 2006/04/10 – 5:46pm

Barry, Hi. TIFF/EP did not

Barry,

Hi. TIFF/EP did not cater for specialised encoding techniques needed, and did not provide for mask area information. Also, manufacturers tend to use EXIF in preference to TIFF/EP (there are differences between them). However, TIFF/EP and EXIF are a good step in the right direction (with DNG being a further step), and manufacturers do use a lot of fields where they can.

Regards,

Stuart

Stuart Nixon – Mon, 2006/04/10 – 6:56pm

But TIFF/EP was for raw!

But TIFF/EP was for raw!

Yes, I agree that TIFF/EP is out of date and inadequate. I doubt if anyone, including TC42, disagrees. But I was responding to your statement "... TIFF/EP and EXIF. The problem is those standards did not contemplate RAW format". Surely TIFF/EP DID contemplate raw format?

Another point of clarification. You said "3. Added formal XMP entries for the above". "Above" appears to include your "2. Added EXIF style fields for common camera information".

There already appear to be XMP-namespaces, used in DNG files, that include some EXIF fields. Is there a problem with these, or do they need extending, or what?

For example, "ns.adobe.com/exif/1.0/":

ApertureValue, BrightnessValue, Contrast, CustomRendered, DateTimeDigitized, DigitalZoomRatio, DateTimeOriginal, ExifVersion, ExposureBiasValue, ExposureIndex, ExposureMode, ExposureProgram, ExposureTime, FileSource, FNumber, Flash (Fired, Return, Mode, Function, RedEyeMode), FocalLength, FocalLengthIn35mmFilm, FocalPlaneResolutionUnit, FocalPlaneXResolution, FocalPlaneYResolution, GainControl, ISOSpeedRatings, LightSource, MaxApertureValue, MeteringMode, Saturation, SceneCaptureType, SceneType, SensingMethod, Sharpness, ShutterSpeedValue, SubjectArea, SubjectDistanceRange, WhiteBalance.

And "ns.adobe.com/exif/1.0/aux/":

Firmware, FlashCompensation, ImageNumber, Lens, LensID, LensInfo, LensSerialNumber, OwnerName, SerialNumber.

(There may be more. Those, over 40 of them, are just the ones I've seen in DNG files).

Barry Pearson – Mon, 2006/04/10 – 8:54pm

Stuart, I appreciate your

Stuart,

I appreciate your newfound enthusiasm for DNG. There are, however, a few comments I'd like to add. You state:

1. Once we have an open SDK where any software vendor can read legacy RAW formats into DNG, and then work in the DNG space, a lot of the software vendor resistance will melt away.

In the long term, this would be the way to go. In the short term, it's unrealistic to expect camera makers to surrender their roadmaps to legacy formats. Particularly as some camera makers have disappeared and others merge with larger entities. The desire for an open SDK should in no way postpone or distract from proceeding with a common raw file format. The good that comes from a common raw file format supercedes legacy issues, although the lessons of that legacy, as you suggest, should be built into that common raw file format.

2. However, I'd like to take the discussion with Adobe and others off-list, and hopefully we can report back in the near future on the positive results of those discussions.

In my view it's important that the "others" you refer to includes representation from all vital components of the ecosystem, and not a limited subset. Adobe has an excellent track record with the publishing, archiving, management, and print industries-- and those will be some of the biggest consumers of DNG. It's to everyone's benefit that sufficient data is embedded upstream before it impacts distribution and production downstream.

Thanks for your views.

the born 2B pragmatic
design guy

nunatak – Tue, 2006/04/11 – 3:18am

DNG SDK now available.

DNG SDK now available.

This thread may be a useful place to record that Adobe have released a DNG SDK. It also includes an XMP SDK.

"Software License. Subject to the restrictions below and other terms of this Agreement, Adobe hereby grants you a non-exclusive, worldwide, royalty free license to use, reproduce, prepare derivative works from, publicly display, publicly perform, distribute and sublicense the Software for any purpose."

http://www.adobe.com/support/downloads/dng/dng_sdk.html

Barry Pearson – Wed, 2006/04/19 – 1:02am

Is it too much to hope

Is it too much to hope Nikon's announced delay in releasing NX is due to such fortuitous timing? Mmmm...probably not. :-(

the born 2 design
design guy

nunatak – Wed, 2006/04/19 – 3:26am

My guess is Canon first

My guess is Canon first

(But your post was a magnificant bit of conspiracy theory!)

This is interesting:

"Canon Consumer Imaging Group Director Chuck Westfall presented the camera vendor perspective. Mr. Westfall compared two possible solutions for raw data; proprietary and open standards. While making it clear that Canon intended to keep its RAW data recording methods proprietary, Mr. Westfall stated that Adobe's DNG file format has excellent features for archival storage and added that Canon might consider the possibility of adding DNG support in future versions of RAW image conversion software."

http://www.i3a.org/virtual/eye_on_imaging_v3_n2.html

Barry Pearson – Wed, 2006/04/19 – 7:19am