Allow module-based corrections for Linear DNG files

Edit: rewrote this post to clarify the feature request first, and put my personal use case/motivation at the end.

Feature request

As a user:

  1. When I have a Linear DNG file opened in PhotoLab;
  2. Given that PhotoLab recognizes the camera and lens from that file’s metadata, and has a matching DxO module for that combination;
  3. I want to be able to apply module-specific corrections such as Vignetting, Distortion, and Lens Sharpness Optimization.

Currently, at step 3 those module-specific corrections are disabled in the PhotoLab user interface: they appear as greyed-out “Auto with DxO module” and “Based on DxO module” options.

Technical limitations?

Do these features require the RAW color filter array (CFA) data to be present? It looks like vignetting and distortion corrections would happen after the demosaicing and denoising steps in the PhotoLab image pipeline, and should work with a linear DNG’s RGB data too. But if anyone knows if there are such technical limitations that I’m missing, I’m all hears.

UX limitations?

PhotoLab’s UI doesn’t explain why those features are not available for linear DNG files. My main theory is that the PhotoLab developers expect that a linear DNG is likely to have been produced by PureRAW or PhotoLab with baked-in corrections already.

The goal is probably to avoid automatically applying module-specific optical corrections when the linear DNG file is first opened in PhotoLab, because that could result in doubling the corrections, which would be bad.

I think it might be possible to prevent this UX issue (and the automatic double-correction problem) without fully blocking access to those features. Maybe blacklist specific corrections from the presets applied when a file is first loaded, but let users apply said corrections manually.

Use cases

My personal use case is that I’m considering using PhotoLab 9.6 to convert a bunch of old RAW files to Linear DNGs with lossy JPEG XL compression. In my tests this does give some nice space savings for at least some of the cameras I’ve used over the years, and the “half-RAW” nature of linear DNGs is good enough for me in the event that I want to exploit those old pictures later on.

Ideally, I would like to apply DeepPrime XD3 and Lens Sharpness Optimization when exporting those compressed linear DNG files, because I’m comfortable baking in those optimizations in the DNG’s RGB data.

But I would like to keep my options open for corrections such as Vignetting and Distortion, because whether I correct those lens characteristics or not (or only partially, e.g. I often set those corrections to 50%) feels like a creative choice. E.g. for Distortion, sometimes correcting the distortion will cut some edge details that I want to keep in the shot; and sometimes the distortion-corrected picture looks a bit unnatural to me (depending on the lens and subject).

If I export a linear DNG from PhotoLab 9.6 without Distortion and Vignetting corrections, and open that DNG in PhotoLab, I lose the option to apply the module-specific distortion or vignetting corrections. That feels needlessly limiting… unless it’s actually a strong technical requirement (e.g. the corrections require CFA data).

Note that other users may also benefit from having those corrections available for linear DNG files, even if their use cases are different. Generally speaking, I feel that it’s best to give generally capable tools to end users, and to avoid forcing them into specific workflows by disabling tools.

Writing here I’ hadn’t seen this post. :slight_smile:

.

I understand your desire to save storage space, but consider this:

With a “HiFi compressed” DNG file, you’re currently still dependent on DxO software,
whereas e.g. the (large) TIFF output format is universally compatible.
Instead of limiting yourself now to noise reduction, but retaining other corrections, simply apply PL to your raw files, work with space-saving virtual copies and then export only the currently important images in your desired output format. – That’s the least effort and you still save archival space.

( While I do printing, for which I still use PS with custom layout, etc., I even keep the finished version into which I’ve invested so much work. )

I would NOT recommend converting original raw photos to DNG just to possibly save space. I doubt the DNG files would be much smaller than your raw files. I believe Canon compress their raw files already (lossless) and they are quite small relative to pixel count.

If you compare raw files to DNG files, the raw file has been de-bayered producing an image with vastly larger data per pixel than a pixel in a raw file. A raw file has an array of red, green and blue pixels which are converted to single pixels with red, green and blue values for each pixel (probably double the data).

As an example for a 10mp raw file with 14 bits per pixel you will have 10 million numbers consisting of 14 bits each.

Now if you convert that to a 16bit DNG file you will have 10 million numbers considering of 3 X 16 bits each. This would be compressed but I don’t believe to less than you original raw file.

I hope that explanation helps clarify the difference between your raw files and DNG files produced by PL9.

1 Like

To test, I exported the 60 test images I mostly use to HiFi DNG and found that space saving was about 10-15 % … but some files got bigger too. Whether linear DNG saves space or not seems to be coincidental - with Canon RAW files.

Downside of the HiFi DNGs is that they are lossy, and while differences are hard to tell, they do exist. Hence, if losing info is acceptable to save space, trashing a few additional files might be more effective.

My recommendation: KEEP YOUR ORIGINAL FILES

1 Like

Whether Linear DNG files are good for saving space is orthogonal to this feature request, it’s just one use case I had in mind for Linear DNG files without baked optical corrections.

I don’t want to derail this feature request by making this entire thread about Linear DNG file sizes. So I’ll just answer this topic once, then leave it at that.

Yes, it is true that demosaicing generates more data, making it a bad idea for space savings—in theory.

But, in practice, it really depends on compression. Linear DNGs with JPEG XL compression, especially with lossy compression, can be competitive if you have:

  • Uncompressed RAW files (though converting to non-linear DNGs with old-school LJPEG-1992 compression would already save some space).
  • Compressed RAW files that use some inefficient lossless compression (camera manufacturers optimize for low battery usage and clearing buffers fast, not for ideally small file sizes).
  • Compressed RAW files with pretty good lossless compression… that still gets beaten by the lossy compressed linear DNG.

I have files from 3 generations of cameras that each match one of these conditions. For the first 2, converting to non-linear DNG with LJPEG-1992 compression is already a small win. Though if I had the tools to convert to non-linear DNG 1.7 with lossless JPEG XL compression or even lossy JXL compression, that would save a lot of space. Sadly, I haven’t found a tool that does that (no, Adobe DNG Converter is too limited), so I can’t even test the resulting files for compatibility in RAW editing software.

The last scenario is my X-T5, which produces lossless compressed files that clock in at roughly 1.1MB per megapixel (beating DNG LJPEG-1992 compression, clocking in at around 1.3MB per megapixel). Meanwhile, linear DNGs produced by PhotoLab 9.6 are around 0.75MB per megapixel, and lossy linear DNGs from Adobe are around 0.25MB per megapixel.

In an ideal world, I would be able to convert proprietary RAW files to non-linear DNG in DNG 1.7 with JPEG XL compression. But I don’t know a tool that does that, nor do I know how well RAW editing software would support it (or not at all). It might be doable with Adobe DNG Converter’s command-line interface (it’s not possible with the UI), but I haven’t tried it yet.

Anyway, back to this feature request: given that the compressed linear DNGs produced by PhotoLab 9.6 and PureRAW 6 are pretty good, it would be great if opening them in PhotoLab did not block access to DxO module corrections.

Yes, and splitting a possible one-step process into two looks strange to me too.

But:

  • If your files aren’t supported (by DxO), I’d leave them as is in order to prevent doing something that cannot be reversed, no matter if space is saved or not.
  • If your files are supported, you can easily correct them and save linear DNGs as intended by DxO. And if you set PL to add black fillers, you’ll not waste a single bit.

Looking at how DxO supports gear, I’d not expect any module based stuff, specially not for any old (unsupported) gear. DxO’s strive for technical excellence is holding back anything that looks like niche input coming from anything else than DxO or, possibly, Adobe. Everything not from DxO means that DxO has to catch up to it.

Nevertheless, DxO has come up with surprises and maybe this FQ will make it.

So what you’re asking for is to make some of the corrective adjustments “now”, then output to compressed DNG, then (optionally) make other corrective adjustments “later”, and these to be automatic.

All of this to save some space.

I don’t think this would be a common use case. Most people would either start editing with the full CFA original or the fully ‘corrected’ (Linear) DNG. In other words… start with the RAW file or start with PureRAW output (albeit from PhotoLab in your case).

What sort of ‘optional’ cases are you anticipating that would involve not correcting distortion? (Vignetting can surely just be re-added creatively.)

Basically, when a shot has a little bit of distortion but doesn’t look obviously distorted, I typically want to keep it not corrected:

  1. When there are elements in the frame near the edges that I think work well in the composition, and that would be cropped or pushed too close to the edge when correcting distortion.
  2. When I feel that correcting the distortion moves things around and upsets the visual balance of the shot.
  3. When I feel that correcting the distortion makes something (typically a face) look a bit unnatural (the rectilinear look might be technically correct, it doesn’t mean it’s always psychologically correct).

(First one is more objective, and the next two scenarios are more subjective and vibes-based.)

1 Like

As a general note, I feel that there is way too much dissection of my personal use case and motivations in this thread, and virtually no discussion of the actual feature request.

Maybe that’s my fault for mixing both in my original post. So I’ve rewritten it to frontload the feature request and the discussion of possible technical and UX limitations, and relegated my personal use case at the end. Hopefully that’s a bit more clear.

Fair, but I think what some of us are saying is that any use case for this is unusual and, importantly, goes against DxO’s intention for the DNG exports.

1 Like

Discussing use cases can help to understand motivations and often provides alternative routes to achieving desired results. Often, motivations and context are easy to understand from an OP’s point of view, but we’re not in your head and try to get some context. What you found is fairly common and is generally not meant to dissect the posters.

Which is always possible - and bears the risk of disconnecting the answers of whatever was present in the original first post.


All things considered, being able to apply module corrections to linear DNGs created by other apps can be desirable.

  • Including the risk of having corrections applied multiple times.
  • I do feel that DxO could drop a few of the limitations built into apps, limitations that are meant to “protect the user” from doing/getting something less than optimal…
  • BUT I do prefer choice over paternalism, no matter how well-intended it may be.
2 Likes