Clipped highlights, but only in Photolab?

Yes I just changed the camera model using exiftool. Here is a command:
exiftool -model=“NIKON Z 6” file.NEF

Originally I suggested that the difference in highlights might be due to
the camera profiles PL uses for Nikon Zf and Z6.
However, based on the discussion, it appears that the difference in highlights is due to a separate (earlier than applying camera profile) processing step in PL. This step also depends on the camera model.

Yes, the s…t happens at the RAW decoding/demosaicking/NR/brightening/WB stage, but it’s DxO/libraw internals, so we can only speculate. Looks like a baby-age of Zf interpreter but maybe the real problem has wider impact (??).

That’s why I initially skipped reading your text too early, as I considered it silly, sorry for that.

If you go back and read the discussion by Noname Captain Obvious, he shows that the problem exists beyond the Zf model. For Zf there is something additionally wrong.

Your example (just changing camera model in the RAW metadata) shows that there is a specific problem with Zf and provides, hopefully, constructive clue to DxO.

Other examples were handmade with some dogma in mind, so I wouldn’t take them seriously. However, the example with whites inserted seems to be informative indeed (but why 81 pixels, wasn’t just one enough, something more under the hood?).

Side remarks:
PL is not an open source, so we can only speculate, more or less (most often less) constructively. Now I’m having some fun with libraw sources and it’s full of ‘devil’s details’. DxO seem to be familiar with some libraw ‘quirks’. For example, the RAW size for D780 is 6064x4040, while DxO generates jpegs with a smaller resolution 6048x4024, just like the camera, or NX Studio. On the other side, some photos taken with D780 (found e.g. on Wikipedia) and processed by darktable 4.2.1 have jpegs with the original NEF resolution, which shows that the raw_inset_crops[0] field was not taken care of. It’s just an example of devil’s details, no FUD for darktable in mind, which might have fixed it by now. BTW, I have my own theory on why these extra pixels disappear from RAWs in cameras jpegs, but that’s a separate story.

One of the reasons that the jpg has a smaller size is that the outer colums and rows are not used for they don’t have surrounding sensels. In your example the difference is 16 or 2x8.
When I remember well the problems with other camera’s where from Fuiji.

George

Indeed, I skipped the normalize step which is done per “photosite” color. So my question is irrelevant.

(I assume you used there the word “pixel” the same way the document use it (= “photosite” digital value), which is not the common way to use it, because there is still no pixel in raw datas since they are not demosaiced, there are only “photosites” values).

Yes, I shouldn’t :blush: .

“brightening/WB stage” happens after demosaick in DxO code and so after the three bugs listed… as for NR - the issue was demonstrated with NR off… re-reading the topic again is advised

As I observed in another thread, if you modify the whitelevel value in the NEF file and set it to 16383, the problem disappears. This help understand at which level the problem occurs.

one problem, two other remains

PS: Nikon NEF (at least from Zf camera and in raw produced by OP) as written by camera does not have a whitelevel tag ( 0xc61d WhiteLevel ), so what exactly are you modifying there then ?

Looking at (the original? The thread is kinda long) NEF file, I never expected this to be this bad.

But… yes, the raw data in the file is close to clipping, but nothing is really clipped. None of the channels.

As others figured out, the whitelevel used in parsing the RAW data can be crucial.

In this case, DxO PL seems to have that wrong. PLEASE REPORT IT AS IT IS A BUG.
From working with Darktable and other opensource raw processors,
every camera model can have different black- and whitelevels. Data you need for correctly parsing the data in the file. Those black- and whitelevels can even change on the same model, depending on ISO and/or shooting mode! It’s a pain to figure this stuff out.

DxO used the wrong whitelevel here, setting values to 100% that shouldn’t be.

Again, this is a bug, please report it!

Remarks about ‘DxO blabla do not use ETTR blabla’ are horribly wrong or misinformed.
I have never seen misrendering like this on my cameras, so it is brand/model dependant.

You should never blindly to ETTR these days anyway, but that’s a whole other topic.
(ETTR risks clipping raw data if you’re not careful, and since no camera has a working/useful raw-histogram while shooting, it can be hard for people to figure out how much ‘to the right’ they should go. Go too far, you have clipped highlights, which can be a real pain. If you do not go far enough to the right, you risk more noise, which is far easier to handle).

Darktable opens this file with black levels of 1008 and white level of 15892. And the highlights look OK.

1 Like

dude, you apparently have no clue about other bugs in DxO code, so you might want to actually study the problem before barging in with clueless statements

sure sure :wink:

please try to inject just one and share your observations … I merely shared mine that 81 sensels are enough to trigger white level detection for Zf raw in DxO PL code based on the equal DN values in those sensels that was above max DN values in the original image data … somebody can repeat experience with different DN values still above what was the max DN in the original frame … the horse is dead , there is no need to beat it … a proper commercial raw converter developer does test actual raw files as written by a camera’s firmware across the range of nominal ISO values and number of other possible parameters ( that might affect the matter ) and uses the white level found that way to validate or override whatever either exif tags tell and/or 3rd party library provides…

I am, obviously, glad that you heed the advice of the wise

Did a little test with a zone style exposure series and -compensation with Lightroom and PhotoLab version 7.4 on Mac. Have a look:

Top row: Bracketed exposures as seen in Lightroom
Middle row: Virtual copies compensated according to exposure offset (Lr)
Bottom row: Compensated copies as seen im PhotoLab 7

While all compensated copies should have close to equal RGB values, we can see some differences between Lr and DPL. Note that the two brightest images have highlights burnt to different degrees, which allows us to see how the apps deal with it.

  • PhotoLab is, in absolute terms, closer to what the RGB values should be (122,122,122)
  • PhotoLab’s third image from right is too bright
  • PhotoLab can compensate ±4EV while Lr goes to ±5EV (corrected the leftmost image in the .dop sidecar file as mentioned here. The rightmost image cannot be lifted with editing the .dop file.
  • The rightmost image is darker because of massively burnt highlights, which leaves only the left (dark) part of the bell-shaped distribution in the histogram when exposure is compensated.
  • Lightroom produces a fairly even set of compensated images, but they are a tad too dark (105 instead of 122)

Imo, Lightroom handles the highlights better in the sense of a more even compensation. DPL has this brighter zone 8 image and its limitations at ±4EV…which can be worked around by setting exposure bias with a text editor in the .dop sidecar(s). Note that both ExposureBias values must be changed (in the Base and Overrides sections)

Images taken with a Canon EOS R7 with a FF 28mm lens. Optical corrections only.

Here’s an image case that appears to show the opposite to the OP’s Nikon photo for the experts to consider… i.e. PL does not show blown out highlights, but FRV does.

https://1drv.ms/f/s!AjxAgDyf1SbdguQiDWRKrr1bUaGsww?e=txGjHS)

FastRawViewer and Sony’s Imaging Edge software show clipping, but to a different extent. Sony’s IE software shows levels of 255 in all 3 channels in more areas, however I thought the magenta shading meant only green clipping. Rather, it appears there is clipping in all 4 channels.

Both Photolab 7.1 (build 151) and Adobe Raw do not show clipping. The highest value I could find in the body of the crane was 252.

While FRV might show some clipping, I “assumed” that since AdobeRaw and PL did not, then these programs better “recovered” these highlights.

Can those more knowledgeable suggest why the differences, and if there really is clipping in this part of the bird?





I use FRV for culling but just downloaded Digger trial for this specific comparison. I do not claim any knowledge RAW photo analysis.

Another possible confounding factor is the “blinkie” settings in camera. Per some advice, I upped the camera’s “blinkie” settings to 108+ from 100 to get more range. (This is the setting that shows overexposure in the camera viewer.) I don’t know if this is used from metadata.

Ultimately, I’d like to learn the true upper exposure limit setting for my camera.
And perhaps this alternate case will help the experts sort out what these programs are doing to set white/black points and dynamic range.

Thanks!

green channels clipped - as you can see with black level off DNs are 16383 = 2^14-1

and somewhere blue channel too and red


the only tool to tell you is the tool that directly displays DN from raw channels - that is rawdigger… the rest are just more or less precise, FRV comes close in rawdigger absence.


here is the spot with all 4 channels clipping actually

and if we want to be very precise - there are only few sensels clipped, manually set clipping indication in rawdigger to max DN value and

those are non-essential practically ( as only literally few of them )

PS: also for A1 camera DxO code clearly has different processing for ARW → DxO → … vs ARW → DNG → DxO → … ( unlike for example for Zf where this is identical processing , as shown = Clipped highlights, but only in Photolab? - #133 by noname ) so regular investigations are not possible… is it a bug OR done for a purpose I have no idea

this “simplified” visual of the difference in linear DNG output (NROC only with No Corrections preset ) =