PL 5 Keywords Handling: Output Files

jch2103 · November 5, 2021, 5:13pm

I’m finding that PL 5 is changing the hierarchical keywords for my output files, which is disconcerting and annoying. Specifically, PL 5 is flattening some of my hierarchical keywords and duplicating them in the output files.

Example:
NEF original (keywords stored in XMP sidecar, attached 2021-10-13 12-23-04 NIKON Z 6.xmp (7.9 KB)):

Vacation|SW w Dave
National Park|Hovenweep National Monument
architecture|cliff dwelling

JPG output file (keywords in JPG file):

National Park|Hovenweep National Monument
Vacation|SW w Dave; architecture|cliff dwelling
National Park
Vacation
architecture

ExifTool output for JPG (keywords/Subject)

[XMP-dc] Subject : Hovenweep National Monument, National Park, SW w Dave, Vacation, architecture, cliff dwelling
[XMP-lr] Hierarchical Subject : National Park|Hovenweep National Monument, Vacation|SW w Dave, architecture|cliff dwelling, National Park, Vacation, architecture

This shouldn’t be happening, but is part of the issue regarding PL 5’s handing of keywords that has been discussed extensively elsewhere on this discussion board. PL 5 appears to be confusing dc:subject with lr:hierarchical subject. They’re related but should be
distinct metadata tags.

Prior to a fix/changes by DxO, how can I prevent this from happening? I don’t use PL 5’s DAM at all (I use IMatch for my metadata handing).

platypus · November 5, 2021, 6:25pm

Indeed, and…

If you prefer to work with iMatch, stick to it and disable synchronisation of XMP in PhotoLab’s preferences/settings…

BUT: PhotoLab 5 still exports keywords in it’s own manner.

Left: Hierarchy written and exported JPEG file by DPL5
Right: Hierarchy written in .xmp sidecar by Lr11, file opened in DPL5 and exported as JPEG

jch2103 · November 5, 2021, 7:25pm

As you say, it’s best to only use one application for metadata editing.

Unfortunately, this is happening even though I have XMP synchronization turned off. PL 5 is changing how the keywords are stored internally, which in turn is affecting keywords in output files. Hence my question to DxO about how I can keep this problem from happening.

platypus · November 5, 2021, 7:35pm

Maybe we need another feature request for it…

jch2103 · November 5, 2021, 8:25pm

I was about to make a feature request, but on further reflection I believe what I’m describing is a bug. That is, if PL5 didn’t alter the keywords (subject and hierarchical), there would be no need to add a blocking feature. I’m fine if PL5 just passes existing metadata from the raw original to the output images (just like PL4 does). It’s when it changes the metadata that there’s a problem.

@joanna - As you know, there are still issues w/ PL5 metadata handling…

platypus · November 5, 2021, 9:56pm

Here’s a collection of how keywords are exported by Capture One 12 (first four lines), Lightroom 11 (blue lines) and PhotoLab 4 (last 3 lines)

There are many ways to write keywords , which one ist the bug?

I’d stick to a feature request asking for an option for passing keywords along as they came in.

Joanna · November 6, 2021, 10:30am

In actual fact, PL5 is only writing out keywords in a “permitted” format.

Your iMatch XMP file contains…

  <dc:subject>
   <rdf:Bag>
    <rdf:li>Vacation</rdf:li>
    <rdf:li>SW w Dave</rdf:li>
    <rdf:li>National Park</rdf:li>
    <rdf:li>Hovenweep National Monument</rdf:li>
    <rdf:li>architecture</rdf:li>
    <rdf:li>cliff dwelling</rdf:li>
   </rdf:Bag>
  </dc:subject>
  …
  <lr:hierarchicalSubject>
   <rdf:Bag>
    <rdf:li>Vacation|SW w Dave</rdf:li>
    <rdf:li>National Park|Hovenweep National Monument</rdf:li>
    <rdf:li>architecture|cliff dwelling</rdf:li>
   </rdf:Bag>
  </lr:hierarchicalSubject>

The exported JPEG contains the equivalent (in XMP format) of…

  <dc:subject>
   <rdf:Bag>
    <rdf:li>Vacation</rdf:li>
    <rdf:li>SW w Dave</rdf:li>
    <rdf:li>National Park</rdf:li>
    <rdf:li>Hovenweep National Monument</rdf:li>
    <rdf:li>architecture</rdf:li>
    <rdf:li>cliff dwelling</rdf:li>
   </rdf:Bag>
  </dc:subject>
  …
  <lr:hierarchicalSubject>
   <rdf:Bag>
    <rdf:li>National Park</rdf:li>
    <rdf:li>National Park|Hovenweep National Monument</rdf:li>
    <rdf:li>Vacation</rdf:li>
    <rdf:li>Vacation|SW w Dave</rdf:li>
    <rdf:li>architecture</rdf:li>
    <rdf:li>architecture|cliff dwelling</rdf:li>
   </rdf:Bag>
  </lr:hierarchicalSubject>

The difference being the xmp-lr:hierarchicalSubject tag additionally contains the root of each hierarchy, instead of just the path to the selected hierarchical keyword.

This is not a bug; it is perfectly normal and follows MWG guidelines. In fact, by doing this, PL5 is ensuring that the recorded hierarchies are more likely to be compatible with other metadata management and search software.

In fact, it is iMatch that is not including the full recommended definitions in the XMP file

jch2103 · November 21, 2021, 9:23pm

@platypus - I’ve made a feature request: Feature Request PL5: Pass through keywords without change
and
Feature Request PL5: Pass through keywords without change

BHAYT · January 27, 2022, 10:18am

@jch2103 I am hopelessly confused by your post and I am sorry that it has taken me this long to get to review it. My confusion is that when I attempt to put the keywords that you have into IMatch I actually get out what you are suggesting PL5 is putting in with the tests you have done!!

So Imatch, with my ‘Preference’ settings is getting (this is Picmeta PIE output)

This is me entering (or attempting to enter!!) your hierarchical keywords into IMatch and it created the very keywords you suggest that PL5 has added, i.e. with my IMatch settings I cannot recreate your NEF .xmp sidecar file this is what I got out of IMatch 2021.14.2 which uses ExifTool 12.39 instead P1074441.xmp (5.2 KB).

This is actually a better match for what you are suggesting PL5 created when it encountered your sidecar rather than the sidecar you published.

Even worse in my tests PL5 did not add anything to your sidecar and the exported JPGs exactly matched your xmp sidecar (certainly in version PL5.1.3) of all the programs I tested only two added the ‘flat’ keywords to the JPG ‘hr’ data namely, C1 and IMatch itself!!!

To summarise:

I cannot recreate your sidecar using IMatch (with my IMatch setting etc.)
PL5 when it encountered your sidecar left things alone in the exported jpg.
C1 and IMatch when using your sidecar settings actually did add the additional keywords to the ‘hr’ but no other programs that I tested did!!

I am truly sorry I ever encountered this and the related posts because my results make no sense against yours or vice versa.

We need a “drains up” chat and then I will eat “humble pie” when you show me the error of my ways!

PIE also uses ExifTool like IMatch and …

Edited 2022/01/27 @ 12:31:-

The following shows my current settings and the keywords that I entered into IMatch for the photo (sorry the photo is nowhere near as interesting as I suspect yours was!)

The following is a comparison of the sidecar files, IMatch of the left - your data on the right!

’dc’:-

’hr:-

jch2103 · January 27, 2022, 11:59pm

It may take a bit to unpeel all this, especially with all the examples of metadata management we’ve all discussed and the number of times the discussion has been sidetracked.

How about this? Here’s a link to a NEF with XMP sidecar plus PL5 DOP sidecar plus output JPG: Microsoft OneDrive

I set a hierarchical keyword in IMatch for lr:hierarchicalSubject (animals|bird|kestrel|american kestrel). IMatch use this to create a set of flat keywords in dc:subject. The keywords are stored in the XMP sidecar file. You can see the hierarchical keywords via the “1 Default” metadata panel; IMatch hides the dc:subject keywords by default, although you can add that tag to the default panel if you want. The hierarchical keywords were animals|bird|kestrel|american kestrel. The dc:subject contains ‘american kestrel’; ‘animals’; ‘bird’ and ‘kestrel’.

I then used PL 5.1.3 (newest version!) to create a jpg output image. The jpg image hierarchical keywords show up in IMatch as animals|bird|kestrel|american kestrel; animals; bird; kestrel. I.e., ‘animals’, ‘bird’ and ‘kestrel’ have been added to ‘animals|bird|kestrel|american kestrel’. (both NEF and JPG have the same dc:subject contents.

I’ve included ExifTool dumps of both the NEF and JPG files. Note that ExifTool refers to [XMP] Hierarchical Subject for lr:hierarchicalSubject and [XMP] Subject for dc:subject.

(You’ll also see that the ‘Date Subject Created’ shows the PL5 bug of adding a ‘+00:00’ time zone in place of the ‘-07:00’ in the orginal NEF. However, the jpg no longer contains Legacy IPTC tags with the 5.1.3 update.)

I’m not sure how the metadata will show up in the tools you’ve been using. I’ve attached screenshots of my IM preferences for Metadata and Metadata 2.

Hope this helps! Let me know if you have questions.

John

BHAYT · January 28, 2022, 9:41am

@jch2103 Sorry that the saga continues but … It not only continues but gets more and more curious. I will write-up this shorter test now although your time zone will mean that you won’t get to this for some time. I will then put the data through the other products and see what they make of them!

Test environment:-

Windows 10 Version 21H2
DxPL Version 5.1.3.4720

Review Products:-

Exif Pilot 6.11.1 (Beta & there appears to be a bug in the HR Subject
display)
PicMeta PIE 7.51.12.37 (FREE Version) (uses ExifTool)
Beyond Compare 4.4.4 (build 26165)
Sticky Preview v2.4 - Jun 5 2020 (to provide view windows)

Test script:

Create appropriate directories for testing to protect original data
Navigate to directory containing photo supplied and associated DOP in PL5
Export the photo to JPG
Review the output and gather snapshot results for Review

Results:-

The test did not produce the same results as the supplied JPG from @jch2103, the additional fields in the output JPG are not present!

Conclusion:-

Unprintable, getting banned from the forum might end this mental torture and make my wife “happier” but…

Snapshots:-

Post test snapshot with overlay of PIE output (blue) and Exif Pilot table and table!

Beyond Compare comparison of the jch2103 JPG supplied and the JPG created by PL5 on my Main system! Only the GPS data has been obscured!

THE SAGA CONTINUES

EDIT 01 - 2022-01-28 @ 12:16:-

One reason for the problems that I have been having is getting different results from the tests with IMatch, a problem that @jch2103 is highly experienced in using and I only bought a license when it was on offer at Christmas, so I am a real novice. However, I hate the idea of running a test twice and getting apparently different results!

I started to rerun part of my battery of tests pushing the jch2103 image through different software to see what the software makes of that data. Both PM and C1 added flat keys to the exported JPG and, worse, also added those keys to the .xmp sidecar file! How I can prevent C1 doing that I don’t know and don’t really care but I will take a look.

However, I had hovered over an icon in the IMatch thumbnail that indicated that there was metadata to be written and I had pressed that Icon and continued with the test. So, I set up a new test with the original data, ignored the icon and attempted an export. IMatch warned me that there were metadata writes pending etc. etc. to which I said NO and wound up with the unadulterated JPG from IMatch.

So IMatch wants to add flat keywords to the ‘hr’ data which it must consider to be the correct form of that data, @joanna may have something to say about that (she has already). But as a result it is possible to get different outputs from IMatch, at least when you are as inexperienced with that program as me!

Test 1 - Metadata updated by clicking on icon:-

Test 2 - Icon ignored and warning ignored:-

Comparison between xmp sidecar files after Test1 and Test 2:-

jch2103 · January 28, 2022, 6:48pm

I’ll check what you report closely. But a note about IMatch data handing: It caches metadata in its database for performance purposes, and then subsequently writes back update changes to files. More details here: IMatch Help System
One of the useful things about IMatch is its superb Help system. There’s a lot of information there (can even seem overwhelming at times) but it explains a lot of how IM works and what it does with metadata.

Metadata is a complicated mess, as you fully know! It may take me a day or two take a closer look at your information.

One question: You refer to Exif Pilot. I assume you’re referring to the one from Editing, Creating and Viewing EXIF data with free Exif editor rather than something from https://www.picmeta.com/?

BHAYT · January 28, 2022, 6:53pm

@jch2103 Whenever you have time.

However the key element is that I imported your file into IMatch which was fine but then unprompted IMatch set the ‘update’ pending icon and the updates that were pending were the very elements that you were concerned about with PL5. C1 also did the same thing and both programs changed the original 'xmp as well as putting the fields in the exported JPG!

Marie · April 7, 2022, 2:38pm

Hello,

PhotoLab 5.2 has been released today and should improve the export of hierachical keywords.

Regards,
Marie

BHAYT · April 8, 2022, 3:49pm

@Marie I wanted to write my shortest post yet which would have been How!?

That is what I want to know so that I can test whatever the improvement is effectively so thank you for the post and the “heads up” but in what way does the handling in PL5.2.0 differ from the handling in an earlier version of PL5?

What example keywords should my test data contain and what will the revised outcome look like?

Marie · April 8, 2022, 4:08pm

Hello Bryan,

What we fixed is preservation of hierarchy in keywords.
There was a confusion on behavior of dc:subject and lr-hierarchicalSubject , result was you got your keywords with hierarchy but also doubled with same keywords all set to the same level.
Behavior in LightRoom is unchanged as it was fine but now C1 and PhotoMechanic get the correct hierarchy in keywords in exported images from PhotoLab.
I’ll ask for an XMP test file.

Regards,
Marie