PL4.3.6.32 Ghosting Ratings for Same image in different directories (Win10)

[Joanna (Joanna Carter)]

… contd…

If I choose not to include the parent keywords…


… I now get three Orange tags with no means of determining which is which without having to invoke the tooltip over each one!

Which creates an XMP file…

         <dc:subject>
            <rdf:Bag>
               <rdf:li>Couleur</rdf:li>
               <rdf:li>Fruit</rdf:li>
               <rdf:li>Orange</rdf:li>
            </rdf:Bag>
         </dc:subject>
        …
         <lr:hierarchicalSubject>
            <rdf:Bag>
               <rdf:li>Couleur|Orange</rdf:li>
               <rdf:li>Fruit|Orange</rdf:li>
               <rdf:li>Orange</rdf:li>
            </rdf:Bag>
         </lr:hierarchicalSubject>

This seems fine but the standalone version of Orange has been added to the lr:hierarchicalSubject as if it were a hierarchical keyword instead of just a standalone.

So, what happens if add in the parents of the three Oranges?


We still get three mentions of Orange in the keywords box and the XMP file now looks like this…

         <dc:subject>
            <rdf:Bag>
               <rdf:li>Couleur</rdf:li>
               <rdf:li>Fruit</rdf:li>
               <rdf:li>Orange</rdf:li>
            </rdf:Bag>
         </dc:subject>
        …
         <lr:hierarchicalSubject>
            <rdf:Bag>
               <rdf:li>Couleur</rdf:li>
               <rdf:li>Couleur|Orange</rdf:li>
               <rdf:li>Fruit</rdf:li>
               <rdf:li>Fruit|Orange</rdf:li>
               <rdf:li>Orange</rdf:li>
            </rdf:Bag>
         </lr:hierarchicalSubject>

The dc:subject tag is still fine but the lr:hierarchicalSubject tag still contains the standalone Orange

---

I think there is a problem with the UI mixing the ideas of hierarchy management with application of keywords in the same tool.

In my software, I have a separate popup window to manage keywords…

All possible keywords are in the right column, whilst the left column shows them arranged into hierarchies. To add a keyword to a parent, you just drag it from the right column to the appropriate keyword in the left column. This way, a keyword can be added to multiple hierarchies and can exist as a standalone keyword at the same time.

The number in the right column shows how many files reference a given keyword at any one time and pressing the spacebar on a selected line shows a Quicklook panel with a preview of all images that contain that keyword, regardless of where they are in any hierarchies

This completely avoids the confusion we presently have in the PL5 keywords palette, by ensuring that the operation of structuring the dictionary is completely independent of the operation of applying keywords to files.

Of course, we still have the mess of not knowing which version of Orange we are searching for…

… without having to guess from a list of three and then invoke the tooltip on the inserted token to see if we were right…

[BHAYT (Bryan [WIN10, PL7, VP4, FP7, G9 & EM1 Mkii, RTX 3060])]

@joanna, @jch2103 I started to write a reply to your excellent tutorials but needed to do some experiments of my own to confirm what was going on - plus I haven’t yet read your two new posts and won’t be able to until later today/early tomorrow.

To this end I set up 4 directories for Bridge (‘dc’ only), PM hierarchical but still set to update embedded xmp in RAW, IMatch and PL5 itself. The directories contained my previous test data i.e. 3 JPGs and 1 RAW

The test key groups were applied to 3 of the 4 photos by the respective software, captured to a “Baseline” directory and then processed by PL5 and all 4 exported to a subdirectory.

The keywords used were

1. animal, mammal, bear
2. animal|mammal|bear
3. left empty
4. animal, mammal, bear, animal|mammal|bear

This first post compares how the different packages went about setting up the “baseline” data and contains numerous snapshots! Please ask for any comparison not shown if you want to see a particular combination.

Test 1 - animal, mammal, bear:-

Test 2 - animal|mammal|bear:-

Test 04 - animal, mammal, bear, animal|mammal|bear:-

[sgospodarenko (Svetlana G.)]

Yes, Joanna, this is exactly the behavior currently represented on Mac and Win (just do not knowif Mac has a tooltip over the suggestions):

@Joanna

The problem is, in adding it as a child, it is lost as the standalone keyword I also intended it to be and I can no longer add it to Fruit as well.

What to do?

• I see the same behavior in Lr, correct me if I’m wrong, please.

@Joanna

I think there is a problem with the UI mixing the ideas of hierarchy management with application of keywords in the same tool.

In my software, I have a separate popup window to manage keywords…

All possible keywords are in the right column, whilst the left column shows them arranged into hierarchies. To add a keyword to a parent, you just drag it from the right column to the appropriate keyword in the left column. This way, a keyword can be added to multiple hierarchies and can exist as a standalone keyword at the same time.

The number in the right column shows how many files reference a given keyword at any one time and pressing the spacebar on a selected line shows a Quicklook panel with a preview of all images that contain that keyword, regardless of where they are in any hierarchies

This completely avoids the confusion we presently have in the PL5 keywords palette, by ensuring that the operation of structuring the dictionary is completely independent of the operation of applying keywords to files.

@StevenL could you, please have a look at it?

Regards,
Svetlana G.

[Joanna (Joanna Carter)]

The first two screenshots are a bit weird because they show three separate, standalone, keywords instead of any hierarchical relationship between them. If there is no hierarchical relationship, the hierarchical subject tag should be empty.

The third screenshot (possibly more correctly) shows no hierarchical relationship.

The second three all deal with a badly formatted keyword of animal|mammal|bear, which, as we have previously discussed is only one word, not three, and more than likely requires the authoring software to correctly interpret it. It is unlikely to be universally compatible.

The third three also contain the same invalid keyword, designed to confuse anything that sticks to the “rules”.

If I remember rightly, this kind of “composite” keyword is a speciality of PhotoMechanic and caused @KeithRJ all sorts of grief when we he discovered it isn’t necessarily compatible with all other software.

[Joanna (Joanna Carter)]

@sgospodarenko

(just do not knowif Mac has a tooltip over the suggestions):

No it doesn’t. Yet another difference between Windows and Mac

@sgospodarenko

I see the same behavior in Lr, correct me if I’m wrong, please.

I can’t as I don’t pay rent for software that isn’t as good at editing images as PL5

If this what Lr does, it makes for a very good reason not to use it. Being compatible with Lr doesn’t mean that PL has to copy what they do. DxO should rather lead the field in excellence in metadata handling rather than sinking to Adobe’s poor and confusing level.

[BHAYT (Bryan [WIN10, PL7, VP4, FP7, G9 & EM1 Mkii, RTX 3060])]

@joanna I am a little concerned that I have not fully understood the rules that you are proposing!? During Beta testing this was mostly new to me, I had never used a hierarchical keyword in my life so the way that I went about testing was to compare various programs and the way that they handled various scenarios to see if I could understand the logic.

I restarted that process in the my post above; I was always concerned about mixing PL5 with its “apparent” obsession with ‘hr’ keywords (subject) with ‘dc’ only programs, Adobe Bridge in this test.

So I used one program IMatch to compare the results with PL5 and also to see how your proposals stack up against what these programs are doing with the data. The comparison is Test 2 - animal|mammal|bear and Test 4 - animal, mammal, bear and animal|mammal|bear all explicitly added by the user (me), principally to see what happens!

One problem I had back in Beta testing was that using this approach threw up more inconsistencies than I expected, if we add your proposals then things get even more complicated. So for test 2 we have

                                     IMatch                      PL5

   Subject                      animal|mammal|bear         animal, bear, mammal
   Hierarchical Subject         animal|mammal|bear         animal|mammal|bear

IMatch fails to include the flattened keywords in the ‘dc’ Subject which PL5 has included but PL5 leaves out the hierarchical key from the Subject field and, as a consequence ‘dc’ only programs have no sight/idea of the hierarchical nature of the keywords!

Surely for compatibility the Subject must also contain animal|mammal|bear? Hence, in this case shouldn’t the keywords be Subject = animal, mammal, bear, animal|mammal|bear and HR Subject =animal|mammal|bear?

When we get to Test 4 with the “overloaded” set of keywords we have

                                      IMatch                         PL5

   Subject                      animal, mammal, bear         animal, mammal, bear
                                animal|mammal|bear             
   Hierarchical Subject         animal, mammal, bear         animal, mammal, bear
                                animal|mammal|bear           animal|mammal|bear

Based upon your “recommendation” you would suggest that the flat keys have no place in the HR Subject (it does make sense) but I would suggest that animal|mammal|bear should be present in the Subject field to maintain compatibility with ‘dc’ programs. During Beta testing I declared such a key in Bridge and by the time that PL5/LR had finished with the photo the displayed photo in Bridge no longer had the (hierarchical) keyword that Bridge had assigned to it!!??

The other comparisons that I can make show what an absolute “pig’s breakfast” this area is, we may criticise DxO for not getting it “right” (and they had plenty of expertise to tap into) but I am not impressed with what some of the other programs are doing with keywords either! I am not a standards expert but when I see a keyword “vanish” from Bridge I feel that something has gone very wrong!

[BHAYT (Bryan [WIN10, PL7, VP4, FP7, G9 & EM1 Mkii, RTX 3060])]

A league table of what is in the photo after each of the programs has finished putting in the test data. Test 03 is left deliberately blank just in case I need another JPG for a further test.

What is the “right” way to populate the Subject and HR Subject is what we are now discussing in this topic.

I have not included IPTC in the spreadsheet which I can on request (please send bitcoin to the value of …)

[Joanna (Joanna Carter)]

@BHAYT

During Beta testing this was mostly new to me, I had never used a hierarchical keyword in my life so the way that I went about testing was to compare various programs and the way that they handled various scenarios to see if I could understand the logic.

As a great friend from the southern US used to say - “Thar’s yer prawblem”

I awoke at around 5:00 this morning, thinking about this, and had to do some research there and then. As a result, I think may have discovered an explanation for the differences you are seeing.

I am now fairly much convinced, but couldn’t find a definitive reference, that adding “composite” keywords to the dc:subject tag may have come about at a time when somebody wanted to, somehow, record hierarchy before Adobe introduced the lr:hierarchicalSubject tag.

From what I can see, apps like iMatch and PM try to be compatible with that practice, even though it was no longer necessary with the arrival of the lr:hierarchicalSubject tag.

@BHAYT

I was always concerned about mixing PL5 with its “apparent” obsession with ‘hr’ keywords (subject) with ‘dc’ only programs
…
IMatch fails to include the flattened keywords in the ‘dc’ Subject which PL5 has included but PL5 leaves out the hierarchical key from the Subject field and, as a consequence ‘dc’ only programs have no sight/idea of the hierarchical nature of the keywords!

I’m not sure what you mean by “dc only programs”

@BHAYT

Adobe Bridge in this test.

Adobe Bridge is definitely not a “dc only” program, it both reads and writes the lr:hierarchicalSubject tag.

PL5 tries its best to be MWG compliant and is correct in not writing composite keywords to the dc:subject tag. The only minor niggle that I find with PL5’s writing of lr:hierarchicalSubject is that it sometimes writes non-hierarchical keywords as well.

To my mind, adding such composite keywords to the dc:subject tag should be regarded as a defunct and purely legacy requirement and, upon finding such, any decent app should update them by moving them to the lr:hierarchicalSubject tag.

I will stick my head above the parapet and dare to say that any app that does write composite keywords to the dc:subject tag is not adhering to MWG guidelines and should not be relied on.

On the other hand, PL5 needs to “tidy up” what it writes to lr:hierarchicalSubject, omitting purely non-hierarchical keywords.

@BHAYT

Based upon your “recommendation” you would suggest that the flat keys have no place in the HR Subject (it does make sense)

Absolutely. It goes against MWG guidelines.

@BHAYT

… but I would suggest that animal|mammal|bear should be present in the Subject field to maintain compatibility with ‘dc’ programs

And I would have to disagree if PL5 is to be seen as a “correct” manager of keywords. Which isn’t to say that PL5 shouldn’t read and convert such into its correct form. If a “dc only” app can’t cope with that, it is that app that is at fault, not PL5.

I did try PM for its free trial period and, from what I remember, it could read lr:hierarchicalSubject. I can’t say for iMatch but, if it can’t, I would regard it as deficient and not MWG compliant.

@BHAYT

During Beta testing I declared such a key in Bridge and by the time that PL5/LR had finished with the photo the displayed photo in Bridge no longer had the (hierarchical) keyword that Bridge had assigned to it!!??

Maybe during beta testing, but now, assigning my test hierarchies in Bridge gives me an XMP file with…

   <dc:subject>
    <rdf:Bag>
     <rdf:li>Satsuma</rdf:li>
     <rdf:li>Orange</rdf:li>
     <rdf:li>Fruit</rdf:li>
     <rdf:li>Couleur</rdf:li>
    </rdf:Bag>
   </dc:subject>
   <lr:hierarchicalSubject>
    <rdf:Bag>
     <rdf:li>Fruit|Orange|Satsuma</rdf:li>
     <rdf:li>Fruit|Orange</rdf:li>
     <rdf:li>Fruit</rdf:li>
     <rdf:li>Couleur|Orange</rdf:li>
     <rdf:li>Couleur</rdf:li>
    </rdf:Bag>
   </lr:hierarchicalSubject>

… and PL5 picks that up perfectly as…

… even though Orange is duplicated in the keywords field.

Even with the matter of an abysmal and cramped UI for keywording, I would say confidently that I would rather use PL5 over any of the others that you have talked about because it is the lost MWG compliant. If the other apps you have tested can’t read what PL5 writes, then they are wrong.

---

N.B. Since the release of PL5, the integrity of keywording has improved on what it was during the beta, but I would still take the recommendation to try and stick to a single metadata manager rather than move back and forth between them. This becomes even more important in the light of your findings.

---

N.B.B. Don’t ever use composite keywords in the dc:subject tag. It is apps that allow this that are the source of a lot of the headaches people are finding with compatibility, not PL5, which is “almost” MWG compliant.

[Joanna (Joanna Carter)]

Here is your table “marked” for correctness

The HR subjects marked as wrong in Test 01 are because those apps are unnecessarily, but not necessarily wrongly, adding non-hierarchical keywords to the HR subject. In any case, adding them will not cause problems for searching or transmission of hierarchical data.

For Test 04, the first three are marked wrong for the same reasons as Test 02 - adding of composites to the Subject is incorrect. The last two are “sort of” right and, as with Test 01, there is nothing that should affect either searching or transmission.

---

Side note

PL5 seems to search based on HR subject. This might not be strictly wrong but it does restrict results and make predicates harder to define. All searches should be based on Subject alone.

[platypus (Canon & Mac (Intel and M1))]

@sgospodarenko

PL5 metadata values are written in .dop only for virtual copies as we can’t create a separate xmp for virtual copies.

I’m not sure if I understand the first part of the statement: I find DPL5 metadata in .dop sidecars, no matter if virtual copies are present or not. This would mean that metadata is written to .dop sidecars unconditionally in order to provide MD to virtual copies that might (or might not) be added later, even if the database had been deleted in the meantime. Note that I have no problem with that.

I do propose that DxO also read such MD from the .dop sidecars in order to restore or complement info to the database. If MD were also present in the image file or .xmp sidecar, some kind of dialog should appear to let the user select what action should be taken.

Imo, a sync warning (@BHAYT) is no valid solution in a world of more than one app editing metadata. Simply overwriting MD read from whatever source has been prioritized by someone who is not the user is not a decent way to act either.

Update: Check this out: PL5 Tag Field not read from .dop file - #37 by Musashi

[BHAYT (Bryan [WIN10, PL7, VP4, FP7, G9 & EM1 Mkii, RTX 3060])]

@Joanna sorry about “causing” you to wake up early and be so “rivetted”/“inspired”/“puzzled”/“cross” (delete or add or replace as appropriate) that you could not get back to sleep. The central heating has had the same effect on me! Finally having plumbed the central heating into the mains water supply (via an appropriate non-return valve) we flushed the system out (and pushed up the water bill) because the new pump was still not coping.

One bottle of X800 cleaner later and slowly, slowly the heat started reaching the farthest parts of the bungalow, just in time because we had a hard frost last night. Today/tomorrow the X800 has to come out to be replaced by X400 cleaner which can stay in for 1 month!

@Joanna

@BHAYT

Adobe Bridge in this test.

Adobe Bridge is definitely not a “dc only” program, it both reads and writes the lr:hierarchicalSubject tag.

I just updated to the latest version of Bridge 11 (not Bridge 12) on the test machine and that has only ever used ‘dc’ fields, at least in all my tests!? Zoner also still seems to stick to ‘dc’ fields even when inputting an hierarchical keyword. By this I mean that those programs only set ‘dc’ fields and only display from ‘dc’ fields. I am currently downloading the latest version of Bridge and when that is installed I will retest and update the post and the table as appropriate.

@Joanna

To my mind, adding such composite keywords to the dc:subject tag should be regarded as a defunct and purely legacy requirement and, upon finding such, any decent app should update them by moving them to the lr:hierarchicalSubject tag.

I understand that now and I am “sorry” for being such a “naughty” tester but if programs that are on the market only use ‘dc’ fields and start using the hierarchical keywords it certainly looks “weird” when the keys you entered vanish after a “visit” to LR or PL5 or …

Sometimes ignorance of the accepted way of how things work is actually an “asset”, it leads to discovering the oddities that those only treading the right path can miss!

@Joanna

I will stick my head above the parapet and dare to say that any app that does write composite keywords to the dc:subject tag is not adhering to MWG guidelines and should not be relied on.

I think there will be problems with users of older products that might be disappointed except that the chances are that they will never have used hierarchical keys!?

@Joanna

it is the lost MWG compliant.

I think that you meant “it is the most MWG compliant”.

@Joanna

N.B.B. Don’t ever use composite keywords in the dc:subject tag. It is apps that allow this that are the source of a lot of the headaches people are finding with compatibility, not PL5, which is “almost” MWG compliant.

Understood and I discovered that they could be added just by experimenting but if there are any real users out in the world then …

An updated version of the table, if any one wants the spreadsheet I can make it available and they can add Photo Supreme, Capture 1 or whatever!

[platypus (Canon & Mac (Intel and M1))]

@BHAYT

spreadsheet

Can you post a link to a share?

[BHAYT (Bryan [WIN10, PL7, VP4, FP7, G9 & EM1 Mkii, RTX 3060])]

@platypus try this Dropbox - Public - Simplify your life it is an xls file and please tell me if it downloads successfully.

[platypus (Canon & Mac (Intel and M1))]

Added listing for C1 and DPL5 (Mac).
meta data setting _01-B.xls.zip (3.1 KB)

Notation is a bit different, but more accurate imo. Change as you think you should…

Note that

• DPL5 reads C1’s keywords correctly.
• LrC only displays “bear” as keyword from DPL .xmp sidecar. LrC’s notation of hierarchical keywords is similar like DPL5’s.

New in Lightroom: lr:weightedFlatSubject

• dc:subject
  rdf:Bag
  rdf:lianimal</rdf:li>
  rdf:libear</rdf:li>
  rdf:limammal</rdf:li>
  </rdf:Bag>
  </dc:subject>
  lr:weightedFlatSubject
  rdf:Bag
  rdf:libear</rdf:li>
  </rdf:Bag>
  </lr:weightedFlatSubject>
  lr:hierarchicalSubject
  rdf:Bag
  rdf:lianimal|mammal|bear</rdf:li>
  </rdf:Bag>
  </lr:hierarchicalSubject>

[BHAYT (Bryan [WIN10, PL7, VP4, FP7, G9 & EM1 Mkii, RTX 3060])]

@Joanna I have read the details about setting hierarchical keywords in Bridge. I have downloaded the latest version on my main machine but no matter what I set I cannot get Bridge to write anything but ‘dc’ fields!!

I set the keywords here:

and have not found any other field that looks a likely candidate!

[BHAYT (Bryan [WIN10, PL7, VP4, FP7, G9 & EM1 Mkii, RTX 3060])]

@platypus I did amend your fine work but also left it intact. I wanted to see the keywords without the added xmp “stuff” they were “confusing” an old man, i.e. I “could not see the wood for the trees”. So here is your version slightly altered;

and I have attached the spreadsheet here Copy of meta data setting _02.zip (12.3 KB).

What is the intended purpose of lr:weightedFlatSubject?

[platypus (Canon & Mac (Intel and M1))]

Expressions in square brackets are like opening and closing brackets (the one with the /) and x stands for the content that is packed between the brackets. That’s not too complicated, is it.

I have no idea what the new tag is about, it’s new in LrC11. Phil Harvey knows about the new tag(s).

attn. @Joanna and @sgospodarenko

[jch2103]

Sorry, I missed the last 24 hours of discussion on this topic (wrong TZ/other things going on).

@BHAYT - I saw your post PL4.3.6.32 Ghosting Ratings for Same image in different directories (Win10) - #73 by BHAYT, but I didn’t understand how you originally generated the dc:subject and hierarchical keywords in your examples (i.e., in which program(s)).

Regarding IMatch specifically, your findings don’t match what I see. I have a NEF raw image for which I used IMatch to add the hierarchical keyword ‘animals|mammals|bear|black bear’ (to the attached XMP sidecar, not the NEF). _DSC0388.xmp (7.5 KB) Note that IMatch records the hierarchical keyword as ‘animals|mammals|bear|black bear’ and dc.subject as:

dc:subject
rdf:Bag
rdf:lianimals</rdf:li>
rdf:limammals</rdf:li>
rdf:libear</rdf:li>
rdf:liblack bear</rdf:li>
</rdf:Bag>

This is as I would expect. Note especially that dc.subject generated by IMatch doesn’t include a hierarchical keyword, again as I would expect, but your example for IMatch showed otherwise. This is why I would like to see how you generated your examples.

I then processed the NEF in PL5. In its Metadata section, PL5 shows only ‘black bear’ in the Keywords tag. In the Keywords section, it shows:

animals
mammals
bear
black bear

I then did an ExifTool dump of the output jpg metadata (ExifTool dump of PL5 output jpg.txt (22.0 KB)
). The only keyword it contains is ‘black bear’ in [XMP-dc] Description (i.e., in dc-subject) and no reference to hierarchical keywords. As has been discussed elsewhere here, the hierarchical nature of the keywords has been completely lost in the output image. PL5 may retain these in its database for the raw NEF, but it’s disconcerting for it to have lost them in the output image! To do so seriously interferes with any efforts to comprehensively manage keywords.

[In addition, PL5 has added an entire Legacy IPTC section to the output file that did not exist in the original NEF/XMP! That’s a problem, although outside this discussion.]

@BHAYT - To recap, given the number of program you’re testing for metadata handling, it’s important to know the processing sequence(s) that you used in your examples, particularly the program(s) used to originally generate the data. I do appreciate the effort you’ve been putting into this.

[BHAYT (Bryan [WIN10, PL7, VP4, FP7, G9 & EM1 Mkii, RTX 3060])]

@jch2103 I will read the post properly later and check my results but most of the programs were run stand-alone, no simultaneous opening of programs and restricted to their own directory, I say most because it is possible that PL5 was open during one test but I will check the results and post any addendum when I can.

[jch2103]

@BHAYT

What is the intended purpose of lr:weightedFlatSubject?

Apparently this is an Adobe-created tag: “LR 11 added a very limited ability to order the keywords for export to Adobe Stock but not other stock agencies.” See https://community.adobe.com/t5/lightroom-classic-ideas/p-custom-sort-order-of-keywords-instead-of-alphabetic/idc-p/12526371/page/5 for more details. Doesn’t look useful for much else due to limited adoption.