Keyword bugs and enhancements

I am a lucky person who does not use any keywords at all :innocent: … and it is a bliss

2 Likes

In Bridge, thus i think also in LR, you can mark parents as visible and as hidden.
So
When i click on [birds] which has [animals] as parentkeyword, both are V-ed but only “birds” is visual beneath the photo in the information.
But i can set this hierarchical tree as full visible. (which i don’t have set by the way.)

Yes, i am have often keywords like person this and that and scene kind tag and a event kind tag. So that’s already a lot of visual keywords by each photo. Thus visualise the hole string of each would be a crowded infoblock.:grin:
But it could be usefull for homing in a image to start high in the treebranches of multiple parentkeywords and from there fine tune the selection.

Bride can do this as i wrote elseware.
In fact you can create many selections which are stored inside the dxoDB as projects. (named as “externals” ) and i didn’t try it yet but i assume you can then change the external name to anything you like.

I have often bridge and dxopl open at the same time. Aslong as you don’t create a conflict by adding info at both side’s before data sync of both your fine.
Keep keywordmanagment outside dxopl and only add things in iptc fields in dxopl. And i think youre fine.

I’m not sure if I understand what do you mean.
This is normally keyword view in LR

Only “Bienenfresser” is assign as keyword to photo, but it’s part of hierarchy.

Why only one part here and other part outside? A little carelessness and the whole job is destroyed.
If someone really wants to have order, then do everything outside of DxO PL and only do the image processing in DxO PL

… amen !


search for “type” doesnt get any result (it’s only a header for a row of keywords)

Does get result because it’s a hierachical row of keywords.

BUT;

these are test files so i don’t care but for my exported archieve:
i have to clean up this keyword propertie mis writing… :cry:
easy by search for “type”, then “select all” then un check “type” done.
( best by some restriction in foldertree to have not too much images in one go.)

That’s why i manage in bridge and only add geotagging in DxO because there i can copy past Googlemaps coordinats and in bridge it doesn’t work. ( wrong type)
rest of iptc fields i manage in Bridge. :slightly_smiling_face:

I say that because it simply doesn’t work for anything other than single condition predicates or compound predicates that have been ANDed. Try looking for " this OR that" and you will find it is impossible.

What I do is do a metadata search in my software (or any other) - select all the resulting files and then “Open with…” PhotoLab.

Warning

Up to PL5, you did not need to create a project - one would be created automatically, named after one of the folders that contains some of your files.

This has now been changed for PL6 and PL7.

You now need to create an empty project, which you can name appropriately and select, before sending the files from your DAM software.


Here, I search for Cactus in my software…

Now, I select all images and use the context menu…

… to open the project in PL7…

I can now select which virtual copies I want to keep and delete those I don’t. Deleting from a project does not delete either original files or virtual copies.


Once you have refined your selection, then just do a bulk export of all those that remain.

now i notised some hicups in my keyword structure i have a quick repair:
advanged search in bridge : in each main folder (second after "foto’s)
if they show up select all and turn of [Type] ,all of them are jpegs so made by dxopl.
have to check if dxo plv7.2 now has better understanding of this “drawer label” stuff

My point is that this requirement for ORing is one you have and I don’t. Therefore, a blanket statement that no one use PL for searches is one I don’t agree with. If the ability to OR terms is necessary, then by all means, avoid PL keywords.

If I wanted to find birds by species and color I would use this hierarchy:

Birds > Species > …
Birds > Color > …

I don’t have a need to find, say, all Thrushes AND all red birds.

Interesting. Until yesterday, I didn’t know that PL could open files from the command line and I’ve never created any projects that I know about. I picked images in different folders to see what would happen and all three images showed up in PL. I’ve looked in the database and the Projects table is empty.

Of course, I wasn’t trying to create a project. What I saw looked more like the results of a search. Since I haven’t found the command line documentation, perhaps you were using some option that creates a project from a set of images.

The workflow you are pushing is not one I want to use. Here’s what I would be doing:

  • Open PL to view a folder.
  • Note that I have at least one image I want to keyword tag.
  • Open some other tool to view the same folder. The images would be shown in their unprocessed form.
  • Go back to PL to find the first image I want to tag.
  • Go to the other tool to find the image. Like you, I might have several images that look very similar in their unprocessed form, so I’m going to have to be careful to match by filename. Many bird photos are shot in dark environments, which makes the unprocessed images even harder to match visually.
  • Tag the image in the external tool.
  • Repeat for the next image.
  • Repeat for the next folder.

Most external tools don’t know about virtual copies. Perhaps yours does? I have a number of virtual copies whose keywords differ from the source file.

Assuming a hierarchical structure of…

Bird | Thrush | Song Thrush
Bird | Thrush | Blackbird
Bird | Thrush | Red-throated
Bird | Thrush | Black-throated
Bird | Thrush | Fieldfare

… and that you want to find all images of more than one type of Thrush (e.g. Red-throated Thrush, Black-throated Thrush)?

How do you specify a search for images that contain both or either of those varieties?

With DxO’s current limited search, you could only specify images that contain both, but never images that contain one or the other.


You don’t have to try to create projects. Just firing off an “Open with…” PL7 from your DAM or Finder is sufficient for PL to automatically create one or more, containing the imported images.

For example, I can also search for files using Finder’s Spotlight search…

… select all, right-click to reveal the context menu and select “Open with…” DxO PhotoLab 7.

As I mentioned in my previous post this will open them in PL7, in a project.

And, as I also mentioned in my previous post, this also works for selecting images from most other DAMs and image managers.

Then you are well and truly stuffed as the only way to work would be entirely within PL.

But, nonetheless, you can still do various searches and add the results to the same temporary project for exporting.

Wel, indeed virtual copy’s are DB/dopfile only and thus exclusive dxopl ecosystem.
If you use those as variations which you won’t export then you don’t have a rgbfile with the properties (jpeg/tiff) but when you use VC’s to create multiple export of the same rawfile you find those and those point towards the rawfile and command open with dxo will direct you to the VC’s.

So i don’t see a closed in functionality. Unless you use VC’s as non exported image vararities storage which i can’t think of why?

1 Like

Just to be clear: I am not saying that some people might find the lack of an OR option restricting. I am saying that it’s not a problem for me.

Given your example, I normally would only be interested in a specific thrush or all thrushes. In the last 6 years, I haven’t had any cases where that’s come up. I get that you find this restrictive enough to write your own tool to work around the problem. For me, it’s not an issue.

Good to know. I was just putting files on the command line. As far as I can tell, this does not put them in a project.

Well, I don’t agree that I am “well and truly stuffed”, but yes, my goal is to work in the PL environment for now. I do want to preserve as much as I can should I switch tools, but if I were to switch tools, I would lose a lot more important stuff than keywords from virtual images (e.g. all the image adjustments).

Your path was to create your own tool to work around PL’s limitation. My path is to create a tool to fix database/DOP/XMP/RGB file keyword inconsistencies, which is the problem I am most concerned about. If it works, my keywords should be usable by any external software–except for the virtual images, of course.

Actually, I started writing my app long before DxO started the beta test for PL5 and it was this that allowed me to contribute so much to the discussion on compatibility and cross-DAM working.

Now that’s what I call hard work and, as I mentioned, totally unnecessary.

Your main problem is that absolutely nothing other than PL can read or interact with DOP files or the database.

I have tried to parse DOP files but, as yet, have not completely succeeded as they are not in an expected format like JSON. The nearest I can get is that it might be an implementation of LUA, but finding an API that will integrate with tha macOS frameworks.

I was testing a feature. If I wanted to do this from, say, a PHP script or some other software program, I could use the PL command line to open images in PL. You can’t exactly select a set of files in Explorer and click “Open with…” from a script.

The point is not how I did it, but that you posted this warning:

I have no idea for what cases your warning might be correct, but it isn’t correct when images are opened through the command line—no project appears to be necessary. There may be other ways for DAM software to signal to PL that it should open various images. Some of those ways may require that an empty project exist.

This is not my main problem. My main problem is that PL doesn’t keep the database in synch with what it writes to the DOP/XMP/RGB files.

It is not even true that “absolutely nothing other than PL can read or interact with DOP files or the database.” I’ve currently interacted with the database using a tool called DB Browser and written an SQL query to get access to all of the filename/keyword info. I am now working on tools to interact with the DOP/XMP/RGB files so as to enforce keyword consistency.

For my purposes , it is not necessary to parse the entire DOP file, just the keyword parts (there is one set per item, and there can be multiple items in a DOP).

…hardly a normal user’s approach to fix DPL’s keyword shortcomings, e.g. in comparison to what Lightroom Classic can do.

Anyway, the title of the thraed says bugs and enhancements…and I think that many of the things mentioned as shortcomings or bugs could lead to separate feature requests for better visibility. Not that this will shorten the time to implementing those features…
:person_shrugging:

I agree, not a normal approach. I wouldn’t expect anyone else to attempt this, but there seem to be a lot of programmers in this group so you never know…

I’d prefer to see DxO fix their bugs. If I can come up with reproducible bugs, I’ll submit them. For the moment, my focus is on seeing if I can straighten out their mess for myself.

I have no idea what Lightroom Classic can do. It’s a subscription product, so that’s where I stop looking.

Before someone suggests some other product, I’ll repeat that I’m doing the work so I can do my keyword searches directly in PL. I have no objections to other people choosing alternate workflows—if you’re happy with what you’re doing, stick with it.

Thanks for the reminder! The initial list of enhancements I placed in the OP received were not discussed much, perhaps because so many people are working around PL’s keywording. At some point, I will post them as separate feature requests.

Lightroom’s Library module can be used for the price of registering. and Bridge can be used without having to pay too. You could try these apps and fix your mess using tools that are better than PhotoLab in this respect - because they feature the necessary functionality.

Though I can understand the thrill of the challenge to fix things with PhotoLab alone, I propose you consider using alternative apps as you consider database editing, which is not really a built-in solution either.

Any alteration of the database requires a deep knowledge, not just of the table structures, but also of any triggers or stored procedures that might be present. Also, you could easily accidentally circumvent any referential integrity constraints that exist by modifying data in one table without regard to other affected tables.

I would never recommend “kludging” a database as the owner of the database design could change its structure at any point and totally invalidate anything you have written.

Then you have the problem that the macOS database is auto-generated from an object model created using Apple’s Core Data mechanism but the same doesn’t apply for Windows, therefore the structures (and especially the table and column names) are going to be different.


Then you have to bear in mind that the only way data integrity between the database and DOP files is enforced is by means of code written in Swift, Objective-C and C++ for macOS; or in C#, C and C++ for Windows. Not discounting any third party libraries that might be written in other languages.


Now, do changes get written to the DOP file first, then propagated to the database, or vice versa? There is a chance that both might apply so, depending on the operation, you might either get a missing update to the secondary destination or a duplicate, triggered by code unknown to you.


Although I don’t agree with @platypus suggestion on using Adobe tools, I do agree that you should use a “DAM” that works in the most standards complant way using either XMP sidecars or directly embedded metadata in JPEG, TIFF and other file formats.

As I have said before and others have noted, I am a strong believer in SPOD (single point of definition). As soon as you make an XMP sidecar the point of definition for your keywords, possibly via an external DAM, then that should be the only place that those keywords exist.

Unfortunately, because Windows doesn’t provide a built-in metadata indexing and searching engine, DxO were forced to use a database as a means of indexing and searching.

macOS, on the other hand, provides the Spotlight mechanism, which is continually indexing automatically, in the background, all files written to the disk. So, on macOS, my app can find any file, of any type, that contains given keywords. See the screenshot in my post here. As soon as I change a keyword in any file, the Spotlight engine notices it and modifies the index, so it is always kept up to date.

As to keywords for virtual copies, since XMP metadata is “extensible”, there is nothing to stop extra tags from being added without having to mess up the DOP files with their implicit confusion with the database.

…and herein lies the problem. Every app says it can do DAM, nevertheless, functionality and transport/metadata exchange are so-so, depending on which apps are used. Everyone can use whatever is around that can fix the issues. For some it’s Lightroom or database editing or self-made apps or whatnot.

Using Lightroom to fix issues of DPL with keywords has worked on my Macs for years, e.g. whenever DPL’s search found nonexistent files, I had Lr rewrite metadata to all files, deleted the DPL database and indexed the photo archive with DPL again. I use Lightroom because I have it and it works as my primary photo manager and editor…and provides such simple functionality that lets me update metadata of all files instead of folder by folder. Any dam app that can do this could be used to fix issues of DPL in this relatively crude way. BTW, I have given up trying to keep DPL’s database free of corpses. I use Lightroom regularly and PhotoLab if I must…and delete its database every now and then. I also set DPL to not sync dop and xmp sidecars, they are irrelevant under this SPOD approach.

As soon as DPL gets a set of useful database and metadata maintenance features including a possibility to remove images from its DB without trashing the files, external app would not be needed to fix DPL’s issues. Sadly, we do have to wait…and wait…and wait… for such functionalities.

What is so frustrating is when Adobe participated in the MWG (metadata working group) to form a standard, then promptly decided to change the way they do things to be different from that standard.

Take the Fruit | Orange | Satsuma hierarchy applied to an image using the MWG guidance…

  <mwg-kw:Keywords rdf:parseType='Resource'>
   <mwg-kw:Hierarchy>
    <rdf:Bag>
     <rdf:li rdf:parseType='Resource'>
      <mwg-kw:Keyword>Fruit</mwg-kw:Keyword>
     </rdf:li>
     <rdf:li rdf:parseType='Resource'>
      <mwg-kw:Children>
       <rdf:Bag>
        <rdf:li rdf:parseType='Resource'>
         <mwg-kw:Keyword>Orange</mwg-kw:Keyword>
        </rdf:li>
       </rdf:Bag>
      </mwg-kw:Children>
      <mwg-kw:Keyword>Fruit</mwg-kw:Keyword>
     </rdf:li>
     <rdf:li rdf:parseType='Resource'>
      <mwg-kw:Children>
       <rdf:Bag>
        <rdf:li rdf:parseType='Resource'>
         <mwg-kw:Children>
          <rdf:Bag>
           <rdf:li rdf:parseType='Resource'>
            <mwg-kw:Keyword>Satsuma</mwg-kw:Keyword>
           </rdf:li>
          </rdf:Bag>
         </mwg-kw:Children>
         <mwg-kw:Keyword>Orange</mwg-kw:Keyword>
        </rdf:li>
       </rdf:Bag>
      </mwg-kw:Children>
      <mwg-kw:Keyword>Fruit</mwg-kw:Keyword>
     </rdf:li>
    </rdf:Bag>
   </mwg-kw:Hierarchy>
  </mwg-kw:Keywords>

All three keywords clearly labelled as keywords and the hierarchical relationships fully described. Plus the requirement that all keywords are also mentioned in the dc:subject tag…

  <dc:subject>
   <rdf:Bag>
    <rdf:li>Fruit</rdf:li>
    <rdf:li>Orange</rdf:li>
    <rdf:li>Satsuma</rdf:li>
   </rdf:Bag>
  </dc:subject>
 </rdf:Description>

So, what do Adobe do?

They comply with the the dc:subject tag but then totally ignore the mwg-kw keywords and hierarchy tags, in favour of their own Lightroom based tag…

  <lr:hierarchicalSubject>
   <rdf:Bag>
    <rdf:li>Fruit</rdf:li>
    <rdf:li>Fruit|Orange</rdf:li>
    <rdf:li>Fruit|Orange|Satsuma</rdf:li>
   </rdf:Bag>
  </lr:hierarchicalSubject>

… which can be, and is often, interpreted in multiple ways, depending on the software writer.

Some say that standalone keywords must also be included as single level hierarchies, which means that they can be confused with the top level of another hierarchy.

Some say that there should only be one line for the entire hierarchy, but that leads to confusion. We can interpret the above tag as…

  • The complete single level hierarchy Fruit
  • plus -
  • The complete fully qualified two level hierarchy Fruit | Orange
  • plus -
  • The complete fully qualified three level hierarchy Fruit | Orange | Satsuma

… or just a fully qualified description of Fruit | Orange | Satsuma, which, according to some software, doesn’t require the first two tags.


Now just look at what happens when we add Orange and Satsuma as standalone keywords as well as parts of a hierarchy…

  <lr:hierarchicalSubject>
    <rdf:Bag>
      <rdf:li>Fruit</rdf:li>
      <rdf:li>Fruit|Orange</rdf:li>
      <rdf:li>Fruit|Orange|Satsuma</rdf:li>
      <rdf:li>Orange</rdf:li>
      <rdf:li>Satsuma</rdf:li>
    </rdf:Bag>
  </lr:hierarchicalSubject>

This was written by PL7, but is it valid for all other DAM tools?

If we refer to the MWG guidance, it is abundantly clear that a keyword must be defined as either just a standalone keyword or, if it is a hierarchical keyword, it must be mentioned in the dc:subject tag and then its place in a given hierarchy must also be defined in the mwg-kw hierarchical tag set.

Unfortunately, the world has moved on since the MWG hierarchy tags and we are unlikely to find any software that will understand them. So we are left with lr:hierarchicalSubject which when correctly used, appears to be fairly universally compatible.


At the other extreme, if we, inadvisedly, use the following preferences in PL…

… we end up with metadata that looks like this…

  <dc:subject>
    <rdf:Bag>
      <rdf:li>Satsuma</rdf:li>
    </rdf:Bag>
  </dc:subject>
  …
  <lr:hierarchicalSubject>
    <rdf:Bag>
      <rdf:li>Fruit|Orange|Satsuma</rdf:li>
    </rdf:Bag>
  </lr:hierarchicalSubject>

… which, with a lot of software, including macOS Finder, can never be searched for by either Fruit or Orange, since they do not appear in the dc:subject tag.


Fine. you can change the preferences to “do it right” but that will all depend on the default setting and few people truly understand the meaning of the flags.

For the curious, who want to use PL for metadata writing and to be as compatible as possible, the settings should be…

1 Like