Why are my tags not working after copying from iCloud?

@platypus I tried an experiment on Wind 10 with two identical directories one on a hard drive (F:) and another on my NAS(Y).

Found both on PL7.4 and added appropriate Keywords “ON F” (the HDD) and “ON Y” (the NAS) and also created appropriate Projects.

I then (very) carefully hacked the database and swapped the two “drive” entries in the Folder table, i.e. after the hack

and in DxPL we have for the “Project on F:”

and for the “Project on Y:”

I have never thought of doing it that way before!?

  1. Not sure what this means
  2. Yes
  3. not sure
  4. yes
  5. yes

Around 10,000 images.

Yes, RAW files.
Right now, no files are showing keywords in finder. If I go to iCloud I can see the.dop files.
DSC_0058.NEF.dop (21.2 KB)

How do I check if I don’t have .xmp files? None show up in iCloud if that’s why you’re asking.

The keywords are in your dop file: goldfinch and reeves reed.
See my former post.

Try edit>preferences>general>load settings from sidecar file and restart PL

George

@aloha

  1. Do you have a dump from just prior to the movement of data from iCloud? - Where is the data that you are now trying to access, on a local disk or still in the cloud and do you have a database from when you were able to see the keywords?

  2. Is the data in iCloud still intact/accessible? Yes

  3. Do you have any xmp files associated with the image? If you have written metadata to the image for RAW files a “.xmp” file will accompany the “.DOP” file and the RAW image file.

  4. Were you adding keywords to the images in DxPL/using DxPL as the only keyword editor.? Yes

  5. We are assuming that “Tags” means keywords, i.e. “Worthing” assigned to an image in my case. Is this assumption correct? Yes

So where is the data you are trying to use now located, my presumption is that you were accessing the data (images and DOPs and …) directly in iCloud originally but that changed for some reason, is that correct!

The DOP you have showed us is O.K. but is that a DOP from the location where the data is now located?

@George’s procedure will/should work if the DOP accompanying the image you are seeing in PL6/7 contains the same data.

With DPL 7.5 on Mac, I see the following pattern:

  1. Keywords in DOP sidecars are ignored, except when DPL is started without database
  2. Keywords in DOP are read when metadata is imported (manually) from XMP sidecars, in which case, the keyword from the DOP (that was attached when starting DPL without database) is detached from the image file, to which the keyword from XMP will be attached instead

Test procedure:

  1. Add a keyword “UNUSED” and make DPL write DOP and XMP files
  2. Quit DPL
  3. Edit keywords in DOP and XMP sidecar files
    DOP: change “UNUSED” to “UNUSED_DOP”
    XMP: change “UNUSED” to “UNUSED_XMP”
  4. Start DPL
  5. Manually import DOP and check the keyword list → unchanged
  6. Manually import XMP and check the keyword list → both “UNUSED_???” keywords have been added, check allocation of keyword to file

The test without database runs along similar lines.

One more thing: macOS keeps a local copy of files and folders stored in iCloud and uses these files and folders which are then synced to iCloud. Find these items in your Library folder, with macOS Sonoma, it’s called “iCloud”, in earlier macOS versions, it was called “Mobile Sync” or something like that.

(Note that “UNUSED” is not mandatory, it simply helps to find and delete the keyword after the test. Use whichever keyword cuts your cake)
:cake: :cake: :cake:

1 Like

A quick test.
Changing the keyword after PL was closed and opening PL afterwards doesn’t show the changed keyword. But file>sidecar>import does show the changed keyword.

George

Another difference between Mac and Win versions of DPL?

@George I did a similar test this morning and got the following

In the tests I had navigated away from a directory I discovered in PL7.4 and hacked a DOP keyword and saved the DOP.

Navigating back to the image resulted in no change (surprising because the date timestamp will have changed but not the modification field in the DOP, and I believe it is the latter that determines if DxPL will reread the DOP) but causing an ‘Import’ of the DOP showed up the “hack”.

@platypus As far as I understand providing the AS flag is OFF then any newly discovered image with a DOP will take both the editing data and the metadata from the DOP, ALWAYS!

I believe this was the guiding principle since PL5.2.0 or thereabouts to “answer” user angst about the loss of this type of functionality when moving from PL4 to PL5, i.e. early PL5 releases stored the metadata in the DOP but it was never used?

So I believe that

  1. If an image with a DOP is discovered by DxPL for the first time then the source of the metadata to be used (image metadata versus DOP metadata) is governed by the AS setting.

a) With AS(ON) the metadata will be taken from the image (if any metadata is present, embedded or in an xmp sidecar file), if there is no metadata then the DOP will be overwritten with the new values of the metadata regardless of its values at this first discovery. Please note that “first time” will be true if the image in now in a directory “unknown” (directory name change/drive change) to DxPL or the name of an existing image has been changed!

b) With AS(OFF) the metadata will be taken from the DOP and any metadata in the image will be ignored unless the user explicitly requests that data should be taken from the image using the 'Read from image" command. The ‘Read from Image’ can be used at any time regardless of the AS setting.

The state of the database has no effect except by virtue of the existence of the directory/image in the database. A new database will contain no entries so everything will be “new”.

  1. If there is no DOP present the metadata will be taken from the image regardless of the AS (ON or OFF) settings but only for the initial discovery thereafter automatic metadata reads (and, unfortunately, writes) are controlled by AS(ON) and ignored by AS(OFF), except for the ‘S’ icon being set.

I will see about running your tests tomorrow and reporting my findings complete with snapshots.

@BHAYT , the “S” badge only appears when metadata has changed in the XMP sidecar. It does not react to changes in DOP sidecars.

“Initial discovery” is what happens when the DB has been deleted. I also tested with automatic import of DOP sidecars…and the behaviour is the same as with manual import.

@platypus I never said that it did. The sentence in which I mentioned the ‘S’ icon was one where I had stated that there were no DOPs.

Initial discovery occurs with an empty database certainly but at all other times when a directory not already in the database is selected for automatic import by the user (via simple navigation), e.g. simply copying a directory with an “-2” added to the name will produce a new discovery when encountered by DxPL.

Hence, it is not a necessary criteria simply one that will “guarantee” all subsequent discoveries will be “first/new discoveries” (until directories already discovered are discovered again)!

I only ever use automatic DOP input (and output) so all my tests and results will be based upon that premise (those premises).

@aloha , what is the current situation, have you been able to resolve the issue?

You have copied your Photos from iCloud to your computer. Do you plan to keep them locally (on your computer) or do you plan to keep them in iCloud later on?

Supposing your computer is a Mac, what version of macOS is installed on it?

Your answers might help us to tailor a remedy to the issues you have/had and to propose steps to a hopefully more robust configuration.

Finally, the issue should really be addressed to support.dxo.com. Apart from providing a fix, DxO should also provide a solution that is able to prevent and/or fix issues related to bulk moving files and folders. As of now, PhotoLab is fairly intolerant to such actions.

@platypus The results of the test are

For an existing, i.e. already discovered directory

With AS(OFF) and Auto DOP Read/Write also off we have this after the re-discovery

With the setting of the various options DxPL has no reason to update anything automatically but did set the ‘S’ icon indicating something of interest had change in the metadata.

Step 5:- Manually import the DOP

Step 6:-
Manually Read the metadata from the image

While @aloha’s issue still remains and we need to understand the steps actually taken that caused the problem, to determine what might have gone wrong.

With respect to your test I am afraid DxPL detected the DOP change after the restart and then overwrote that when I Read the metadata from the image.

The DOP change results didn’t fit with my previous results but there was a DxPL restart in the middle of the later test which was missing from my original test.

PS:- I will repeat the test to see what might have gone wrong and with a new discovery scenario.

On further testing, I saw that sidecars are only exported or imported when focus is changed, e.g. by selecting a different folder (and back) in DPL’s browser. This confirms what @Joanna told me in a FaceTime chat a few days ago. Moreover, the transport of keywords through DOP sidecars seems to be shaky, see below for details.

Further testing:

  • DPL6 and DPL7 set to automatically import and export DOP sidecar files.
  • Add keyword in DPL6, change focus and back: All images are said to have the keyword
  • Focus on DPL7 and check keyword list: Not all images have the keyword
  • In DPL7, select a different folder and back, wait and check: All images have the keyword, but DPL7 also has a duplicate keyword - and a DPL restart does not clean that out.
  • Selecting, in DPL7, one image after the other causes the sort order to change in the keyword list tool.

BTW: Similar issues between DPL5 and DPL6 :nauseated_face:

I’ve contacted support.dxo.com
(Note to myself: https://support.dxo.com/hc/requests/465033)

@platypus That has never been my experience! Typically image metadata is updated immediately and DOPs are queued for the periodic (20 Seconds or thereabouts) update cycle.

Changing directory focus might accelerate that process but I can’t say that I have ever seen DxPL not update the DOPs (although I seem to remember the odd problem which could be explained by such an issue).

You do not mention in your tests whether AS(ON) or AS(OFF), you do mention Auto DOP Load & Save (ON). How many images were included in your tests?

I tested with PL6 adding two keywords to each of 13 images and then discovering in PL7 and everything was fine (I executed the focus switching you indicated)?

I could imagine problems with large number of images but I need guidance before I test that and a suggestion of how many images.

@BHAYT , again, I’m on Mac, you’re on Win and we know that DPL is not the same on both platforms. That being said, the following applied to my last tests.

  • DPL 6 and 7 set to automatically import and export DOP files (read my previous post)
  • 60 test image files, Canon RAW files from EOS 5D, 5D3, M6 and R7 bodies

I choose to NOT activate XMP sync in order to see what DPL does when it only gets DOP sidecars (read my previous post for findings).

If DPL only gets DOP sidecars, keywords are read, but in kind of mixed blessing ways. If DPL gets DOP and XMP sidecars, behaviour changes as reported earlier. My respective lesson learned: Don’t use DPL to manage keywords.

And once more: This is all on Mac and YMMV because you’re on Win.

I don’t specially care for DPL doing this or that with keywords because I do it all in Lightroom. Whether my findings will help @aloha remains to be seen … and we’ll have to wait for @aloha to re-emerge in the thread.

@platypus My experience on Win 10 does vary somewhat so here is my experience so far.

Summary:-

With the test scenario outlined by @platypus in Why are my tags not working after copying from iCloud? - #35 by platypus I was unable to reproduce the problem encountered on the Mac on my Win 10 system with the latest versions of PL6 and PL7.

But relying on DOPs to preserve metadata runs into problems when taking them backwards from release to release because while DxO provides compatibility with older DOPs, there is no backwards compatibility between major releases!

It should work if the DOPs are “hacked” but that simply adds to the complexity of the process!

Test Scenario:-

  1. 150 images, auto DOP read & write set, AS(OFF).
    PL6:-
  2. Add “Brighton” to all images in PL6, navigate away and back to the images.
  3. Result at this point is 150 DOPs and all appear to have “Brighton” where it should be and the keyword count for “Brighton” is 163 because I didn’t clear the database before I started and had the previous test results as well (from another directory)!!
  4. Navigate away from the directory and
    PL7:-
  5. Open PL7 with an empty database!
  6. Check settings to replicate PL6 then navigate to the “Brighton” directory.
  7. All keywords present and correct and taken from the DOP.
  8. Restarted this morning and all are still O.K.
  9. Added “East Sussex” to keywords of all images and navigated away and back and away before starting PL6.
    PL6:-
  10. Because I had some “East Sussex” images in PL6 I cleared the database and discovered the directory anew.
  11. No keywords were detected because the DOPs are now PL7 not PL6 and therein lies one major issue with using DOPs as the carrier of keywords. The DOP is forwards compatible between releases but not backwards compatible (I believe that they are typically backwards compatible within a major release but am not sure that can be guaranteed!?)
  12. The old DOPs will be O.K. until an edit is undertaken when DxPL deletes the old DOP and makes a new one, i.e. when I added a keyword of “test” to the first image.
    PL7:-
  13. Returning to PL7 the DOP containing “test” was not automatically read but a forced ‘Import’ of the first image resulted in this

because the import has created a Virtual Copy, i.e. part of the update of the DOP on PL6 added a new UUID to the image 1 DOP and that results in VC[1] when imported into PL7.

Nothing new here, but some further detail on how keywords migrated through DOP files appear. Procedure with all DPLs set to NOT automatically import/export/sync DOP and XMP files:

  1. Add keyword to 60 images in DPL5 and manually export DOP
  2. Duplicate and rename DOP file
    for backup and illustration
  3. Manually import DOP in DPL6
  4. Add keyword to 60 images in DPL6 and manually export DOP
  5. Duplicate and rename DOP file
    for backup and illustration
  6. Manually import DOP in DPL7
  7. Add keyword to 60 images in DPL5 and manually export DOP
  8. Duplicate and rename DOP file
    for backup and illustration

Keywords in DPL6 after manual import of DOP :-1:

Keywords in DPL6 after adding keyword to 60 images :+1:

Keywords in DPL7 after manual import of DOP :-1:

Keywords in DPL7 after adding keyword to 60 images :+1:

And just for the fun of it all - the DB entries :exploding_head:

There should now be enough food for thought for DxO bugbusters…

I’m doing my best to understand what everyone is saying :slightly_smiling_face:
I want to keep them on the computer.
Sonoma

Honestly I trust this forum more. I’ve had negative experiences with DXO support in the past and you guys have helped me more than them.