Unwanted Virtual copy generation

On Mac, there is a way. Ctrl-click on the frame of the VC (or press command-ENTER) and rename the file. It doesn’t matter if the file is real or virtual. The new name will be visible in the .dop sidecar file.

The order in which master and virtual copies are shown in a .dop sidecar is fairly random. The main structure looks like this:


(.dop file of a Master with 8 VCs. Respective portions highlighted in blue)

The master can be identified by its timestamp: It’s the oldest

If one deletes the portion with the oldest creation timestamp, all VCs will be promoted and the now oldest item will be the new master.

Thank you, however, using a PC, not Mac. From what I gather, renaming VCs is not possible when using a PC. It’s simpler to generate a separate image file (with the cost being the storage requirement and DAM management. I’m not a capable programmer, so parsing the DOP is possible, but tedious for me.

As for the OP question, I’ve noticed unwanted, and lost VCs, on PL updates before.
My current habit is to create separate folders with duplicated/renamed images for each software version. This allows for easy separation and labeling. I ignore database issues and have not noticed an issue this way. For example, now working with 20 images to compare PL7, PL8, and PR4/LR. Each software has a separate folder(s) and appropriately named images. Then generate TIFFs in a "compare folder for review. Nothing unusual in this approach, so nothing to help the OP.

Thank you for the feedback.

@platypus
Thank you for sharing the Mac method!

@swmurray I am a PC only user with no access to a Mac system so what I write is related to testing on PCs including between PCs.

Just like the Partition GUID fix program I mentioned in a post above I have started but not completed a Backup program and one thread of that program will be the ability to copy (backup) DOPs to a “safe” place, that all depends on me not finding something else to distract me, like the release of PL8 has.

You mean like this

2024-09-29_170340_

In this articular case the same image was used and I haven’t added PL7 and PL8 to the test, oops.

To be honest a practice I don’t always follow (hence the program) is that before starting with a new release a user should do at least the following

  1. Backup and retain the old database (use it to test that the new release automatically picks it up)

  2. Select the directories that are going to be used for the initial evaluation and backup at least the initial DOPS and xmp sidecars. Possibly the images as well, item 13 to be added to my drop-down list.

  3. Undertake whatever testing the user deems necessary before paying for the new release etc…

Under no circumstances risk losing DOPs by using the newest release without a backup until you are sure that everything is as it should be.

Sadly I haven’t completed my program in time for PL8 and all the items of the list really need to be included. Its easy to plan how it might look but harder to put a program together to include all the elements.

Hi BHAYT,

What is the conclusion of your test ?

A quick test on a Windows 10 PC with PL8. Ctrl+click on a VC allows you to change the name of the VC - but when you hit enter it changes the name of the Master as well.

I just learned that DXO saves unique metadata for each virtual copy, which is saved by PL8 for the next session as well as exports. Windows11, PL8

For example, I entered “Master”, “Virtcopy1”, “Virtcopy2”, “Virtcopy3” in the IPTC Headline field for the respective master and virtual copies.

@SAFC01

Understood. Hence the question of how to uniquely annotate each VC.

Thank you!

Not sure. I was comparing PL8 XD2s and optical corrections to PL7 and PR4. Was not investigating the VC and .DOP differences/challenges. Thank you for pointing out some DOP info!

“My” Process (probably very similar to most)
-Verify DB and images+dops are backed up.
-Copy a desired test images over to new directories in my Testing folder, one directory per software version. Rename each image set to reflect software used as needed.
-Download the new software separately so as to run in parallel, if needed/possible.
-Run desired trials on the lset of images. If not parallel, then first generate a set of text export TIFs from current software as baseline.
-Generate TIF results with the edit treatment notes in the export suffix using descriptors, an A,B,C code, and the VC _# additional suffix.
-Compare TIFs in latest PL version as well as overlayed as stacked layers in PS. In PS, use the layer blending modes, including a subtract equation to highlight areas of difference for study as needed.

In this case the PL8 tests have been over multiple short sessions while travelling, so it’s getting a bit confusing to remember the various suffix codes and notes. :crazy_face:

Anyway, my VC naming question was answered - Not possible on PCs, sort of possible on Mac. Will try the IPTC field idea going forward.

Thank you for allowing me to butt into the thread.

@Bencsi Hi

Firstly I have experienced genuinely unwanted or unexpected VCs only on a very few occasions, in spite of the fact that I typically use the same images again and again and again in a set of tests so the image will be in the database many times over but attached to a different directory and don’t appear to get UUID clashes which leads to VCs!

I had one incident back on PL5, I think, when every image in a directory got a VC in spite of the fact that the directory had been deleted before being re-added, somehow the deleted directory entries were still in the database and caused the UUID clash!??

I never encountered the problem again and never got a satisfactory reason from DxO, and they were at least talking to the Forum back then!

With respect to my tests I ran PL6.18.0 and PL8.0.0 using the same directory the tests are described in detail below but the shorter summary goes something like this.

Do not edit an image that has a later release DOP associated with it using an earlier release because you may regret the outcome, which in one test not only left the PL8 image with a VC but also with a PL6 DOP?

Test Details:-.

  1. Open image in PL8 and make an edit to the image, which will result in a PL8 DOP being created (and the PL8 database being updated).

  2. Open the image in PL6 and examine, no edits will be found because the PL8 DOP will be ignored. At this point we have two copies with their own databases which will be sharing the same directory containing the RAW image and a single PL8 DOP.

  3. Both versions remain showing the same data.

  4. Edit the image in PL6 and wait (DOPs are not written immediately and can take up to 20 seconds to be written, generally as a batch of DOP updates).

  5. Check PL8 and after a period the update made by PL6 is seen by PL8 when it recognises that a PL6 DOP has been created and written to disk. PL6 will treat the PL8 DOP as if it doesn’t exist and will simply overwrite it with the PL6 edit, complete with a new UUID for the entry made in the PL6 database…

  6. However, PL8 will see the PL6 DOP, when it is finally written to disk, and will see an edit with a new UUID so it will create a VC to handle the new edit from the PL6 DOP.

  7. It appears that at this stage PL8 does not create a new DOP so we have the PL6 database with an entry and a DOP that relates to that database entry and a PL8 database that contains the [M]aster (from the PL8 edit) and VC[1] from the PL6 edit but that state is not reflected in the DOP anywhere!

I repeated the test on the second image and we have the following

I then repeated the test with a third image added to the original 2 test files, with a utility called File Monitor running, which monitors a chosen directory or directories for file activity using either the same method that DxPL uses or one very similar.

I got the following log from Folder Monitor and an investigation revealed that all the DOPs at the end of the tests were PL6 DOPs!!??

That comes as a bit of a surprise.

I re-opened PL8 and nothing happened with the DOPs or the directory so the DOP actually shows one image with the PL6 edit but the PL8 database has both its original edit and the PL6 edit.

Hmmmmmmm!?

Continued the test so I opened PL8 and made a change to the [M]aster and then re-opened PL6 and made a change to the image.

PL8 detected the change and changed VC[1] because the UUID matches the entry now in the database (which came from the PL6 DOP) from the earlier test. The DOP is now a PL8 DOP!

So @Bencsi I am sorry to say I am no closer to understanding how you wound up with unwanted VCs but have found a situation which means that “playing” with an old version of DxPL when the image has been edited with a later version is an absolute no-no.

Sorry been there, done that and concluded we couldn’t change the name on DxPL(Win).

@swmurray I have also been “stealing” IPTC fields for some time to identify copies during testing but it is worth mentioning it here and I forgot to use Folder Monitor on my early tests and didn’t use the IPTC “trick” at all.

Sorry this response has become very long and it creates more questions than it answers.

I use 100% Jpegs to keep the space requirements down.

Use a field that you are not going to use for anything else and preferably one that is easy to include when taking a snapshot. Unfortunately that is complicated because DxPL moves the IPTC fields from one side of the display to the other between PhotoLibrary and Customize.

Is working with several photolab versions on the same images a safe process ?
No chance of a bug ?

@JoPoV is there any obvious reason why it won’t?

Arguably yes! Intuitively there certain reasons why it might not work but until you test as I did above, the exact issues are not that obvious.

My concern about what I found is that it means I have a situation where the DOP and the database are definitely not in sync, i.e. neither can be trusted to preserve the correct status of the edits that have been processed.

The PL8 database in the tests I conducted is the most reliable and at times the DOPs are the least reliable but that state is not constant.

Forcing a DOP write after the PL8 discovery of the PL6 edit should ensure that at least the database and DOP status are inline .

I understand that going from v6 to v8 should be a safe process.
But what if, for example, some images need finally dpxd and are processed with v6 after they have been processed first with v8 ?
With 2 possible cases :

  • linear dng from v6 with dpxd and process in v8 to keep all v8 functions unavalaible in v6.
  • full process in v6.

Is this still safe (to go back and forth) ?

Yes, forwards compatibility is assured.

Intuitively, that’s definitely not wise … because PLv8 correction details (whether from the db or a sidecar) will contain references to tools/options that PLv6 (or any earlier version) will not recognise and will not be able to handle … with risk that it will choke/freeze.

It seems, I did throw a stone into the water.
Meanwhile the situation is simple for me.
If I find an image in my original image library and open that library, how can I avoid automatic VC generation ? Here is my DxO output library set:
image
The folders incl. several DxO version images. In spite of I can identify the DxO version of the image edited, I can not open the related version. E.g. the OpticsPro v5-v8 was working on Win XP only. An image from this age is not editable anymore with latest version of PL ? Certainly must be editable without any side effect.
The DxO latest version user have to take care of the earlier version sidecars before load a folder content to PL 8 ? It is not very easy.

I think, I found the solution: Here is a folder incl. one image DSC_6507 with DOP file. The last edit made in 2019.
image

Now I will load this folder to PL v8. The opening doesn’t made any change. Here is its reason, the .dop file write is not enabled.


However, if I enable the sidecar write, the .dop file to be overwritten.
image
At the same time the additional VC generated
image

It seems, the best way to avoid unwanted VC generation is disable the .dop file write. In this case no VCs will be generated.

I think the problem solved - at least for me. Disable the sidecar write.

However, if a sidecar is generated by the PL8, it is impossible to simple remove - in case of the sidecar write is still enable. After remove ( Shift+Del )
image
the sidecars disappear. Not forever. Jump to another folder and jump back, the database remember the VCs existence and display them. What does it mean ? Without sidecar write enable the sidecars are frozen to the database, impossible to remove ! The indirection is: first remove all unwanted sidecars with enabled sidecar write, second disable the sidecar write in the Preferences.

Endre

Hi again, Endre … I’m curious about your workflow;

  • Are you using multiple, different versions of PL ?
    – If so - - why ?

  • Why not simply use the latest version for which you have a license (such a PLv8) ?
    – Because, then, PL will not be forced to create extra VCs.

  • Do you have a need for the database - - such as to create “Projects” and/or to search for images by keyword.
    – If not, then deleting the database would make your “unwanted VC problem” simply go away !

I never get VCs, even though I test a lot of different versions against each other (I keep old versions for fallback and occasionally install all for some tests) by setting PhotoLab to NOT automatically import and export .dop sidecar files.

Hi John,

No, I try to use the latest. Actually the v8 is the latest. Just for testing sometimes compare to the previous versions and I have valid license from OP v5 - v12 and PL 1 - 6 only. PL6 is my working version actually.

I have to print recently 8 images from my life. Most of them comes from 1990 till today. Most of the images has been edited with DxO applications. I search images with Google Picasa, which recognize similar faces and allow to select easily. After find the proper image, I open the latest PL and drag the image to the desktop shortcut - regardless which application has been used to edit. And the PL v8 open the folder and add VCs without my intention.

I use the Project feature, because the Project allow to add images from different folders ( e.g. on an event more camera used and additionally mobile phones too ). This a part of my Project tab


There are more than 200 projects from the past. I always load the latest Database to the new PL just after first installation. This an important compatibility test for me.

You are right, if I delete the Database, the VC problem disappear. However, if a feature is existing, the user have the right to utilize, rather than neglect, because it is not reliable. There are features in PL which are not useful, therefore better to disable ( e.g. Enable open CL … ).

Endre

Hi Platypus,

Try to enable the sidecar write feature among the Preferences.

Endre

@Bencsi Endre read the rest of this post when you have time but in the meantime please to the following

  1. However you normally drag and drop the images to the Desktop for printing please drop to a new Directory e.g. “_Images for printing - 01” and then open that directory in PL8 and see what happens.

I believe you have been copying the same images to the Desktop and they have never been deleted by PL6 etc. and are still in the database and that is what is causing the VCs

The original Post begins here:-

@Bencsi Endre I thought I had a problem yesterday loading an OpticsPro 11 (OP11) image into PL8 but I tried it again today and it seems fine.

So I went back to the earliest release I have installed which is OP9 added two images to a new test directory, discovered the images in OP 9 and applied some fairly obvious edits and moved away from the test directory in OP9.

I discovered the directory in PL8 complete with the (OP9) edits and edited one in PL8 successfully.

I also have OP7 in a Trial mode so I repeated the test with that which also seemed to work O.K…

In both cases the tests today were on JPGs and

  1. The DOP was immediately converted to a PL8 DOP so no using that on an old release again. I have both the read and write DOP settings on and always have done.

  2. No VCs were created in the process which is what I would expect to happen.

All old DOPs should be loadable into PL8 and their edits shown as has been the case since … “forever”. and that still a seems to be the case for me at least.

I then added PL7 to my DOP test list and cleared the PL7 and PL8 databases, anything I want to keep is already backed up.

I then discovered each images in Pl7 then PL8 and compared to the original state of te directories on another machine and got the following
PL7:-

PL8:-

Both PL7 and PL8 change some of the DOPs (I made no edits in either release or exports in either release, either action will cause a new DOP to be created.)

DxPL has also caused changes to some of the JPGs which appear to be time changes i.e.

I will be reporting to DXO Support

  1. The situation where PL8 doesn’t create a DOP after discovering the PL6 change as discovered in my test above.

Endre I don’t like the idea of disabling the DOP write at all.

As has been stated by a number of users here they rely on the DOPs and not the database, i.e. they remove the database after an editing session and re-discover the edits via the DOP as and when (and if) they need to.

But I certainly don’t like the idea of relying solely on the database as the only source of the edit data.

It will indeed but if you still have the original in a folder and the change is made to an image on the desktop does that actually matter?

I just did that with PL8 and had no problem and no VC. Which version do you use to drag and drop the image, PL6 or PL8?

This shows two images dragged to the desktop and opened in PL8, the Acacia tree image was dragged and dropped by PL8 , the view of London from Southwark was dragged and dropped by PL6 and PL8 is treating both identically and no VC in sight.

Both DOPs are now PL8 DOPS.

It should work and it does work but not in your case. The images you are dragging to the desktop already exist in the database and that is what I believe is happening.

You have done this before on one release or another and you are doing it again, either overwriting the existing images in the drag and drop operation or (more likely) you have deleted them from the desktop in the past but they are still in the database!