Greg,
Yes I agree with the point about not having PL open when you Restore - this way you’ll avoid PL possibly creating a DOP that doesn’t contain your changes that were in the deleted DOP. However, none of this material if you’re trying to recover VCs as PL’s changes to the DOP as you delete an image make any recovery impossible.
While not conclusive, i.e. because I was monitoring file activity not file contents and this is an annotated (by me) copy of the relevant section of the Log file.
So we have me making a change to force the DOP to be updated then copying the original DOP and then deleting the [M]aster in the test directory.
The outcome has already been documented, only the [M]aster data exists in the recovered DOP.
@SAFC01 Thank you and an update on locating copies, “Albums =” is the start of each copy and the “Should Process =” doesn’t quite end each copy.
“Albums” is not a field that is used by DxPL(Win) and tests on PL7 indicated that high jacking the field to use it to name things (via a text editor or programmatically) didn’t appear to cause DxPL any “stress”!?
Thank you everyone for checking. I’ve had no issues in the past, so not sure why it is happening now.
The issue does not occur in version 7.10.0 Build 287 (tested 5 files).
The issue occurs in version 8.1.0 build 434 and is repeatable (5 files, same results). Again, working from a Windows 11 PC, not Mac.
As a “workaround” the error does NOT occur when making the file restore when PL8.1 is CLOSED. The error occurs when PL8 is OPEN. I haven’t tested if the error occurs if I simply point PL8 to a different folder.
The virtual copy information has been removed from the DOP file for the properly functioning situations.
Folks who suggest that PL8 is attempting to build the dop files when the image is restored and then finds a dop conflict are likely correct.
My lesson learnt is to close PL when attempting to restore deleted items from the recycle bin. By extension, I should be structured to close PL, or change the active folder, whenever making file changes from the system’s File Manager (i.e. moving files outside of PL). This has been a topic before.
Perhaps there is a bug which DxO can fix, but for now I can live with the limitation to avoid PL - File Manager conflicts.
That is an image with just one copy not three, the VCs went from the DOP before the image made it to the Recycle Bin.
But that doesn’t fit with my comment above, the extra VCs cannot emerge from nothing so one of us is doing the tests incorrectly.
The notion that PL8 is suddenly going to get confused does not make sense.
When the deletion is executed in PL8, all entries are removed from the database.
The evidence seems to show that what is in the ‘Recycle’ Bin is a DOP with only one copy the [M]aster copy.
DxPL rediscovering that returning (Restored) image is going to create no more confusion than discovering any new image being added to a directory already open in DxPL and I have done that many times in my testing.
DxPL is not magical nor particularly sensitive.
Like the FolderMonitor program I used above it is always “watching” the directory currently open in DxPL, no more but certainly no less and it will see items leaving and arriving and, in my tests, tends to handle those events successfully (unless the ‘Date Modified’ doesn’t look like something beyond the last time it looked when it will ignore the item as if no event occurred at all!?)
So after the Remove, with the directory open in PL8, we have
Always a wise move but in my tests PL8 and PL710 were left open and with the directory from which the data had been removed still selected, to see if anything untoward was going to happen.
I’m confused. Perhaps we are struggling with different issues.
In my case, I accidently deleted a “master” image along with a virtual copy. I intended to delete the VC only. My workaround is to close PL before recovering the accidently deleted files. This recovers the “master” file and adjustments, but no VCs.
This does not appear to overlap the issue others are reporting of multiple VCs created…
…
For anyone interested in studying the dop changes made by PL during this recovery process, here are the “as deleted” and “after recovery” versions from a re-test today. (tested on a culled scrap image with arbitrary edits)
Both image versions had the local adjustment labeled “Hawk2” at line 638 in both files, with VC containing the 2nd local adjustment labeled “SKY” at line 1198 as seen in the “as deleted” version. There are other corresponding differences consistent with the data for the VC version dropped.
I don’t have the software/system chops to study further.
Thank you all for checking your systems and offering suggestions. I am comfortable with the “workaround” of recovering the “master” image adjustments only (no VCs).
But I appear to have caused you upset by my excessively long analysis, sorry.
Because I was running the monitor software it slowed things up a bit and I saw VC[2} being deleted (and removed from the screen), followed by VC[1} and then followed by the [M]aster, until the Test directory was empty.
Unfortunately each of those actions appears to have created a new DOP, which overwrites the previous DOP, so at the end of the exercise only the image and the DOP in its final state DOP reside in the Recycle Bin and are available for restoration.
It is thanks to @SAFC01 that the DXPL(Win) situation was evaluated.
My posts were to simply to show exactly what is possible with PL7-10 and PL810, i.e. that you can recover only the [M]aster after the situation reported by you.
At least that way the all important image is recoverable, complete with the edits made to that particular copy.
No, not upset at all. I just don’t have the skills to track what is happening at the file creation level or in the database. That part is, unfortunately, a black box to me.
So far, others don’t seem to have the same error on a recovered file. Don’t know why, but in my blissful, non-software world, just glad to have a work around.
While trying to stumble on a workaround I poked my nose into the dop files and noted the dop changes. The deleted/recovered dop files checked before reopening PL8 showed lines with the VC-only local adjustment name I was using to check. These were recovered. The dop files did not “update” before the delete/recover. The dop changes occurred after PL8 was opened and the file(s) were “re-discovered”. This seems different than what you describe, so simply offering as a data point. I cannot explain this behavior.
Actually, I appreciate that you and other software-skilled folks are investigating these type issues. That helps me, and perhaps others, learn a bit about the “black box”.
This is a lot different and makes no sense according to my tests and their outcome. Having the database open when I do the ‘Restore’ should have no impact on the data, i.e. the only data available is what is in the DOP.
So a repeat test concentrating on DOP contents rather than on DOP files being created and deleted
with the [M]aster, followed by VC[1] and finally VC[2] and this is the summary from an analysis program I wrote a few months ago, but extended to include ‘ColorLabels’, if any are found.
I made a copy of the DOP at that stage and it is named “P1112346.JPG.dop.sav”.
The output from the program contains a lot of debugging information so that I can verify its processing.
So I then delete the [M]aster, but whatever you do don’t use the Shift Delete command (I believe my testing showed that if you did that while in PL8 the image is deleted and nothing is placed in the Recycle Bin), which will delete the image and the DOP and it should find its way to the Recycle Bin
The image in the Recycle Bin has a ‘Rating’ of “1” and a ‘ColorLabel’ of “Red”, i.e. it is the [M]aster image that remains intact in the DOP.
@swmurray Quite what you are doing to get the result you appear to be getting I do not know because each and every test that I run yields the same results.
This time I closed PL8 and Restored the image and DOP, re-opened PL8 and navigated to the original directory and found this
Looking at the sidecars, I found that the “as deleted” copy contained the settings of the VC, while the “after recovery” copy had no settings for a VC.
In my opinion, this can mean that PhotoLab
overwrites the sidecar with old information (not likely) - or
ignores (on automatic import) entries for newer “Albums” hence removing VCs
My tests (on macOS) that I did with disabled automatic import and export of sidecars showed no unexpected behaviour. I’d therefore recommend that someone with a Win computer (and some spare time) test the recovery with manual import and export of .dop files. My bet would be that under these settings, the restore should also bring back the VCs. Can someone please check this?
My examples above show that the DxPL(Win) DOP in the Recycle Bin has only the [M]aster edits so no amount of clever importing can bring back something which is no longer there.
The second program output is of the DOP as it lies in the Recycle Bin and it has only one set of edits.
@BHAYT , can you please test with manual import and export of sidecars?
This could give us another peace in the puzzle…that DxO should solve asap.
Steps:
turn off automatic export and import of .dop files
take an image, create a VC, export the sidecar from the menu
check the .dop file for the VCs
have DPL move the test image to the trash
→ is the .dop not containing the VC again?
Update
Original text: Just ran a test with multiple deletes and restores and automatic I/E…which resulted in a big mess and mixup of images with no VCs or multiple VCs, even though each image had exactly one VC before deleting.
Correction/comment: I’ve re-tested different scenarios and found that every combination I tried delivered the expected results as reported here.
@platypus Apart from the FolderMonitor program not updating the logfile like it normally does.?
What you suggested in your script is going to cause the age old problem with manual DOP I/E, actually with manual DOP Import.
DxPL discovers an image with a DOP but is not allowed to read and Import the DOP so it allocates a new UUID
The user then imports the DOP and instant Unwanted VC results, at least one reason for not having manual imports. A slightly more sound strategy would be automatic imports and manual exports
With manual DOP export the deletion will not cause any automatic DOP exports except that the final deletion [M]aster will result in the deletion of the image file and the DOP, which will be in its last manually exported state, i.e. with all three copies in my case.
As for what happens using this as some sort of long term strategy I am not sure. But it is not the way most run DxPL which is with automatic Import and Export set.
With Auto Import set the unwanted VC, a copy of the original [M]aster, would be avoided and the DOP would be protected from the current cascading deletion of VCs until only the [M]aster is left.
But users have then to remember to periodically export the DOPs otherwise they will be left in some indeterminant state.
PS:- If the export wasn’t made when it was then who knows what might be in that DOP, just the [M]aster, some updates to the VC edits but not all of them, it is potentially a game of Russian Roulette!
PS:- After setting on Auto DOP Load, clean up of [M]aster unwanted VC and change to the colour of VC2
DPL in macOS does not create any additional VC in my tests. Again, I deleted the images with the respective menu item of DPL and restored the images when DPL was running or not running. In all those “manual I/E” tests, I got no unexpected effects, also not with auto-I and manual-E settings.
The Save settings in sidecar file (.dop) automatically option is responsible for deleting virtual copies and local adjustments such as … (the images belong to @BHAYT)
I can confirm that Wolfgang’s finding also holds for me. The deleted DOP when using Remove is the same as the DOP I had PL8 export prior to using Remove to delete the image. A simple Recycle Bin Restore of the RAW and DOP resulted in the Master and its 2 VCs being accessible again in PL8.
As I already have an open ticket about this issue I’ll just add these findings to the ticket.
I still consider it a bug, as this isn’t the PL (MAC) behaviour. Furthermore, as I rely on PL8 creating DOPs automatically (I save the RAW images and DOP for long-term retention and don’t rely on the database at all) I wouldn’t find this workaround acceptable. It would place too much onus on remembering to manually force the creation of a DOP at the end of making changes to a RAW image.
In these tests with DPL 7.10 on macOS, things worked as expected, no VCs were lost on the way to the trash and back to the original folder. No additional VCs were created. Moreover, results seemed consistent, no matter whether I had DPL on or off when I moved the deleted items back to their original folder.
Other than on Mac, win seems to follow this route
delete command taken
VCs deleted from DB → change, therefore-dop-update
rest of files deleted from DB → nothing there, so no dop-update