Oh, I’m sorry. I forgot the average Brit could only speak one language 
Delete Master…
Delete VC…
1 Like
Don’t forget us Aussies!
Ah yes, Australian - the language where every sentence sounds like a question 
1 Like
@Joanna Merci or should that be Mercy!
I am sorry but I stopped learning French after my O levels, I was 15 at the time so that would be about 62 years ago.
You obviously have never experienced hearing a California valley girl accent from the 1980s-1990s. The valley referred to is the San Fernando Valley. The people who spoke that way very often came from very privileged backgrounds. The accent is generally demeaned in popular culture.
Everything out of their mouths sounded like a question. Valleyspeak has a linguistic affectation called uptalk where the pitch of the voice rises towards the end of declaritive sentences and makes them sound more like questions.
Mark
It is strange that we rely on a database that doesn´t support cascading updates and deletes in 2024. No wonder why there are orphaned data garbage in our databases.
I think I’ve posted this before but it’s worth repeating…
- If you speak three or more languages, you are multilingual
- If you speak two languages, you are bilingual
- If you only speak one language, you are British
1 Like
And if you speak only one language – and poorly – you are probably US American! The incredible amount of poor grammar everywhere in this country, even in commercials where the communications “experts” should know better, probably has most English language teachers questioning their choice of profession!
2 Likes
…and the last syllable of the last word in every sentence is unnaturally drawn out.
It’s gittin’ pretty common nowadays, everywhere.
2 Likes
@stuck That might need to be modified to English in order to not insult a sizable chunk of the Welsh, some Scots and some Northern Irish!
@platypus I think you published something like your post above before and I am sorry if I appeared to ignore it I didn’t but I did, i.e. I ran some tests of my own which had differing results between the tests I ran and then reran and because I could not come up with a definitive strategy I failed to publish, sorry for apparently ignoring you previous post!
So I will try to document what happened to my tests here but must emphasise that I have run into yet more problems when I attempted to repeat my tests for this post.
Unpublished Post in response to forum.dxo.com/t/kludge-how-to-make-a-master-of-a-vc-on-mac/38581:-
I have always felt that DxO could have provided coding to completely prevent the “Unwanted Virtual Copy” scenario!
I also believe they have changed the product somewhere along the way so that the product does not automatically assign a new UUID to an image when a DOP is present unless the one being presented is a UUID already in use (I would expect this to be extremely rare but not impossible).
This makes “round-tripping” an image with edits, e.g. from laptop to the desktop and then back to the laptop, less likely to result in “unwanted VCs” but they still crop up from time to time!
However, a simple test in DxPL with a request to the user to make a choice instead of automatically creating a VC, could avoid the current fallout from the problem.
With respect to the old scheme that @platypus remembers (as described in this topic and the reasons behind it in your post above) I am not sure that will ever return, so simplifying a collection of edits by deleting is only going to work easily for the VCs and the [M]aster is always going to remain intact or be completely deleted from the database and disk.!
Any attempt at “external” adjustment would lack the main feature of being able to actually see the images rendered which is essential to aid the user to make the decision as to what is going and what is to remain.
The best scheme that I can suggest is to leave the [M]aster intact and create VCs for editing and creating alternatives. When the time comes to “thin” the edits down then copy the best corrections to the [M]aster and delete all or most of the VCs.
To swap corrections between VC’s or between a VC and the [M]aster an additional VC is needed to provide the required “swing space”, i.e. copy edits, Iptc and keywords from A to “Swing” VC, copy edits etc. from B to A and then copy edits etc. from “Swing” to B and delete “Swing” when no longer required…
Because both your proposed technique and @platypus’s technique are excruciatingly boring and tedious as is the process I have outlined below, when executed manually.
However, the process can be automated with the help of coding and that makes it possible to run through an entire directory of hundreds or thousands of images as and when required, e.g. when an entire directory becomes full on “unwanted VCs”!
@platypus Why bother to destroy the database!?
Here is a test directory complete with [M]aster and VC[1]
and also complete with projects
The way that the data is arranged in the DOP is as follows
All the metadata and edit data are to be found after the “Album” heading in the DOP, that includes all the metadata, so having ensured that DxPL is not using the directory to be adjusted and having secured a copy of the database before potentially “trashing” it by accident, it is possible to fiddle with the DOP before it is then re-discovered in DxPL.
Intentionally trashing the database would destroy any projects and on the Mac, advanced history as well. If it is possible to avoid any unnecessary destruction of the database or DOPs etc. then I believe that to be the better course of action.
I did the following with DxPL(Win) running
-
Extract the [M]aster details
-
Extract the VC[1] details
-
Create a new DOP containing the Header, VC[1] details,[ M]aster details, + the Trailer
-
Swop the UUID between the, now re-located, [M]aster and VC[1} entries so that the (new) [M]aster contains the VC[1] data but with the [M]aster UUID and VC[1] contains the old [M]aster data but with the UUID of the old VC[1]
-
Adjust the 'Modification time" up by 1 (I need to verify that this really triggers DxPL to reload the DOP when automatic DOP loading is set in the ‘Preferences’).
Rediscover the directory and hope for miracles.
Via the ‘Projects’
with the “BaseLine” directory still showing the original layout
By retaining both copies it is possible to review the outcome and easy to delete [1] which was originally the [M]aster entry or not!?
Now all I need to do is write the PureBasic program to automate the function., including securing the database and the DOPs before starting the switch for all images that have two entries in the DOP!
Update, with an update to that:-
However, during that test I managed to hang DxPL but on restart got the swap that I expected and have achieved in the past.
I then tried to repeat the tests and go further but made the mistake of reversing the images because I felt it looked more appropriate to have the muted image as the [M]aster and the brighter image as VC[1] but that just made things even more complicated,
I also created additional image copies, i.e. copy the image with a new name and take one of the VC entries to create its DOP by adding the header and the trailer.
That appeared to work at the time, however I failed to repeat the results I got with the DOP swop and gave up.
Update to the update:-
I tried again with the example I used at the beginning of this post, with the ‘Ratings’ altered slightly so they go from 0 for the [M]aster to 4 for VC[4].
I then created 5 images from the original one image and 5 (hand) manufactured DOPs and got
The edits look to be intact but the ‘Ratings’ etc. are there in the DOPs but not in DxPL after discovery, even after clearing the database and locating only that directory!?
So that will remain as is for now while I get on with the program to do the DOP manipulation automatically (it is way to prone to error doing it manually) to see if I can discover what I have done wrong or which fields are causing the metadata to be ignored, i.e. there isn’t much to cause problems other than the date fields (or my formatting)!?
Sidecar = {
Date = "2024-08-11T07:15:39.8424695Z",
Software = "DxO PhotoLab 7.8",
Source = {
CafId = "C61004c",
Items = {
{
Albums = "",
CreationDate = "2024-08-11T07:14:01.5966400Z",
IPTC = {
contentHeadline = "[M]",
}
,
Keywords = {
}
,
ColorLabel = "Red",
ModificationDate = "2024-08-11T07:15:39.8384705Z",
Name = "P1-0.RW2",
Orientation = 1,
OutputItems = {
}
,
ProcessingStatus = 1,
Rating = 0,
Settings = {
I even changed the image name in the DOP, I have previously believed the field is basically “ignored” by DxPL(Win) but without any success.
In order to prevent conflicts…and to make the edits work. The annoying thing about editing the sidecar is the master’s bunch of edit entries which can be on any position from first to last.
The distinction between master and vc seems to be in the database rather than in the sidecars. Haven’t bothered to search for it, even though rhe db on Mac looks rather clean and simple in comparison to what I remember from checking dbs of Mac vs. Win.
Any hack, be it with editing a sidecar or db is a workaround at best. Any softwareupdate could render all of it useless, e.g. if sidecars were binary or signed.
I have a very simple technique for dealing with accidentally deleting a master instead of a VC - I just never edit the master, creating a VC before doing anything else.
@platypus the answer is yes and yes but actually the basic coding, i.e. minus all the good checks, proper timestamps etc. etc. is simple enough and currently I have the following for a DOP from another user
So ahead of the first “Albums” entry is the header and the trailer is between the last but one “Uuid” entry (Line 1105) and the end of file and includes the last “Uuid” shown at line 1110
As far as I am concerned the lines between the “Album” and associated “Uuid” entry are the details for the copy, [M] or VC or … and it should be possible to do a number of things with that data, allegedly!!
@Joanna As do I and many others but I am trying to see how close I can get to a working “Unwanted VC” fix as possible. My ideal fix is an intact database with intact ‘Projects’ and both copies but reversed and I have managed that manually a number of times until I didn’t for some reason(s)!?
Plus I want to avoid going anywhere near the database if possible, other than automatically backing it up, along with the original DOPs and sidecar files etc.
The task of doing the job by hand is error prone, hence the coding which is easy to fix and try again and again and …
Its a hot day here in the South of the U.K., I still have 4 rack bolts to fix but only mad dogs and Englishmen go out in the midday sun, I have returned to some coding!
@RexBlock Basically my eyes, i.e. countless tests and inspections of the resulting DOPs.
The analysis may or may not be completely accurate, for Macs in particular. with respect to the ordering of the copies in the DOP image, i.e. I have assumed that the order in the DOP was the same as the order in the database, i.e. [M]aster first, then VC[1} then etc etc, but this might not be the case for Macs and possibly not for Windows!?
I have written a program that shows the following for a Mac DOP is particular
and looking for patterns
Does this help!?
PS:- The program makes no assumptions about what it will find, it reads each line of each DOP and reports on “lines of interest” as they are encountered, i.e. a line with “Albums”, a line with “Dates”, then split into “Creation Date” and “Modification date”, and a line with “Uuid”.
As I wrote, a copy (exactly which copy may be in doubt?) exists starting with “Albums” and ending with the “Uuid” for the copy, and then repeats again and again with 1 entry for [M] and then entries for VC[1], VC[2] etc.
An additional “Uuid” is part of the “Trailer” as I called it and I need to see if that points to the “Uuid” of the “Sources” entry, which I once thought it did but haven’t bothered to confirm recently. Regardless of what it refers to it exists and I believe that DxPL will expect such an entry in the “Trailer” of any DOP.
However, there are issues with my analysis because it is purely empirical and while I have succeeded with my tests getting the desired results from time to time I have also failed with my hand created DOPs.
I am close to being able to create a new DOP from an existing DOP, e.g. a [M]aster only DOP, a VC[1] only DOP programmatically, i.e. the errors will be made in a program which can be “easily” rectified and tested again and again quickly instead of putting the DOP together by hand and finding it doesn’t work because of some stupid mistake!
PPS:- the database schema is available via “DB Browser for SQLite” available for Mac and Windows and opening the database gives
on Windows .
It does, but I’m on holiday and I need to return home and clear the holiday brain before digging into this more deeply. This is too much to entertain while listening to the crickets and owls.
My curiosity was piqued earlier in this thread with a couple of screen shots that came out of a DB analysis tool. That was my paying job back in the day and it reminded me of a tool I once used called Toad. It was for Oracle RDBMSs but I’m sure there are similar tools for MySQL DBs.
I was wondering if you managed to open the DxO MySQL DB with the tool, and what tool it was. Thanks, your work is quite enlightening.
1 Like
@RexBlock Enjoy the rest of your holiday.
Having started as a Database Consultant for Burroughs (which became part of Unisys) and working for them for 36 years with various job titles, mostly involving a product called DMS II, I should have realised that you were referring to a database schema rather than the DOP “schema”.
So the product of choice for the PhotoLab database is “DB Browser for SQLite” available for Windows and Mac (and Linux etc.) and free (or Patreon sponsorship). It provides access to PhotoLab DB and gives an insight into what DxO are doing with the database.
I decided to get back into programming a couple of years ago and started to learn Python but stopped for a bit and then developed my own Watchdog program (built around someone else’s core code, of course), before discovering PureBasic (PB) which I am using to further investigate the “ways” of the DOP.
I have written the basis for a program in PB to “swap” drives which allowed me to start amending the PhotoLab database, albeit only the ‘Folders’ structure in Windows PhotoLab.db,. There appear to be fairly substantial differences between the DxPL(Win) database and the DxPL(Mac) databases.
That program needs finishing before it can be used for an “emergency” disk replacement.
The program that I used to investigate the contents of the DOP is useful and I have started to modify that to provide the ability to create new DOPs prior to investigating swapping the [M]aster entry with VC[1], or rather extracting the DOP element for the [M]aster and VC [1] etc. and rebuilding a “replacement” DOP with the data reversed.
I need to verify that the order of DOP data placement is as I originally thought, on Windows at least, with the order of Copy data within the DOP being the same order as we see in the display or by determining what strategy DxO uses for extracting the data from a DOP to reflect the order in the display.
PhotoLab for macOS creates dop files in which the section describing the master’s customising is in any seemingly random position in relation to the virtual copies.
The master and the copies cannot be identified directly by a special entry, but by the creation timestamp. only. We could also use manually added data like colour tags or changed file names, but I consider this to be less practical and reliable in comparison to a timestamp written by PhotoLab.
Deleting the section of the master from a dop file will “elevate” the remaining oldest entry to “master”.
Note that some of the above was discovered and verified in cooperation, via private messages, with @BHAYT .
Interestingly, delete and remove are used in the same context. Delete rather means “put in the trash” while remove implies complete elimination. Also, the dialogs says that only virtual copies are deleted from projects 
. Mhhh, all rather inconsistent.
@platypus Thank you for the update and the technique of colour Tags + Rating + IPTC field would help to verify that the timestamp, which requires no special action by any user, is doing what you believe it is and logically the [M]aster exists before VC[1] which exists before VC[2] etc. so the ‘Creation timestamp’ should be correct.
I need to verify that works for Windows and understand the implications for any DOP Copy swap strategy.
But that needs to wait a while, we are off to Hastings to see one of our granddaughters taking part in “Charlie and the Chocolate Factory”, a bit part but you have to start somewhere.