New user - very slow and wondering what to expect

@noname they and you are free to have that opinion but it is not one that I personally share, I prefer both plus I believe that DxPL (and DxO OpticsPro before it) are built around the database with the added facility/freedom of the DOP, effectively a database and a “partial” audit trail, arguably a full audit trail if you don’t need ‘Projects’ etc…

While I love your wonderful script how often is that actually executed? While I was sweltering at a garden in the U.K. yesterday I took 262 photos and typically I would look to use an original preset or two and add a couple of new ones and that would be done in DxPL.

I have FRV and could pass the image from that “browser” but tend to use FSIV which can only pass a single image and then load the directory.

But the “only” issue that poses any real problem is if images move from one system to another and back again when “Unwanted VCs” can result in certain (not all) circumstances.

I do “destroy” the database frequently when testing since the database is one of the most powerful tools with respect to the functioning of DxPL but only if I can see the “wood for the trees”. i.e. the test images from the rest.

However, I destroy the database by switching to an empty database, taking a backup copy before the switch if I want to retain the previous database. It needs a DxPL restart but your script requires DxPL to be down at the time of the database destruction anyway!

However, the good news is that you can freely practise your ritual and I mine!

@mwsilvers I would have agreed but if you look at my latest post DxPL 6.9 has lost database entries in my latest tests!?

@convergent it is round tripping that poses the biggest risk, i.e. process on A, then from A to B, more editing on B and then transferring back to A that poses the biggest risk, although some tests that I did worked just fine!?

@convergent If DxO got their act together the DAM features in DxPL could have been good enough for many, not as heavyweight as IMatch, a copy of which I own but rarely use.

PhotoMechanic has the advantage of metadata presets and although XnView does a reasonable the presets do not contain “dynamic” data, e.g. the file name, dates etc. as Photo Mechanic does.

DxPL has not such presets at all!

As do I but both for our digital images and the photos I am currently scanning we need to be able to catalogue family photos in particular i.e. who, what, when, where, why etc…

It was followed by “…” i.e. “but …” so the sentence trailed off, principally because I don’t actually have all the information necessary to inform your opinion further but I am concerned about how straightforward it actually will be and what pitfalls (if any) might lurk there now or in the future, you could try asking DxO whether your proposed strategy will work!?

I have run tests where the images and the associated DOPs made the round trip successfully (between two windows systems in my case) but it is a real pain if/when any don’t! Indeed I have processed on A and shipped to B, undertaken more processing on B and then shipped back to A with no problem and repeated the exercise again!

However, that was between machines of the same “persuasion” (operating system) I have never transferred between a PC and a Mac and then tried to come back to the PC.

If we could persuade a Mac user (@platypus) to supply two images and the DOPs I could try to see what happens when I add them to my system and whether the UUid changes before returning them to @platypus to see if they make it back into the Mac successfully?

every single time… I use FRV and when I need DxO PL a single raw file will be fed to the script by FRV (fastrawviewer), it does a lot more work in the rest of it … ( I also have another one if I need to open more than one file at a time - like several selected files )

I have so far not seen any loss of database entries as a result of 6.9, but admittedly my testing has not been exhaustive.

Mark

DOP sidecar files seem to be interchangeable. I have several times sent RAW files + DOP created on Windows to other users using Mac: we never saw anything weird.

At least this! :smiley:

there are matters of course when I probably better need to make sure DxO PL is NOT running to do certain things with DOP files… like batch generate them first (using DxO PL6 first time for the whole directory , all files in it ) and then update them outside of DOP with some other script, etc… so I rather need to be sure that all of .DOP files indeed appeared in the filesystem and will not be overwritten after I modify them

@noname I am not sure exactly what you are doing with the DOP files but the only reason for concern is if you change the UUid in the DOP so that it no longer matches the database entry.

I have not experimented with other fields in the DOP but forcing an unwanted VC is as simple as changing this UUid in the DOP

2023-09-07_234237_

which matches

Re-opening the image in DxPL after such a change will leave the database entry as the [M]aster and the DOP entry will become a Virtual Copy [1] or [2] etc. depending on how many VCs there are already.

If your scripts change that field in any way then you need to keep clearing the database but if you don’t then the database does not need to be cleared continuously.

In tests I carried out images from system A were accepted on System B with the same UUid providing that Uuid does not already exist in the System B database. It will then return to System A with the same UUid and no unwanted VCs will be created!

There should certainly be a feature added to the product to allow such situations to be resolved but the chances of that happening any time soon are next to zero I believe.

We were told during some testing that DOP writes are essentially batched and written every 20 seconds or before if circumstances warrant but I feel that is not quite correct using a product called Folder Monitor to “watch” image directories (it is thanks to a request from me that the log has an extra digit for the seconds, plus the actual work from the developers of course).

By that I mean that DOP writes might be slightly longer than 20 seconds although typically not by much.

@Lucabeer Thank you for the comment I believed that to be the case but had no way of verifying it.

@mwsilvers I have not experienced problems in the past but DxPL(W)6.9.0 has crumbled in my hands in a rather “scary” way as I indicated here DxPL(Win) 6.9.0 Loses all image entries in the database & Hangs when trying to access Projects.

Sadly as I attempted to recreate the problem in the hope of being able to determine the exact steps that cause it I unearthed another similar variant (lost image records from the database), a lost/empty directory in the display and the product hangs as I reported elsewhere also turned up.

Mac Mini is in use now so a couple of things…

  • Speed of DeepPrimeXD is not a concern, which was the point of this thread. I did a couple of images and it took 10-17 seconds per image. On the Surface Pro was 5-10 minutes per image.

  • I intentionally went back and forth between the two devices and edited images round trip a couple of times. I made sure to close PL6 on one machine before opening it on the other. I thought it was working fine as I managed several times without issue. Then on one edit it did not have the edits when I went over to the other machine. It had the “master” icon on that image, as if there should be a virtual copy, but there wasn’t but one DOP file. So that seems like what some of you were suggesting could happen. I re-edited it and it seemed to be OK, but it still has the master icon and only one DOP file.

With the overall speed of the Mac vs. Surface Pro (just talking about Deep Prime XD) I would likely not use the Surface Pro a lot for editing but it is my portable device so that is what would go with me on the rare time that I travel. And, I don’t have 10GBE setup yet which I expect will speed up going through all the images. I am not sure how I feel about the unpredictability of this.

1 Like

@convergent This is much the same experience as I’ve had trying to edit images using two machines (Mac Mini and Dell laptop), I have my images stored on an external SSD.

It worked a few times, and then other times it didn’t.

Sadly I’ve given up trying to figure out how to make it work reliably.

Previously I used Exposure X4 that also used sidecar files to store editing info and that worked seamlessly between computers, so that shows it can be done. But not with Photolab :frowning:

Photolab produces significantly higher quality edits than Exposure though.

rumors are they are out of business or in zombie mode ( that is just collecting money if somebody still buys the software )

Interesting, that does help explain why they haven’t updated their application for two years now.

@convergent This is what the Virtual Copies look like (contrived by me by changing the UUid of the DOP)

image

and both edits will reside in the same DOP, i.e. the DOP will (should) increase in size in-line with the number of Virtual Copies plus the [M]aster.

In this case the [M]aster contains the edits from the database and [1] the edits from the DOP which are the same in this case!?

Closing the database is a wise precaution because it will “flush” any pending DOP writes to disk. DOP writes are essentially “batched” and there can be a delay of 20 or so seconds before an updated DOP is written to disk!

This is the size of a DOP with a very simple edit applied

and this is the size of one contains [M] and [1] or rather it should have been but the disk appears to show only enough disk for one copy

After closing DxPL I have exactly the same size DOP containing only one set of information.

I had to re-open DxPL(W) and force the DOP to be written before I got this

which I believe should be unnecessary @DxO_Support-Team.

@convergent and @CHPhoto I cannot test your exact scenario(s) because I do not have a Mac I can use, all my machines are Win10, but even though it is my Birthday today if you supply snapshots of the incidents that have you worried and explain your exact procedure/intended procedure I will try to recreate the problem you are encountering, with the caveat that there will be no Mac in the mix!?

If I was working on a PC then I could choose to keep the data one an SSD and pass it between two systems with two separate databases but issues have arisen in the past when edited images do a round trip A to B to A to …

If I was working on a PC only I could also choose the keep the database with the data on the SSD but that won’t work in this case because a Mac requires the database in a prescribed location plus the databases are simply not the same anyway!

@CHPhoto How did the failure(s) show up?

If DxPL(M) is prepared to read DxPL(W) DOPs (and vice versa) then the “only” real issue is VCs, except if the versions of DxPL are different between the systems there can be issues, normally between different major releases rather than interim releases, but my experience does not extend to the same or roughly the same Mac release versus Win release!

How easy it is to keep both systems in lock-step I am not sure.

it is difficult to be another kid of the block - there are too many LUTs and plug-ins and whatever to twist the colors, add the grains, frames, light leaks, etc, etc and they do not have really any outstanding core value in terms of raw conversion or post processing features … hopefully somebody will buy them to augment own product if they decide to sell cheap enough for transaction to make sense

1 Like

I’d perform an edit in Photolab on one machine and when I opened it (in Photolab) on the other machine the image would display as if it hadn’t had any edits applied.

P.S. Happy birthday!

I appreciate the generous offer but I don’t know yet what I’m doing and need to do some more controlled testing. After looking at your explanation, what may have happened on that one file was I didn’t wait long enough between the edit and closing PL6 on one machine before opening it and editing it on the other.

Your example also had JPG files for the master and virtual and in my case I didn’t export a JPG, I just edited it and then closed PL6. I didn’t realize the DOP contained all the versions in there… I expected a new DOP for the virtual copy, so that will give me something to look further at.

Same for me.

@convergent if you had asked before my test I would have stated categorically that DxPL will “flush” all pending writes when it is shut down and generally long before that!

My tests showed that this was not the case and I believe that to be a very large error! Without that guarantee then those who “throw” away the database and keep the DOPs are in for a nasty surprise!

To force the writing of the DOPs select any or all images and

image

when DxPL will respond with

image

and since we are trying to force the writes of all the DOPs then the answer is ‘Yes to all’.

But this should not be necessary @DxO_Support-Team, unless I have been lucky in all my previous tests until today, or particularly unlucky with my tests today!?

The DOP looks like this

which corresponds to the two database entries

That sounds more like an issue between versions of software typically DxPL is upwards compatible but that does not guarantee being backwards compatible.

If a DOP is present it should be used but it sounds as if it is being rejected/ignored.

I have seen this in the past when testing two versions next to one another, going from an earlier release to a later release works but not necessarily going the other way and all my tests have been without the complication of different versions running on different operating systems!

So try to repeat your tests but forcing the writing of the DOP and check if moves from one system to another work but not when the direction of the move is reversed.

I have been looking for a new laptop but I am not in the market for a MAC I have way too much invested in Windows software!

PS:- I presume you both have these two items set

1 Like

It needs time. It’s the same issue using Topaz Sharpen or anything else.

As an update on my progress, I now have the Mac Mini M1 with 10GBE that I’m testing on, and I’ve got all my master RAW files in folders on my Synology DS1522+ NAS which also has 10GBE. I can now go through and cull my images very quickly directly in PL6. Also reported earlier, Deep Prime XD exports take 10-20 seconds. So my initial thread title about being very slow is no longer the case with the M1 and 10GBE combo.

My remaining issue is when I go back and forth between two machines and make edits most of the time there isn’t an issue but occasionally it is. I’ve not had a chance to do more testing on that.

PhotoLab writes .dop files asynchronously. Edits are therefore not written to .dop as you do them, but with some delay. This could be a reason for occasional issues.