When is a user edit made in PhotoLab actually written to the DOP?

I wrote this some time ago in response to a post by @justinwyllie, namely Does PhotoLab have a separate database for each major version? - #7 by justinwyllie but never posted it until I just re-discovered it and decided it might be of interest to some forum users.

Sorry I left what I refer to as the “DOP write cycle” out of my discussion.

That “discussion” was at Does PhotoLab have a separate database for each major version? - #6 by BHAYT.

Exactly what DxO call the process of “batching” (some) DOP writes I am not sure, but a DxO Software engineer told me that DOPs are written up to 20 seconds after they have been updated and it looks to me (and others have also commented on the behaviour) as if they are written as a batch of DOP updates.

So DOPs can be written immediately or delayed by up to 20 seconds (or more, please see below), and any time in between.

In truth I have seen some writes that have taken longer than 20 seconds but if you want to ensure you are looking at the latest DOP then wait at least about 20 seconds before looking or close DxPL, which will force all pending DOP writes to be completed (allegedly and I have no evidence to suggest otherwise).

Folder Monitoring:-

Alternatively use

  1. Nodesoft “Foldermonitor” (freeware or make a donation, which I did when they kindly increased the timing field in the log to include milliseconds) or
  2. “my” Python script or
  3. “my” PureBasic program

but the latter 2 programs can only monitor 1 directory at a time where FolderMonitor can handle many!

  1. or all 3!!

The statement that the PureBasic program can only monitor a single directory doesn’t appear to be exactly correct. I removed some code from the PureBasic monitor program, downloaded from the PureBasic forum, and it now leaves the button to select a directory “active” and it appears that a number of directories can be selected! and monitored?

The two that I “wrote” are based on code that I “borrowed” and then modified , i.e. definitely not all my own original work but a useful way of learning about the processes involved!

The process that I believe they all use is based on FileSystemWatcher Class (System.IO) | Microsoft Learn.

What’s important is that I believe that DxO use the same process to monitor the directory currently “in scope”, i.e. open in PhotoLab, to detect any activity in the directory that might be of interest, e.g. another program making a metadata change, such as a keyword addition/change or the user deleting an image with another piece of software etc…

So I set all three to monitor the same directory which was open in PL8.6.0.

The directory contained a single image and I triggered a DOP write by toggling the NR button and got the following

1. From the PureBasic program

2. From the Python script running in the “Spyder” IDE

***********************************************************************
* P F M - P Y T H O N   F O L D E R   M O N I T O R - S T A R T I N G *
***********************************************************************
Log file title =  f:\___Python\T\P-log_2025-05-27 09-10-35.txt
 parameter lines = {'DEBUG': 'ALL'}
 parameter lines = {'DEBUG': 'ALL', 'DIR': 'F:\\___BETA DXO PL8 - TESTS\\TEST 42 - MOVING DOPS BACKWARDS'}
 parameter lines = {'DEBUG': 'ALL', 'DIR': 'F:\\___BETA DXO PL8 - TESTS\\TEST 42 - MOVING DOPS BACKWARDS', 'LOG': 'SAME (E:\\PFM LOG 2023-11-19 11-42-00)'}
p_lines = {'DEBUG': 'ALL', 'DIR': 'F:\\___BETA DXO PL8 - TESTS\\TEST 42 - MOVING DOPS BACKWARDS', 'LOG': 'SAME (E:\\PFM LOG 2023-11-19 11-42-00)'}
p_line_count = 2
p_lines_sorted {'DEBUG': 'ALL', 'DIR': 'F:\\___BETA DXO PL8 - TESTS\\TEST 42 - MOVING DOPS BACKWARDS', 'LOG': 'SAME (E:\\PFM LOG 2023-11-19 11-42-00)'}
 P_lines = DEBUG, ALL
 P_lines = DIR, F:\___BETA DXO PL8 - TESTS\TEST 42 - MOVING DOPS BACKWARDS
 P_lines = LOG, SAME (E:\PFM LOG 2023-11-19 11-42-00)
 P_lines_sorted = DEBUG, ALL
 P_lines_sorted = DIR, F:\___BETA DXO PL8 - TESTS\TEST 42 - MOVING DOPS BACKWARDS
 P_lines_sorted = LOG, SAME (E:\PFM LOG 2023-11-19 11-42-00)
key = DEBUG value = ALL
DEBUG command entered = ALL
DEBUG = ALL
key = DIR value = F:\___BETA DXO PL8 - TESTS\TEST 42 - MOVING DOPS BACKWARDS
Log file title  =  E:\PFM log 2025-05-27 09-10-35 .txt
ELog file title =  E:\PFM Log 2025-05-27 09-10-35 Elog.txt

Watcher Running, Directory = F:\___BETA DXO PL8 - TESTS\TEST 42 - MOVING DOPS BACKWARDS

2025-05-27 09:13:45.826579 - FILE - F:\___BETA DXO PL8 - TESTS\TEST 42 - MOVING DOPS BACKWARDS\P1137136.RW2.dop - FILE - deleted
2025-05-27 09:13:45.836588 - FILE - F:\___BETA DXO PL8 - TESTS\TEST 42 - MOVING DOPS BACKWARDS\P1137136.RW2.dop - FILE - created
2025-05-27 09:13:45.849600 - FILE - F:\___BETA DXO PL8 - TESTS\TEST 42 - MOVING DOPS BACKWARDS\P1137136.RW2.dop - FILE - modified

From the main Log file of that program

From the Log file where the full message from the Windows OS is logged

3. From NodeSoft Folder Monitor:-

Folder Monitor logs to a screen window and to a log file and can be set up to monitor many directories at the same time.

At this point I got ChatGPT to create a PureBasic “TickTock” program that would change the ‘Date Modified’ for a text file in the same directory as the image that I was changing for the test. This would trigger a file event, i.e. it would be detected by ‘FolderMonitor’ etc. and logged.

I took note of the time when I toggled the NR selector and got this time period from 17:14:50.962 to 17:15:21.367 which is more than 20 seconds?

I was concerned about how long it took so I stopped the “TickTock” program and used a stopwatch and it took 42.129 seconds!? Another test gave 25.046 seconds.

So the 20 second “rule” I was told about appears to be a general guideline only!?

So the actual time an edit will be reflected in the DOP will be immediately or up to 20 seconds or considerably more in some cases.

Export/update of DOP files usually take about 2 seconds on my Macs (when I watch) and there seems to be an “update all pending” when I quit PhotoLab.

This is how I found out

  • Quit DPL
  • Open its database
  • Set to 1 all fields of “ZSIDECARNEEDSTOBEUPDATED” in the ZDOPSOURCE table
  • Quit DB editing
  • Open DPL (nothing happens just yet)
  • Quit DPL → .dop sidecars are exported for all the files that had the field set to 1

Checking the DB again, all the fields I had set to 1 were reset to 0.

Check out the video:

@platypus I would normally agree and always felt that the DOP write was almost always immediate but had noticed and issue some releases back and queried a delay with DxO, when they were interested and monitored the Forum and got a response that suggested a 20 second maximum delay.

When @justinwyllie made the comment I showed above in his topic, I started doing some tests with the three folder event monitors at my disposal.

It gave me an opportunity to revisit Python given that our eldest Son wanted me to help with my Grandson’s preparation for his “mock” GCSE exams which includes Python!

So I had the three programs monitoring a simple “NR Toggle test” on my 5900X.

So running I had PL8.6.0, the three monitor programs which are all waiting on an Event to be caused, i.e. they are idle and not “buzzing” the folder looking for something to happen, and my turning NR on and off and …

From those tests I obtained the long wait times for a DOP write that I showed, which were unexpected given the power of the machine and the lack of any real processor activity.

Having three programs and the “TickTock” program running isn’t really going to be tying up any resource to any great extent, i.e. the 5900X has 12 cores and 12 additional multi-threading cores available!!

I will set up the test again, selecting one or other of the Folder Event monitor programs, but only one at a time and repeat the test with and without “TickTock” running and see what I get.

Actually I will set FolderMonitor to monitor the database Folder, on N:\ a fast NVME drive and the test directory on F:\ an old fashioned HDD at the same time and start with an empty database.

Sadly the database monitoring worked to show the files being created but no further activity was then seen by the monitoring software, firstly FolderMonitor on its own and then joined by PureBasic FM!

So back to the drawing board and one new ChatGPT program later with a proper timing routine that actually returns milliseconds correctly we have the following

To stop the clock I need to move from DxPL, having toggled NR on or off, to the timer window and press the space bar to freeze the timer and then let the monitoring program detect the event and report and log it!

From the first experiment with this new setup we seem to have

i.e. 15:50:41.936 - 15:50:15.797 = 26.139 seconds before the DOP is written!?

For me, the delay does not matter because of the export at quit. Whether switching folders or any other action triggers the export, I have not tested.

The important part imo is, that DPL should not drop any sidecar update under normal conditions. So, unless one force-quits DPL or with a crash, sidecars should contain current edits and metadata. I‘ve not found a means to force export to xmp though.