Exporting to disk takes extremely long with DeepPRIME XD

My take on it is that the studio drivers are more firmly established as stable builds for non-gaming applications, and are therefore not updated as often. However, with all these changes of behavior from one PL minor update to another, I recall the principle advocated by many OEMs: update only if you have problems. So I’m starting to believe that the game-ready driver is the one to go with. I’ve been using the studio drivers for some time now, but have not enjoyed a more stable experience as a result.

This is my rule too.

Saying update to the latest version as DxO does is not enough.
Other softwares I use say : from version xxx at least : so we know with which version they have made their optimisation.

For those who use their workstation for professionnal use with other gpu intensive softwares, studio drivers are generally the best or only option.

But I think DxO is not part of those companies which collaborate with NVidia to elaborate those studio drivers.

@Lucabeer My tests involved a 1050Ti and a 1050 as well as a 3060 is there any issue with my results, i.e. the problem may be widespread but not everyone is experiencing it.

@Joli but not mine it seems!?

@Joli agreed.

Apparently I need to rephrase: Well, it looks like only some older nvidia GPUs are affected,
There are of course lots of variables that may or not affect the outcome of this issue. For instance W10 vs W11 (although I see you’re using W10 as well), but there are many others.

BTW And I see you’re also an MFT shooter :grinning:

@Joli Indeed my standard rig is a second hand G9 with a 12-200 (24-400) Olympus lens. I also have an Olympus EM1 MKII (part of a special package with the lens) and recently traded my GX80 for a second hand GX9.

But the worse one if surely

image

Why in a package that seeks to “automate” so many things, including the importation of images, must that question be asked time and time and time and time and … again and again and again and … until the user is driven crazy (enough to write this)!

Provide a ‘Preference’ that states I will always use ‘Use unique names’ or at least a session option that provides that facility.

I will admit that the performance tests I ran earlier in this post are one of the few occasions where I am tempted to use ‘Overwrite’.

However, I am concerned about the fact that some users are experiencing the slowdown but I am not why the differences in results? My PL6.10 “rig” is not that different from the author of this topic an i7 4790K with a 1050Ti versus an i7 3770K with a 1050!

@TomS I ran a test with PL5.13 which was on my system, DP first then NO NR (No noise reduction) but the images have no editing because the original work was done with PL6.4 and the DOPs don’t work when carried to previous release, i.e. DOPs are forward compatible but not backwards compatible between major releases.

The result was
2023-09-29_235819_PL5.13

I then downloaded and installed PL5.15 and got

2023-09-30_002113_PL5.15

They are both almost identical!?

@BHAYT Maybe the problem was caused by the time I edited photos in the program. I did this for many hours. And then I did the export. About 450 photos were saved for 44 minutes. This resulted in 25 errors during process, most of them in images using DeepPrime denoising.

I exported photos marked with exclamatiom mark three more times. Each photo took about a minute to save.

Today I exported the same 450 photos again. There were no errors. And the saving time was 17 minutes.

@TomS My tests are basic, the ones in my PL5.13/PL5.15 tests are even more basic because the edits were not present at all. But my tests are intended to see if I can re-produce any issue and see if we can gain any insight into what might be going wrong.

I am concerned about the errors you encountered during the “long” export but which did , I have encountered errors during export once or twice in the past! I was going to say “typically when I started multiple export runs that overlapped” but DxPL queues such runs, i.e. one export does not start until the previous one completes!

From the length of time of the previous run with the errors and the latest run without errors (and with a greatly improved run time) it suggests some sort of clash for resources that resulted in the error report but what they might be I cannot guess! Did the error report give any sort of clue?

I am concerned about the errors but have no way of reproducing them and that is unlikely to help anyway, this is one for DxO.

That might have been the case but that would be a “bug”, i.e. regardless of what you did during the editing that should not impact the export and cause delays and errors, unless you start/continue editing the same images after they are already queued for export!

Personally I would raise an error report with DxO giving as much information as possible and either

  1. Terminate DxPL after a long editing session and restart for an “export” session and/or

  2. Using your timings for the 450 image export of 17 minutes, monitor the export and if the time starts to greatly exceed your estimate and/or errors start appearing then cancel the export, terminate DxPL and restart and start the export session.

Either restart from the beginning or from an “appropriate” place, unfortunately DxPL does not provide an ‘Ignore’ option when it encounters duplicates you must either
image

and both are a waste if you have exported many images already!

Sorry I can’t be any more help.

Regards

Bryan

If in preferences, an option to keep things as they are should be provided too, because it is usefull for some of us.
As always, let user CHOICE.

Guess we’ll have to wait a while for a solution. I suppose DxO is currently more focused on PL7.

PS I see I messed up the layout of quotes within quotes within quotes, hence the italics.

@TomS My tests are basic, the ones in my tests even more so because the edits were not present at all. But are intended to see if I can re-produce any error and see if we can gain gain any insight into what might be going wrong.

I am concerned about the errors you encountered during the “long” export, I have encountered errors during export once or twice in the past but

@JoPoV sorry I always assume that as standard. Any additional facility should always provide a method to retain the status quo. However, the chance of any change is remote as is DxO these days arguably for the reasons @Joli stated

but they have been largely absent throughout the life of PL6!

It’s the same with me but my biggest problem is that my CPU is running at 100%. I have relatively slow GPU NVIDIA 1030 but never had issues until now. The process time is around 8-9 minutes as well.

From: https://support.dxo.com/hc/en-us/articles/4432757357073-DeepPRIME-DeepPRIME-XD-hardware-acceleration

For a better experience, be sure at least to match our recommended requirements:
Microsoft Windows recommended requirements: NVIDIA RTX™ 2060, AMD Radeon™ RX 6600 or better with the latest drivers

Based on what @kal said (Hello @kal :slight_smile: ), could it be that the older CPUs now struggle because the code has been optimized to use functions they don’t have and that the changes in processing times have nothing to do with older GPUs that weren’t really doing anything with DeepPrime anyway?

EDIT to add: Perhaps the code using CPU for processing is broken and only GPU processing works correctly. Can anyone verify this comparing older and current versions of PhotoLab with CPU only processing on a curent CPU?

@kal I gave figures earlier in this post that showed that as far as I could tell there was no issue with exporting on my machines.

However, after behaving for a while PL6.10 started sending the processor usage to 100% and causing the processor fan to become more audible. Just clicking on an image would cause a spike for a while and then the machine would calm down and then another click would start the whole thing off again.

This has nothing to do with exporting but appears to be some sort of looping in PL6.10 and makes the machine unusable during each spike @DxO_Support-Team.

Out of curiosity: in Edit > Preferences > Performance, is the video card still selectable for DeepPRIME acceleration?

Yes, that’s exactly my experience too.

@kal Just started PL6.10 up and it started on an image that has been causing problems in particular.

The system slowed, then I jumped to another test image and that was fine, but that might deteriorate later, returned to the first image that cause the issue and lost mouse control for a short period.

Process Lasso only stalled slightly this time and the graph shows this

but the third incident shows the responsiveness of the system, shown in red, vanishing for a short time.

I will create a topic and then make a formal support request later this morning.

As I stated before this is not anything to do with exporting but simply traversing images, in this case with VCs and I was testing HSL in particular.

Selecting VC[1] is shown in the graph below and the width of the peak is narrow because Process Lasso stalled completely, please note the PhotoLab processor consumption at that point to simply render the thumbnail and the image (which is actually not in the display)!

it takes you 8 min per each file to export to DNG DeepPrimeXD?

You made me curious … and exported a 64MB raw-file as 16bit TIFF / AdobeRGB

  • PL5.15.0 Build 4902
    with DeepPrime → 20-21 sec

  • PL6.10.0 Build 284
    with DeepPrime → 16 sec
    with DeepPrimeXD → 41-42 sec

  • PL7.0.1 Build 76
    with DeepPrime → 16 sec
    with DeepPrimeXD → 41-42 sec

In all cases the task manager showed (more or less) full use of GPU !


i7-8700 / 3,2GHz // 32GB
NVidida GTX 1060 6GB / Studio driver v536.99
Win10 pro, latest version

Checked DPL 5, 6 and 7 on macOS Monterey on iMac 2019 (8 core i9, Radeon Pro 580X 8 GB)
The times I got with No Correction and virtual copies (master with HQ, copy with DP)
produce the following average values.

  • DPL5: 5.75 seconds/image
  • DPL6: 5.10 seconds/image
  • DPL7: 5.10 seconds/image

Test with 60 image files, 1.3 GB total