Massively Slow Export on a relatively decent machine

DxO PhotoLab macOS topics
21

@Sean.Wash Because you have invested in Macs already that is an understandable choice.

I considered a Mac mini when I was looking to upgrade earlier this year but decided I had way too much investment in Win 10 software!

So I first upgraded to an RTX3060 with my i7 4790K but then started “lusting” after a new machine so I built a Ryzen 5600G and moved the 3060 to that machine which processed your image in

image

for NO NR, DP and DP XD respectively.

I still use the i7 4790K as my daily machine so I recently upgraded that with an Ebay purchased 2060 which gave the figures I posted earlier.

Hope you find the resources to purchase a suitable export system at a reasonable price.

1 Like
22

It would be an easier transistion and set up, especially networking wise. It would be super easy to set up a Mac Mini on my network and share files between them over the air.

But I’m absolutely not apposed to building an inexpensive Window’s rig either if the price and workflow is right. There’s some additional research that I will have to look into as well as shopping around.

But this has been really helpful and while I don’t have a quick solution I have some paths to go down through and again, I can bring down those render times down if I’m a little more selective and keep the simaltanous exports down.

23

@Sean.Wash

All of my tests were done with 2 simultaneous exports on both the i7 and the 5600G but that won’t show until a batch is submitted for export. Bye for now.

Regards

Bryan

1 Like
24

Oh…the Mac Mini is a great idea. The M2 Mac Mini is just $479 at B&H now. Some say it is better to get 16GB instead of 8GB. And, apparently the SSD doesn’t read/write as fast as the M1 did with 256 GB RAM unless you get 512GB on the M2. But neither of those things will probably make much difference where processing Deep Prime is concerned. The Mini shouldn’t throttle even with hundreds of photos, since it has a fan. Good luck on what ever you decide. The newer M3 Mini is rumored to be announced in March 2024.

1 Like
25

@Sean.Wash
On my mid-range PC/Win11, i7-14700KF+RTX-4070, PL7.1.0, the processing times are 6s and 4s for DPXD and DP respectively. With my 24MP camera, for batch exports (100-2000 images) it’s typically 3s and 2s per image on average, but that’s probably mostly due to parallel processing with 2 threads. Unfortunately I have zero experience with Macs to help you with the hardware.

As @platypus has pointed out, I don’t see any point of using DP XD for ISO 160 (and no extreme shadow recovery). Until you use 5-digit ISO, the only difference between DP and DPXD is the processing time. As a rule of thumb I would say, that people’s photos which are not good enough with DP (non-XD) should land in trash, unless you are a photojournalist targetting smartphone users. You may see some difference between DP and DPXD on very high ISO only, but then there’s a danger of AI inventions that will not suit you. DPXD can save your picture indeed, but only in quite specific cases.

DP (and DP XD) adapts to ISO settings, so you can use it quite safely until some ISO limit. Low ISO photos without any NR and with DP do not show any noticeable difference, at least in my case. For higher ISOs (5-6-digit), DP takes care for some (controllable) luminance noise to remain, which is good for me, but might be bad for others. So, use DP, which makes exports usually run 1.5x faster, compared to DPXD.

BTW, I had a hard time trying to figure out what is the main topic of the photo - is it the car, the kiss, or both. I don’t get the blacks setting and strange tonal curve - somehow the asphalt distracts me. But don’t feel offended, that’s just my personal impression.
The God has so various inhabitants :wink:

26

Thanks for the insight @Wlodek. I’m new to using DXO and I thought the overall results of DPXD were really impressive so I was a little greedy and trigger happy to use it (Also a little lazy, I just like to select all the images and press the button.) But I obviously need to be more selective.

It’s admittedly not the best photo from the session composition wise. But it was very important to get a photo with the couple and the car as it was highly requested. Looking back, I wish I had worked more with the car compositions earlier in the session than saving it for last.

Initial focus was supposed to be on the car and which then would lead up to the couple kissing on top. I was hoping the “coning” of the wide angle lens would help the flow of the image some.

Also, interms of style, I always lift the blacks on the curves. I’m a weirdo who doesn’t like their blacks to be black, lol.

Obviously, there are more touching up I would like to do before delivering this photo (this is a WIP file) but I’m totally open to suggestions on editing, color and croping even. I’d love to get your perspective and try out different editing styles.

27

I’m on the opposite side, but luckily we are all different.
I did just a few weddings and didn’t try your method. Maybe it makes the pictures more relaxed…

1 Like
28

Definitely. It would be boring if everyone approached their photos the same exact way. As for me, I really should open and try to do some edits without my usually approach to blacks and contrast to expand my style capabilities and creative vision.

You have it exactly right though, it gives it a more relaxed feeling. It also technically hides detail since you’re technically shrinking the dynamic range, but in return you get what I like to call a painterly feel. But it’s an overall subtle effect, but I got really comfortable with it. Maybe too comfortable though haha.

29

…key differentiators…if you can get them to work reliably. Handling PhotoLab is just somethimg you’ll have to learn and get comfortable with. Depending on how you best learn, you can check the user guide, respective material on youtube - and lots of trial and error.

If you like painterly/movie looks, you might want to check out Fuji’s film emulations and run a trial of FilmPack.

1 Like
30

As a Fujifilm shooter, I’m very familiar with their Film Emulations! Also, I bought the Film Pack as part of a bundle when I got DxO PhotoLab 7. I knew I was going to like it haha!

But thank you for the encouragement! I’m always trying to learn after all.

31

@Sean.Wash Here are the figures for a batch of 50 of your images with your DOP from both machines, I forgot @platypus’s suggestions that batches of 60 make calculations easier!

image
image1519×688 250 KB

5600G (+3060) on the left and i7-4790K (+2060) on the right and the CPU charts should be self explanatory (and I missed the problem shown in the snapshot of the i7 that I spotted and corrected on the 5600G, and I am not going to run the test again just to get a better snapshot).

Both machines were running 2 copies of the export, the 5600G should be able to handle 3., it can but to no great advantage in elapsed time

image

The 5600G manages 5.68 seconds per image versus 9 seconds when exporting just 1 image (both with 2 copies). The 5600G is just over twice as fast as the i7 and the 2060 is about 80%, maybe a bit more, as powerful as the 3060 and all the components are “relatively” cheap, i.e. the motherboard is capable of taking the 5900X, twice as powerful as the 5600G should I get greedy!

One of our son’s uses Macs for his video work but I don’t think I can persuade him to bring it along at Christmas just so that I can “play” with it!

1 Like
32

@BHAYT Wow! Thank you so much for running these tests. They’re so thorough. I’ll definitely look into building/buying a dedicated rendering computer. If I buy used, I think I could get something for not a whole lot of money, but will really speed up my workflow.

33

@Sean.Wash You are welcome! I woke early and couldn’t get back to sleep so it helped pass the time until my wife woke up.

Except that I had to remember the process for creating batches quickly from a single image, once remembered it was easy but waiting for the tests to finish reminds me of what it was like with my 1050i (even more tedious), which was still way better than trying without a graphics card, but the 2060 is a useful addition to the i7.

UPDATE:-

Something went wrong with the 5600G keyboard so I rebooted and reran all the tests with 3 export streams and got

2023-12-19_103600_5600G with 3060 DP XD with 3 copies
2023-12-19_103600_5600G with 3060 DP XD with 3 copies794×260 85.3 KB

so the additional copy benefits NO NR and DP but the DP XD figures now start to stand out. There is sufficient CPU power with the more CPU intensive streams (No NR and DP) to improve the export times with the additional copy to give 3 exports active but the 3060 is holding back the times with the (even) more GPU intensive DP XD.

The 3060 was arguably the cut-off point for price at the time that I bought that card, i.e. there was a big jump to the 3070 etc…

The processor charts look like this

2023-12-19_104943_5600G with 3060 No NR 3 exports
2023-12-19_104943_5600G with 3060 No NR 3 exports1904×305 179 KB

2023-12-19_102851_5600G with 3060 DP
2023-12-19_102851_5600G with 3060 DP1897×264 148 KB

2023-12-19_103428_5600G with 3060 DP XD
2023-12-19_103428_5600G with 3060 DP XD1895×303 177 KB

1 Like
34

“ The recommended number has always been 2 as is the default. You must have inadvertently increased it at some point.”

Wrong. I’ve never altered, either by design or accident, the default which is also 7 on my MacBook Pro.

35

I’m on Windows. Maybe somebody on a Mac can chime in on this…

Mark

36

I’ve just changed my settings to 2 at a time. I did a parade photoshoot recently and it took all morning to process the pictures. It didn’t occur to me that DXO would set a suboptimal setting by default.

I’m running a test now on 37 images, R5 raw, with deepprime XD on. Curious to see how long that takes 2 at a time.

37

@626ds

That seems to be very ambitious, you can see from my tests above what happens on a reasonable PC (my 5600G with a RTX3060) as the number of export threads increases from 2 to 3.

Sadly there is zero chance of me being to upload the 50 copies I used for the test but it would be interesting to see what a MacBook Pro can do with at least 10 of the images.

However, what happens on a PC is that DxPL starts with one copy and when that finishes it adds another copy and repeats that adding an additional copy until the maximum is achieved or DxPL needs to start decreasing copies much the same as it increased them!

With a batch of 10 images, DxPL would never get 7 copies running before it was time to start closing them down!

The 5600G was still running from this mornings test so I ran tests with 7 export copies and got

2023-12-19_152737_3 export copies versus 7
2023-12-19_152737_3 export copies versus 7736×397 90.6 KB

So No NR was slightly faster, DP was fractionally slower and DP XD was slower still!

The problem with the Google spreadsheet, other than the fact that it doesn’t split out the ‘No NR’ (NO Noise Reduction i.e. CPU only activity to apply the edits to the image) is that it doesn’t include batches of a suitable size to actually determine the real power, or lack of it, of a CPU + GPU combo, i.e. if there is enough CPU and GPU to process multiple threads simultaneously then the overall throughput shoots up or stays where it is or declines.

So the next GPU up from the 3060 is the the 3070 with a quoted games benchmark score of 148% (the 3060 is rated as 100%) and costs about £200 more (£270 versus £470 which is 100% versus 174%) so the price starts outstripping the GPU performance, and yes I do know that the RTX 4000 series has replaced the RTX 3000 series!

PS:-

If I bought the hardware for the 5600G today it would cost about £331 (CPU, Mboard, 32GB memory, Cooler) which is roughly what it cost me some months ago and was the price I paid for a RTX 3060 GPU, which would now cost about £280 or £295 for an RTX 4060 and all but the RTX 4060 are generally available second hand.

To those costs would need to be added a case and power supply but I already had them and then an OS licence, which can be picked up for a variety of prices, with or without Office 2021.

38

It just never occurred to me to alter the default. It took 43 min 41 seconds to process 37 Canon R5 raw. I’m now running the same conversion with 5 at a time.

39

Still more than a minute per image. DeepPRIME XD is not really meant to be the default denoiser…and it painfully shows with weaker GPUs.

Today, I ran 60 copies of the RAF file with the respective edits, DeepPRIME and export set to 4 parallel tasks…which gave 7.25 seconds per image. With DPXD, exports happen around 30 seconds per image.

40

I use DXO PL7.2 on a windows pc. In Preferences, the number of images exported simultaneously is set to 2.My Cache is set to 5000. Your cache setting is 1000. The next time you run a batch, think about increasing cache and click on clear. Make sure the cache location set in preferences has enough room. When I use batch mode to create Jpegs in Deep Prime, I usually do ten at a time. I have never timed it, but my guess is that each image was less than a minute, maybe 30 seconds max.

Others with more technical experience can make better comments about the role that cache settings may or may not be playing.