Author Topic: Number of CPU Core Utilization for DxO Optics Pro 11  (Read 12115 times)

GSB2

  • Jr. Member
  • **
  • Posts: 14
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #15 on: November 07, 2017, 08:05:13 am »
A very intersting thread this. I am choosing the spec of a new desktop PC for myself at present. Home use, general purpose, but with the specific intent of speeding up export of raw to jpg with prime NR enabled a lot of the time.

The above is something my aging first gen. i7-860 (4 core 2.8Ghz), 8GB RAM, AMD Radeon 6670HD struggles with. A 20mp image with Prime NR taking approx 80 seconds to export in Optics/PhotoLab.

Now everything I had read until now (including a long and detailed thread on here) was telling me how great DxO was at utilising cores and threads when exporting with Prime NR which is very CPU intensive (and this specific function not using GPU).

However like someone else in this thread, my workflow is quite serial in that I will work on an image with specific corrections, export, check and move on to the next. I have never batch processed corrections.

So the performance increase I am looking for is exporting a SINGLE image with Prime NR. What I am reading in this thread is that a higher base frequency more important than multiple cores for this task. Have I got this right please? Any fresh input able to confirm or deny?

My choices:
A) i7-8700. 6 core, 12 thread, base frequency 3.20 GHz, turbo 4.60 GHz.
B) i7-8700K. 6 core, 12 thread, base frequency 3.70 GHz, turbo 4.70 GHz.
C) i7-7800X. 6 core, 12 thread, base frequency 3.50 GHz, turbo 4.00 GHz.
D) i7-7820X. 8 core, 16 thread, base frequency 3.60 GHz, turbo 4.30 GHz.


I was looking at D) with my "must get an 8 core processor for DxO" mindset from what I had previously read, which actually still looks like an excellent all round option. However option B) would be the best performer for my one  image at a time exporting, although perhaps negligible difference.

Which would you guys chose for my requirement?

This will be coupled with 16GB RAM, an SSD for OS, 7200rpm data drive, and at least an Nvidia 1060 3GB (maybe a 6GB).



mavricioh

  • Jr. Member
  • **
  • Posts: 11
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #16 on: November 07, 2017, 03:28:07 pm »
Forum Member "wyliec2" has done a lot of investigation which I have found very useful and very helpful:

I assume this is the other thread you refer to:
http://forum.dxo.com/index.php/topic,12262.0.html


Based on his testing and my understanding of it, it has been my experience as well that a multicore CPU with a high clock speed has the most impact in terms of performance as opposed to just more cores at lower clocks.

Personally, I would go with the 8700K, especially because of your workflow description. A higher clock speed will have more effect on the "live" corrections and single export and more cores will affect the export time on multiple images. But hopefully wyliec2 will chime in with a better explanation.
ASUS M32CD4 | Intel i5-6400 | Intel HD 530 | 8GB RAM | 960GB SSD | 350W PSU

Bencsi

  • Hero Member
  • *****
  • Posts: 1603
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #17 on: November 07, 2017, 04:36:53 pm »
Hi GSB2,

Refer to your subject, I made a test with a D-800 RAW file ~43 MB size. The export formats made as follows:

8 cores with High NR       14 sec
8 cores with Prime NR     60 sec
4 cores with High NR       24 sec
4 cores with Prime NR   105 sec
2 cores with High NR       53 sec
2 cores with Prime NR   243 sec

It seems, the core number has a proportional rate to the export processing. If you would like to test by yourself:
Ctrl+Shit+Esc -> Processes Tab -> select DopCor.exe -> right mouse click on -> select Affinity -> change the ticked CPUs quantity.
 The CPU cores' number have priority to CPU clock speed, because increase the clock speed is more limited. The total number of rendering commands not depends on the core number, therefore to execute all command in the export processing cue is mainly depends on how many cores are able to share the task.

Let me comment your export processing practice. Is there any advantage to make the export immediately after the editing finished ? I think, the main advantage of the DxO application is the export job separation from the edit job. The editing need my stay at the desk, the export not at all. It is up to you, whether utilize such advantage but e.g. Photoshop in this respect is far behind. The Local adjustment use the full rendering for edit, it makes the job far more slow. In the other tools the changes are more quick, because DxO working on the preview only, not the full resolution image. That is the advantage of DxO to some other competitive app.

Endre

Win7/64 PC, i7-3770, 3.9GHz, 24G RAM, Intel HD-4000 GPU, 27" calibrated LG monitor 1920*1080 px res. 82 DPI

GSB2

  • Jr. Member
  • **
  • Posts: 14
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #18 on: November 07, 2017, 05:03:36 pm »
Hi GSB2,

Refer to your subject, I made a test with a D-800 RAW file ~43 MB size. The export formats made as follows:

8 cores with High NR       14 sec
8 cores with Prime NR     60 sec
4 cores with High NR       24 sec
4 cores with Prime NR   105 sec
2 cores with High NR       53 sec
2 cores with Prime NR   243 sec

It seems, the core number has a proportional rate to the export processing. If you would like to test by yourself:
Ctrl+Shit+Esc -> Processes Tab -> select DopCor.exe -> right mouse click on -> select Affinity -> change the ticked CPUs quantity.
 The CPU cores' number have priority to CPU clock speed, because increase the clock speed is more limited. The total number of rendering commands not depends on the core number, therefore to execute all command in the export processing cue is mainly depends on how many cores are able to share the task.

Let me comment your export processing practice. Is there any advantage to make the export immediately after the editing finished ? I think, the main advantage of the DxO application is the export job separation from the edit job. The editing need my stay at the desk, the export not at all. It is up to you, whether utilize such advantage but e.g. Photoshop in this respect is far behind. The Local adjustment use the full rendering for edit, it makes the job far more slow. In the other tools the changes are more quick, because DxO working on the preview only, not the full resolution image. That is the advantage of DxO to some other competitive app.

Endre

Point taken on the work flow. It's preference rather than necessity.

Thanks also for the test. So certainly I should I think expect the i7-7820 8 core 3.6Ghz to perform better than the i7-8700K 6 core 3.7Ghz.

Mike1

  • Jr. Member
  • **
  • Posts: 70
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #19 on: November 07, 2017, 05:26:46 pm »
While its top clock speed lags a little compared to Intel ('only' 4GHz under boost, and 4.2GHz for 4 cores under XFR), it may also be worth considering AMD's 16-core 32-thread Threadripper 1950X, which has some impressive reviews.

Beware thought that you'd need to install the unofficial Zeffy patch to stop Microsoft blocking updates on anything lower than Windows 10 (https://www.extremetech.com/computing/248069-unofficial-patch-unblocks-windows-7-8-1-updates-kaby-lake-ryzen)

GSB2

  • Jr. Member
  • **
  • Posts: 14
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #20 on: November 08, 2017, 02:02:32 pm »
Hi GSB2,

Refer to your subject, I made a test with a D-800 RAW file ~43 MB size. The export formats made as follows:

8 cores with High NR       14 sec
8 cores with Prime NR     60 sec
4 cores with High NR       24 sec
4 cores with Prime NR   105 sec
2 cores with High NR       53 sec
2 cores with Prime NR   243 sec

It seems, the core number has a proportional rate to the export processing. If you would like to test by yourself:
Ctrl+Shit+Esc -> Processes Tab -> select DopCor.exe -> right mouse click on -> select Affinity -> change the ticked CPUs quantity.
 The CPU cores' number have priority to CPU clock speed, because increase the clock speed is more limited. The total number of rendering commands not depends on the core number, therefore to execute all command in the export processing cue is mainly depends on how many cores are able to share the task.

Let me comment your export processing practice. Is there any advantage to make the export immediately after the editing finished ? I think, the main advantage of the DxO application is the export job separation from the edit job. The editing need my stay at the desk, the export not at all. It is up to you, whether utilize such advantage but e.g. Photoshop in this respect is far behind. The Local adjustment use the full rendering for edit, it makes the job far more slow. In the other tools the changes are more quick, because DxO working on the preview only, not the full resolution image. That is the advantage of DxO to some other competitive app.

Endre

Point taken on the work flow. It's preference rather than necessity.

Thanks also for the test. So certainly I should I think expect the i7-7820 8 core 3.6Ghz to perform better than the i7-8700K 6 core 3.7Ghz.

Any agreement or disagreement with this?

I am very interested if anyone with an i7 close to either of these two processors two can quote a export time for a circa 20MP raw image to 100% jpeg with Prime NR please.

wyliec2

  • Jr. Member
  • **
  • Posts: 52
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #21 on: November 10, 2017, 05:44:36 am »
My normal processing involves 24MP RAW images into 90% JPEG with PRIME NR.

You pose an interesting question on single image processing.  I have benchmarked on I7 4770K and 5960X CPUs overclocked to 4.3 GHz on all cores (8 - 16 threads available).

For processing multiple images with a concurrency setting of 1 - which I think would be essentially equivalent to exporting images singly, I have found DxO to use a maximum of 8 threads - so the 4770K runs at 100% use of 4 cores/8 threads and the 5960X runs at 50% so only using half of the 8 cores/16 threads available.

I will try running some tests this weekend.  Something I have found with PRIME is that the darker the image and/or higher the ISO, the longer the export time since there is more noise to deal with.  I took several pictures of the same image with varying amounts of light and different ISOs to come to that conclusion.
Win 10/64, I7-5960x 4.3 GHz,  32Gb 2666 RAM, Sapphire R9-390 GPU, Dell 25" 2560x1440 monitor 117DPI, m.2 NVMe OS

Bencsi

  • Hero Member
  • *****
  • Posts: 1603
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #22 on: November 10, 2017, 08:40:13 am »
Refer to the ISO value vs. file size it is true, the same image with higher ISO has generally a bigger file than lower ISO. In my practice Canon EOS-5D II 5616x3744  (~21 Mpx) file sizes are as follows:
ISO   125      21.5 MB
ISO 1000      22.9 MB
ISO 3200      26.2 MB
Of course, the images are not perfectly identical but similar, having a ~30% black part. The images made is series  with 2-3 second time difference. Certainly the black part represents the most noisy rate.
On Dpreview site the published test image comparison also confirm my results. Here is the EOS-5D II same RAW test image sizes: ( https://www.dpreview.com/reviews/canon-eos-5d-mark-iv/9 and select EOS-5 II - or any other - camera files from the list )
ISO     100   27.3 MB
ISO     400   28.1 MB
ISO   1600   29.3 MB
ISO   6400   32.7 MB
ISO 25200   39.3 MB
Generally the more noisy images RAW files are bigger. The PRIME noise reduction process certainly need more correction steps in the cue, therefore its export processing take more time significantly on higher ISO images.

Endre
« Last Edit: November 10, 2017, 11:47:32 am by Bencsi »
Win7/64 PC, i7-3770, 3.9GHz, 24G RAM, Intel HD-4000 GPU, 27" calibrated LG monitor 1920*1080 px res. 82 DPI

GSB2

  • Jr. Member
  • **
  • Posts: 14
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #23 on: November 10, 2017, 09:23:48 am »
Thanks both.

My images generally not high ISO, but I shoot with a 1" sensor so I suppose I am using Prime NR to compensate for that light performance compromise (even though I shoot in good light typically). Testing with a low noise image is fine for my purposes.

I have two build options priced up, the 8 core just a few hundred pounds more than the 6.

Bencsi's test is steering me toward the 8 core, and I have a feeling the X299 series better for future proofing.

wyliec2 if you can share your test results that would be appreciated.

wyliec2

  • Jr. Member
  • **
  • Posts: 52
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #24 on: November 11, 2017, 07:41:43 pm »
Thanks both.

My images generally not high ISO, but I shoot with a 1" sensor so I suppose I am using Prime NR to compensate for that light performance compromise (even though I shoot in good light typically). Testing with a low noise image is fine for my purposes.

I have two build options priced up, the 8 core just a few hundred pounds more than the 6.

Bencsi's test is steering me toward the 8 core, and I have a feeling the X299 series better for future proofing.

wyliec2 if you can share your test results that would be appreciated.

Thanks for prompting an interesting question - this was a benchmarking exercise I hadn't pursued - disabling cores and/or hyperthreading to evaluate performance impact.

Baseline info:
- 24 MP RAW images (file size 24,608 KB)
- 3 copies of the same image - ISO 200 with moderate customization in DxO
- 5960X CPU @ 4.2 GHz clock; 32 Gb 2666Mhz RAM; normal configuration 8 cores/16 threads
- total of 44 timing tests at different combinations of active cores/threads
- JPEG 100%
- HIGH NR

NOTE: The first export after launching DxO has a delay that subsequent images do not have.  I presume there is some overhead in getting code into memory and preparing the threads for processing – this overhead adds 80%-100% additional time to the first image.  GSB2 asked specifically about exporting single images – if the use case involves starting DxO, exporting one image and closing DxO each time, the 1st image time will always apply.  If the use case is to open DxO and keep it open, editing and exporting images singly, the second and all subsequent images will process much faster.

8 cores/16 threads
-   12.2 sec First Image
-   6.0 sec Subsequent Images
-   50% Processor Usage

8 cores/8 threads
-   11.8 sec First Image
-   5.9 sec Subsequent Images
-   80% Processor Usage

6 cores/12 threads
-   12.4 sec First Image
-   6.4 sec Subsequent Images
-   70% Processor Usage

6 cores/6 threads
-   12.0 sec First Image
-   6.4 sec Subsequent Images
-   100% Processor Usage

4 cores/8 threads
-   14.0 sec First Image
-   6.9 sec Subsequent Images
-   90% Processor Usage

4 cores/4 threads
-   13.7 sec First Image
-   8.0 sec Subsequent Images
-   100% Processor Usage

3 cores/6 threads
-   15.3 sec First Image
-   8.1 sec Subsequent Images
-   100% Processor Usage

Given how hyperthreading works, DxO sees the following combinations as equal:
-   8 cores/8 threads (hyperthreading disabled; 4 cores/8 threads (hyperthreading enabled)
-   6 cores/6 threads (hyperthreading disabled; 3 cores/6 threads (hyperthreading enabled)
DxO assigns work to threads, however, even though the above combinations appear equal to DxO, in actuality, 8 threads on 8 cores is more powerful than 8 threads on 4 cores.

Looking at the test results, I infer that DxO is most efficient using 6-8 threads per image (concurrent image testing I have done suggests this as well).  In past testing, DxO seems to ‘pick’ threads and stick with them – I’m not sure what happens when DxO use 8 threads on an 8 core machine with 16 threads – if it were to randomly assign it might assign 2 threads to a single core when it would be more efficient to use one thread on each of the 8 cores.  This might explain the slight advantage of 8C/8T over 8C/16T – the difference was slight but it was consistent.

If the use case was edit/export 1 image at a time, arguably one could choose an I5-8600K 6C/6T CPU at 3.6 GHz and not lose much in comparison to higher core/thread CPUs.  I would presume that someone editing in this manner is not doing much multitasking.  I often export in batches of 20-50 images and will navigate to websites, Word, Excel, etc. while the export is running – in this case, hyperthreading and more cores are useful.
« Last Edit: November 12, 2017, 10:33:03 pm by wyliec2 »
Win 10/64, I7-5960x 4.3 GHz,  32Gb 2666 RAM, Sapphire R9-390 GPU, Dell 25" 2560x1440 monitor 117DPI, m.2 NVMe OS

GSB2

  • Jr. Member
  • **
  • Posts: 14
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #25 on: November 11, 2017, 11:35:38 pm »
Thanks for taking the time to test and share this. The above tests are with or without Prime NR please?

wyliec2

  • Jr. Member
  • **
  • Posts: 52
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #26 on: November 12, 2017, 10:32:13 pm »
Thanks for taking the time to test and share this. The above tests are with or without Prime NR please?

Well, I completely blew that benchmark - I used images I had not processed and did not select PRIME NR....(I always use PRIME for pics I finalize)....    :-[  :-[

So NEW numbers with PRIME (JPEG 100%) are:

8 cores/16 threads
-   31.5 sec First Image
-   24.1 sec Subsequent Images
-   53% Processor Usage

8 cores/8 threads
-   31.5 sec First Image
-   24.0 sec Subsequent Images
-   100% Processor Usage

6 cores/12 threads
-   36.6 sec First Image
-   29.7 sec Subsequent Images
-   70% Processor Usage

6 cores/6 threads
-   41.4 sec First Image
-   31.1 sec Subsequent Images
-   100% Processor Usage

4 cores/8 threads
-   48.5 sec First Image
-   39.3 sec Subsequent Images
-   100% Processor Usage

4 cores/4 threads
-   49.6 sec First Image
-   42 sec Subsequent Images
-   100% Processor Usage

3 cores/6 threads
-   62.8 sec First Image
-   50.6 sec Subsequent Images
-   100% Processor Usage

DxO maxes out on processing at 8 threads/image and 8 cores.  There is no difference between 8 cores/8 threads and 8 cores/16 threads.
Win 10/64, I7-5960x 4.3 GHz,  32Gb 2666 RAM, Sapphire R9-390 GPU, Dell 25" 2560x1440 monitor 117DPI, m.2 NVMe OS

GSB2

  • Jr. Member
  • **
  • Posts: 14
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #27 on: November 13, 2017, 07:04:52 am »
Thanks again wyliec2, those results with Prime NR are very useful. These correlate to Bencsi's results as well, but the greater detail and image size closer to what I am working with helping a lot.

The results are steering me to my 8 core choice. Some assumption but I hope that given the CPU only 53% utilised using all 16 threads I will see turbo side clock frequencies.

wyliec2

  • Jr. Member
  • **
  • Posts: 52
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #28 on: November 13, 2017, 01:18:21 pm »
GSB2 - are you looking at I7 processors (ie. I7-7820X) or Xeon??

Are you getting a pre-built system or building yourself?

Given the CPU intensive nature of PRIME, I suspect there will be minimal turbo clocking.

If building, it's normally relatively easy to achieve turbo clock frequency on all cores as a base overclock. By this, I mean with simple air cooling and minimal voltage change.
Win 10/64, I7-5960x 4.3 GHz,  32Gb 2666 RAM, Sapphire R9-390 GPU, Dell 25" 2560x1440 monitor 117DPI, m.2 NVMe OS

GSB2

  • Jr. Member
  • **
  • Posts: 14
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #29 on: November 13, 2017, 05:26:15 pm »
GSB2 - are you looking at I7 processors (ie. I7-7820X) or Xeon??

Are you getting a pre-built system or building yourself?

Given the CPU intensive nature of PRIME, I suspect there will be minimal turbo clocking.

If building, it's normally relatively easy to achieve turbo clock frequency on all cores as a base overclock. By this, I mean with simple air cooling and minimal voltage change.

The i7-7820X is what I am pretty much settled on. This will be built by a reputable custom build company, that will over clock of I wish (closed loop cooler in the spec).

 

photography
photography