Author Topic: Number of CPU Core Utilization for DxO Optics Pro 11  (Read 855 times)

traderjay

  • Jr. Member
  • **
  • Posts: 6
Number of CPU Core Utilization for DxO Optics Pro 11
« on: July 31, 2017, 10:52:51 pm »
I've been testing DxO Optics Pro 11 on my XEON workstation which has the E5 2696v3 18 Core CPU. When exporting images it seems only less than 8 physical cores are used and I am wondering if this is a software limitation?

sgospodarenko

  • DxO tester (Win version)
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 994
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #1 on: August 01, 2017, 08:34:11 am »
Hello traderjay,
You can set manually the number of cores (simultaneously processed images) in Preferences. I've attached a screenshot for you.

Regards,
Svetlana G.
Regards,
Svetlana G.

wyliec2

  • Jr. Member
  • **
  • Posts: 46
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #2 on: August 02, 2017, 01:35:11 pm »
There are several factors impacting Export processing, the two key items are:

1) Noise Reduction - PRIME is nearly 100% CPU based; HIGH much less so and will use GPU.
2) Concurrent Image setting - max setting will equal the number of threads supported - I have only been able to test this up to 16 concurrent images as my biggest PC has 8 cores/16 threads.

If the Concurrent Image setting continues to scale up, I would think you could set this to 36.  This likely would not be the most efficient depending on the number of images you normally export at once and your available memory.

http://forum.dxo.com/index.php/topic,12262.msg65108.html#msg65108
Win 10/64, I7-5960x 4.3 GHz,  32Gb 2666 RAM, Sapphire R9-390 GPU, Dell 25" 2560x1440 monitor 117DPI, m.2 NVMe OS

traderjay

  • Jr. Member
  • **
  • Posts: 6
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #3 on: August 02, 2017, 06:25:57 pm »
I tried changing the core count to 30 and the utilization is still stuck at 30%. So I guess for a single image processing there is a core cap limitation?

graphitepaddle

  • Jr. Member
  • **
  • Posts: 14
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #4 on: August 03, 2017, 01:05:43 am »
So up to 8 cores helps?  Trying to configure a new Window tower, and DxO Optics Pro will drive the hardware requirements:

  • Useful # of cores & CPU speed?
  • How much RAM?
  • Which NVIDIA graphics card?

It's a given that I'll need at least 1/2 G SSD.

Looking down the road at Nikon D850 FF body & how big its raw files will be ... can only speculate now.  Given that PRIME noise is taxing, but don't care because most of the time I shoot at 100 ISO.

wyliec2

  • Jr. Member
  • **
  • Posts: 46
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #5 on: August 03, 2017, 05:40:14 am »
So up to 8 cores helps?  Trying to configure a new Window tower, and DxO Optics Pro will drive the hardware requirements:

  • Useful # of cores & CPU speed?
  • How much RAM?
  • Which NVIDIA graphics card?

It's a given that I'll need at least 1/2 G SSD.

Looking down the road at Nikon D850 FF body & how big its raw files will be ... can only speculate now.  Given that PRIME noise is taxing, but don't care because most of the time I shoot at 100 ISO.

So D850 speculation is in the 45MB raw size - I take your comment to mean you will not use PRIME NR shooting at ISO100.  My experience with 24MB raw images is 2 - 4 concurrent images on an 8 core/16 thread CPU work best for me - I periodically export 30 - 50 images at a time and I want them done pretty quickly.  I barely use more than half of my 32 Gb RAM.  I use PRIME exclusively.

Unless you are a professional exporting several hundreds of images every day, IMHO a high-end XEON (14+ cores) is probably not called for.  I have been eyeing the new i9-7920X 12-core CPU coming soon - clock speed is a definite factor with most DxO export time.

I would think 32Gb RAM would work for 45MB RAW files with 2 - 4 concurrent image setting.

For GPU there is something I'm not 100% clear on - DxO favors OpenCL which is associated with AMD GPUs.  I'm not sure how this translates with Nvidia CUDA cores.  Non-PRIME (HIGH) NR export utilizes the GPU; Image rendering during the edit process also leverages OpenCL.

The posts in the link in my earlier reply have a lot of discussion about this topic...




Win 10/64, I7-5960x 4.3 GHz,  32Gb 2666 RAM, Sapphire R9-390 GPU, Dell 25" 2560x1440 monitor 117DPI, m.2 NVMe OS

traderjay

  • Jr. Member
  • **
  • Posts: 6
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #6 on: August 03, 2017, 04:50:43 pm »
Can the developers confirm on the number of cores DxO can utilize when doing a SINGLE image processing or export? This is important to me because I will spend the time to perform the adjustments and print them one by one so my workflow is serial in nature. The print part also require the processing effects to be applied but its taking quite long on my XEON 18 core machine. For some reason, DXO runs faster on my older six core 3970x machine with much faster clock speed. So I am suspecting Mhz matters more than core count?

Bencsi

  • Hero Member
  • *****
  • Posts: 1458
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #7 on: August 03, 2017, 09:15:44 pm »
Hi Guys,

 I'm also interesting the CPU thread number and clock speed influence on the export timing. Let us make a set of test on our PC. Please find an Excel sheet for testing notes. If you find some details are missing, please add. The sheet include now sample data only, if we can agree about the form, let us do it.

Endre
Win7/64 PC, i7-3770, 3.9GHz, 24G RAM, Intel HD-4000 GPU, 27" calibrated LG monitor 1920*1080 px res. 82 DPI

wyliec2

  • Jr. Member
  • **
  • Posts: 46
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #8 on: August 04, 2017, 04:37:12 am »
Can the developers confirm on the number of cores DxO can utilize when doing a SINGLE image processing or export? This is important to me because I will spend the time to perform the adjustments and print them one by one so my workflow is serial in nature. The print part also require the processing effects to be applied but its taking quite long on my XEON 18 core machine. For some reason, DXO runs faster on my older six core 3970x machine with much faster clock speed. So I am suspecting Mhz matters more than core count?

I am not a DxO developer but I have done significant testing on i7-4770k and i7-5960x CPUs and I have never found a case where it was faster to process 1 image at a time.  With PRIME NR a single image will use about 50% of the cpu (aggregate percentage from AIDA 64).  With HIGH NR a single image generates a peaky sawtooth graph and offloads some of the processing to the GPU.

Since I use 'k' and 'x' CPUs I have also benchmarked with CPU clock varying from 3.0 to 4.5 - given the CPU intensity and minimal benefit in scaling beyond 4 cores/8 threads for a single image, minimizing export of single images will entirely a factor of clock speed.  Total throughput over time will be the combined factor of cores + clock speed.  So given roughly equivalent IPC, you'll get about the same output processing two images concurrently on a 8 core 4 Ghz machine as processing four images concurrently on a 16 core 2 Ghz machine.

I believe processing a single image on an 8 core 4 Ghz machine will be twice as fast as on a 16 core 2 Ghz machine.  In pondering a high-end machine for DxO processing, the Xeon E5 2687 v4 with 12 cores @ 3.0 Ghz seemed optimal.  The new i9 Intel machines open that up with 12, 14, 16 and 18 core 'x' CPUs coming.

In any case, you are better off exporting batches of multiple images and controlling concurrency in preferences than simply exporting a single image at a time - the export process seems to have startup overhead in the first image.
Win 10/64, I7-5960x 4.3 GHz,  32Gb 2666 RAM, Sapphire R9-390 GPU, Dell 25" 2560x1440 monitor 117DPI, m.2 NVMe OS

graphitepaddle

  • Jr. Member
  • **
  • Posts: 14
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #9 on: August 04, 2017, 04:59:01 am »
Thanks Wyliec2,

Should have reviewed that initially.  Combine that info with the following being infrequent: a) Exporting over 8 images at once, and b) Using PRIME, getting the idea that w. budget limitations, preferences are:
  • Max out CPU clock speed
  • RAM: 16G okay, but 32G much better
  • SSD: More capacity is better,  to support the pagefile ... UNLESS RAM is plentiful
  • Lowest priority: More CPU cores; Quad Core probably okay, 6 cores better, and 8+ is a luxury.

Would be great to max out of those items, but in my case it won't happen.

Niels


wyliec2

  • Jr. Member
  • **
  • Posts: 46
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #10 on: August 05, 2017, 01:25:07 am »
Hi graphitepaddle,

In response, here are my suggestions:

1 - Max out CPU clock speed - YES!!

2 - RAM: 16G okay, but 32G much better - 16Gb is probably fine for 2x concurrent images even with 45 MB raw images.

3 - SSD: More capacity is better,  to support the pagefile ... UNLESS RAM is plentiful  - Unless you have a lot of other stuff running besides DxO, you shouldn't be swapping much.  That said, ever PC I have has the OS on SSD.  The only DxO use for SSD is for the Cache location (Preferences) where it builds the thumbnail filmstrip images.  I have found no measurable difference between source and output locations on hard drive or SSD - even NVMe SSD.  No if you were using USB drives maybe a different story - all may HDs are SATA 3.

4 - Lowest priority: More CPU cores; Quad Core probably okay, 6 cores better, and 8+ is a luxury. - If my target use was outputting small numbers of images as quickly as possible, I would target a 7700k and run 2x concurrent images.
« Last Edit: August 05, 2017, 01:33:33 am by wyliec2 »
Win 10/64, I7-5960x 4.3 GHz,  32Gb 2666 RAM, Sapphire R9-390 GPU, Dell 25" 2560x1440 monitor 117DPI, m.2 NVMe OS

graphitepaddle

  • Jr. Member
  • **
  • Posts: 14
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #11 on: August 05, 2017, 01:33:53 am »
Quote
Max out CPU clock speed
RAM: 16G okay, but 32G much better
SSD: More capacity is better,  to support the pagefile ... UNLESS RAM is plentiful
Lowest priority: More CPU cores; Quad Core probably okay, 6 cores better, and 8+ is a luxury.

Thanks Wyliec2, that's what I thought.  So here's another one: When comparing CPU's, which is more important: Base clock speed, or Turbo clock speed?  Put another way, how often will a raw conversion take advantage of the CPU's turbo speed?

wyliec2

  • Jr. Member
  • **
  • Posts: 46
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #12 on: August 05, 2017, 02:32:02 am »
I overclock so I run all cores at higher clock than Turbo would be - Turbo varies based on how many cores are being heavily used - the fewer cores in use, the higher the turbo will be.  I presume you are not overclocking but you might still consider a 'k/x' level CPU - they may come with a faster base clock than non-'k/x' CPUs.  If you're building a new platform, the x299 i7-7740x looks pretty interesting at 4.3 base and 4.5 turbo - high base and you don't lose much if many cores are in use.  I wouldn't think PRIME NR would turbo since it uses heavy CPU.  HIGH bounces around a lot as opposed to a steady, heavy CPU load.

If you primarily use HIGH, then a good GPU that is optimized for OpenCL will help - doing searches on this, Nvidia seems sketchy on OpenCL...  Testing with GPU + OpenCL with HIGH NR compared to No GPU/No OpenCL produces times that are nearly double with them disabled.

A good GPU will help with rendering high pixel images.  Not sure what monitor resolution you are using - I would never go back from the 1440p I'm using.  4k might be overkill but it's certainly not overly expensive any more, however, it will tax a lesser GPU.
Win 10/64, I7-5960x 4.3 GHz,  32Gb 2666 RAM, Sapphire R9-390 GPU, Dell 25" 2560x1440 monitor 117DPI, m.2 NVMe OS

graphitepaddle

  • Jr. Member
  • **
  • Posts: 14
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #13 on: August 05, 2017, 07:49:10 pm »
Thanks Wyliec2,

Have a billion color monitor (NEC PA272W), and had to upgrade my video card to get a Display Port.  On my quadcore 3GHz machine w. 8G of memory, going from a GeForce GTX560 to a GTX780, made a HUGE difference in raw conversion speed.  (Shame that I never benchmarked.  :()

Bottom Line: No proof if the turbo helps DxO raw conversion?  More to the point: Comparing 2 quadcore CPUs, which is the faster option for raw conversion --> a) Higher base clock speed, or b) Higher turbo speed?

Appreciate that someone is looking at the speed issue empirically.   :)

traderjay

  • Jr. Member
  • **
  • Posts: 6
Re: Number of CPU Core Utilization for DxO Optics Pro 11
« Reply #14 on: August 07, 2017, 05:13:47 pm »


I believe processing a single image on an 8 core 4 Ghz machine will be twice as fast as on a 16 core 2 Ghz machine.  In pondering a high-end machine for DxO processing, the Xeon E5 2687 v4 with 12 cores @ 3.0 Ghz seemed optimal.  The new i9 Intel machines open that up with 12, 14, 16 and 18 core 'x' CPUs coming.


This is what I am experiencing as well. My E5 2696V3 CPU (18 core) doesn't turbo very high due to the core count and export has been quite slow. My other PC which uses an older 6 core 3970x CPU that turbos to 4 GHz performs faster when doing single image processing and print.

I use DXO primarily to correct my images to print and seldomly needs to export the images. But this still require the image to be processed prior to sending to the printer and lack of multi-core optimization for a single image processing is slowing down my workflow quite a bit :(

 

photography
photography