Most of my images aren't worth a 15-minute loading time

[Stenis (Sten-Åke Sändh (Sony, Win 11, PL 6, CO 16, PM Plus 6, XnView))]

In iMatch: Activate the “Favourite Panel”. Then Drag a shortcut from windows for"Photolab.exe" to that windows and configure it.

Then configure it with a Short Cut so you can activate the transfer to Photolab quickly

In Photolab: You shall look for External Searches in the left panel when PictureLibrary is active. Try to open any number of files with iMatch or install the free File Viewer XnViewafter configuring it and configure it too to open Photoplab 9. Switching with 100-300 files is instant. It is what I use to use normally.

If you want to go back to an earlier search it is just to click on that in the list below.

As already said External Searches uses exactly the same tabels in PhotoLibrary as the Projects but they are configured slightly differently.

Example of “Project”-table: The red rows are part of a Project all the rest are entries fron External Searches

[Stenis (Sten-Åke Sändh (Sony, Win 11, PL 6, CO 16, PM Plus 6, XnView))]

Wrong of me - shall of course be PhotoMechanic.l

[Stenis (Sten-Åke Sändh (Sony, Win 11, PL 6, CO 16, PM Plus 6, XnView))]

Yes projects works fine too. No problem.
I just use to avoid them since Projects means administration and External Searches does not and tends to befar more effective to use.

The only administration you need to do with External Searches is to delete the ones you don´t need anymore.

Tips for an effectice workflow with iMatch:

- Both Photolab and the external DAM or Viewer you use shall be open in parallell
- Avoid having more windows open than these two if you want to be max effective
- Use the old Windows command Alt+Tab to switch between those two windows.
- Add Photolab to iMatch as a Favourite and configure a short cut too

In this example I have selected some pictures in iMatch.
Please note: I have RAW in main folder and JPEG-derivates in a subfolder and iMatch can display RAW+JPEG side by side.

I then press my iMatch shortcut F3 TWO TIMES quickly and then follow quickly with Alt+Tab too, in order to switch to Photolab

… and if you look carefully now you will be able to se both the RAW and the JPEG in pair side by side, something Photolab normally is unable to do and some users have asked DXO to add as a new feature by a feature request.

If you want to you can recall any earlier External Selections any time just by clicking that historical entry in External Searches Here I have switched to 405 pictures organised side by side RAW+JPEG with just one click and they hget displayed instantly witout any delay at all.

I don´t think you can match this iMatch/Photolab-workflow in any way in Capture One or Lightroom when it comes to speed, the picture quality or the efficiency when it comes to XMP-metadata management. And if you do it this way even the culling can get very effective even within Photolab, since we now kan handle both the RAW and the JPEG in parallell. I normally have my camera configured to take RAW+JPEG

External Searches is sadly overlooked and never used by many users, but it is in fact a killer-feature that eliminates all the downsides with using high res previews even with instant Deep Prime-rendering active. It gives direct access in Photolab to folders and files in the filesystem without using any slow and inefficient import-processes like the ones in Capture One or Lightroom.

I use it all the time!

[zkarj (macOS, PLe+FP+VP, Pentax)]

You can pay for another tool to manage your images. I don’t think I’d do that unless it did a lot for me. Because so far I’ve found nothing (on Mac) that can keyword images like I can in Lightroom. Not even close.

And Lightroom… as purely a DAM… is not only cross platform but free.

The import process is ridiculously easy, and fast. Thereafter I can browse hundreds or thousands of images with ease.

[SchorschGaggo]

Thanks for the many contributions. However, I didn’t expect to have to justify the number of my images, their storage, or their use.
I upload 3,000 images in one step from a cheap 128GB uSD to the NAS. This isn’t a burden on the camera. It scrolls through the images without any noticeable delay.
My NAS stores around 800,000 RAW/JPG pairs. 400,000 images in 50 years – is that unusual? I also don’t scroll through them every 1 second for 9 days, but rather display them on multiple screens/photo frames at 10-second intervals. In 6 weeks, all the images will be gone through – on every screen. Every now and then I find a picture that I want to improve.

I’m not complaining about how quickly or slowly PL9 builds the image library. It can take as long as it takes. I don’t need this library. I use other programs for management and categorization.
But I complain that PL9 prevents editing the image until all images in the folder have been read and preprocessed. And if a camera/lens plugin is required, I have to wait again until it has been applied to all images.

Many posts recommend using external browsers. That’s exactly what I want to do. The first, and most widely available, is Windows Explorer. It lists 1,000 folder names in less than a second. It reads the directory of the image folder containing 3,000 images just as quickly. Then I open an image via the context menu for editing with PL9. I’ve already stopped waiting 15 minutes before I could actually work on the image. Of course, I’m not working with Explorer, but with Faststone and JRiver MediaCenter. But the effect when I open PL9 is the same.

Analysing it with Microsoft ProcMon shows that all image files in the folder are read before the selected image is opened for editing.

I’m now interested to know whether this is a bug that we can expect to be corrected soon, or whether this is in all seriousness the expected behavior.

In this thread, I read from users who share my experience and deny PL9 any suitability for larger image collections.

But there’s also @Lucabeer, who can work with a folder of 3,000 images without any noticeable lag. However, an SSD isn’t that much faster than a NAS to explain the difference of a few seconds compared to 15 minutes. This makes me hope that these are simply bugs. But if they’ve been known for years, how many more years will it take to fix them?
The bug theory is also supported by the fact that during the extremely long wait time, my CPU usage is < 2% and I get almost no LAN/disk activity.

I’ve submitted this, along with various logs, to support and hope someone responds.

@Stenis You write ‘No need either to create separate intermediate ad hoc folders for this processing.’ That sounds good. That seems to be what I need. But it takes minutes before I can operate anything in PL9. How can I work around this and switch to ‘External Searches’?

[Lucabeer (Luca Chichizola)]

@SchorschGaggo

But there’s also @Lucabeer, who can work with a folder of 3,000 images without any noticeable lag. However, an SSD isn’t that much faster than a NAS to explain the difference of a few seconds compared to 15 minutes.

Well, NAS are usually based on traditional mechanical hard disks. And usually they are slower (5400 RPM) mechanical hard disks.

The difference in speed between a 5400 hard disk and a SSD is STAGGERING even when both are installed locally (SATA). When we also add a LAN (no matter how fast) in between, it can only get worse. SATA is 6 GB/s, LAN is 1 GB/s maximum.

Please make at least the experiment of trying to open a HUGE folder of images from a local SSD, to see if it makes a difference.

[Fineus]

As much as that’s true, I find PL still slow to open, allow editing, and (since it likes to work this way) constantly generate previews of any given image on my local SSD.

[Joanna (Joanna Carter)]

@SchorschGaggo

My NAS stores around 800,000 RAW/JPG pairs

Patient : Doctor, doctor, it hurts when I hit my head on the door.
Doctor : Well stop hitting your head on the door.

Why oh why are you taking JPGs as well as RAW files? Do you not know that every RAW file contains a JPG? So you are effectively doubling the number of files that PL has to index.

Simple answer to halve the time taken is to stop taking, or at least, storing unnecessary JPG files. If you really need them, just export them from PL. if you really feel the need to keep all those OOC JPGs, at least put them in a separate sub-folder, so PL doesn’t see them, whilst editing the RAWs.

Then there’s the question of the NAS.

1. Does it contain HDDs or SSDs? The average SSD is 15-20 times as fast as an HDD.
2. How is the NAS connected? Cable, Wifi or optical?
3. How fast is the network card in your computer?
4. How many devices are accessing the NAS at any one time?
5. How many devices are accessing the network at the same time?

@SchorschGaggo

The first, and most widely available, is Windows Explorer. It lists 1,000 folder names in less than a second. It reads the directory of the image folder containing 3,000 images just as quickly.

But Windows Explorer is not trying to build an index of file metadata so that you can search on metadata. And it is not trying to show a rendering of the edited images.

[vainol]

@SchorschGaggo

But I complain that PL9 prevents editing the image until all images in the folder have been read and preprocessed.

RE: “But I complain that PL9 prevents editing the image until all images in the folder have been read and preprocessed.” You do not have to set PL9 to read any folder and process any folder at all (I could quite well disrecard my own folder Vestibule in that regard). When I set IMatch to open on image in an external application (that is PL9) then PL9 opens with only that one image, it does not read or process any folder, just one image.

[Stenis (Sten-Åke Sändh (Sony, Win 11, PL 6, CO 16, PM Plus 6, XnView))]

If just speed is your primary objective I suggest that you start using CameraBits PhotoMechanic and rigg the function “Open with…” to open Photolab 9. Nothing is faster than PhotoMechanic to create previews optimized for speed and make a new folder with pictures searchable and the reason for that is basically that a DAM like PM Plus do not need to compromise like a general purpose RAW-conveter that needs hig res previews too. You can avoid clumsy but competent “ingestion -functions” like Import (Lightroom) and Ingestion (PhotoMechanic) and use “Scan to Catalog” instead - it is the fastest. “Ingest” is more for automation of metadata flows with variables.

But, it is a fact that no professional DAM (for big organisations and enterprises) with lots of data will allow the use of named folders or folders with more than 1000 pictures plus or files of any type really for that matter, since they can handle all sorts of data. Their systems autogenerates new folders when they reach 1000 as a “general Best Practise” and that is a pure general file system thing. So this is not a recommendation for the use of a Photolab-workflow but a general recommendation. What I have written above in my example of a iMach/Photolab - is a suggestion how you dramatically can speed up a flow despite high res previews in PL. That does not change that a limit of 1000 is far better.

Always place your archive on an SSD and also check all your data disks for fragmentation too once in a while especially if you have a habit of deleting a lot of files too.

[Fineus]

When I set IMatch to open on image in an external application (that is PL9) then PL9 opens with only that one image, it does not read or process any folder, just one image.

Quick question on this:

It sounds great if you want to edit just one image, but what happens if you (for example) shoot 500 photos and want to look through them, editing somewhere between 5 and 10?

Do you have to close down PL and re-open it again for every individual image? (Sounds tedious.)

Does it (PL) just remain open and somehow pick up the images you pick in (presumably faster for DAM/browsing work) IMatch, as you pick them?

Do you still get a .DOP file reflecting all your edits, placed in the same folder as each image?

[George]

Just tried with IrfanView. It takes the usual time to open PL for the first time, then it stays open and you can select another image which comes direct.

George

[BHAYT (Bryan [WIN10, PL9, VP5, FP7, G9, 5060Ti(16GB), 3060(12GB)])]

@SchorschGaggo

You write ‘No need either to create separate intermediate ad hoc folders for this processing.’ That sounds good. That seems to be what I need. But it takes minutes before I can operate anything in PL9. How can I work around this and switch to ‘External Searches’?

@SchorschGaggo Firstly when you either navigate to a directory or use the ‘Load original image folder’ all images in a directory will be opened one by one and the image added to the database, so the location of that will have an impact on speed.

For a load test the database is on a normal SSD (not an NVME) in its default location and the images are on an USB3 attached SSD. Load time for 7,000 images from the SSD was a tortuous 3 mins 52.347 seconds.

The machine is a 5600G with a 3060(12GB) GPU.

Every time I move away from that directory and return PL9.1 will re-discover the images and that took 14.143 seconds no AI is involved in these timings and the images are .ORF .

@Joanna

Why oh why are you taking JPGs as well as RAW files? Do you not know that every RAW file contains a JPG? So you are effectively doubling the number of files that PL has to index.

@joanna It contains a a reduced JPG image, effectively a glorified thumbnail. I take both JPGs and RAWs and resent any form of chastisement from a RAW only photographer.

My images are segregated when imported so there is no doubling of any load time unless I choose to discover both directories.

@Fineus

Do you have to close down PL and re-open it again for every individual image? (Sounds tedious.)

@Fineus I included a snapshot of a number of external selections and stated that they are maintained in the database just like a ‘Project’, just not titled as such, but if you use the ‘Load original image folder’ that will cause the whole folder to be opened just as if you had selected the directory as normal.

which in the case of this image will re-discover the directory of 7,000 RAW (ORF) images.

[Fineus]

@BHAYT

I included a snapshot of a number of external selections and stated that they are maintained in the database just like a ‘Project’, just not titled as such, but if you use the ‘Load original image folder’ that will cause the whole folder to be opened just as if you had selected the directory as normal.

Got it I’m just trying it now:

• It’s taking the usual time to load PL first time.
• It’s much quicker in actually getting me to work on (an individual image)
• PL still slows to a snails-pace when introducing AI masks at all.
• Export time is 28 seconds for one image with XD/XD2s denoising and masking inc. AI masks.
• The exact same photo with masks disabled takes just 11 seconds.

So launching from an external DAM is a great tip and I’ll keep that in mind for actually getting to work on an individual image… if only there were a way to speed up the impact that masking has on editing/exporting.

It’s worth noting that launching another image to edit from the DAM (in my case Photo Mechanic) sees PL running relatively quickly again, up until masks are re-introduced.

It’s a shame I use masking quite a lot.

[BHAYT (Bryan [WIN10, PL9, VP5, FP7, G9, 5060Ti(16GB), 3060(12GB)])]

@Fineus

So launching from an external DAM is a great tip and I’ll keep that in mind for actually getting to work on an individual image…

@Fineus It is useful, the 7,000 image directory was created by me to test handling of large quantities of data, It was an 11,000 directory until this morning when I split the JPGs away from the ORFs I intend to put it on a larger SSD and have a 10,000 image directory of RAWs (ORF or RW2) and a 10,000 image directory of JPGs for stress testing.

Actually the stress comes from renumbering all the duplicate images so they go from 00001 to 10000.

FasRawViewer is not a DAM it is a useful RAW viewer and currently less than £18 (its on offer currently), IMatch is a full blown DAM and costs about £125. They have different purposes and I have licences for both, I just haven’t invested the time in mastering IMatch!?

@Fineus

It’s a shame I use masking quite a lot.

Sadly AI masking in particular is putting a real strain on a lot of users hardware (and patience)

[Fineus]

FWIW I’m still using Photo Mechanic as the trial is working and it does a good job, but I appreciate FasRawViewer isn’t a DAM!

I’m using (and can wholeheartedly recommend) IrfanView for viewing any image filetype on my PC from JPEG to .CR3 RAW files. It’s crazy fast for flicking through, but I find myself preferring to use Photo Mechanic for actually vetting images after a shoot (and I’ll give IMatch a try too).

[Joanna (Joanna Carter)]

@BHAYT

@joanna It contains a a reduced JPG image, effectively a glorified thumbnail

In my Nikon D850 files there are three embedded images…

• [EXIF] Jpg From Raw : (Binary data 7037847 bytes, use -b option to extract)
• [EXIF] Other Image : (Binary data 840553 bytes, use -b option to extract)
• [EXIF] Thumbnail TIFF : (Binary data 57816 bytes, use -b option to extract)

The Jpg From Raw is exactly the same pixel size as the containing RAW.

@BHAYT

I take both JPGs and RAWs and resent any form of chastisement from a RAW only photographer.

Then you have to ask yourself why you are filling up your memory cards and hard disks with, arguably, redundant files.

The only reason I could think of to justify such a practice is when you need to send copies by email as soon as they are taken.

@BHAYT

My images are segregated when imported so there is no doubling of any load time unless I choose to discover both directories

And this is one of the tips I mentioned.

[George]

Just be aware that the edits will not be visible in your browser. You’ll have to export the image first and go to that exported image in your browser.

George

[Stenis (Sten-Åke Sändh (Sony, Win 11, PL 6, CO 16, PM Plus 6, XnView))]

No, it is possible to open at least 400 pictures in my system regardless of external source used for the selection. Read what I and others have written and also shown with screen captures. How did you end up in that end?

[Stenis (Sten-Åke Sändh (Sony, Win 11, PL 6, CO 16, PM Plus 6, XnView))]

Or just pick it it in External Searches if it has been opened before.