I don’t understand what I’m supposed to do if I want to do large rearrangements of my folders in PhotoLab, in order to prevent “lost” or “missing” images in my search results.
When I installed and started using PL, I had my entire collection of photos in a folder called “Image Folders” on my iCloud drive, but then later on I decided I do not want it there anymore, so I moved the top-most containing folder somewhere else.
No problem (at first, it seemed) as DxO is supposed to be catalog free and so “sees” changes made in the Finder automatically.
Mostly, everything is fine, but I now have an issue when searching keywords where thousands of missing photos show as thumbnails with a question mark. And these show first, and since PL can only show 1000 images at a time, I literally cannot see any photos at all when I use more broad searches like “RAW” or searches for a focal length.
So far, after an hour of searching, the only “solution” I can find is to delete my database entirely and start over again. I’m pretty sure this gets rid of all my Projects as well, and probably other unfavorable consequences?
It seems like it is not extremely uncommon that one might migrate their entire photo library to another location, drive, etc. What is the common solution in the face of this? I cannot move folders within PL, so it seems to me so far that there is 100% no way to do what I want without screwing up the database forevermore until I delete it and start over.
As you describe, your rearranging of folder locations has lead to duplicate entries in PhotoLab’s database. PhotoLab does not track folders, specially when it’s not running…and now, the mess is already done.
There is no way to fix this with something that PhotoLab can do, but the situation is not hopeless. As you say, deleting the database and re-indexing the photo archive will again establish the order. Depending on how PhotoLab was set, re-indexing can be almost lossless, or, in the worst case, lose all the metadata you added like e.g. keywords as well as the customising of the images.
If you have left DPL’s default settings, all image files you edited have the settings stored in them or will be accompanied by a settings sidecar file that ends with .dop. The sidecar file brings back your metadata and image edits upon re-indexing and the same goes for .xmp sidecars.
Now, before you rush deleting the database and re.indexing, make sure that your new photo archive is organised properly so that you don’t have to cycle through the fixing process again and again.
Note that every time, you rearrange folder and file names and locations, DPL’s database will get off-track again.
Seems silly to me, I wish DxO would spend even a tiny bit more effort on making the photo management part of PL at least a little better.
I’m worried that down the line I will likely want to move the location of my photos again, if I get a NAS. I guess I’ll have to delete my database all over again when I do?
Doesn’t seem very nice of DxO to build it this way.
Anyway, you are helpful with your answers, so thank you!
Do I delete all three of these files to “reset” the DB?
Or do I only need to delete the first here?
On what is tat number stated? I just made some timelaps and edited some 8000 images in one directory. It just takes a lot of time.
PL identifies an image with path/image. So when you copy the image outside PL to a new directory you’re just creating a new virgin image.
I just tried . You can select individual images or a group of images and copy them to another directory shown on your file browser. However they are copied and not moved. I don’t think that’s what you want.
Delete the database directory, so everything.
Not all is lost. Once you have indexed your photos and are happy with them, then in future if you want to move or rename photos or folders do it inside PL, in other words use PL as your file manager. This way PL will know what you have done and will update the database accordingly.
That will also delete the other folders that contain presets. You only need to delete the main database file plus the .shm (shared memory) and .wal (write ahead log) temporary files.
In principle, as long as you don’t use projects or edit history, you can manage perfectly well with the database, which seems to be for caching search results, etc.
As long as you ensure to write to the DOP files for editing and the XMP files for metadata, you should have everything you need to move the image plus those two files to any location without a problem.
But you will need to reindex the first time you access a selected folder.
There is no database directory in macOS. @Joanna is right, I think if I deleted all these files/folders in this directory I would clear those away too. But I deleted the three files I had selected above and that worked for what I was trying to accomplish of clearing the database.
Lol, I can’t reply more than once on my “first day”?! Have to edit this to add my reply to @KeithRJ :
Hmm, I’m not sure about your copy of PL, but in mine I’m unable to move folders using PL’s file manager. So if, in the future, I have many folders/subfolders, then the only way to do what you are saying is to recreate the entire folder structure on the new drive within PL’s file browser, and then manually move all the images from each folder, one folder at a time, from within PL’s file browser.
That’s maybe okay if there are not too many folders, but it is a bit crazy if I start to have scores of project folders and/or sub folders.
There are many differences between DPL on Mac and Win. DPL for Windows can change folder names, DPL for Mac can’t.
Renaming image files (including the sidecars) can be done though: command-Enter will produce the respective popup.
As @Joanna wrote, the database can be trashed at any time, provided you … (see Joanna’s list) and re-indexing can be run through the night, no need to watch it.
Re-indexing the whole lot is only necessary if you want to find images based on filename, keywords or metadata etc., but DPL is, imo, not a reliable asset manager due to its lack of DB maintenance functionality. Many of us therefore use something else for that. I use Lightroom Classic. Others use other apps.
Indexing time depends on several things. On my Macs, Indexing a pool of about 30k image files (most of the RAWs) is done within 30 minutes. When sidecars are present, indexing can take longer.
That seems to be an indexing limit. No practical use if so many images are selected on your criteria. I see now that’s what you meant.
But for editing I have not found any limitations yet.
Curious this limitation, I have just done a search on a very common keyword for me and the result only shows me 1000 images and tells me “+ display the 7850 images”.
And if I click on it, it shows me all of the images, the 7850 images.
After spending years dealing with a growing library and periodic needs to reorg its structure, I’ve found the best way to move folders is to create the new folder outside PL (using Finder, for example), and then in PL, grab all the files you want to move and haul them over.
Anything moved without the direct knowledge and supervision of PL will not move properly. Instead, you will see duplicate entries and orphaned entries. The state of projects and keywords is unpredictable. If your edits are recorded in sidecars, those will remain intact.
If you move inside of PL, the software properly handles the changed location in the DB.
Certainly an easy to do way for a move here and there.
Probably too cumbersome for moving a load of folders and files though. My photo archive has 368 folders containing 29’130 files and re-indexing is a lot less effort.
As a general rule, it pays to think first and then find a structure - and stick to it. Adding metadata to file names should be avoided imo.
There’s another way to avoid issues/corruptions/hassles with the database …
IF you don’t plan to use Projects (the ability to create logical groupings of images) and you don’t care about the ability to search images in the database by Keyword(s) … then you can delete the database - and rely only on the sidecar/.dop files associated with each source image to retain and apply corrections to the image;
I work this way by running PL via a wrapper that deletes the database (and the cache) before invoking each new PL session.
PL will then recreate a new database just for the current session (before it’s again deleted the next time that PL is run).
Provided one does not need projects and keyword searching, this works very well.
Thanks everyone. I want to give this single reply to everybody just in case I still can’t reply more than once.
As was pointed out, I do care about all the features that keeping the DB provides - searching, projects, etc.
My main concern right now is that I know sometime in the near future I’m probably going to invest in a NAS or home server. I will want to offload my entire photo collection from my internal drive, or more likely just part of it, onto that.
I’m coming to PL from Lightroom Classic, and what I’ve been used to is having all my photos organized into this structure: [Main Folder] > [YYYY] > [YYYY-MM-DD]
However, I realized that I have no practical reason to keep it this way (because I never go looking for a particular date using the folders), and given the idiosyncrasies of PL’s file/folder management I’ve decided my structure will simply be [Main Folder] > [YYYY] and leave it at that. Then I will use PL’s projects - or perhaps an album/collection feature of some other DAM that doesn’t touch my folder structure, to further organize.
This way, when it comes time to move photos off my drive and onto a NAS or similar, I don’t have to go through endless folders - I can just go into the limited [YYYY] folders and move them in big chunks.
Thank you everyone for your input and insight! It did help me figure out what to do. And though I’m still flabbergasted at DxO’s choices and/or lack of effort when it comes to the “features” of the Photo Library, and hope they make improvements in the near future, I’m at least a bit less frustrated and think I have a handle on it now.
Folder structures don’t matter that much, if your application(s) can maintain a healthy catalog and provide useful features for finding images with keywords or other metadata.
Lightroom Classic does a pretty good job (although not perfect) with that while PhotoLab has the issues mentioned in more than this thread.
I advise you don’t set up a structure that contains folders with several thousands of files, DPL can get reaaalllllyyyyyyyy slow under such conditions. While you could do this in Lightroom, you’ll probably find that you need finer granularity to work around PhotoLab’s properties, quirks, shortcomings, issues and whatnot.
If possible, test your intended structure by creating the yearly folders on an external SSD and copy the respective files into those folders. Then check out how DPL handles the load.
On my Mac, I’ve moved my photo structure to a separate volume on the internal SSD.
Right, I understand this. The structure to me matters only insofar as performance (which you mention it is not good to have too many photos in one directory) and for future portability without damaging the database’s linking.
Perhaps I can split my [YYYY] folders into “chunks” of 1000, or something like that.
I don’t want to have individual dates or too many other project-named folders because I know myself, and I am not good at sticking to anything in particular. So I’m 100% sure that some time down the road I will be porting my entire library of folders onto some other drive or location; it’s just going to happen .
So – I’m trying to make that process as pain-free as possible by reducing the number of folders.
Interesting idea with volumes. I only have a 500gb SSD as of now, so I don’t know that creating extra volumes is a great idea for me right now - maybe in the future when I get that NAS I’m talking about. Which again, is the reason for wanting to simplify my folder structure as much as possible now so that I can make the file migration later within DxO PL with the least headaches.
.