New database - no keywords

DxO PhotoLab
1

I’ve just installed PL7 on my new laptop, pointing to the same image folders and database (on my NAS) as the copy on my old laptop. After a few uses it gave a message saying I appeared to be using an obsolete format of database and would I like to have it create a new one for me. I allowed it to, and now find all my keywords are gone. I put in many hours of work into keywording my photos.

My preferences / metadata has no tick for “Synchronise metadata wit XMP sidecar files” nor for “Write keyword hierarchies in the XMP dc:subject tag” as I didn’t understand the jargon and the manual was of no help so I left the default setting.

Judging from this thread Exact rules for the creation and use of sidecar files , I’m far from alone in not understanding the synchronisation process

The old format database appears to be still in the relevant folder on my NAS, renamed.

Anyone know what’s gone wrong, is it my fault and is there a way of recovering my keywords and their associations with my photos?

2

Start by making a copy of the old format database. Ideally, make it read-only. If you try anything, try it with a copy of that read-only copy.

You could make a support request to DxO. They shouldn’t throw away keywords when they update a database, so they clearly have a bug. If they fix the bug, then you could try the database upgrade again, this time using the fixed software.

Another thing you could try is to find one of the files whose keywords you know and look for those keywords in the DOP file (it’s just a text file). If so, you might be in luck. Delete the new database, find the top level of the folder that contains your images, right click on it in the Folder pane in PL7 and select Index a folder. There’s a chance that this will recover your keywords. I haven’t tried this, though.

Someone else may have a better idea. Good luck!

1 Like
3

Just realised that indexing might work. I don’t really know what this does nor why it sometimes seems to index of its own accord but perhaps worth a try?

4

Isn’t that what I said in my reply to your original message?

Indexing is what creates and adds to the database.

When PL encounters any image file for the first time (i.e. if the file is not already in the database), it adds it to the database. When you enter a folder (using the Folder pane) for the first time, every image file gets indexed. When you select “Index a folder”, I believe that PL will index every image in the folder and any sub-folder.

If all your images are stored within some root folder, then deleting the database and indexing that folder should re-create the database using information from the image files and any associated XMP and DOP files.

Your keywords are likely not in the original image files. If you didn’t ask for XMP files to be created, they won’t be there either. But I think that PL always writes keywords to DOP files. There’s a chance that re-indexing will restore those keywords despite their absence elsewhere.

The way to check is to look in your DOP files to see if there are any keywords. If you can’t find any keywords in the any of the DOP files, then there’s no point re-indexing.

Keywords should appear near the beginning of the DOP file. I added a top-level keyword called “Test keyword” to an image and it shows up in the DOP file as:

Keywords = {
{
“Test keyword”,
}
,
}

Checking the DOP files will be easier if you remember, for some images, what keywords you assigned to them. You don’t have to check every DOP file, of course–just enough to verify that the keywords were saved.

5

Hi again. Yes, thanks , it was one of your suggestions. I tried a few DOPs in WordPad (never opened one before except in PL) and didn’t find the keywords, but now I’ve actually indexed and found some I know to have keywords, I see they are there.

Still mystified by the documentation which suggests indexing automatically happens as soon as you visit a folder, but - for some reason - they provide a tool to do it manually. I’m pretty sure it doesn’t always happen as soon as you visit a folder. Also wondering whether indexing subfolders would obviate the need to index the whole top level folder which is huge. My top level folder structure looks as below, and if I deliberately index the top level it takes ages (and ends by crashing).

image

6

The new database does seem more compact than the old one, though I don’t know for sure what the different files and extensions in this folder are, and which one or ones represent the current “database” …

image
image837×242 7.52 KB

7

If you have put many hours into adding keywords to your pictures they are there in your picturefiles and RAW-sisecarfiles.

The simplest way to get them into your database is to index all your picture-folders. It is easiest if you make a topfolder, if you do not have one before and put all your picture folders in that if you haven´t done so before.

If you already have a topfolder and all your picture-folder in that you just need to select the topfolder and then right click it and select “Index a folder”. If you proceed Photolab will index all your picture-folder. You can also index a special folder that of some reson needs a resynch.

Since Photolab totally lacks a possibility to import or export your thesaurus/vocabulary that is really the only choise you have.

image
image583×1131 49.5 KB

1 Like
8

It should happen as soon as PL needs to interact with an image file. When you select a folder in the Folder pane, PL starts displaying images in the image browser. As it prepares to display an image, it has to first look for it in its database. If it’s not there, it adds it. It will read metadata from the image. If it finds an associated XMP file (for RAW images), it uses the metadata from that. If it finds an associated DOP file, it uses that data as well.

So when you visit a folder, it is indexed. Sub-folders are not indexed unless you visit them as well. If you manually request indexing on a folder, it and its sub-folders are indexed.

Before keywords and metadata searches, you could live entirely without the database. PL would still index image data (place it in the database), but you could delete the database on a regular basis without issues. The keyword structure and metadata searches require knowing something about more than just the images you are currently looking at. If you were to delete the database, you would not be able to search for metadata across all images, nor would you be able to view your complete keyword structure.

If you delete the database and index from the top level, PL will know about all the images within that top-level folder. If you index other folders, it will only know about those. However, since info is accumulated with each indexing operation, you could index each sub-folder independently and it would be the same as indexing just the top-level. It requires more manual work and it takes as long or longer, but it might not crash on you.

I suspect the crashing is occurring because of lack of memory, although it could be something else. If it’s a memory problem, indexing the sub-folders might squeak you by.

Photolab.db should be your main database. Photolab.db-shm and Photolab.db-wal are auxiliary files. The three files should be considered a unit. I suspect that these can be deleted (not while PL is running!) and will then be re-generated the next time the database is accessed, but I won’t swear to it. PL uses SQLite for its database. You can get an SQLite DB reader and take a look at what is stored in the file if you want.

Keep in mind that if you delete or rename files outside of PL, their database entries will live on. If you then delete the database and re-index, the database may, indeed, be a lot smaller.

1 Like
9

Thanks to you both for valuable and helpful insights. I think some of that info could usefully be added to the User Guide.

I spent some time yesterday working on this and I think the crash during indexing may be due to the age of my NAS drive (7 years) which is leading to erratic / erroneous data or stalling from a tiny number of the folders.

1 Like