Using an SQL Lite tool, I recently looked at the contents of my PL database in order to investigate problems with settings that didn’t seem to be sticky.
I discovered that a lot of images had 2 entries in the Items table. Same file name but different ID and different containing folder ID. And the information about the image was not always the same in each entry. Similarly, I could see that many folders also had a “double identity” : 2 different entries in the “Folders” table, same name but different folder ids for their parent folders actually pointing to the same physical folder. And no, this is not a problem with virtual copies or with image files having different types. I’m really talking about unique image files.
To make it short, a whole mess and a database that has grown much too big (about 80% of the images had 2 entries). I deleted the database, re-indexed all folders and everything seems to be clean again now (XMP sync was on but I have lost my Projects).
That 2 items (images) having the same filename and being stored in the same physical folder could eventually get 2 different IDs is a clear sign that something is plain wrong in the PL database management. Ditto for some unique physical folders having multiple IDs in the “Folders” table. One could wonder which entry was taken into account when working on a given image.
So, if you detect some strange behavior of PL about your settings, you should look at the DB using a tool like SQLite Expert (free).
I’m reading this now… what if you select, say, two different images and “organize” them in a project. Do you then see the “duplicate” entries in the database? Just wondering…
Investigating virtual copies, I found that the DB had several entries with the same filename and different IDs. What other procedures could cause this, I cannot say. But as long as the database can get off track, PhotoLab can never be used as a reluable asset manager.
Current workaround is to sync both the dop and xmp sidecars in order to loose as little information as possible when a cleanup is performed as described above.
In that case, the image ID is different but the SourceID is the ID of the source image (so, they are identical). This was not the case for the huge number of duplicate rows present in the database, which was much greater than the actual number of existing virtual copies.
Well, I’ll not be able to reproduce, I guess. This was just posted as a warning.
As for the lost projects, they were all coming from Lightroom collections, so I could easily import them.
Stenis
(Sten-Åke Sändh (Sony, Win 11, PL 6, CO 16, PM Plus 6, XnView))
6
There is a thing with the “Project Items” too since it also adda “External” search items in that table too and the problem reindexing a new databas to replace the old is that you will lose the items in that table.
Some people might miss their “Projects” and “External Search” items when they start using that new database.
Despite that I have used that technique a lot when I have wanted to clean up the database. Another anoying thing I think all these obsolete Temporary .DOT-files are, that we have to clean up from time to time.
@Pat91 Did you by any chance replace your old drive with a nice new faster one?
The scenario you describe is typically what happens when users replace an old drive.
DxPL is not driven by drive letter it is driven by the drive UUID.
If the UUID of the disk changes it will not give a warning but simply discover the “new” directories, which are identical to the old directories and use any relevant DOPs, if they exist, so everything can look the same while using DxPL but actually any old ‘Projects’ will fail to find the Items (images) that resided on the old drive if it is not still online.
@Pat91 If you have a dump of the old database I would welcome the opportunity of obtaining a copy to test my programs on, DM me if you don’t want to make it public.
@platypus You are describing the characteristics of VCs the difference between the [M]aster entry and the VC is “subtle”, i.e. basically non existent except by date and the fact that VCs have higher ids. than the [M]aster.
I believe that DxPL database can get off track, @Bencsi kindly lent my his database which spans many generations of DxPL and I believe that my count program indicates some anomalies but I am not convinced that is happening here, nor do I know whether the count mismatches I “discovered” with @Bencsi’s database are real or just my understandings about the workings of DxPL are (slightly) wrong!
