Hi all,
I use some backup drives and a NAS which is not allways running for saving my images. When i search for a keyword or folder in the search field, DXO shows always also images on these drive. When the external drive / NAS is not mounted, an exclamation mark appears as search result (when the path is not accessible)
Question: Can i exclude drives fix for showing in the search results. I don’t wan’t do see duplicate images on Drive1, Drive2 and NAS…
Thanks
Bruno
I just tried it. Deleted the database, I don’t use it. Opened PL and went to an USB drive containing some pictures. In Photolibrary right click on the folder or drive and choose indexing. Only those images on that drive or (sub)folder are indexed. Maybe something went wrong when building the index, such as including the backups.
PL8 and Windows.
George
@George 's approach seems to be the most promising solution.
There are a few catches though:
CAVEAT: Do not delete the database files, unless you’re sure that settings and metadata have ben saved and that you have a backup of the database. Otherwise, deleting the database files will effectively destroy all the things you did in DPL.
Note that I use DPL on macOS, on Windows, YMMV.
By deleting the database you will lose details of any “Projects” (logical groupings of images) that you may have created - and the ability to search for any keywords that you may have assigned to images … but, provided you have not de-activated PL’s default setting to create sidecar/.dop files associated with each source-image, you will not lose any of the details related to corrections/edits you did in PL.
PS. I know that you know this, Mr. P … but just aiming to help others who may misunderstand the details of your CAVEAT.
@Brunok1 As has been pointed out in the earlier posts, the search is of the database, the whole database and nothing but the database!
As @George and @platypus have indicated you need a copy of the database that relates to one drive but then that drive needs to be attached at the time of the search.
Having a mixed estate of images from one drive or another or another plays havoc with the counts returned by DxPL for any search!
The entries in the database get there when you discover a directory , regardless of whether you edit the images or not (they will be “edited” automatically using the ‘Preferences’/‘Correction Settings’ the first time they are “discovered” directly by the user or via indexing).
DxPL is an editor not a viewer, so every image “viewed” in DxPL is automatically added to the database. If you view images in the same directories on Drive 1, Drive 2 and NAS or ‘Index’ the images on any of those drives they will all be added to the database.
If those directories are actually copies of one another then you could wind up with as many as three entries in the database for the “same” image.
All searches are made via the database and if any drive is absent you will get the nasty surprise that you showed in your post.
So in my case I had done a bulk load of images as part of a timing test and the drive was switched of when I did a ‘Search’ and I got the following when I did a search on “orf”, the extension for images from my EM1 MKii, taken some years ago and used to create a large batch.
Please note that I had loaded a smaller batch from my NVME drive of the same but differently identified images and they are still on the machine. If those images had been named identically I would have had thumbnails for those on "N:" mingled with those with the “surprise” thumbnails for those on "I:"!
‘Sorting’ provides these options, not particularly useful to “cluster” the images in a useful way, but adding the ‘Drive’ as an options available would be useful.
and filtering provides these options which don’t offer such options as ‘Present’ or ‘Absent’ or ‘Drive Letter’ or …
As I suspect you have realised already there is not any way that I can see that you don’t retrieve what is not present in a ‘Search’ and cannot ‘Sort’ or ‘Filter’ out those that are present from those that are not present or vice versa.
I tried searching on "N:" the NAS and "I:" the (currently) missing drives but neither search turned up anything
Throwing the database away simply means that no searches within DxPL are available and ‘Projects’ (‘Projects’ and ‘Advanced history’ on the Mac) will be lost and the proposal is then to only edit on one drive and to index that drive.
Editing on more that one drive would be a potential nightmare anyway but the only way to view what edit you actually have on any drive (copy) for an image is to open the image in DxPL and it is immediately found in or added to the database!
Searches are done with an index file. That index file is built with the right click on a folder and choose “index a folder”. When the index file doesn’t exist it’s created, when it does exist the new folder is added. Sub folders are indexed automatic. So it’s not a matter of pointing to the drive but to the root folder of your images and eventual adding another (root) folder.
Filtering is working on the regular images. Maybe it also works when using a search. I didn’t try.
George
@George Searches are done via the Tables of the database which contain details of the image and its status, the metadata assigned or obtained from the image and the IPYC data. The database is the index you are referring to.
Data is added to the database in two way:-
‘Filtering’ works on anything that you have used to select the contents of the thumbnail window (as does ‘Sorting’, I believe) be that
It is the database that is driving the process, repeat it is the contents of the database that is the data source behind all enquiries, other than when the user selects a specific directory to view, at which point the entire contents of the directory will be read and compared to the contents in the database (if any) for that directory.
If the directory or any images in the directory is new to the database, a DOP and xmp sidecar (if either or both are present) and any embedded xmp and Iptc data in the image will be used to help populate the fields in the database Tables, i.e. ‘Sources’, ‘Items’, ‘Metadata’, ‘Iptc’ and ‘Keywords’ (and ‘ItemsKeywords’) (the names here used are Windows, the Mac database has different naming conventions.
If there is no DOP present then the ‘Preferences’ ‘Preset’ are applied as the image is “absorbed” into the database.
If the directory is new to the database it will be added to ‘Folders’.
When the directory has been fully “absorbed” into the database it can be searched as part of the contents of the database not as the data directly from the disk.
So there is no index file but there is a database and DxPL only works from the contents of its database, of which there can be many but only one at a time.
In that case PL has to go through the database everytime I do a search. A time consuming work when one has many pictures. I don’t believe that. A normal database creates index files with a search item and a pointer to the record in that database. When the index is sorted alphabetical it’s just a matter going through the index until you find a match with the search item. With the indexing feature I can add images to the index file that weren’t opened before in PL so not yet in the database. I believe it’s adding these files to the database at the same time, otherwise an index file will be impossible.
I am not quite familiar with the names in mysql.
Back to @Brunok1 . When he opens a file in his backup it’s added to the database and index file. So back to the beginning 
George
@George Its way faster than searching images on disk. The images searchable data is added to the database when discovered by user browsing or because the user has already indexed the directory/directories.
The good news is that it does and that speed (or otherwise) is what you see every time you do a search with DxPL.
Where is this index you keep referring to? It does not exist or rather it does as the database , they are one and the same entity.
When you index a directory the data is added to the database that is what indexing means in this case. If it isn’t in the database then DxPL knows nothing about it so you ensure it is there already by executing the indexing command.
With respect to @Brunok1’s problem he has entries in the database for every directory he has visited (or indexed) on the three disks he has spoken about and these entries are there (in the database) whether the disk is mounted or not.
Hence, a search can turn up entries that point to off-line disks or where images/directories have been renamed, moved or deleted outside DxPL so DxPL has the data and edits for an image but no image is available to apply the edits to!?
Just downloaded a SQL viewer and opened the PL8 database. I see 40 index files.
It seems all related to the search function.
George
@George I wasn’t trying to complicate things by adding extra detail but this is from DB Browser for SQLite
and some indexes provide the interlinks between the structures. others have obvious keys, e.g. the Name and some assist searching but some searches can only be satisfied by reading the Table structure from front to back, still way faster than reading the image records on disk.
So the database is the “index” and when it is destroyed all searches are also lost, along with ‘Projects’ (and ‘Advanced History’ on the Mac, it is lost every time DxPL(Win) is closed.)
Maybe another interpretation of ‘database’'. In the old dbf terminlogy the database itself contains the adresses of the records, the indexes are used to traverse through the database within a selection and sequence of these records. And that’s how I read these tables.
George
@George The database I worked with for 36 years had indexes and datasets and pointers and …
In the case of DMSII each structure, data or index occupied their own file and the largest database that one of my customers had contained 34 million entries, one for each household in the U.K (at that time) in one of its major structures with associated indices.
SQLite puts all the database data into a single file and the software handles mapping the data and indexes onto that file. Given that the edit list are variable in length (and become most of the DOP) it is working with variable length records.
So a database contains data and indexes and that is the way I have considered it for about 52 years (I retired in 2009).
The indexes allow fast access based upon full keys, slower access based on partial keys (binary chop followed by a linear search of the index) and an even slower access if a linear search of the data is necessary.
PS:-
The database is effectively an index to the images that are individually stored on disk so the index and the database are one and the same.
Back to the beginning, keywords and other searchable data are indexed, be it a separate file or part of the DBS.
Anyway I learned some more about the searching with PL and I hope others too.
George
@George That is good and I was going to leave it at my previous post because none of this is going to provide a solution to @Brunok1’s problem, sadly.
However, I had loaded another SQLite utility some time ago, “SQLite Expert Personal”, and it showed the extent of the indexes quickly and easily so we have the following
There is an opportunity to use the indexes on ‘Items’ (which holds the basic image data), the ‘Iptc’ data can only be obtained by searching because there is only a simple index to link an ‘Iptc’ entry to its ‘Items’ entry and some of the ‘Metadatas’ is certainly accessible via indexes.
‘Items’ don’t link directly to folders, there is another structure between ‘Items’ and ‘Folders’ called ‘Sources’. The original ([M]aster) image shares a single ‘Sources’ entry with any and all VCs the user might create for a given image.
But the ‘Sources’ indexes are of limited use for searching but essential to get from an ‘Items’ entry back to its ‘Folders’ entry and vice versa.
This is the output from a program I wrote in PureBasic, in this case it is accessing a database that has been used with many versions of DxPL and “loaned” to me by another user.
The program is still a work in progress and to get some of the data requires a linear search of one or more of the Tables in particular look at the timings when obtaining the “Picked” counts. The database is held on a NAS JBOD volume, the slowest drive I have but it provides access to more than one machine.
PS:- the timings for obtaining “Picked” items counts needs investigating!? So I copied the database to a local NVME and ran it again and it improved somewhat
