Exact rules for the creation and use of sidecar files

Automatic Synchronisation

Thanks. It’s not a short for the used name.

George

One reason why I use macOS, because Finder and Spotlight continually indexes all sorts of metadata from all sorts of files without my having to think about it

1 Like

@George AS stand for Automatic Synchronisation (Auto Sync or AS) and that was what the feature was called originally and arguably it still is

and I am afraid that is all anyone is going to get out of me today. Our youngest sons family brought some flu bug with them and shared it with us!!

My wife succumbed yesterday and yesterday evening I started to run a high temperature and I feel awful today.

Bye for now.

Just FYI: I had a lot of trouble figuring out exactly what you wanted. I can see a lot of eyes glazing when they try to read what you wrote, which might reduce the number of votes you get. I wasn’t sure myself whether I should give your feature request a vote.

Joanna does seem like a reliable source, but I disagree with her on this.

  • The keywords/metadata need to be in the database for fast access.
  • The keywords/metadata need to be in the DOP file in case the database is ever deleted or corrupted and needs to be re-created. However, if PL were to always enable XMP synch, then the DOP file could be skipped.
  • The keywords/metadata need to be either in an XMP file (for RAW files) or within the image (for RGB files). This is for compatibility with the rest of the world.

All these sources need to stay synchronized, which is what I’m proposing. By synchronization, I meant that all the sources should match and that PL must recognize changes made by external tools, even while PL is running.

All metadata that is “global” (keywords, star ratings, colors, etc.) must be properly synchronized. PL should never wipe out metadata in XMP files that it doesn’t recognize–it should only alter the metadata it recognizes.

If Program X changes the star rating of an image (a change to the XMP file or the RGB image), then PL should pick up this change and update the DOP and database to match. Of course, external programs need to do the same. If you then change the star rating in PL, the external program needs to pick up the changes. If you use an external program that doesn’t follow the rules, then any problems are on you.

Right now PL doesn’t follow the rules so it’s reasonable for people to avoid using it if they have a workable alternative.

Yikes! Hope you feel better soon.

Hopefully you and your wife feel better soon.
There is in the Netherlands a bactery which causes lung / windpipe infection and a corona virus at the same time very easy to catch up too. I spent a week in bed and an other to be able to work again. So take your time.

@freixas why would you use dopfiles to store keywords and other data as iptc? In rawfiles you use xmp and rgb files you store it inside.
When the DB crashes you can re index and retrieve the data again from xmp and propperties of rgbfile so the dopfile isn’t needed. Keep in mind dopfiles arn’t readable for any other applycation so any syncronisation is only through dxopl.

About my proposal:
Key feature i need is
Read only sync
Read and write with control of writing sync.
Full automatic sync (which we have now only)

An other key feature which is connected to the three above is folder selectivity.
Hide and ignore folders who arn’t containing images for dxopl to see.
And the folders which dxopl need to be seen needs selectivity in syncronisation.

Me i would like read only or read and write with selection list before writing.
I never would like to have dxopl to be sync in full automated way aslong as an other application is used as DAM.

If you have seen how Syncback free works you understand the mirror function and the different values in that sync command.

I tried to lay out a path of control which DxODAM communicate with files and thus other possible DAM’s.

This is exact the problem.
Which DAM functionality is the right way?
If both are poking in your database/library you will get a mess.

A simple “read only auto sync” checkbox would be enough for most i think.
That way dxopl is always up to date for information wile using dxopl for editing and then it is not important how dxo pl handles the xmp and rgb imagedata inside it’s own ecosystem/database. Every screwup stays inside so no problem.

It’s too easy to state it’s the users problem if things go wrong.

Search speed. I just tried creating a Smart Collection in Adobe Bridge to find all images matching the keyword “Bird”. It’s been running for a while, and it’s only up to my 2009 folders (2005 is the first year). Without a database, the search has to be repeated every time.

Well, that’s where one needs a standard. I think you can get pretty far with:

  • Metadata in XMP files for RAW files. Adobe started this, most others follow this because…well Adobe sets the standard.
  • Metadata in RGB files.

No one shares database files. Programs with databases should keep up with metadata changes to the XMP/RGB files made by any other tools at any time. I realize this is easy to say and hard to do.

My proposal was for a method that fixes problems as they are found. A re-index that uses timestamps to resolve all problems might be the solution for resolving the problems all at once.

It’s easy and it’s true. If things go wrong, then the user has a problem. I think you meant to say that it’s not the user’s fault. I agree—if software works improperly, it’s the fault of the people who wrote the software.

On the other hand, if you know a program has a problem and decide to use it anyway knowing that it will screw up, then it’s on you.

I am using PL for metadata handling. I know it screws up. I can submit bug reports for the problems I can replicate and I (like you) can make feature requests. I hope things will get better.

At the moment, I’m working on ways to fix the database/file mismatches myself. Not ideal, but I don’t want a workflow that involves using a separate keywording/metadata tool.

I don’t do anything with any other metadata and so have not noted if there are problems elsewhere.

which will be the same reading the database so dopfile isn’t needed. :slightly_smiling_face:

Most of what you write i agree with in some or more precentage. I see iptc/xmp/propperties data as a database on its own. (if it changes it changes on many places too when reading this changed file.)

So any program who changes this data by just turning it on ( activate it) and don’t have a "we noticed changed data should i write from DB to file? yes/no "can be causing big problems for the users personal data.
And DxOPL does this with Dopfiles and xmp/propperties if autosync is active.

i don’t know from which version you stepped in but many of us have seen the birth of dxoDAM including al the childdeceases with it. v7.2 is rather grown to puberty but not to a adult level.
That’s why many of us still uses workarounds and external DAM’s.

true

True again and i always try to motivate DXOdevelopementstaf to make things better.
I am not an expert in any way so i look at it as a plane user.
And the easiest way to cut out 50% of the trouble is make autosync simplex. only in, in my honest opinion.(edit: As checkbox option ofcaorse so full duplex is possible for those who wants it. )
This two advantages:
1 any screwup inside DxOpl’s DAM doesn’t hurt the users library (of external DAM’s) but we can report it.
2 because of the read only more people are probably using it for simple searching which can reveal those quirks and bugs.

So DxO’s DAM can take some time to grow without people turning away from it and shutdown any syncronisation besides DOPfiles

i am not negative about DxO only carefull and rather busy to repair my xmp and file propperties keyword like. ( found a lot of “problems” old and dxo new like when i was searching for hicups.) and now i am re tagging all my exported jpegs. :sweat_smile:
This cost me a few days to repair.

And then i have to check my source files too… :face_with_peeking_eye:

And when i am done that i probably re-edit all rawfiles in v7.2 which are done in Silkypix and rename the export folders xxxx v7.2 so i know that they are up to date…

i think i don’t have time for rebuilding the kitchen this year…

:rofl: :crazy_face:

PL Version 3; pre-any DAM, I believe.

My prior keywording was from 2019 or earlier (using Adobe Bridge CS6). I haven’t done any since, until just a few weeks ago.

1 Like

Hi @OXiDant @Joanna

Firstly thank you to you both for all your valuable contributions to the forums - all very helpful.

In response to both you recent posts in this thread, and perhaps I am “late to the party”, in my quest to find the perfect DxPL compatible DAM (which doesn’t exist) I noticed a subtle distinction creeping into how various software packages are being described and realised that Excire describes “itself” as used for “Photo Management” - so clearly no intention to be a DAM.

At this stage, I am leaning towards digiKam as it seems on balance to do most of what I need for DAM albeit no automatic keywording from what I can tell. The journey continues and all part of the fun!

1 Like

This reply is perhaps “Off Topic”…yet no need for a new thread.

I am speculating here but I would say serious influencers have it all professionally managed and less serious it’s possibly a “dogs breakfast”.

Further speculation - on assumption most influencers come from a younger age cohort they may not care about future retrieval of base content - once it’s “posted” - move on.

A bigger concern for me is the concept of “dark data” - I read a few months ago that something like 44% of all digital data is categorised as unlikely to ever be viewed again! To me that’s “digital waste” that would be contributing to global warming via the energy to keep the physical storage devices running.

Relying on the PL database for all metadata exposes the user to a single point risk of irretrievable loss, unless the user implements a comprehensive backup strategy. Things can go wrong besides a deliberate deletion of the database. Unfortunately, I don’t think it’s clear to most users how to do this (including where the database file is located for backup).

Also, the XMP sidecar file (in addition to providing data redundancy) offers the option of having metadata available to other programs. Among other things, this helps avoid the user being locked in to a single proprietary program for access to image metadata.

1 Like

Yes, ExifTool is a great tool. A number of programs rely on it, including IMatch (photools.com) which I use as my DAM. However, I’m pretty sure DxO has written its own metadata handling routines, for better or worse.

1 Like

Not having backups of one’s computers is foolish imo, but I also notice that many people I know are happy enough without backups.

PhotoLab can back up (and restore) its database, but there is no provision to automate the creation of backups. But the location of the backup is left to the user, everyone should therefore be able to select the location that best suits personal needs or preferences.

XMP sidecars are meant to store metadata in an orderly (as defined by respective standards) way, which helps to transport metadata between apps. Due to the situation that manufacturers interpret standards differently, not all metadata is transported without issues between any two or more apps. Therefore, it’s good practice to test what one’s environment will and won’t do. Only then can one decide if the transport includes all needed information an that the information reaches the targeted application correctly. It also makes sense to limit the number of metadata editors, preferably to one.

1 Like

Agree and i afriad i am one of them but then local in a Nass. So it’s running already for other data so …:sweat_smile::zipper_mouth_face:

From what I can see, Excire is mainly about using AI to find images using AI, not keywords or conventional metadata.

Be warned, DigiKam has its own dedicated XMP tags, with which PL will not play.

@freixas and @OXiDant thank you for your messages. I had my flu jab in one arm and Covid jab in the other arm, which doesn’t mean you are not going to catch the virus but might help the immune system to fight the bug more effectively.

Getting up and moving about has caused my temperature to drop from the high 30s (touching 40 on some occasions) down to 36.7 which is still a little high for me but shows I am hopefully over the worst but not really in a fit state to re-enter the forum, which when written like that seems to imply a combative forum rather than a friendly space for the exchange of ideas!?

Take care.

Regards

Bryan

@Joanna Apart from spending hours with my flu fuddled brain trying to add my beta test images to digiKam, at this point it is digiKam 1 (at least) and BHAYT 0!

However I set digiKam to absorb all my images some years ago and whenever I return to it from time to time is catches up!

So I picked a directory from digiKam and added two keywords in a JPG image and got this in PicMeta PIE

and in DxPL I got this

With RAW I got this

and this, and it is not the simplest xmp sidecar file I have ever seen (!?)

and from DxPL

So it seems to work but I am concerned about all the “chaff” in the sidecar file!

I am using the latest version of digiKam

and I don’t find it as intuitive as I might like to!

Great that you are past the worste part, your now in the cotton in the head part?

Combat? Maybe a bit more in the discusion section about the exchanged idea’s… :yum:.

this header should be:

Exact rules for the creation and use of sidecar files arn’t exactly that exact as you would like…

:grin:

Sins i am involved in using meta data back in pse 7? For library function, i am switched several times from idea’s, applications and level of detail.
Only in jpegs burnt by pse database push, before i used rawfiles they where the only library, i did my thing on the oocjpeg and got 1234.jpg and 1234_editted.jpeg. In the same folder.
Then we got rawfiles, so i editted after export the propperties of exported jpegs for tags and keywords.
Later i discovered xmpfiles, finaly a standard which holds in the future!
Eh nope.
Same as a pdf isn’t made by one standard either.
So we dragging our feet through the mud with flipflops in order to keep our metadata library crisp clean wile every application which claims to use a worldwide standard tries to destroy it :joy: :crazy_face:

And every one has his or her own secret swamppath where the mud is reasonable thinner then elsewhere.

My swamppath is Adobe Bridge for now. Until i find a better one that is.

:grin: