@Musashi, @sgospodarenko Thank you for your post and for providing a sneak preview of elements of the FAQ but I am afraid I strongly disagree with your opening statement which according to my tests is positively the wrong thing to do!!
Namely,
“Since version 5, PhotoLab now reads rating from .xmp files and no more from .dop files”
-
This statement is factually incomplete, PL5 reads from .xmp sidecar files and from the embedded ‘xmp’ data contained within the photo (RGB or RAW format) and no more from .dop sidecar files.
-
This statement casts many users coming from using PL4 adrift by actually changing the way that ‘Ratings’ data is handled and may require users to change their work flow. This change should have been highlighted at the time of the release not months later.
-
Arguably worse than any of that, having collected the ‘Ratings’ data from a new ‘xmp’ source PL5 then “destroys” the data in the DOP, potentially eradicating that data forever. Unfortunately users have been used to DxPL never destroying their data from release to release until along comes PL5 and many may not have backups (because there has never been any need for them, until now)!
-
The PL4 DOPs get “special” treatment, at least on Win 10, according to my testing but Joanna’s testing on the Mac shows that the Mac release is apparently ignoring the PL4 DOPs i.e. apparently adopting the new, PL5 strategy! The handling of PL4 .dop sidecars is not shown in the table.
I ran a test on PL4.3.6 which turned out to be something of a “mixed bag”! Why did I run such a test on a previous release and the answer is because I thought it would be quick and give some insight into the upgrade process and the thinking behind the PL5 strategy. Simple it was not!!
-
PL4 can read the ‘Rating’ (Rank) from either the .dop or the .xmp sidecar file (if one exists) with RAW photos, I didn’t test it with embedded ‘xmp’ metadata (laziness).
-
I started with all 4 RAW files containing no ‘Ratings’ (Rank), actually PL4 refers to ‘Rating’ in the product but stores as ‘Rank’ in the .dop sidecar.
-
I added ‘Ratings’ in PL4 and .dop sidecar with the correct ‘Rank’ files were created with the correct ‘Rank’but no .xmp sidecar files were added. With this test the DOPs were created when I closed PL4.
-
I deleted the images in PL4 and added them back and assigned ‘Rating’ values in FRV which only creates .xmp sidecar files for RAW images.
-
PL4 took the ‘Rating’ from the .xmp sidecar and showed them in yellow, no DOPs were created at this point (no other edits had been made)
-
Changing the .xmp ‘Ratings’ externally was immediately picked up by PL4 and shown correctly.
-
Setting the ‘Tag’ in PL5 resulted in the creation of the .dop sidecar (DOP) to hold the new flag but the DOPs had Rank= 0 at this point! Is there a flag in the DOP to tell PL4 that the ‘Rating’(Rank) if being externally set? If the ‘xmp sidecar is lost at this point the DOP will be of no use for recovery.
-
I changed the ratings of two of the photos externally and the DOP of those two photos changed, the ‘xmp sidecars remained unchanged (PL4 immutability at work perhaps) and the “star” colour changed from yellow to white.
-
I changed the values of the ‘Ratings’ in all photos externally as might be guessed those with PL4 assigned values showed no change but the other two were changed in line with the external product used
-
Mixing and matching PL4 ‘Rating’ assignment with external assignment means that ‘Ratings’(Rank) is spread between sidecars types, i.e. .dop sidecars and .xmp sidecars
So at the end of the test PL4 might believe one thing and the external program another if the two methods of assigning ‘Ratings’ are used!
More importantly PL5 needs to be able to handle the situation of this potential mixture of sidecar files in the migration from PL4 to PL5. Arguably PL5s initial source of all such data is the PL4 database but as some of us found that upgrade didn’t go as well as it might have and the database was “lost” to PL5 during the upgrade . PL4 keywords were only held in the database (as far as my tests show).
I managed to recover my database but the loss of the database came as a real shock to longtime DxPL users.
Effectively prior to PL5 the 'Ratings’ (Rank) field was not held external to the PL5 system because of the “immutability” mandate of pre-PL5 releases with respect to JPGs etc so the .dop sidecar was treated as the primary (or only) source. PL4 did not write the ‘Rating’ (Rank) to the .xmp sidecar file as well as the .dop sidecar file as I thought earlier in my testing so there is now a potential conflict between the differing data that can be held.
Hence, many users coming from that background were not really treating ‘Rating’ (Rank) as anything other than DxPL metadata held in the DxPL database and, for portability, in the .dop sidecar.
I am afraid that I would suggest that such an expectation would be carried forward by those users into PL5.
Suddenly changing the rules because the ‘xmp’ data is now available to PL5 for update and that is, arguably, “where that data belongs” is unacceptable without any suitable warning. Forcing a change in work flow is arguably unacceptable (Skylum tend to do it often), changing it without adequate warning is …!!
Hence, taking a null or 0 value from the ‘xmp’ embedded’ or sidecar data is rather short sighted because of the heritage of PL4 which many users are coming from. Users may not want to use PL5 ‘xmp’ metadata and actually considered ‘Rating’ (Rank) as DxPL metadata.
The decision has effectively cut DOP ‘Ratings’ “loose” and made it a WOM (Write Only Memory) field like the keywords which are also stored in the DOP.
However, the important thing is that the old PL4 processing flow involved both the .dop and the .xmp sidecar files and and the new PL5 processing flow completely ignores the .dop with respect to ‘Ratings’.
Given that PL5 cannot distinguish between a Null and a 0 ‘Rating’ entry, plus the fact that NEF and my Panasonic camera set the value to 0 in the original photos the adopted strategy is doomed to corrupt the data of unsuspecting users, which is exactly what has happened.
The current PL5 algorithm is wrong, in my opinion, it needs to be more sophisticated in handling the data potentially stored in .dop sidecars, .xmp sidecars (for RAWs) and embedded in the photo for (RGB and potentially RAWs as well).
In addition it is possible that a different set of rules should be in force for the first introduction of a directory to PL5 (i.e. the database) versus subsequent navigation to the directory. However this still poses the problem if the .xmp data (sidecar and/or embedded) contains any ‘Rating’ information! Effectively the metadata dates should be taken into account and the .dop sidecar should also be considered as a candidate for the data, certainly on the first discovery of the directory and, potentially every time!
Therefore discover the potential Rating data available and rank (not Rank) that data based upon the date/timestamp from latest to first, i.e. determine the latest data available and that should include keywords, GPS data etc… (I haven’t tested them in this context at all yet!!)
While one alternative solution might be to force a write of the .xmp sidecar data even when the ‘Preferences ‘Sync’ option is off and the customer has not (and possibly never will) use the ‘Metadata’ ‘Read from disk’ and ‘Write to disk’ commands PL5 could force a write to secure the ‘Ratings’ in the .xmp sidecar but users needs to be aware that these need to be secured in any backups and that they have value and that doesn’t help any users currently “trapped” in the situation of PL5 DOPs with ‘Ratings’ in them!
The current algorithm is way too simplistic (I feel like saying naive but that would be rude), it does not take into account the potential loss of the database (in the PL4 to PL5 upgrade or later because the database becomes damaged/lost etc.). It fails to cope with the possibility that users may not embrace the new features of PL5 and start using PL5’s new found “powers” of being able to store metadata and continue to believe in the power of the DOP as a backup for the database!
Particularly if there was no timely warning about the potential consequences of the PL5 changes.
Please change the strategy and/or provide a ‘Preference’ to allow PL5 to use the PL4 strategy and please discuss this with the users, you created the product and it is excellent but we use it, many on a daily basis, in a wide variety of ways and we might just have something constructive and useful to say!