Rotation not kept in dop’s why?

@platypus & @john regardless of the semantics of the situation and given that just about every piece of software I have refers to ‘Rotation’ for the act of flipping an image around, I found the little arrow symbols showing how to “turn” images and that is what I pressed!

@john sorry but I have forgotten what platform you are on, I know @platypus and @joanna are Mac and I am Win 10 (I think) but I have forgotten what you use. Photo Supreme is both, FastRawViewer is both so no clues there!

What it is called does not change the “problem” but does at least give me a another search word to use when looking for similar issues etc…

The following doesn’t really help a whole lot either!

Test:-

Opened JPG image in FastRawViewer and did a ‘Rotate 90 CW’ (another package that has got it wrong!)

The following detected the change

  1. PIE shows a 'Rotate 90 CW)
  2. PL5
  3. Zoner
  4. Exif Pilot shows an ‘Orientation’ of 6
  5. Photo Supreme

The following did not

  1. Photo Supreme - removed
  2. Photo Mechanic
  3. FastStone Viewer
  4. ACDSee
  5. XnViewMP
  6. Adobe Bridge
  7. Affinity
  8. ExifPro

I then repeated using FastStone Viewer ‘JPEG Lossless Rotate’ (!?):-

Found:-

  1. PL5
  2. Photo Mechanic
  3. Zoner
  4. FastRawViewer
  5. Affinity
  6. Adobe Bridge
  7. ACDSee
  8. XnViewMP
  9. Photo Supreme

Not Found:-

  1. PIE
  2. Exif Pilot
  3. Photo Supreme - Removed

I should repeat the experiment with other software doing the rotation (sorry re-orientation) but it is lunch time!

However, I think that the major takeaway is don’t rotate/re-orientate/orientate your images until you have checked what might or might not be detected by other software in your work flow!

EDIT 01:-

Sorry but there are errors in the above. Photo Supreme leaves the displayed thumbnails alone but double clicking to access the additional image editing options then detects the change in the images.

PL5 changes appear to be detected by all programs!

This statement needs to be amended as shown

“However, I think that the major takeaway is don’t rotate/re-orientate/orientate your images until you have checked what might or might not be detected by other software in your work flow and understand how that detection shows itself!”

Im W10 and I found oftern PS leaves thumnails wrong (but not always). I was told “Best is that you correct the wrong orientation tag in the Image Details->Technical.” Its more constant but odd range of rotations

As I use FastStone Image Viewer to do my inital weeding I tried rotation in that. Its intresting as its not effecting the thumbnail in W10 or Affinity Photo but in PL5, FastRawViewer and Photo Supreme 5 thumbnails were correct as were full screen displays. Using Photo Supreme the W10 thumbnails were right (correcting through Technical). So for me it looks like doing corrections at my inital stage with FastStone would be woth doing as I don’t have many to do so the very miner inconvenace of W10 incorrect tthumbnails isn’t a problem. Its more constant that changing in PL5 that requires me changing the orintation again when copied to the 2nd PC as I found the change in PL5 wasn’t carried over when copying between PC which is why I started this query on rotation

There are clearly a multitude of varying ways the programs are righting this things and reading in many cases just some of them.
Out of intrest I have copied the test to the laptop and W10 is showing it correcty rotated

When I do this with 2 JPEG files (1 in portrait, one in landscape orientation, both exported with DPL5), I get

  • previews that are rotated in FRV
  • jpeg files with untouched orientation tags
  • .xmp sidecar files with orientation tags that reflect the re-orientation

All the apps I used to open the two images seemed to ignore the sidecars and presented the JPEGS correctly - as exported by DPL5.

@platypus FRV seems to want to produce xmp sidecars in all cases at the “drop of a hat” but this can be prevented by selecting the following in the ‘Customisation’/‘Preferences’ option

The ‘Restore original JPEG file date’ may or may not play havoc with PL5’s detection of changes, I must remember to try that one day!

@John7 thank your for the update! We just had a rain shower here so I “amused” myself by using Photo Mechanic to “rotate” all images by 90 clockwise, as you do, and the outcome was as follows;

  1. PL5 missed one update in the middle of the group! The display initially showed an ‘invalid image’ and the directory contains 4 orphaned DOPs related to the way that PM makes updates - please see PL5.2.1.4737 (i.e. PL5.all releases) on Win 10 has Metadata change detection issues! for a ridiculously long description of how this happens!

  2. Photo Supreme also got caught out and wound up with some of the thumbnails on display being intermediate temporary JPGs created by the Photo Mechanic update mechanism.

Once PL5 fails to detect a change @sgospodarenko that is it! Try not to have PL5 open on the directory you are changing at the same time you are making the changes, I never follow this advice when testing because I am looking for the timing holes in the software so that I can “vent my spleen” and have a good moan and try to persuade DxO to build a better product, one day it might work (and “pigs might fly”)!

After all the testing that I have been doing I am thoroughly disillusioned by how many software companies manage to code metadata handling with so many variants and create so much havoc! DxO had (arguably still have) an opportunity to get it right but I am running out of patience and other more pressing matters need attention.

This is one of the matters that needs my attention, I started chiselling out an area that looked like it was rotten below the paint surface and wound up removing half of the timber sill. The repair now needs to be completed, re-pointed and painted!

The following is a composite of real-time snapshots of the situation after the “rotation” by PM.

The strange thing is that PIE is not showing orientations for most of the images and the centre image for PL5 is “unmoved”!

This is what happened with Photo Supreme thumbnails

I just “flipped” all the images copied to a new directory with FastStone and file manager shows all the JPGs flipped, FIV did object to trying to change the one RAW in the directory!

Quick! Put in a feature request for PL 10 :crazy_face:

@joanna why PL10 in particular, it isn’t going to take that long to finish that particular job and we need to have moved long before that!

Or are you using it as a metaphor, i.e. that every time I start digging into a “problem” with PL5 it gets bigger and bigger, until half the product needs replacing, which fortunately isn’t true!

The rot (in the sill) actually extends for most of its length and it looks as if the bitumen damp course under the wood has failed but having dug out a small area and then discovered another hole from the top the holes all joined up, so out went half the sill and in went a new piece of wood screwed and glued in place (PU glue actually likes, i.e. is activated by, damp).

The new part of the sill needed to be remodelled and rotated, sorry, orientated, actually rotated (slightly) is probably the correct term before being glued, clamped and screwed in place.

Or …??

and a new release of IMatch is installing as I write, it is all too …!

Coming back to the OP statement that orientation is not stored in DOP files. Here is what I found:

DOP sidecar with changed orientation vs. original state (Rotation tag)

XMP sidecar with changed orientation vs. original state (Orientation tag)

How I tested

→ all original RAW files came up with Orientation “normal” in XMP and Rotation “0” in DOP

  • exported sidecars, duplicated them and renamed the copies to “AsShot”
  • adjusted the orientation in DPL so that the scissors pointed downwards (normal gravity here…)
  • exported sidecars, and renamed them to “Orient”

Summary: DPL stores Rotation/Orientation changes in both sidecars under the conditions outlined above.

What I learnt:

  • A shot taken with the camera held “belly up” lists as taken in “normal” orientation
  • Maybe it’s best to switch off automatic rotation in camera, if possible?
    → all shots will list as “normal” and might necessitate manual re-orientation.

Interesting, I have found no problem with RAW only jpg and I fear my phone doesn’t have any orientation options. In PL rotation was OK , but this wasn’t displayed in PS or when copied to anther laptop in PL. I at least now know to ignore rotation in PL but do in at the initial weeding stage in FastStone whose rotation works in PS, PL and my other programs and even in the W10 thumbnail’s usually AND it works copied between my PC’s in PL

@platypus this statement is nonsense and I have stated more times than I care to mention (and again in the post above Rotation not kept in dop’s why? - #25 by BHAYT) that ‘Rating’ and ‘Rotation’ previously stored and read back from pre-PL5 DOPs are still stored in the DOP by PL5 for all copies of an image, the [M]aster and the [1], [2] etc Virtual Copies but will never be read back from the PL5 DOP by PL5 for the [M]aster (frequently the only copy).

Time and time again I have used the WOM term, Write Only Memory, when referring to this but thank you for confirming that I have been correct all along!

The storing of the data is only half of the story and there is no change between PL4 and PL5 in that respect except that the DOP now holds a lot more metadata fields than before. The big change is in the reading of that data which for PL5 is now replaced by reading such data from the embedded or sidecar metadata for the [M[aster copy and not from the DOP! The Virtual Copies continue to rely on the DOP for storage and retrieval of metadata.

Indeed I started this asking how to overcome rotation not working with copies on another pc. Clearly for some odd reason V5 isn’t reading all the data in dops. Now with PL rotation needs to be done befor the imige is opened in PL by another program. Hope if ever they sort out this stupidity they will inform us but untill then really users are best to not use rotation in PL. As it impact on using PL I still regard it as a bug even if DxO for some odd reason see it as a improvement.

I have just done a quick test, prompted by what you wrote.

I took a JPEG, made four virtual copies of it and rotated each one once, twice, three times and four times.

This gave me five rotations in the DOP file, the last being the same as the first - as expected.

All five rotations were also written to the database.

Because I used a JPEG file, the orientation for the master was also written to the original image file.

I then closed PL5, copied the JPEG file and its DOP to a new folder, deleted the PL5 database and reopened PL5 with the new folder. The master and all virtual copies appeared correctly rotated.

Finally, I closed PL5, used ExifTool to change the orientation of the JPEG file and reopened PL5. The rotations for all the virtual copies were correctly read, as was the altered rotation from the JPEG.

So, it would seem that the only WOM rotation in the DOP its for the original file - otherwise the other rotations are both read and written. My guess is that there is a template that says that something has to be written to the rotation in the DOP for every copy, including the master. It’s just that, whether the orientation gets written to the original file or to an XMP, from the first writing, the rotation for the master is only there for simplicity of maintenance and is ignored.

@John7 I am sorry about my “brutal” post to @platypus but he is simply re-hashing stuff that I have done before without referencing that work DxO Photolab 5 Image Rotation Lost in Backups - #3 by platypus.

I apologise for not responding to your original post earlier but I find it difficult to read a post and “fully see it for what it is” when I am already working on something else so I saw it but didn’t fully take in what it was about at the time!

My response to @platypus was that he had seen this before some of this before and repeated tests I had done back in February as if he was discovering this for the first time, without referencing my earlier posts. The rest of @platypus’s post is very much his own work and nothing I have attempted in any of my posts.

Now back to your concern that DxO have engineered a bug! They would argue that they have put metadata where is belongs, namely, into the image metadata (embedded for JPG, TIFF, DNG and sidecar for RAW)!

In fact prior to the February post there had been numerous arguments about why the PL5 DOP contained superfluous metadata at all which “violated” the “rules” of data separation and data “duplication”.

The original reason that the data resided in the DOP was because of the “so called” immutability constraints that DxO were following at the time. I believe that Capture 1 maintains such constraints and that all metadata changes for all image types are held in xmp sidecar files, only being “merged” back with the image when it is exported (or at least that is what appears to happen).

DxO decided to use their own sidecar, the DOP, and you have been using that feature as a useful tool, as indeed had some others. When DxO finally decided to handle metadata more like other programs they continued to store the now expanded metadata in the DOP but for the [M]aster copy that is never used as the source of the metadata, the source is now the image (JPG, TIFF, DNG & RAW) and the xmp sidecar file (RAW) which then completely undermines you carefully crafted strategy.

Others detected the problem because they conserved only the DOP in PL5 as they had previously in PL4 and discovered ‘Rating’ turned to 0 when PL5 read the PL5 DOP or rather didn’t read the DOP but read the metadata which had not actually been written or preserved (e.g. saving the xmp sidecar) . For any metadata changes made in PL5 to be preserved they must be written back to the metadata with AS(ON) in the ‘Preferences’ or ‘Write to image’ with AS(OFF).

@joanna as it should be, the [M]aster data is coming from the image metadata and the Virtual Copies from the DOP.

Only with AS(ON) or ‘Write to image’ I believe.

Thank you for adding that bit which confirms my statement and the little chart that @Musashi provided some time ago which is now in a FAQ on the website I believe!

We are now on the same page but unfortunately it leaves some users bereft of a “feature” they have been using to their advantage.

Fortunately all the data is present in the DOP if DxO decided to change their minds and start re-using the DOP for the [M]aster but I would suggest that needs to be carefully engineered, preferably as a special option so that @John7 and others can use it to their advantage while others who have become used to the way PL5 currently works can continue to use the PL5 way of working without discovering any unwanted side-effects @sgospodarenko @alex.

I consider the implementation of the metadata changes made in PL5.2.0 to be a “knee-jerk” reaction to a genuine issue in an attempt to assuage the ire of some users, which didn’t fix the cause of that ire and actually had a retrograde impact on keyword management. It effectively de-implemented the current way of working and didn’t provide a facility to allow those who were using the product to choose which course of action they wanted to follow!

I don’t think I have ever seen such wanton, inexcusable “vandalism” in years!

Where I am confused is if the information is added by PL5 to the jpg why on the 2nd PC PL5 doesnt read it and rotate rather have to do it again on the 2nd PC. If it did readf I would agree better to have the information in a form other programs could (but not all) read. But as I now have a wayu of avoiding the proiblem I am happy (sort off)now, just wish they had not createde a problem for some of us by not reading from the jpg.

@John7 Because of the change then two things need to happen;

  1. The metadata needs to be written, either AS(ON) in ‘Preferences’ or AS(OFF) but use ‘Write to image’ if neither is used the metadata will not be written and the image will not have the data. But beware because PL5 can change metadata in unexpected ways, part of my rant about the “ire” of some users. So please tell me what your metadata looks like before it goes into PL5 on the first system.

  2. If the data goes into an xmp sidecar file for RAW then that sidecar file + DOP + image will need to be secured and shipped to the other system

This has been the subject of other rants (particularly from me), the lack of communication from DxO about the potential impact on users workflow!

But it should read from the JPG if the data is there but beware if any PL4 DOPs are involved the ‘Ratings’ and ‘Rotation’ (and ‘Tag’) will be taken from the PL4 DOP and ignored from the JPG!

No worries, we’re all doing what we do for what ever our motivation might be. Mine is to progress PhotoLab and I try to comment in a way to make things lean, understandable and without bashing others.

I did the test at different times and in different manners, which brought up results that were unexpected and not always the same, We also have to note that DPL5 has seen a few releases since February and some of the differences might be attributable to changes in the respective releases too.

Anyway, DPL is not my main photo app, I mostly use it as an addition to Lightroom. I was tempted to drop Lightroom when it turned subscription about 6 years ago, but PhotoLab did not have the capabilities to replace Lr then, and even now, I find DPL to lack the capabilities to make it a viable replacement. The main gaps in relation to what I want are a) stability and b) metadata handling.

@john Just to be clear the following should happen.

For the data to be available of the second system it must have found its way into the JPG on the first system.

If the ‘Rotation’ (‘orientation’) data is coming from PL5 then it needs to be written back to the image metadata. If the mechanisms required for that writing back are blocked then it won’t be transferred back into the image so either

  1. In ‘Preferences’, ‘Always Synchronize’ = '‘x’ i.e. AS(ON) when all changes between the image and PL5 will be trapped (but I have seen some missed) and written back (I have no evidence that the backward path hasn’t worked on all occasions)

  2. with AS(OFF) in ‘Preferences’ then nothing will be automatically detected in either direction except that when PL5 first discovers a new image it automatically does a ‘Files’/‘Metadata’/‘Read from image’ (or equivalent) to pull the metadata from the image into the PL5 database (and thence to the DOP - never to be read again on any system for the [M]aster). The only thing that can block part of this metadata read is a PL4 DOP; if one exists for an image the ‘Rating’, ‘Rotation’ and ‘Tag’ will be taken from the PL4 DOP and the rest of the metadata from the image!

So for the metadata to be available on System 2 it needs to have been put there on System 1, if it hasn’t been written on System 1 it won’t be there to be read on System 2!

As I understand your old workflow you used Photo Supreme to ‘rate’ and ‘Rotate’ you JPG images from your phone but as we have discovered PS ‘Rotation’ (‘orientation’) data is not understood by other software (!!??) hence you second rotation in PL4 which was previously then stored in the PL4 DOP and carried to System 2!

With PL5 the rotation in PL5 must now be “flushed” back to the metadata of the JPG using one of the two methods I described above. This should result in any PL4 DOP being updated to a PL5 DOP, please note that if there are no metadata changes or editing changes made in PL5 then the DOP will not automatically be updated to PL5 I believe, i.e. I have seen PL4 DOPs survive after my testing when I have made no changes to some images!!

So in your case if you set AS(ON) or perform a ‘Write to Image’ then the metadata will be updated on System 1 and can be carried along with the PL5 DOP to system 2. I have not had a single situation where PL5 has failed to update the metadata when discovering an image for the first time (e.g. on System 2.

My occasional “missed” update captures have always occurred when PL5 is running at the same time as the software making the changes and (typically) the directory where the changes are being made is open in PL5 (or PL6 Beta).

BUT PL5 does make changes to the metadata and that has annoyed some users, me included, my complaint is that if I start with simple metadata in the ‘dc’ fields then PL5 automatically starts duplicating them in the ‘hr’ (hierarchal) fields as well! Others have complained about the “excess” creation of ‘dc’ entries for each element of an hierarchical key, arguably the correct thing to do, which PL5 changed on release 5.2.0 (in a way that it shouldn’t, in my opinion) but the main thing is that users want the metadata left exactly as their DAM placed it.

What this means is that to use and retain PL5 metadata changes may (or may not) have unwanted side-effects! Hence, my concern over what your metadata looks like!

There are two pieces of freeware that I use for peeking under the metadata bonnet in Win 10, Exif Pilot and Picmeta PIE but they do require some setting up!

I have a copy of Photo Supreme Lite installed on both my Main and Test machines and all my photos are taken as JPGs and RAWs so with a little effort I could reproduce you exact workflow and assess the implications on your metadata!

I hope that the above is not too boring and is understandable.

@platypus Thank you for your courteous reply my concern is also to try to get DxPL in a stable and usable (by all) state.

The following was intended for a Topic but I will start it here!

Some issues with the metadata launch were “blown out of proportion” “simply” because of the lack of information about the implications of the PL4 to PL5 DOP usage change and the others being mainly about the changes that PL5 can make, of its own accord, to the metadata, although some elements of that must have existed with the exports from PL4 (but at least the original image metadata was left intact)!?

The “DAM sandwich” approach suggested by some users should not be necessary, i.e. use a DAM, use PL5, Export, re-DAM the output to restore the desired metadata configuration.

Instead of the pass through approach DxO changed the product in PL5.2.0 to reduce the amount of ‘dc’ data generated for any ‘hr’ keys @Joanna !? This was arbitrarily executed with no option to retain the original working @Marie and doesn’t fix the problem so when they do listen to forum posts they arbitrarily change the product and de-implement something that users may have been using as a feature, i.e. a lose, lose, lose scenario (the third lose is for any confidence one may have in DxO)!

The PL5 Database cannot support anything more sophisticated with keywords:-

Unfortunately I am being a little harsh on DxO because like most of the other products the ‘Keyword’ structure is basically a simple repository of any keyword encountered in the image or created in PL5 with no distinction of its origin with respect to the fields it came from ‘dc’ or ‘hr’ or whether it was from the image or created in PL5.

So my request for a “simple” pass through is simply not possible (with the current database structure) because PL5 can’t tell one keyword from another and knows nothing about its “heritage”/“lineage”. Keywords are “flattened” and stored in a one dimensional structure with pointers that allow hierarchical keys to be recreated. @Joanna that should make searching keywords as simple as …

So when it comes time to export it is up to PL5 to decide what goes where, so my simple ‘dc’ keywords wind up in ‘hr’ fields @Joanna and all the keywords in a hierarchy wound up in ‘dc’ fields prior to PL5.2.0 which then changed the rules and only now includes the lowest level keyword of the hierarchy.

To provide a “pass through” requires PL5 to retain the origin of each keyword, i.e. image or PL5 + field location ‘dc’ or ‘hr’ or both and be able to use these to prevent “corrupting” the image metadata directly (AS(ON) and/or ‘Write to image’) or indirectly via the export option.

The issue of what happens if a user starts making changes within PL5 still applies, with AS(OFF) then the image should be protected and the data would “only” find its way into the export.

Use an Export option:-

I think it was me that suggested that retaining the metadata data could be accomplished by providing a metadata “take from image” option in the ‘Export’ options. At the time it seemed a little “off the wall” but it could be implemented without touching the database and would avoid the need for the “DAM sandwich” and all the effort that entails for the user @sgospodarenko.

There could then be another Export option to ‘retain metadata structure’ with the revised database design to allow DxPL to retain the essence of the original and add any new metadata in an appropriate manner (needs more thought)!?

EDIT 01:-

The current keyword structure on Win10 looks like this

The Id relates to the number assigned to the keyword, the ‘Value’ and ‘NormalizedValue’ are the actual keyword and the ‘ParentId’ provides the mechanism to reconstruct an hierarchical keyword.

The structure is accessed via the ‘ItemsKeywords’ which is a really “complex” structure linking an ‘Item’ (image) to its ‘keyword(s)’ and looks like this

image

Where ‘ItemId’ is a number allocated to an image and the KeywordId’ points to the keyword(s) in the ‘Keywords’ structure.

The suggestion I am making is that two fields are added to the ‘Keywords’ structure e.g. ‘KeywordOrigin’ (‘Image’ or ‘PL5’ or …) and ‘FieldLocation’ (‘dc’, ‘hr’, ‘dc’+‘hr’) or one for each type of field location ‘Fielddc’ and ‘Fieldhr’ and whatever else is appropriate.

When populating the fields for output (back to the image or for export) then if the “Global” or “local” option states ‘Preserve original format’ then use ‘KeywordOrigin’ to help locate/filter all ‘Image’ keywords for the image and the ‘FieldLocation’ or ‘Fielddc’ and ‘Fieldhr’ to put the keyword “back” where it came from!

There is a potential issue with hierarchical keywords that can start out in ‘dc’ fields and with hierarchical keywords in general that are typically stored in PL5 in the same way as stored in most products that use a database, i.e. “flattened”!

Plus handling ‘dc’ keywords that can also come from the flattened elements of an hierarchical keyword, i.e. my favourite test of animal, mammal, bear, animal|mammal|bear arguably in my scheme you should wind up with 6 keywords in the database so when you delete ‘hr’ animal|mammal|bear the animal, mammal, bear in the ‘dc’ keywords should remain intactt which they didn’t when I ran a test with PL5 on this set some time ago @Joanna.

Most DAM software asks questions about ‘structured keywords’ and the elements of ‘structured keywords’ which PL5 simply does not do, so what you get is what you get, until they change it mid release @sgospodarenko!?

This needs reviewing but “normal” life (or divorce) calls.

I am leaving PL5 alone as a DAM. Its to unreliable and as has happened keeps changing and with resulting unwanted effects on some of us. I think presenting PL with jpg images rotated and key words, geotagging etc. done allows DxO messing about to have the minimal possible effect. I still feel and more so now that it was a disaster to develop PL into a DAM its caused endless problems and I don’t think DxO will ever have the resources to make it fully work for most of those who want a DAM. My experience with Photo Supreme is I use only a small part of its ability’s, that its evolving faster than PL is and shows how much work is needed to keep up with the changing requirements of the manty sorts of users that a DAM has.
So for me its easy I just isolate PL as a DAM as much as possible and use it for what its good at (and without the going into the DAM would undoubtedly be better)RAW processing program and I just hope the DAM development doesn’t lead to the same fatale problems as the iPhone camera did!