Zoom amount changes with move tool

When zoomed in to 200%, if I use the hand tool to move around the image I find that the screen randomly goes back to 100% or even 6% on occasions. There doesn’t seem to be any specific cause for this & it is completely random. I’m using a 24 inch M1 iMac with 16gb ram. Has anyone else noticed this?

I forgot to mention that this is with PL5 updated to the latest release.