Calibration behavior sometimes fails for reasons that are not easy to understand. But I always managed to get a profile after a few tries.
In your case, it is probably indeed the shadow of the hand on one of the boxes. I also noticed that the order of positioning of the points can have an impact depending on the orientation of the chart in horizontal or vertical.
Generally the charts have crosses at the corners to position the grid. This is not the case for this Datacolor charter. On other charts, these crosses are closer to the colored squares than the edges of the chart, and the edges are wider. As a result, there is less risk of the hand shadow causing a problem
I suggest you start your calibration again by positioning the points near the exterior angles of the color patches: this will offset the measurement squares (white frame) in relation to the color patches, but there is no impact as long as that the white frame remains in the patch. This might help resolve the problem.