ios – Entity Does Not Remain in Position

I am tracking an image and having a ball follow it very simply, but want it so that when the StateHolder lockedPostion is true, the ball stays in the place—in 3D space—at the very position and scale where it left the tracking reference image the frame prior to said state change.

So for example, if I lock the position, and don’t change the tracker reference image’s position in IRL too, when you move the camera around the ball, the expected result should be as if the ball remains at the center of the tracker reference image—where it was when tracking—however, this is not the case. You’ll notice the ball now appears it is the same size, but not at the center of the tracker reference image in 3D space anymore. The ball is actually the exact distance away from the camera as to appear the right size from the camera as it was the frame prior to locking, and so after locking the ball can be considered ‘mis-aligned’ in 3D space from where it should be; which is still at the position in 3D space at the center of the tracker reference image.

I believe this must be something to do with physical scale and the relationship of how the camera to the ball in space is being calculated i.e. it is not being treated in the same way as it was when the reference image was being actively tracked and the ball scaled/positioned to match it, and now it thinks the ball is further back according to its scale(?). The ball should stay in place, in the same world position to the camera as it was, no matter where the camera moves to and from.

        func makeBall(radius: Float, color: UIColor) -> ModelEntity {
            let ball = ModelEntity(mesh: .generateSphere(radius: radius),
                materials: [SimpleMaterial(color: color, isMetallic: false)])
            return ball
        }
        
        var imageAnchorToEntity: [ARImageAnchor: AnchorEntity] = [:]
   
        func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
                        
            guard !parent.lockedPosition else { return }
            
            anchors.compactMap { $0 as? ARImageAnchor }.forEach {
                guard let currentImageAnchor = $0 as? ARImageAnchor else { return }
                let width = Float(currentImageAnchor.referenceImage.physicalSize.width * 1.03)
                let anchorEntity = AnchorEntity()
                parent.arViewAnchor = anchorEntity
                let modelEntity = makeBall(radius: width, color: .systemBrown)
                parent.selectedEntity = modelEntity
                anchorEntity.addChild(modelEntity)
                arView.scene.addAnchor(anchorEntity)
                anchorEntity.transform.matrix = $0.transform
                imageAnchorToEntity[$0] = anchorEntity
                parent.arViewImageAnchor = imageAnchorToEntity[$0]
            }

        func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {
            
            guard !parent.lockedPosition else { return }
            
            anchors.compactMap { $0 as? ARImageAnchor }.forEach {
                let anchorEntity = imageAnchorToEntity[$0]
                anchorEntity?.transform.matrix = $0.transform
            }
        }
    }

I have tried adapting the scale of the ball and that does shift the resulting distance the ball will be away from the reference image on locking, but this alone does not yield the intended behaviour. I cannot find how to get it to scale and position like the ball was when tracking; it is as if the entire interpretation of the 3D world space and objects within it shifts unexpectedly.

I am perplexed as to how to get this seemingly simple intended result.

Is this description clear enough to understand the issue? I appreciate your help to solve this with me.