three.js drag and drop common solution causes objects to "jump"

Question

I have been experimenting with three.js for the past few days making an interactive environment where you can pick up a watering can(drag and drop), click on it to make it spout water, and a mango on a tree in the scene will grow.

I have implemented drag and drop successfully due to their being a lot of tutorials on it all sporting basically the exact same code. I feel like there is a bug with this solution though and I have found it in other peoples jfiddles etc so in my limited experience it seems like a problem with the actual code and not just my implementation of it.

the common implementation I have found on here goes like this:

let plane;
function createPlane() {
  plane = new THREE.Mesh(new THREE.PlaneBufferGeometry(WIDTH, HEIGHT, 8, 8), new THREE.MeshBasicMaterial({ color: 0xffffff, alphaTest: 0, visible: false }));
  scene.add(plane);
}

let selection;
function onDocumentMouseDown(e) {
  e.preventDefault();

  const mouseX = (e.clientX / WIDTH) * 2 - 1;
  const mouseY = -(e.clientY / HEIGHT) * 2 + 1;

  const mouse3D = new THREE.Vector3(mouseX, mouseY, 0.5);
  mouse3D.unproject(camera);

  raycaster.set(camera.position, mouse3D.sub(camera.position).normalize());

  const intersectedObjects = raycaster.intersectObjects(objects, true); 
  if (intersectedObjects.length > 0) {

    selection = intersectedObjects[0].object;

    const intersectPlane = raycaster.intersectObject(plane);
    offset.copy(intersectPlane[0].point).sub(plane.position);
  }
}

function onDocumentMouseMove(e) {
  e.preventDefault();
  const mouseX = (e.clientX / WIDTH) * 2 - 1;
  const mouseY = -(e.clientY / HEIGHT) * 2 + 1;

  const mouse3D = new THREE.Vector3(mouseX, mouseY, 0.5);
  mouse3D.unproject(camera);
  raycaster.set(camera.position, mouse3D.sub(camera.position).normalize());
  raycaster.setFromCamera( mouse3D.clone(), camera);
  if (selection) {
    const intersectPlane = raycaster.intersectObject(plane);
    selection.position.copy(intersectPlane[0].point.sub(offset));
  } else {
    const intersectedObjects = raycaster.intersectObjects(objects);
    if (intersectedObjects.length > 0) {
      plane.position.copy(intersectedObjects[0].object.position);
      plane.lookAt(camera.position);
    }
  }
}

function onDocumentMouseUp(e) {
  selection = null;
}

With this implementation, when you click on the object and dont move it for a second, the object "jumps" back to its original position. also, if I have dragged the object across the x-axis and I try to drag it again from that position, it "jumps" to (0, 0, 0) which is the more annoying part.

I am not familiar with how to make a jfiddle and I think I have too much code going on for it to be successful anyways, but you can see what i mean in this other jfiddle which thank you to whoever made it, it was really helpful to me!! and sorry I am using it as an example of what I think might be a bug.

If you play around with the box for a bit, you can see that it "jumps" back to the center. The code for this jsfiddle is essentially the same as every other solution I have found on stackoverflow and other three.js specific tutorials. Has anyone implemented drag and drop in a different way and avoided this problem? Is this unavoidable as it seems everyone uses this method despite this issue?

I would be very very grateful for any help anyone is willing to give. Thank you for your time in reading and thinking about this!