touchMove - change direction during gesture

Question

I'm using this code to move an image around inside a div on a tablet, using touch events. I've found that when testing on an iPad, the image can be moved quite happily either vertically, horizontally, or both (if you start by going diagonally). What I've also found however is that if you start by moving the image either vertically or horizontally, you can't change direction until you finish that gesture and begin another.

It seems that once you start moving on one axis, that takes precedence over the other axis, no matter what the co-ordinates of the touch point are. Is there a way to override this behaviour to move in any direction no matter how you started, or is it just how Safari interprets touchMove?

More detailed code can be found in a previous question here, if that helps.

var clicking = false;
var previousX;
var previousY;

$("#scroll").touchStart(function(e) {
    e.preventDefault();
    previousX = e.clientX;
    previousY = e.clientY;
    clicking = true;
});

$(document).touchEnd(function() {  
    clicking = false;
});

$("#scroll").touchMove(function(e) {
    if (clicking) {
        e.preventDefault();
        var directionX = (previousX - e.clientX) > 0 ? 1 : -1;
        var directionY = (previousY - e.clientY) > 0 ? 1 : -1;
        $("#scroll").scrollLeft($("#scroll").scrollLeft() + (previousX - e.clientX));
        $("#scroll").scrollTop($("#scroll").scrollTop() + (previousY - e.clientY));
        previousX = e.clientX;
        previousY = e.clientY;
    }
});

$("#scroll").touchLeave(function(e) {
    clicking = false;
});

Edit: having just checked, it works correctly on an Android tablet, so just the iPad is causing problems.