You're mixing up two different ways of zooming in d3 -- zooming with scales ("semantic" zooming) and zooming with transforms ("geometric" zooming). See this answer for a detailed breakdown of the differences.
You've attached your scales to the zoom behaviour, so when you zoom or pan your scales get modified to reflect the change. However, you're only using that modification to redraw the axes, for the plot you're creating the zoom effect with a geometric transformation. That works until you try to redraw any of your plot (or, draw a new shape). When you do that, the modified scales and the transformed plot add together to double the zoom and pan effects, resulting in shapes that are out of position with the axes.
The solution is to simply do one or the other. Either connect your scales to the zoom behaviour, and then use those scales to redraw all your shapes after every zoom event; or use transformations -- but then your axes won't automatically change the tick spacing as you zoom in and out.