Let's say you're in position (x,y)=(7,8) of color blue, the algorithm turns the (8,7) (green) pixel to blue.
When your loop arrive at point (8,7) it will find it blue from the prior assignment and you don't have anymore the original color (green). As a result, you end up with the top right diagonal unchanged and its reflection on the bottom left.
In order to restore the original values of the bottom left, you need to store them somewhere and use them for when x > y (bottom left section of the picture)
[EDIT] Note that depending on the geometry of your picture, the transformation (x,y) -> (y,x) may point to a point that doesn't exist. Consider a 50px x 100px picture, when you're at point (50,100) you're calling the pixel (100,50) to change its color while it does not exist in your original picture