I banged my head on this for a bit today. Right-clicking a file in Project Explorer or Package Explorer, then selecting "Team" > "Show Annotations" does not work (at least in version 4.5.1). So don't do that. I found the easiest way to show annotations is just click on the vertical ruler then select "Show Annotations" from there.
By the way, the ignore whitespace bug that @Derek mentions has been fixed. From the bug ticket,
After updating to the 3.4.0.201405071430 showing the annotations worked also without setting the property "Ignore whitespace changes"
So the bug seems to be fixed.
I verified that I am not impacted by the bug in 4.5.1