It's because the image and title of a button are centered together by default.
For example, here's a 300px wide button with an image and text, no insets:

Looks like I need the button contents shifted left by 60 pts, so -60 left inset and 60 right inset.
button.imageEdgeInsets = UIEdgeInsetsMake(0.0, -60.0, 0.0, 60.0)
button.titleEdgeInsets = UIEdgeInsetsMake(0.0, -60.0, 0.0, 60.0)

Edit:
These insets would also work for the above example.
button.imageEdgeInsets = UIEdgeInsetsMake(0.0, -120.0, 0.0, 0.0)
button.titleEdgeInsets = UIEdgeInsetsMake(0.0, -120.0, 0.0, 0.0)
The button is centering its contents, so if you say "there are 2 more pts on the left" the button will move its contents over by 1 pt to keep them centered.
The rule is
-leftInset+rightInset=leftMargin*2