Generally, the rule is to support the lowest version of the OS that provides the capabilities that enable you to deliver the product you want to. I made all of my applications require iPhone OS 3.0 about a month after that launched because I needed to support custom copy and paste, as well as in-application email. For the free version of one of my applications, I wanted to try out iAds, so I made that 4.0-only.
The possibilities for internal code improvements that are provided by a new OS are a little trickier to make rules about. As Joe points out, the time you spend struggling against an old way of doing things, that could be saved by moving to a new OS version, is time that you aren't fixing bugs or adding that next great new feature. At some point, the elegance of a new way of doing things (like blocks and GCD) is just too compelling to ignore, even though it may not directly add any new features to your application.
On the Mac, many developers maintain support for the current version of the OS plus the previous one released by Apple (Snow Leopard and Leopard, at this time). Only when a new OS comes out do they drop support for the previously one-behind version. Others, like Wil Shipley, advocate jumping on board the new OS exclusively right away and ignoring old versions. The argument here is that people who won't pay for Apple's new OS versions or who don't keep their systems up to date are much less likely to buy your third-party application.
I've seen this as well on iOS. For example, I ran a test of ads targeted to different versions around the launch of 3.0 (when iPod touch users still had to pay for their OS updates). While downloads of a free version of my application were relatively consistent between 2.x vs 3.x users, almost no 2.x users paid for the application while 3.x users did.
No one complained among my paying users about the early move to 3.0 (they did speak well about the new features, though) and only two people have complained about my free version going 4.0-only, compared to 56,000 that have upgraded without a problem. While I would give a little time for a transition period, I believe in moving to new OS versions relatively soon after they launch. Right now, I'm in the process of making everything of mine 4.0-only so that I can modernize the codebase.
Finally, one last benefit you get from going with the new OS version is that you become much more attractive to Apple, who always wants to promote applications using new features in their new devices and OS versions.