It is not necessary, but it is, like you said, considered a best practice.
1+2) Having same controllers with respond_to/respond_with logic is a nice idea at first sight. But, from my experience, I can say, there always comes a day where API code start to differ with HTML client code. The mobile client might have a different UI and it is just natural that it will expect to consume your data another way as your web client does. The web client is specialized to one use case where an API should be more generic allowing multiple consuming ways.
The second issue that will arise is the fact that you cannot rely on your mobile users to always have the latest app version where with a webapp you can. So for the HTML app you can easily introduce non-compatible changes because you are delivering a proper client right within where for the mobile API breaking the API is at least concerning. Perhaps, you will want to maintain a backwards compatibility which will make your all purpose controllers ugly as hell. And without a proper api/v1 namespace you even can't have two different API versions at the same time.
You can avoid duplication of your logic by keeping your controllers very skinny and move the logic out into models (Service Objects are models too, not only Active Records).
3) Your mobile HTTP lib will to a high probability have a proper automatic cookie management. Having token based authentication is just again a best practice. If it is just a token vs session_id within cookie, there will be not much win. I can only think that it will be automatically secure against CSRF attack and you can disable this protection entirely for the API because your website users won't be allowed to consume the API, just by logging in to the site (an additional benefit perhaps). With session based authentication you will have to generate a CSRF token on first API request and set it within X-CSRF-Token
cookie.
The big advantage of token based authentication is that it is extendable to more security, like introducing expire tokens, HMAC tokens etc, whereby session authentication is not.
See Using Sessions vs Tokens for API authentication
I would also encourage you to look at json:api. It comes from the creators of ember.js, who have thought about minifying decisions to take, when building APIs. Another interesting thing is an active_model_serializers gem. An intro to it is given within Rails: The Next Five Years by Yehuda Katz