You lose your CallContext if ASP.Net switches threads. In an asynchronous model, the asp.net task scheduler will take care of joining async calls back to a request thread with the same HttpContext, but not necessarily the same thread.
Example: A request starts and then you go off to asynchronously wait for some slow IO before returning - while you're waiting for that slow IO there's no reason for your request thread to be sitting around doing nothing so it may get used for another request.
ASP.Net is a big exercise in Thread Agility
(google it), and there is also a great discussion about this here: CallContext vs ThreadStatic vs HttpContext