I'm not sure if you are asking about whether it is too many to have 20 handlers defined, or have 20 handlers invoked from jQuery, so I will address both.
In terms of defining many handlers, a generic HTTP handler (ASHX) is similar to the ASP.NET page handler (ASPX), but more lightweight in that it does not have the full lifecycle of a page, and is not intended for returning UI. Many large-scale applications have hundreds of ASPX pages defined, which is consistent with the design intention of ASP.NET Web Forms, where every UI page is a distinct ASPX. So, to have hundreds of ASHX, would be even less heavy than hundreds of ASPX, and no problem at all.
In terms of invoking 20 handlers, here we get into the conversation about "chunky" versus "chatty" interfaces. When interfacing over a WAN (i.e., between browser and server), a "chunky" interface (one which makes a smaller number of heavier calls) is better: when you try to scale your application, a "chatty" interface (one which makes a higher number of lighter calls) will hold open many more connections on the server, will often cause more load on the database in terms of a higher number of transactions and a higher number of open simultaneous connections, and therefore will generally not scale as well on the server side.
On the browser side, the news is even worse. Per HTTP specification, browsers limit you to two simultaneous requests, so if you mean to fire off your 20 requests all at once, it will not happen, which means you may get some performance problem from having so many jquery get/post calls queueing up at one time.
The tradeoff, of course, is that often the programming is often cleaner with a "chatty" interface. So here you must make the judgement about your future scaling needs, versus the importance of cleaner code.
I would say if you're building an application that, for its expected life and evolution, can comfortably run all of its traffic on a single web and single database server; AND, your browser code is set up in such a way that the 2 simultaneous requests will not cause you any performance issue, then it is reasonable to go for the "chatty" interface if it gives you much cleaner code.
But if you expect there to be a need for scaling beyond a single server; OR, there are common use cases where many of these jquery get/posts will be invoked simultaneously and hamper performance, then by all means I would refactor to a more "chunky" interface, which would mean not calling more than 20 handlers from a single page via jQuery.
If you've read this and still can't decide which is right, then I would recommend refactoring the interface to make it more "chunky".
Hope this helps, and best of luck to you!