You don't need to maintain multiple connections for what you're trying to achieve. Javascript's asynchronous nature allows the connection to stay alive while other things are being processed. You may have thought that javascript's XHR's were blocking because it is single threaded.
The XHR's in javascript are non-blocking because of the event loop model - the javascript engines are constantly in a loop, checking to see whether registered calls have completed and whether its callbacks should be processed. This allows its operations to be non-blocking and thus allowing a single javascript application to process multiple XHR long polling requests.
If you are able to use jQuery for the request, it wraps the XHR quite nicely with this function: http://api.jquery.com/jQuery.ajax/. With this, you can define a timeout of 20 seconds and handle it immediately (to reinitiate the connection with your server).
Aside - You may wish to consider your server stack to optimize long polling. Make sure that your web server does not spawn a thread per request (like Apache 2.2) - otherwise you will quickly run out of system resources! If you can use node.js (which is great for handling many simultaneous requests), look into socket.io library as a server-side and client-side solution (http://socket.io/#home).