"In my experience, servers that are heavy on dynamic content tend to be impacted less by HTTPS because the time spent encrypting (SSL-overhead) is insignificant compared to content generation time."
"Making lots of short requests over HTTPS will be quite a bit slower than HTTP, but if you transfer a lot of data in a single request, the difference will be insignificant."
So there should be no major performance hit to your server, but the request times will be longer as you need to handshake every request. BUT you should read for more information:
HTTP vs HTTPS performance