How to get the HTTP response string using Curl in C++

Question

I'm very new to HTTP commands and the libcurl library. I know how to get the HTTP response code but not the HTTP response string. Following is the code snippet that I wrote to get the response code. Any help on how to get the response string will be highly appreciated!!!

curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
CURLcode ret = curl_easy_perform(curl);

if (ret != CURLE_OK) {
    LOG(INFO) << "Failed to perform the request. "
              << "Return code: " << ret;
    return false;
}

std::unique_ptr<int64_t> httpCode(new int64_t);
// Get the last response code.
ret = curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, httpCode.get());
if (ret != CURLE_OK) {
  LOG(INFO) << "curl_easy_getinfo failed to retrieve http code. "
            << "Return code: " << ret;
  return false;
}

I tried doing this as well to get the HTTP response string in readBuffer.

static size_t WriteCallback(char *contents, size_t size, size_t nmemb, void *userp)
{
    ((std::string*)userp)->append((char*)contents, size * nmemb);
    return size * nmemb;
}

std::string readBuffer;
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &readBuffer);

CURLcode ret = curl_easy_perform(curl);
cout << readBuffer << "\n";

But the readBuffer is empty. I don't understand where I am going wrong. Any pointers on how to solve this will be really nice!

Why do you need an `int64_t` pointer to capture a number? Just use `int status` and then `curl_easy_getinfo(..., &status)`. — tadman, Jul 09 '17 at 07:51
Thanks, I would do that but that does not solve the problem of how to get the HTTP response string! — Simran, Jul 09 '17 at 07:54
Possible duplicate of [Http status code with libcurl?](https://stackoverflow.com/questions/290996/http-status-code-with-libcurl) — tadman, Jul 09 '17 at 07:56
Again, that is about HTTP response **code** and not HTTP response **string**. — Simran, Jul 09 '17 at 07:58
Stuff like that is locale specific and isn't in the library as far as I can tell. Use a `std::map` to convert from the response code to the string you need. — tadman, Jul 09 '17 at 08:01
The HTTP response code is just a number like 200 or 404. I don't want to convert it into a string. I want to dig deeper and get the HTTP response string which will tell me what went wrong while sending the HTTP request. Converting the response code to the string is not what I want! — Simran, Jul 09 '17 at 08:05
You just said you wanted a string, and now you're saying you don't want a string? These things need to be interpreted, as while there are standards the server can return any number it pleases. Your code needs to understand what those mean as it pertains to the HTTP service you're calling. — tadman, Jul 09 '17 at 08:26
I guess there is some misunderstanding. I resolved it anyway. Had to replace `CURLOPT_WRITEFUNCTION` with `CURLOPT_HEADERFUNCTION` and `CURLOPT_WRITEDATA` with `CURLOPT_HEADERDATA` — Simran, Jul 09 '17 at 09:12

score 19 · Answer 1 · answered Jul 10 '17 at 16:46

There doesn't look to be much wrong with your code. I ran the following code, based on yours, to read the front page of the BBC news website:

#include <iostream>
#include <string>
#include <curl/curl.h>

size_t WriteCallback(char *contents, size_t size, size_t nmemb, void *userp)
{
    ((std::string*)userp)->append((char*)contents, size * nmemb);
    return size * nmemb;
}

int main(int argc, char * argv[])
{
    curl_global_init(CURL_GLOBAL_ALL);

    CURL* easyhandle = curl_easy_init();
    std::string readBuffer;

    curl_easy_setopt(easyhandle, CURLOPT_URL, "http://www.bbc.co.uk/news");
    curl_easy_setopt(easyhandle, CURLOPT_VERBOSE, 1L);
    curl_easy_setopt(easyhandle, CURLOPT_PROXY, "http://my.proxy.net");   // replace with your actual proxy
    curl_easy_setopt(easyhandle, CURLOPT_PROXYPORT, 8080L);
    curl_easy_setopt(easyhandle, CURLOPT_WRITEFUNCTION, WriteCallback);
    curl_easy_setopt(easyhandle, CURLOPT_WRITEDATA, &readBuffer);

    curl_easy_perform(easyhandle);

    std::cout << readBuffer << std::endl;

    return 0;
}

... and I got the full HTML response. NB I had to use a proxy, and I enabled verbose mode to see what was happening. Also NB; the HTTP server might be fussy about the URL you give it: if I replace http://www.bbc.co.uk/news with http://www.bbc.co.uk/news/ (i.e. with a trailing slash) then I get no data back (a response length of zero), as noted by the verbose curl output:

Host: www.bbc.co.uk
Accept: */*
Proxy-Connection: Keep-Alive

< HTTP/1.1 301 Moved Permanently
< X-Cache-Action: PASS (no-cache-control)
< Vary: Accept-Encoding
< X-Cache-Age: 0
< Content-Type: text/html;charset=utf-8
< Date: Mon, 10 Jul 2017 16:42:20 GMT
< Location: http://www.bbc.co.uk/news
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< Content-Length: 0
< X-Cache: MISS from barracuda1.[my proxy].net
< Connection: keep-alive
< 
* Connection #0 to host [my proxy] left intact

Here, Content-Length is 0 and the WriteCallback() function is never called. Hope this helps.

Thank you for the slash info! :) – Simran Jul 11 '17 at 09:21 — Simran, Jul 11 '17 at 09:21

score 4 · Answer 2 · edited Feb 20 '21 at 15:27

For the numerical response code, getinfo with CURLINFO_RESPONSE_CODE is the way to go:

long response_code;
curl_easy_getinfo(handle, CURLINFO_RESPONSE_CODE,&response_code);

However there is no equivalent getinfo capture for the server's response text. If you need the server's text, inspect the raw HTTP headers. There are two ways to do this:

Enable writing headers to the payload with CURLOPT_HEADER, then extract the headers from the combined payload, splitting the body on \n\n
Set a header callback function with CURLOPT_HEADERFUNCTION and parse directly from that

How to get the HTTP response string using Curl in C++

2 Answers2

Linked