cURL 404 behind proxy

Question

Just a simple file download test. Not working. Probaby because of the proxy at a Company. How can I configure the proxy? Before adding "127.0.0.1:8080" I always got error 7: Could not connect to server. Now I always get return code 0, but the test file json.txt contains only a 404. But the url is obviously working in my browser. I also tried to get proxy settings from system via this, but I failed.

HTTP/1.1 404 Not Found
Connection: Keep-Alive
Server: Embedthis-http
ETag: "0-0-56d6b6ea"
Cache-Control: no-cache
Last-Modified: Wed, 02 Mar 2016 09:48:26 GMT
Date: Fri, 19 May 2017 14:33:57 GMT
Content-Length: 167
Keep-Alive: timeout=60, max=199

<!DOCTYPE html>
<html><head><title>Not Found</title></head>
<body>
<h2>Access Error: 404 -- Not Found</h2>
<pre>Cannot locate document: /</pre>
</body>
</html>

Here is my cURL attempt:

CURL *curl;
FILE *fp;
char url[] = "http://www.carqueryapi.com/api/0.3/?callback=?&cmd=getMakes&year=2010&sold_in_us=1";
char outfilename[FILENAME_MAX] = "C:\\temp\\json.txt";
curl = curl_easy_init();
if (curl) {
    fp = fopen(outfilename,"wb");

    CURLcode res;
    curl_easy_setopt(curl, CURLOPT_URL, url );
    curl_easy_setopt(curl, CURLOPT_NOBODY, true);
    curl_easy_setopt(curl, CURLOPT_HEADER, true);
    curl_easy_setopt(curl, CURLOPT_HTTPGET, true);
    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
    curl_easy_setopt(curl, CURLOPT_WRITEDATA, fp);
    curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1 );
    curl_easy_setopt(curl, CURLOPT_PROXY, "127.0.0.1:8080" );

    res = curl_easy_perform(curl);
    cout << "Curl response code " << res << ": " << curl_easy_strerror( res ) << endl;

    curl_easy_cleanup(curl);

    fclose(fp);
}

UPDATE

The internet settings are configured to use the wpad.dat file to detect proxies. If I enter one of those, I get a response from the server, but it says the access is denied because the website is blocked. But my browser can display this site.

On the contrary, if I enter netsh.exe winhttp show proxy it says that the current winhttp proxy settings are "direct access (no proxy server)".

UPDATE 2

I finally got it working with those options: Used Verbose to get more helpful output; output showed some auth method like negotiate and ntlm; the ntlm option did not work but the negotiateone together with the userpwd : which takes the credentials from the system, if libcurl has the sspi module.

    curl_easy_setopt(curl, CURLOPT_URL, url );
    curl_easy_setopt(curl, CURLOPT_VERBOSE, 1 );
    curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1 );
    curl_easy_setopt(curl, CURLOPT_HEADER, 1);
    curl_easy_setopt(curl, CURLOPT_HTTPAUTH, CURLAUTH_ANY );
    curl_easy_setopt(curl, CURLOPT_PROXY, <"host:port"> );
    curl_easy_setopt(curl, CURLOPT_PROXYAUTH, CURLAUTH_NEGOTIATE );
    curl_easy_setopt(curl, CURLOPT_PROXYUSERPWD, ":" );

Edson K. Cury · Answer 1 · 2017-05-19T15:09:29.413

Try user agent and define header.

#define HEADER_ACCEPT "Accept:text/html,application/xhtml+xml,application/xml"
#define HEADER_USER_AGENT "User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.70 Safari/537.17"

struct curl_slist *list = NULL;
list = curl_slist_append(list, HEADER_USER_AGENT);
list = curl_slist_append(list, HEADER_ACCEPT);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, list);

You may need a real proxy, find a proxy sites, and retry with another ip

cURL 404 behind proxy

1 Answers1