Using boost asio to get web page

Question

I'm trying to build a program that would take in a stock ticker, run a google search for it, and output data (current price, high, low, percent change, etc.). I am trying to use boost asio and it is not returning any data from the server.

#include "stdafx.h"
#include <iostream>
#include <istream>
#include <ostream>
#include <string>
#include <boost/asio.hpp>

std::string getStockPage(std::string ticker) {
    boost::asio::ip::tcp::iostream stream;

    stream.connect("www.google.com", "http");
    std::cout << "connected\n";
    stream << "GET /search?q=" << ticker << " HTTP/1.1\r\n";
    stream << "Host: www.google.com\r\n";
    stream << "Cache-Control: no-cache\r\n";
    //stream << "Content-Type: application/x-www-form-urlencoded\r\n\r\n";
    stream << "Connection: close\r\n\r\n";
    std::cout << "sent\n";

    std::ostringstream os;
    //os << stream.rdbuf();
    char buffer[100];
    os << stream.readsome(buffer, 100);
    return std::string(buffer, 100);
}

int main() {
    std::cout << getStockPage("$tsla");
    std::cout << "done\n";
    std::string temp;
    std::getline(std::cin, temp);
    return 0;


}

I tried to read just the first 100 characters to see if it was having issues outputting the response, but it only outputs null characters. I want it to output the entirety of the google page "www.google.com/search?q=$tsla"

Any help would be greatly appreciated!

http://www.boost.org/doc/libs/1_65_1/doc/html/boost_asio/example/cpp03/http/client/sync_client.cpp — Henri Menke, Dec 08 '17 at 03:13
Possible duplicate of [Sending http GET request using boost::asio, similar to cURL](https://stackoverflow.com/questions/28728347/sending-http-get-request-using-boostasio-similar-to-curl) — Henri Menke, Dec 08 '17 at 03:13

score 1 · Answer 1 · answered Dec 08 '17 at 07:48

std::istream::readsome is allowed to just always return 0 bytes. Then, it appears as if you received NUL bytes, because you did

return std::string(buffer, 100);

instead of

return std::string(buffer, stream.gcount());

Really, just use the other approach

std::ostringstream os;
os << stream.rdbuf();
return os.str();

This works for me when testing. Note you could add a flush:

stream << "Connection: close\r\n\r\n" << std::flush;

The resulting program

#include <boost/asio.hpp>
#include <iostream>
#include <string>

std::string getStockPage(std::string const& ticker) {
    boost::asio::ip::tcp::iostream stream;

    stream.connect("www.google.com", "http");
    stream    << "GET /search?q=" << ticker << " HTTP/1.1\r\n";
    stream    << "Host: www.google.com\r\n";
    stream    << "Cache-Control: no-cache\r\n";
    // stream << "Content-Type: application/x-www-form-urlencoded\r\n\r\n";
    stream    << "Connection: close\r\n\r\n" << std::flush;

    std::ostringstream os;
    os << stream.rdbuf();
    return os.str();
}

int main() {
    std::cout << getStockPage("$tsla");
}

Is printing

HTTP/1.1 302 Found
Location: http://www.google.nl/search?q=%24tsla&gws_rd=cr&dcr=0&ei=3EMqWrKxCILUwAKv9LqICg
Cache-Control: private
Content-Type: text/html; charset=UTF-8
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Date: Fri, 08 Dec 2017 07:48:44 GMT
Server: gws
Content-Length: 288
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Set-Cookie: NID=118=MsVZZpoZFEz4mQDqDuuWFRViB8v8yEQju7FPdOw8Rr7ViQ1cJtF6ZeN9u-dSRhGMT4x8F8yDilk9FqsoTkO8IsoQX-YvHXRcCoHcOLk0p4VOTn8AZoldKeh84Ryl0bM0; expires=Sat, 09-Jun-2018 07:48:44 GMT; path=/; domain=.google.com; HttpOnly
Connection: close

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="http://www.google.nl/search?q=%24tsla&amp;gws_rd=cr&amp;dcr=0&amp;ei=3EMqWrKxCILUwAKv9LqICg">here</A>.
</BODY></HTML>

Using boost asio to get web page

1 Answers1

Linked