1

I am making https synchoronous call using the boost asio as described in Boost Asio https synchronous call- Error code 400 bad request and now we are getting response code but while reading it from the socket, i am getting short read error:

   boost::asio::write(socket, request_);
   boost::asio::read_until(socket, response_, "\r\n");
    string res=make_string(response_);

    // Check that response is OK.
    std::istream response_stream(&response_);
    std::string http_version;
    response_stream >> http_version;
    unsigned int status_code;
    response_stream >> status_code;
    std::string status_message;
    std::getline(response_stream, status_message);

    if (!response_stream || http_version.substr(0, 5) != "HTTP/")
    {
            PBLOG_WARN("Invalid response\n");
    }
    if (status_code != 200)
    {
            fast_ostringstream oss;
            oss << "Response returned with status code: " << status_code << "\n";
            PBLOG_WARN(oss.str());
    }

    boost::asio::read(socket, response_, boost::asio::transfer_all(), error);
    if (error != boost::asio::error::eof)
    {
        fast_ostringstream oss;
        oss << "Error : " << error.message();
        PBLOG_WARN(oss.str());
        return false;
    }
    else
    {
    //parse the original resposne
    }

in the above logic, it goes to if loop and getting the error as Error : short read. Please help me in fixing this issue.

Thanks

  • Come on. I've been asking you for self contained code. Not only do you not provide that (while I practically include one ready-made in my [previous answer](https://stackoverflow.com/questions/63300832/boost-asio-https-synchronous-call-error-code-400-bad-request#comment112112210_63304226)), but you leave out the essentials even here. It makes a huge difference how you implement `make_string`. Something tells me you are doing that incorrectly since you don't pass the return value of `read_until`. Of course, you MIGHT be doing it correctly still by duplicating the `\r\n` delimiter logic. – sehe Aug 14 '20 at 11:16

1 Answers1

1

As I explained in the other answer, short read ("stream_truncaed") is to be expected when servers are implementing shutdown by simply closing the connection. The regular SSL shutdown is missing in such a case, prompting a "Short Read".

The issue I linked back there describes how to treat the error in the context of HTTP response parsing - and concludes that it is safe to ignore when the response has been parsed completely.

In your case, you DO NOT explicitly parse the response, so you can't really make an informed decision. This leaves you open to potential abuse where you can be made to accept malformed requests by an attacker.

1. Tired (workaround)

Uour naive workaround would be:

Live On Coliru

std::istream response_stream(&response_);

if (std::getline(response_stream >> http_version >> status_code, status_message)
        && http_version.substr(0, 5) == "HTTP/")
{
    if (status_code != 200) {
        std::clog << "Response returned with status code: " << status_code << "\n";
    }

    boost::system::error_code error;
    boost::asio::read(socket, response_, boost::asio::transfer_all(), error);

    if (error != boost::asio::error::eof &&
        error != boost::asio::ssl::error::stream_truncated)
    {
        std::clog << "Error : " << error.message() << std::endl;
        return 1;
    }
    else
    {
        //parse the original resposne
        
    }
} else {
    std::clog << "Invalid response\n" << std::endl;
}

Prints

Payload:
----
<!DOCTYPE html>
<html>
<title>Coliru Viewer</title>
<head>
    ....
    ....
</body>
</html>

----

2. Wired: correct implementation

However, I suggest you up your game and use Beast to parse the response. This has the benefit that you can trust the response. It parses the expected protocol, rather than blindly waiting for EOF (which might never come?) and scratching your head why that results in short reads.

Since we now stop reading once the response is complete, we don't encounter EOF nor the Short Read.

Note: this method also makes a lot of other things less error-prone and way less insecure. Things like encoding header values (which you don't), proper use of CRLF (which was an error in your previous question), correct calculation of content length etc.

On the response parsing you - randomly - check that HTTP version starts with "HTTP/". However, using Beast you will actually check input sanity and change behaviour accordingly. This means that if your server does something you didn't expect (like, you know, actually using HTTP/1.1 features like chunked encoding) you will still read the response, as if you totally knew about that.

Conversely, if someone attacks your server by sending invalid responses, you have a better chance of not crashing or worse.

Live On Coliru

#include <boost/asio.hpp>
#include <boost/beast/http.hpp>
#include <boost/beast/core/flat_buffer.hpp>
#include <boost/asio/ssl.hpp>
#include <iostream>
#include <iomanip>

namespace http = boost::beast::http;
using boost::asio::ip::tcp;

// https://postman-echo.com/post see https://docs.postman-echo.com/?version=latest
static std::string
    verb = "POST",
    server_endpoint = "/post",
    hostname = "postman-echo.com",
    port_no = "443",
    authorization_token = "c3RhdGljIGNvbnN0IHN0ZDo6c3RyaW5nIGF1dGhvcml6YXRpb"
        "25fdG9rZW4gPSAiQXV0aDogIj"
        "sK",
    client_name = "demo program 0.01",
    req_str = R"(name=blabla&password=bloblo)";

int main() {
    boost::asio::io_service io_service;
    boost::asio::ssl::context ctx(boost::asio::ssl::context::sslv23);

    ctx.set_verify_mode(boost::asio::ssl::verify_peer);
    //ctx.load_verify_file("ca.pem");
    ctx.add_verify_path("/etc/ssl/certs");

    ctx.set_options(boost::asio::ssl::context::default_workarounds |
                    boost::asio::ssl::context::no_sslv2 |
                    boost::asio::ssl::context::no_sslv3);
    boost::asio::ssl::stream<boost::asio::ip::tcp::socket>
        socket(io_service, ctx);

    {
#ifdef COLIRU
        verb = "GET";
        server_endpoint = "/a/cf2748285fa3343a";
        hostname = "coliru.stacked-crooked.com";

        socket.set_verify_mode(boost::asio::ssl::verify_none);
        socket.lowest_layer().connect({ boost::asio::ip::address::from_string("173.203.57.63"), 443 });
#else
        tcp::resolver resolver(io_service);
        tcp::resolver::query query(hostname, port_no);

        tcp::resolver::iterator endpoint_iterator = resolver.resolve(query);
        tcp::resolver::iterator end;

        boost::system::error_code error = boost::asio::error::host_not_found;
        boost::asio::connect(socket.lowest_layer(), endpoint_iterator, error);
#endif
    }

    {
        socket.handshake(boost::asio::ssl::stream_base::client);

        using http::field;
        http::request<http::string_body> request;
        request.method_string(::verb);
        request.target(server_endpoint);
        request.version(11);
        request.set(field::host, hostname);
        request.set(field::accept, "*/*");
        request.set(field::authorization, authorization_token);
        request.set(field::user_agent, client_name);
        request.set(field::content_type, "application/x-www-form-urlencoded");
        request.set(field::connection, field::close);
        request.body() = req_str;
        request.prepare_payload();

        write(socket, request);
        // std::clog << request << "\n"; return 1;
    }

    {
        boost::system::error_code ec;
        using http::field;
        http::response<http::string_body> response_;
        boost::beast::flat_buffer buf;
        read(socket, buf, response_, ec);
        
        {
            //std::clog << "Response: " << response_ << std::endl;
        }

        // Check that response is OK.
        if (!ec && response_.version() == 11) {
            if (response_.result() != http::status::ok) {
                std::clog << "Response returned with status code: " << response_.result_int() << "\n";
            }

            std::clog << "Payload:\n----\n" << response_.body() << "\n----" << std::endl;
        } else {
            std::clog << "Error: " << ec.message() << "\n";
        }
    }
}

Summary

Lessons learned:

  • Don't underestimate the complexity of HTTP or SSL
  • Don't reinvent the wheel - use libraries to your advantage!

Happy learning!

sehe
  • 374,641
  • 47
  • 450
  • 633
  • thanks sehe. I will ahve a look to change to boost beast and i have raised another query on the extra character https://stackoverflow.com/questions/63417956/boost-asio-synchronous-https-call-json-response-have-unintended-character. will you able to have a look on that? – Krishna Moorthy Aug 14 '20 at 18:17
  • @KrishnaMoorthy I'd really much prefer to get a self-contained example, or just the task. You know, it's one thing we have to come up with the solutions ("doing your work"), but it's a little weird that we're **also** having to imagine the problem with it. You're making more expensive than necessary to answer your questions, I put my $0.02 there because I think it might well be the same solution. – sehe Aug 14 '20 at 19:26
  • Please also consider to vote/accept https://meta.stackexchange.com/questions/5234/how-does-accepting-an-answer-work – sehe Aug 14 '20 at 19:27