Boost Asio https synchronous call- Error code 400 bad request

Question

we are migrating from http to https boost asio sysnchornous call and I am using the below code to make https synchoronous call with ssl certificate validation. we got our client certificate issued by certiticate authority and we downloaded it in .pem format. we have the following questions:

1.) how to load the certificates in boost asio; can we load the certificate file with the path as below:

boost::asio::streambuf response_;
boost::asio::ssl::context ctx(boost::asio::ssl::context::sslv23);
ctx.set_verify_mode(boost::asio::ssl::verify_peer);
//ctx.set_default_verify_paths();
**ctx.load_verify_file("/tmp/cacert.pem");**
ctx.set_options(boost::asio::ssl::context::default_workarounds |
       boost::asio::ssl::context::no_sslv2 |
       boost::asio::ssl::context::no_sslv3);
boost::asio::ssl::stream<boost::asio::ip::tcp::socket> socket(io_service,ctx);

2.)what is the purpose of peer verification in synchoronous https call; can we make handshake without peer verification as like below?

tcp::resolver resolver(io_service);
tcp::resolver::query query(hostname, port_no);

tcp::resolver::iterator endpoint_iterator = resolver.resolve(query);
tcp::resolver::iterator end;

boost::system::error_code error  = boost::asio::error::host_not_found;
boost::asio::connect(socket.lowest_layer(), endpoint_iterator, error);
socket.handshake(boost::asio::ssl::stream_base::client);

3.) I am getting Bad request Error code 400 when i am hitting endpoint url with ssl verification. Please verify the below code and let me know if i am missing related to ssl certificate part(note: request header and message worked fine before changing to https):

boost::asio::streambuf response_;
boost::asio::ssl::context ctx(boost::asio::ssl::context::sslv23);
ctx.set_verify_mode(boost::asio::ssl::verify_peer);
ctx.load_verify_file("/tmp/cacert.pem");
ctx.set_options(boost::asio::ssl::context::default_workarounds |
       boost::asio::ssl::context::no_sslv2 |
       boost::asio::ssl::context::no_sslv3);
boost::asio::ssl::stream<boost::asio::ip::tcp::socket> socket(io_service,ctx);

std::ostream request_stream(&request_);
request_stream << "POST " << server_endpoint << " HTTP/1.1\n";
request_stream << "Host: " << hostname << "\n";
request_stream << "Accept: */*\n";
request_stream << authorization_token << "\n";
request_stream << client_name << "\n";
request_stream << "Content-Length: " << req_str.length() << "\n";
request_stream << "Content-Type: application/x-www-form-urlencoded \n";
request_stream << "Connection: close\r\n\r\n";
request_stream << req_str << "\n";
tcp::resolver resolver(io_service);
tcp::resolver::query query(hostname, port_no);

tcp::resolver::iterator endpoint_iterator = resolver.resolve(query);
tcp::resolver::iterator end;

boost::system::error_code error  = boost::asio::error::host_not_found;
boost::asio::connect(socket.lowest_layer(), endpoint_iterator, error);
socket.handshake(boost::asio::ssl::stream_base::client);

Thanks

Answer 1

Your request is bad :)

HTTP requires CR+LF line ends. So, everywhere you use \n alone it needs to be \r\n.

Complete Working Sample

I completed your sample in order to test it. I may (?) have answered your question about certificates - or partly - by using

    //ctx.load_verify_file("ca.pem");
    ctx.add_verify_path("/etc/ssl/certs");

This uses the location where most Linux systems will store the default certificate store, meaning the certificates that are typically trusted system-wide.

Listing:

#include <boost/asio.hpp>
#include <boost/asio/ssl.hpp>
#include <iostream>
using boost::asio::ip::tcp;

// https://postman-echo.com/post see https://docs.postman-echo.com/?version=latest
static const std::string
    server_endpoint = "/post",
    hostname = "postman-echo.com",
    port_no = "443",
    authorization_token =
        "Auth: "
        "c3RhdGljIGNvbnN0IHN0ZDo6c3RyaW5nIGF1dGhvcml6YXRpb"
        "25fdG9rZW4gPSAiQXV0aDogIj"
        "sK",
    client_name = "User-Agent: demo program 0.01",
    req_str = R"(name=blabla&password=bloblo)";

int main() {
    boost::asio::io_service io_service;
    boost::asio::ssl::context ctx(boost::asio::ssl::context::sslv23);

    ctx.set_verify_mode(boost::asio::ssl::verify_peer);
    //ctx.load_verify_file("ca.pem");
    ctx.add_verify_path("/etc/ssl/certs");

    ctx.set_options(boost::asio::ssl::context::default_workarounds |
                    boost::asio::ssl::context::no_sslv2 |
                    boost::asio::ssl::context::no_sslv3);
    boost::asio::ssl::stream<boost::asio::ip::tcp::socket> socket(io_service,
                                                                  ctx);

    {
        tcp::resolver resolver(io_service);
        tcp::resolver::query query(hostname, port_no);

        tcp::resolver::iterator endpoint_iterator = resolver.resolve(query);
        tcp::resolver::iterator end;

        boost::system::error_code error = boost::asio::error::host_not_found;
        boost::asio::connect(socket.lowest_layer(), endpoint_iterator, error);
    }

    {
        boost::asio::streambuf request_;
        socket.handshake(boost::asio::ssl::stream_base::client);
        {
            std::ostream request_stream(&request_);
            request_stream << "POST " << server_endpoint << " HTTP/1.1\r\n";
            request_stream << "Host: " << hostname << "\r\n";
            request_stream << "Accept: */*\r\n";
            request_stream << authorization_token << "\r\n";
            request_stream << client_name << "\r\n";
            request_stream << "Content-Length: " << req_str.length() << "\r\n";
            request_stream << "Content-Type: application/x-www-form-urlencoded \r\n";
            request_stream << "Connection: close\r\n\r\n";
            request_stream << req_str << "\r\n";
        } // forces flush()
        //std::cout << &request_;
        //std::cout << "--------" << std::endl;

        write(socket, request_);
        //socket.lowest_layer().shutdown(tcp::socket::shutdown_send);
    }

    {
        boost::asio::streambuf response_;
        boost::system::error_code ec;
        read(socket, response_, ec);
        
        std::cout << "ec: " << ec.message() << "\n";
        std::cout << &response_ << "\n";
    }
}

It uses the postman online sample service and prints:

ec: stream truncated
HTTP/1.1 200 OK
Date: Fri, 07 Aug 2020 16:23:04 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 519
Connection: close
ETag: W/"207-JMbCSlSXSCnZPMi2WQ8SuP+keys"
Vary: Accept-Encoding
set-cookie: sails.sid=s%3AFBof16WW2UeR2Si6dtf9WRUfKiJbpIhH.O2YgXPhClKJKnJ0bmTFuyl%2FyKNyS3oADFbDHHt4UKX8; Path=/; HttpOnly

{"args":{},"data":"","files":{},"form":{"name":"blabla","password":"bloblo"},"headers":{"x-forwarded-proto":"https","x-forwarded-port":"443","host":"postman-echo.com","x-amzn-trace-id":"Root=1-5f2d7fe8-0348dee860e746ac828f4d80","content-length":"27","accept":"*/*","auth":"c3RhdGljIGNvbnN0IHN0ZDo6c3RyaW5nIGF1dGhvcml6YXRpb25fdG9rZW4gPSAiQXV0aDogIjsK","user-agent":"demo program 0.01","content-type":"application/x-www-form-urlencoded"},"json":{"name":"blabla","password":"bloblo"},"url":"https://postman-echo.com/post"}

The stream truncated is probably expected: https://github.com/boostorg/beast/issues/38