0

I've written a http server that only sends back compressed http responses:

https://github.com/ericcurtin/asio/commit/1d37a1d225d1e812a747b595c02f9770ebd75dd0

So if you you use curl to request the data and decompress the response by piping through gunzip it works fine:

curl -x "" 127.0.0.1:5000/main.cpp --output - | gunzip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   758  100   758    0     0   740k      0 --:--:-- --:--:-- --:--:--  740k
// g++ -O0 main.cpp server.cpp connection_manager.cpp request_handler.cpp
// connection.cpp reply.cpp mime_types.cpp request_parser.cpp -lboost_system
// -lpthread -lz
//
// run like: ./a.out 0.0.0.0 5000 .
//
// main.cpp
// ~~~~~~~~
//
// Copyright (c) 2003-2017 Christopher M. Kohlhoff (chris at kohlhoff dot com)
//
// Distributed under the Boost Software License, Version 1.0. (See accompanying
// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
//

#include <iostream>
#include <string>
#include <boost/asio.hpp>
#include "server.hpp"

int main(int argc, char* argv[])
{
  try
  {
    // Check command line arguments.
    if (argc != 4)
    {
      std::cerr << "Usage: http_server <address> <port> <doc_root>\n";
      std::cerr << "  For IPv4, try:\n";
      std::cerr << "    receiver 0.0.0.0 80 .\n";
      std::cerr << "  For IPv6, try:\n";
      std::cerr << "    receiver 0::0 80 .\n";
      return 1;
    }

    // Initialise the server.
    http::server::server s(argv[1], argv[2], argv[3]);

    // Run the server until stopped.
    s.run();
  }
  catch (std::exception& e)
  {
    std::cerr << "exception: " << e.what() << "\n";
  }

  return 0;
}

But if you use curl using --compressed which works with other http servers like the one at example.com it fails after the first 512 bytes:

curl -x "" 127.0.0.1:5000/main.cpp --compressed
// g++ -O0 main.cpp server.cpp connection_manager.cpp request_handler.cpp
// connection.cpp reply.cpp mime_types.cpp request_parser.cpp -lboost_system
// -lpthread -lz
//
// run like: ./a.out 0.0.0.0 5000 .
//
// main.cpp
// ~~~~~~~~
//
// Copyright (c) 2003-2017 Christopher M. Kohlhoff (chris at kohlhoff dot com)
//
// Distributed under the Boost Software License, Version 1.0. (See accompanying
// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
//

#include <iostream>
curl: (23) Failed writing received data to disk/application
#include <string

Any idea on how my compression could be fixed?

Some programmer dude
  • 400,186
  • 35
  • 402
  • 621
ericcurtin
  • 1,499
  • 17
  • 20
  • What headers are you sending? How do you send the headers? How do you compress the data being sent? How do you send the data? – Some programmer dude Jul 04 '19 at 09:58
  • from curl doc "--compressed (HTTP) Request a compressed response using one of the algorithms curl supports, and save the uncompressed document. If this option is used and the server sends an unsupported encoding, curl will report an error." – RaGa__M Jul 04 '19 at 09:58
  • This is how I send the data 'curl -x "" 127.0.0.1:5000/main.cpp --compressed'... My http server sends compressed data back regardless of the headers – ericcurtin Jul 04 '19 at 09:59
  • I am used a supported encoding (gzip) and it at least starts to decompress when using --compressed, but fails after the first 512 bytes. I forgot to add I'm on xubuntu 18.04, although that should be of little difference – ericcurtin Jul 04 '19 at 10:01
  • 1
    "This is how I send the data 'curl -x "" 127.0.0.1:5000/main.cpp --compressed'... " No, that's how you *receive* the data. Your program is sending it some way, how do you send it? And again, what headers do you send? How do you send the headers? – Some programmer dude Jul 04 '19 at 10:08
  • Maybe you should take some time to refresh [the help pages](http://stackoverflow.com/help), take [the SO tour](http://stackoverflow.com/tour), read about [how to ask good questions](http://stackoverflow.com/help/how-to-ask), as well as [this question checklist](https://codeblog.jonskeet.uk/2012/11/24/stack-overflow-question-checklist/). And lastly please don't forget how to create a [mcve] to show us. – Some programmer dude Jul 04 '19 at 10:09
  • On I see my bad, I thought you were an about how I send the request. It's all in the git commit above, these are the headers I send: rep.headers[0].name = "Content-Length"; rep.headers[0].value = std::to_string(rep.content.size()); rep.headers[1].name = "Content-Type"; rep.headers[1].value = mime_types::extension_to_type(extension); rep.headers[2].name = "Content-Encoding"; rep.headers[2].value = "gzip"; – ericcurtin Jul 04 '19 at 10:11
  • This is a minimal reproducible example, unfortunately you cannot write a http server in C/C++ in less than 100 lines of code. So I shared you a simple patch to reproduce the issue. – ericcurtin Jul 04 '19 at 10:14
  • Possibly related: https://stackoverflow.com/a/8365089/85371 – sehe Jul 04 '19 at 10:58
  • curl --compressed "http://example.com" works fine as expected, there's something wrong with my code change in the git commit, that I can't put my finger on. – ericcurtin Jul 04 '19 at 13:10

0 Answers0