8

I think it is better if I explain the situation so this doesn't seem too arcane a question. I want to release some starter code for a project I want some of my students to work on. The project involves scraping through some internet webpages and as such, I want to provide them with a URLStream class that will download the html of an input url and return it as a string to them.

The issue is that I can't seem to find a particularly nice way to deal with networking in a way that will be cross platform (the students have mac/windows/linux machines). I know of libraries like Boost asio and libCurl, but the issue with using these is that I can't enforce all my students download them. So my question is twofold:

  1. Is there any nice way to provide them this cross platform networking code?
  2. If a library is the only way to do this, is there any way to attach the library to the starter project so that students don't have to download it? I know this might be a stupid question but I can't seem to find out if this is possible.
gowrath
  • 3,136
  • 2
  • 17
  • 32
  • What is the goal of the exercise? For the students to learn low-level network programing? Learn the HTTP? Or something else? Unless the goal is for the students (or for *you*) to learn low-level network programming, then I strongly suggest you use third-party libraries (like libcurl). It will make it much easier for both you and your students. And it will be much less maintenance needed from you, much less support and problems for you. – Some programmer dude Apr 21 '17 at 08:46
  • @Someprogrammerdude The goal of the exercise has nothing to do with networking. They are messing around with html at most. I just want them to be able to focus on the assignment without having to worry about networking at all. I am more than amenable to using 3rd party libraries, but this would involve me having to tell them to download and set up the libs (right?) - something I was trying to avoid because they shouldn't have to worry about these details. Can libcurl be attached to the starter project so they don't have to download it? – gowrath Apr 21 '17 at 08:50
  • If you have access to most of the systems the students have access to (a Windows system, a macOS system and a Linux system) then you can always pre-build a static library of e.g. libcurl that you distribute with your starter code, and give simple instructions on how to use the library when building. – Some programmer dude Apr 21 '17 at 08:56
  • But you *can* make your own simple HTTP functionality using low-level sockets, especially if you only want to make simple `GET` requests. The good news here is that all major platforms support [Berkeley sockets](https://en.wikipedia.org/wiki/Berkeley_sockets), it should be a one-to-one match on Linux and macOS, with Windows being only a little different. The downside being that if you haven't done low-level network programming yourself before it might take some time to get if right, and there will be a lot of boiler-plate code that you need to write. – Some programmer dude Apr 21 '17 at 08:57
  • @Someprogrammerdude Thank you for the detailed response. If you want to make it into an answer I can up vote it. Some of my requests have to go to wikipedia pages and I was having trouble getting those to work using raw sockets. Guess I'll double down my efforts. – gowrath Apr 21 '17 at 09:08

3 Answers3

5

Boost.Asio is really not suitable for your needs as it involves huge Boost and building at least some of its non-header-only libs. You can still consider Asio lib that can be used w/o Boost and is header-only lib, so much less hassle for you and your students. As it's probably the most popular and modern networking C++ lib this exercise can provide some useful experience to the students. Asio examples also have a simple HTTP client.

As a side note, are you bound to C++ for this assignment? It would be much simpler in Python or similar languages that provide networking out of the box.

Andriy Tylychko
  • 15,967
  • 6
  • 64
  • 112
  • so what's the topic of this assignment? *They are messing around with html at most.* maybe just give them offline HTML and call it done? – Andriy Tylychko Apr 21 '17 at 16:54
  • they are writing a page rank type program, so need to be able to follow arbitrary links. – gowrath Apr 21 '17 at 17:33
4

The Berkeley sockets API is the most common low-level socket API. It is supported on all POSIX platforms which means both Linux and macOS will have it.

Even Windows have it, but with a slight twist since sockets aren't descriptors like they are on POSIX systems.

Using sockets directly will lead to more boiler-plate code, but it is definitely possible to use it to make a simple HTTP client that supports only simple GET requests.

There are many tutorials and references on using sockets. Beej's Guide to Network Programming seems to be a popular tutorial, which should have notes about the tweaks needed for Windows.

Some programmer dude
  • 400,186
  • 35
  • 402
  • 621
0

cross-platform C++ library for network programming

asio is a cross-platform C++ library for network programming that provides developers with a consistent asynchronous I/O model using a modern C++ approach. It has recently been accepted into Boost.

I copied that from the info window in Synaptic. If you're using Linux, install the library (and its documentation) thus:

sudo apt-get install libasio-dev libasio-doc