What is the best module for HttpRequest in OCaml

Question

I wish to use OCaml to access the Yahoo Finance API. Essentially, it will be just a bunch of HTTP requests to get quotes from Yahoo Finance.

Which module I should use?

I wish to have async HTTP requests.

Did you consider `Ocamlnet` http://projects.camlcity.org/projects/ocamlnet.html — Basile Starynkevitch, Jan 03 '13 at 06:37
@BasileStarynkevitch Not really, I am a total new learner and know nothing. Is Ocamlnet the best? — Jackson Tale, Jan 03 '13 at 06:38
I've used to ocamlnet for this exact purpose recently there was no async ability. — rgrinberg, Jan 04 '13 at 19:54
@BasileStarynkevitch could you please give me a simple sample code fragment for: loading a url, get the http page string out of it? — Jackson Tale, May 14 '13 at 10:40

score 23 · Accepted Answer · edited Jan 13 '13 at 14:32

23

There are possibilities using lwt:

ocsigen has a quite complete and a bit complex implementation
cohttp is a bit simpler but lacks some usefull parts

using opam to install:

$ opam install ocsigenserver cohttp

For instance in a toplevel:

try Topdirs.dir_directory (Sys.getenv "OCAML_TOPLEVEL_PATH") with _ -> ();;
#use "topfind";;
#thread;;
#require "ocsigenserver";;
open Lwt

(* a simple function to access the content of the response *)
let content = function
  | { Ocsigen_http_frame.frame_content = Some v } ->
      Ocsigen_stream.string_of_stream 100000 (Ocsigen_stream.get v)
  | _ -> return ""

(* launch both requests in parallel *)
let t = Lwt_list.map_p Ocsigen_http_client.get_url
  [ "http://ocsigen.org/";
    "http://stackoverflow.com/" ]

(* maps the result through the content function *)
let t2 = t >>= Lwt_list.map_p content

(* launch the event loop *)
let result = Lwt_main.run t2

and using cohttp:

try Topdirs.dir_directory (Sys.getenv "OCAML_TOPLEVEL_PATH") with _ -> ();;
#use "topfind";;
#require "cohttp.lwt";;
open Lwt

(* a simple function to access the content of the response *)
let content = function
  | Some (_, body) -> Cohttp_lwt_unix.Body.string_of_body body
  | _ -> return ""

(* launch both requests in parallel *)
let t = Lwt_list.map_p Cohttp_lwt_unix.Client.get
  (List.map Uri.of_string
     [ "http://example.org/";
       "http://example2.org/" ])

(* maps the result through the content function *)
let t2 = t >>= Lwt_list.map_p content

(* launch the event loop *)
let v = Lwt_main.run t2

Notice that an implementation of cohttp for jane street async library is also available

edited Jan 13 '13 at 14:32

Thomas

5,047
19
30

answered Jan 03 '13 at 10:08

Pierre Chambart

1,469
12
17

2

The "+../toplevel" is a bit misleading here: this is not necessary for compilers installed by OPAM and it won't work for the system compiler (you should use `try Topdirs.dir_directory (Sys.getenv "OCAML_TOPLEVEL_PATH") with _ -> ()` instead). – Thomas Jan 03 '13 at 12:31
@Thomas I think your suggestion is very good. And I can't run the codes, but can with your `try Topdirs.dir_directory (Sys.getenv "OCAML_TOPLEVEL_PATH") with _ -> ()`. Could you please modify his other code? – Jack Jan 13 '13 at 13:36
@Thomas I am a newbie, could you please tell me what is `Some (_, body)`? – Jackson Tale Jan 14 '13 at 09:26
It's a pattern matching against an optional pair. In this case, the first element of the pair is discarded and the second one is bind to the name "body". Read about "pattern-matching" if you want to know more about that. – Thomas Jan 14 '13 at 11:04
@Thomas thanks for your reply. So `Some` is kind of keyword in `pattern-matching`? – Jackson Tale Jan 14 '13 at 15:02
@Thomas what is the meaning of `t >>=` in `let t2 = t >>= Lwt_list.map_p content`? – Jackson Tale Jan 14 '13 at 16:48
1

@JacksonTale It is some sugar for Lwt.bind. See [Lwt doc](http://ocsigen.org/lwt/api/Lwt#VALbind) – Pierre Chambart Jan 16 '13 at 10:18

score 3 · Answer 2 · answered Jan 09 '13 at 13:49

3

Just for the record, there is also ocurl with curl multi API support.

answered Jan 09 '13 at 13:49

ygrek

6,656
22
29

What is the best module for HttpRequest in OCaml

2 Answers2