OCaml: Lwt and non blocking socket

Question

I wanted to try the Lwt_unix module for a simple client that reads data in a socket until there is nothing to read. Some told me that Lwt create non blocking sockets but with my code, it is still blocking:

open Lwt
open Unix

(* ocamlfind ocamlc -o lwt_socket_client -package lwt,lwt.unix,unix  -linkpkg -g lwt_socket_client.ml *)
let host = Unix.inet_addr_loopback 
let port = 6600

let create_socket () =
  let sock = Lwt_unix.socket PF_INET SOCK_STREAM 0 in
  Lwt_unix.set_blocking sock false;
  sock

let s_read sock maxlen =
  let str = Bytes.create maxlen in
  let rec _read sock acc =
    Lwt.ignore_result(Lwt_io.write_line Lwt_io.stdout "_read");
    Lwt_unix.read sock str 0 maxlen >>= fun recvlen ->
    Lwt.ignore_result(Lwt_io.write_line Lwt_io.stdout (string_of_int recvlen));
    if recvlen = 0 then Lwt.return (acc)
    else _read sock (acc ^ (String.sub str 0 recvlen))
  in _read sock ""

let socket_read sock =
  Lwt.ignore_result(Lwt_unix.connect sock @@ ADDR_INET(host, port));
  s_read sock 1024 >>= fun answer ->
  Lwt_io.write_line Lwt_io.stdout answer

let () =
  let sock = create_socket () in
    Lwt_main.run (socket_read sock)

If I try this example with in a term:

echo "totoche" | netcat -l 127.0.0.1 -p 6600

then the result is :

./lwt_socket_client
_read
8
_read

Which block until I hit Ctrl+c.

I have tried both with :

Lwt_unix.set_blocking sock false;

and

Lwt_unix.set_blocking sock true;

and of course without this line, but it is still blocking. What am I doing wrong?

For more informations, one of my previous question : OCaml non-blocking client socket

Note: `ignore_result x; ...` means "do this in the background without waiting for it". You probaby want `x >>= fun () -> ...` (wait for x to finish, then ...) — Thomas Leonard, Sep 25 '16 at 20:32

antron · Accepted Answer · 2016-09-27T14:14:32.963

Conceptually, Lwt_unix.read always blocks the Lwt thread, but never blocks the whole process – unless the process is waiting for that Lwt thread, and there are no other Lwt threads to run. Lwt_unix.set_blocking does not affect this behavior. It just changes the settings on the underlying socket, and therefore the strategy used internally by Lwt to avoid blocking the process.

So, as mentioned by @ThomasLeonard, the "idiomatic Lwt" way to do a non-blocking read (from the perspective of the process) is simply to run additional Lwt threads concurrently with the Lwt_unix.read.

Concerning the specific code in the question, the underlying read system call fails with EAGAIN or EWOULDBLOCK (depending on the system) if the underlying socket is non-blocking, but no data is available – rather than succeeding with zero bytes read, which indicates the socket has been closed.

Unix.read converts this into an exception Unix.Unix_error Unix.EAGAIN (respectively, Unix.Unix_error Unix.EWOULDBLOCK). Lwt_unix.read retries Unix.read in this case. So, you cannot (currently) directly respond to non-blocking reads that fail in this way if using Lwt_unix.read.

If you do want/need this level of control on a socket created with Lwt_unix, you can do this:

Lwt_unix.set_blocking sock false;

try
  Unix.read (Lwt_unix.unix_file_descr sock) str 0 maxlen
with Unix.Unix_error (Unix.EAGAIN | Unix.EWOULDBLOCK) ->
  (* Handle no data available. *)

EDIT: Also, as mentioned by @ThomasLeonard, some uses of ignore_result in your code should probably be e >>= fun () -> e' instead. This forces Lwt to wait for e to complete before running e'. In particular, you should do this for Lwt_unix.connect.

score 1 · Answer 2 · answered Sep 25 '16 at 20:50

1

On OS X, I get:

> ./lwt_socket_client 
_read
8
_read
0
totoche

Which seems to be what you're asking for. However, I'm not sure this behaviour is useful, since it depends on how the kernel schedules the work. What are you trying to do? If you want to e.g. get on with something else while waiting for input, just run a second Lwt thread in parallel with the (blocking) read.

answered Sep 25 '16 at 20:50

Thomas Leonard

7,068
2
36
40

Did you just copy the code I post? because I am on ArchLinux x86_64 and it didn"t work. – cedlemo Sep 26 '16 at 07:55
The reasons: I try to do a mpd client. In a mpd connection, _the connection is initiated and terminated by the client. The client connect, read a status message from mpd, then send a command, get a result, and so on until the client close the connection_. My problem is in the reading, I stop reading and return the mpd message. For this I can use **two events** : there is **no remaining data** in the socket (which is the possibility I test with this code) or I **rely on the mpd protocole** which says that every mpd server message is terminated by "\n" (that I have not tested yet). – cedlemo Sep 26 '16 at 08:03
"no data available" doesn't tell you anything useful, only that the kernel isn't ready to give you the next bit right now. The only thing you can do in that case is call `read` again to see if more has now arrived. `read` will always return immediately if it can, so just process as many complete messages as you can from your buffer each time `read` returns. Assuming that a non-blocking read of 0 means you have a complete message may work during testing, but will fail eventually. – Thomas Leonard Sep 26 '16 at 08:24

OCaml: Lwt and non blocking socket

2 Answers2