Receive packet from socket in Erlang active mode with ASCII packet size header

Question

I have an Erlang SSL TCP socket which has a permanent TCP connection to the other party. We use a protocol similar to the ISO8583 protocol where the four first bytes are the packet size, which is ASCII encoded.

Based on Erlang inet documentation (https://erlang.org/doc/man/inet.html) it only supports unsigned integer for packet size.

The header length can be one, two, or four bytes, and containing an unsigned integer in big-endian byte order.

Right now I use :gen_server.handle_info, as soon as I receive a packet I read the first four bytes and compare it to Binary size and if the binary size is small I do nothing and put received binary to LastBin and wait for rest of packet and if more than one msg is in packet I call read_iso packet several times, short sample of what I do is like this:

handle_info({ConnType, _Socket, Bin}, {BSSl, BProtocol, Psize, OSocket, LastBin})
    when ConnType =:= ssl; ConnType =:= tcp ->
    logger:debug("mp_back:Got response from backend~w", [Bin]),
          read_iso_packet(Bin, {BSSl, BProtocol, Psize, OSocket, LastBin})
    end.
read_iso_packet(Bin, {BSSl, BProtocol, Psize, Socket, LastBin})
    when size(<<LastBin/binary, Bin/binary>>) < 5 ->
    {noreply, {BSSl, BProtocol, Psize, Socket, <<LastBin/binary, Bin/binary>>}};
read_iso_packet(Bin, {BSSl, BProtocol, Psize, Socket, LastBin})
    when size(<<LastBin/binary, Bin/binary>>) > 4 ->
    Packe_Size = get_packet_size(binary:part(<<LastBin/binary, Bin/binary>>, 0, 4)),
    logger:debug("mp_back:packet_size==~w", [Packe_Size]),
    logger:debug("mp_back:bin_size==~w", [size(Bin)]),
    read_iso_packet(Packe_Size + 4 - size(<<Bin/binary, LastBin/binary>>),
                    Packe_Size,
                    <<LastBin/binary, Bin/binary>>,
                    {BSSl, BProtocol, Psize, Socket, LastBin}).

read_iso_packet(0, _Packe_Size, Bin, {BSSl, BProtocol, Psize, Socket, _LastBin}) ->
    do_somthing(server_response, CSocket, Bin),
    {noreply, {BSSl, BProtocol, Psize, Socket, <<>>}};
read_iso_packet(SS, Packe_Size, Bin, {BSSl, BProtocol, Psize, Socket, _LastBin})
    when SS < 0 ->
    do_somthing(server_response, CSocket, [binary:part(Bin, 0, Packe_Size + 4)]),        
    read_iso_packet(binary:part(Bin, Packe_Size + 4, byte_size(Bin) - (Packe_Size + 4)),
                    {BSSl, BProtocol, Psize, Socket, <<>>});
read_iso_packet(SS, _Packe_Size, Bin, {BSSl, BProtocol, Psize, Socket, _LastBin})
    when SS > 0 ->
    logger:debug("mp_back: Small data going to read next~w", [Bin]),
    {noreply, {BSSl, BProtocol, Psize, Socket, Bin}}.

get_packet_size(Bin) ->
    {ok, [A], _} = io_lib:fread("~16u", binary_to_list(binary:part(Bin, 0, 1))),
    {ok, [B], _} = io_lib:fread("~16u", binary_to_list(binary:part(Bin, 1, 1))),
    {ok, [C], _} = io_lib:fread("~16u", binary_to_list(binary:part(Bin, 2, 1))),
    {ok, [D], _} = io_lib:fread("~16u", binary_to_list(binary:part(Bin, 3, 1))),
    A * 1000 + B * 100 + C * 10 + D.

My questions are:

1. Is there a better way to do this? Last time I made a mistake and read some messages more than once (I fixed that bug, but did not test my code in production yet, but it seems OK in tests). When packet size is an unsigned integer Erlang can handle this for me but no success with ASCII encoded packet size.
2. I tried to use {active, once} and gen_tcp:recv but it did not work properly for me. Is this a safer approach?
3. Is gen_server:handle_info synchronous?

Answer 1

Question 1: Your packet protocol doesn't fit with erlang's packet protocol, so I think you need to read from the socket in raw mode by specifying {packet, raw} or equivalently {packet, 0}, see https://erlang.org/doc/man/inet.html#packet.

I'm not sure how you are using handle_info() to read from the socket. Are you setting {active, true} so that data sent to the socket lands in the genserver's mailbox? If so, I don't think that will work because {active, true} tells erlang to automatically read N bytes from the socket, where N is specfied by {packet, N} when you open the socket. In your case, N will be 4. Erlang then uses the integer contained in those 4 bytes, let's call it MsLen, to read MsLen bytes from the socket. Erlang then combines all the chunks it reads from the socket into one complete message and places the complete message in the genserver's mailbox. However, your MsLen will be wrong because it's not an unsigned integer, rather it's an ascii encoded integer. Therefore, I think you need to open the socket in passive mode, {active, false}, read the first four bytes using gen_tcp:recv(), decode to get the integer length, then call gen_tcp:recv() again to read that many bytes from the socket.

Or, you could specify {active, true} and {packet, raw} so that any data sent to the socket will land in the genserver's mailbox. In that case, a message will consist of whatever size chunk happened to be sent to the socket by the underlying transport mechanism. So, you would need to use a loop around a receive block to keep extracting messages from the mail box until you got enough bytes for a complete message.

Question 2: When you open a socket in active mode, {active, true}, erlang automatically reads N number of bytes from the socket, where N is specified in {packet, N}, then erlang combines the chunks into a complete message and places the message in the process mailbox, which can only be read by a receive clause. Calling gen_tcp:recv() reads from the socket, which is of no help in that case. See details here: Erlang client-server example using gen_tcp is not receiving anything.

Specifying {active, once} tells erlang to open the socket with {active, true} for one message, then the socket switches to {active, false} or passive mode. In passive mode, a process needs to read directly from the socket by calling gen_tcp:recv(). You can specify {active, once} when you want to prevent a hostile actor from flooding the socket with millions of messages, which in turn would fill up the process mailbox and cause the process to crash. Will a hostile actor be able to flood the socket with millions of messages?

Question: 3 Synchronous with what? When you use ! to send a message to a genserver process, the message send is asynchronous because the process that sent the message does not wait for any type of response from the genserver process, so execution continues unabated in the sending process. With a genserver, we don't ask whether handle_call() is asynchronous, instead we ask whether the process that calls gen_server:call() is asynchronous with the genserver process. The process that calls gen_server:call() stops execution and waits for a response from the genserver process, so we say that the process that called gen_server:call() is synchronous with the genserver process.

Is a process that calls gen_tcp:send() asynchronous with the genserver process? gen_tcp:send() returns ok or an error message, so it does not wait for a reply from the genserver process, so a process that calls gen_tcp:send() is asynchronous with the genserver process.