Recording messages received on a port with SOCAT

Question

I have a server with an open port which receives between 50 and 1000 messages per second. By message I mean that a single line of text is sent.

Essentially we want to record these messages in a file which will be processed every hour (or x minutes).

I have created a bash script (see below) which runs in the background and it works except when I kill the socat process (so I can take the file for processing and it can start a new file) we get part of a message, plus I am sure we are losing messages during the split second that socat is down.

DELAY="3600"
while true
do  
    NEXT_STOP=`date +%s --date "$DELAY second"`
    (
        while [ "$(date +%s)" -lt "$NEXT_STOP" ]
        do
            killall socat
            socat -u TCP-LISTEN:6116,reuseaddr,keepalive,rcvbuf=131071,reuseaddr OPEN:/var/ais/out_v4.txt,creat,append
        done
    ) & sleep $DELAY ; killall socat

    mv /var/ais/out_v4.txt "/var/ais/_socat_received/"$(date +"%Y-%m-%d-%T")"__out_v4.txt"
done

Is there a way to:

Get socat to rotate its output file without killing the process
Or can we purge the content of the file whilst SOCAT is writing to it. e.g. cut the first 10000 lines into another file, so the output file remains a manageable size?

Many thanks in advance

On further research I suspect the solution lies with using the EXEC function rather than open. So this test writes to the file but again it doesnt rotate (i.e. create a new file every second).`socat -u TCP-LISTEN:124567 EXEC:"tee -a /root/socat_out_{$(date +"%Y-%m-%d-%T")}.txt"` — user1844937, Nov 22 '12 at 16:05
Is that one message per TCP connection, or do you want it to rotate halfway through handling a bundle of messages from a single connection? The latter's more fiddly. The former shouldn't be so hard: set up socat as a forking server, invoking a script. The script will just cat stdin to a file. When you want to change the file written to, write a new script that contains the new filename, and mv it on top of the old script. The listening socat will seamlessly hand over connections to be dumped in the new file. Make sure to use O_APPEND etc. — Nicholas Wilson, Nov 22 '12 at 16:06
Hi Nicholas, the server sending the messages is persistent so it stays connected for hours / days at a time (sending millions of messages), hence we need to rotate files part way through. Any ideas? — user1844937, Nov 22 '12 at 16:28

score 7 · Accepted Answer · answered Nov 23 '12 at 13:19

For anyone interested the final solution looks like the following, the key difference to Nicholas solution below is that I needed to grep the PID of the socat process rather than use $?:

#!/bin/bash
DELAY=600
SOCAT_PID=$(/bin/ps -eo pid,args | grep "socat -u TCP-LISTEN:12456" | grep -v grep | awk '{ print $1 }')

while `kill -0 $SOCAT_PID`
do  
  touch /var/ais/out.txt
  NEXT_STOP=`date +%s --date "$DELAY second"`
  while  `kill -0 $SOCAT_PID` && [ "$(date +%s)" -lt "$NEXT_STOP" ]
  do
    head -q - >> /var/ais/out.txt
  done
  mv /var/ais/out.txt "/var/ais/_socat_received/"$(date +"%Y-%m-%d-%T")"__out.txt"
done

In addition adding the start script within an infinite while loop so that when the client disconnects we restart socat and wait for the next connection attempt:

while true
do
socat -u TCP-LISTEN:12456,keepalive,reuseaddr,rcvbuf=131071 STDOUT | /var/ais/socat_write.sh
done

score 2 · Answer 2 · answered Sep 04 '14 at 11:22

2

Instead of reinventing the wheel, you could use rotatelogs or multilog, both of which read log messages on std input and write them to log files with very flexible rotation config.

Even one step higher, the functionality you described is very similar to what rsyslogd and the like do.

answered Sep 04 '14 at 11:22

Aryeh Leib Taurog

5,370
1
42
49

Or `cronolog` or others discussed on [Cronolog vs logrotate - Server Fault](https://serverfault.com/questions/20337/cronolog-vs-logrotate). IMHO cronolog uses a logic as simple as it can get while being very versatile. – Stéphane Gourichon Aug 04 '16 at 06:01

score 0 · Answer 3 · answered Nov 22 '12 at 17:00

0

Not so obvious! socat doesn't have any options for changing what it does with the connection halfway through. That means you'll have to be a little bit sneaky. Use socat with the output as STDOUT, and pipe to this script:

#!/bin/bash
rv=0
while [ $rv -lt 1 ]
do  
  NEXT_STOP=`date +%s --date "$DELAY second"`
  while [ "$(date +%s)" -lt "$NEXT_STOP" ] && [ $rv -lt 1 ]
  do
    head -q - >> /var/ais/out_v4.txt
    rv=$?
  done
  mv /var/ais/out_v4.txt "/var/ais/_socat_received/"$(date +"%Y-%m-%d-%T")"__out_v4.txt"
done

Totally untested, but looks reasonable?

socat -u TCP-LISTEN:6116,reuseaddr,keepalive,rcvbuf=131071,reuseaddr STDOUT | ./thescript.sh

answered Nov 22 '12 at 17:00

Nicholas Wilson

9,435
1
41
80

Actually, head's return value is useless. As a hack, you could check that the size of /var/ais/out_v4.txt grew, and if not, exit. There must be a neater way... Also, with a subshell, you can put the socat line in the same script. – Nicholas Wilson Nov 22 '12 at 17:07
Your solutions works from the point that it does do the rotation perfectly, there is a slight issue with the pipe buffering collecting 4kb before sending but that I can fix. I assume in your comment you mean exit if the file hasnt changes size because when the client (server sending messages) terminates our server is still connected and the client can't re-connect to send more messages. Presumably this is because the script is still looping? – user1844937 Nov 22 '12 at 17:27
Within the script how would I get the pid of the socat script? whilst everything is running doing a `ps -A | grep socat` returns `23218 pts/0 00:00:00 socat 23219 pts/0 00:00:00 socat_write.sh ` and when the client breaks the connection pid 23218 is terminated so in the script it could loop while pid 23218 is running? I assume that is what `rv=$?` is trying but `$?=0` and `$$=23219` – user1844937 Nov 22 '12 at 18:51

Recording messages received on a port with SOCAT

3 Answers3