Persistent socket in Lua in parallel with other Lua code

Question

I am implementing sockets in Lua, and the example code I'm working from uses the following method to keep the connection alive:

while true do
  -- handle socket traffic here
  socket.sleep(1)
end

The loop obviously prevents the rest of the project code to be run, but if I exit the loop the socket server immediately says that the connection was closed.

So how do I keep the socket open simultaneously as the rest of my Lua code runs as normal? (Is there some sort of background job support? Can coroutines be used for this purpose?)

If it is, you can take a look at the luasocket's `listener` code example on [github](https://raw.github.com/fire/luasocket/master/samples/listener.lua) — hjpotter92, Feb 22 '13 at 17:59
This one seems to wait until it has received all the data, then it prints it. I need socket code that will be running continuously "in the background", handling incoming data, while the rest of the project code is running "in the foreground". (If there is such as thing as background and foreground in Lua) — forthrin, Feb 22 '13 at 18:19
[OpenResty](http://openresty.org/) does what you describe, i.e. it uses coroutines to run background tasks while a socket blocks, as long as you use [its I/O functions](http://wiki.nginx.org/HttpLuaModule#Nginx_API_for_Lua). Consider using the openresty server instead of the standard interpreter. — finnw, Feb 23 '13 at 18:54
possible duplicate of [Lua sockets - Asynchronous Events](http://stackoverflow.com/questions/12889361/lua-sockets-asynchronous-events) — Chains, Feb 25 '13 at 15:47

score 0 · Answer 1 · edited May 23 '17 at 11:49

I used Lua Lanes to start a thread that is doing the socket i/o and running in the background as you stated.

http://kotisivu.dnainternet.net/askok/bin/lanes/

Take a look at this answer, which gives info on using Lua Lanes and sockets.

LuaLanes and LuaSockets

The Dual-Threaded Polling solution provided there is probably the most viable, but, there's information about coroutines there also.

score 0 · Answer 2 · edited May 23 '17 at 12:27

(Your question is similar to this question (and I have appropriately flagged it as a duplicate), but here's a copy of my answer for your convenience!)

There are a various ways of handling this issue; which one you will select depends on how much work you want to do.^*

But first, you should clarify (to yourself) whether you are dealing with UDP or TCP; there is no "underlying TCP stack" for UDP sockets. Also, UDP is the wrong protocol to use for sending whole data such as a text, or a photo; it is an unreliable protocol so you aren't guaranteed to receive every packet, unless you're using a managed socket library (such as ENet).

Lua51/LuaJIT + LuaSocket

Polling is the only method.

Blocking: call socket.select with no time argument and wait for the socket to be readable.
Non-blocking: call socket.select with a timeout argument of 0, and use sock:settimeout(0) on the socket you're reading from.

Then simply call these repeatedly. I would suggest using a coroutine scheduler for the non-blocking version, to allow other parts of the program to continue executing without causing too much delay.

Lua51/LuaJIT + LuaSocket + Lua Lanes (Recommended)

Same as the above method, but the socket exists in another lane (a lightweight Lua state in another thread) made using Lua Lanes (latest source). This allows you to instantly read the data from the socket and into a buffer. Then, you use a linda to send the data to the main thread for processing.

This is probably the best solution to your problem.

I've made a simple example of this, available here. It relies on Lua Lanes 3.4.0 (GitHub repo) and a patched LuaSocket 2.0.2 (source, patch, blog post re' patch)

The results are promising, though you should definitely refactor my example code if you derive from it.

LuaJIT + OS-specific sockets

If you're a little masochistic, you can try implementing a socket library from scratch. LuaJIT's FFI library makes this possible from pure Lua. Lua Lanes would be useful for this as well.

For Windows, I suggest taking a look at William Adam's blog. He's had some very interesting adventures with LuaJIT and Windows development. As for Linux and the rest, look at tutorials for C or the source of LuaSocket and translate them to LuaJIT FFI operations.

(LuaJIT supports callbacks if the API requires it; however, there is a signficant performance cost compared to polling from Lua to C.)

LuaJIT + ENet

ENet is a great library. It provides the perfect mix between TCP and UDP: reliable when desired, unreliable otherwise. It also abstracts operating system specific details, much like LuaSocket does. You can use the Lua API to bind it, or directly access it via LuaJIT's FFI (recommended).

_{^* Pun unintentional.}

score 0 · Answer 3 · edited Jun 20 '20 at 09:12

The other answers are nice, but kind of miss the most important point here:

There is rarely a need nowadays to use threads when dealing with sockets

Why? Because multiple sockets are so common, that the OSes (most notably *ix systems) implemented the "multiple poll" in the form of epoll function.

All high-performance networking libraries such as ZeroMQ keep only a few threads, and operate inside them. That lower the memory requirements, but doesn't sacrifice speed.

So my suggestion would be to hook up to OS libraries directly, which is really easy in Lua. You don't have to write the code yourself - quick google search brought me this epoll wrapper [1] You can then still use coroutines to read only from sockets that actually have some data.

You might also want to take a look at ZeroMQ library itself.

[1]_{Neopallium created Lua bindings for ZMQ, so I think it's legit.}

score 0 · Answer 4 · answered Feb 25 '13 at 10:56

0

You can indeed use coroutines for that purpose. This is what the popular library Copas does.

Depending on your use case you can use Copas or look at its source code to see how it does it. You may also look at lua-websockets which uses Copas.

answered Feb 25 '13 at 10:56

catwell

6,770
1
23
21