7

I want to download a large file and concurrently handle other things.

However, luasocket.http never calls coroutine.yield(). Everything else freezes while the file downloads.

Here's an illustrative example, in which I try to simultaneously download a file and print some numbers:

local http = require'socket.http'

local downloadRoutine = coroutine.create(function ()
    print 'Downloading large file'
    -- Download an example file
    local url = 'http://ipv4.download.thinkbroadband.com/5MB.zip'
    local result, status = http.request(url)
    print('FINISHED download ('..status..', '..#result..'bytes)')
end)

local printRoutine = coroutine.create(function ()
    -- Print some numbers
    for i=1,10 do
        print(i)
        coroutine.yield()
    end
    print 'FINISHED printing numbers'
end)

repeat
    local printActive = coroutine.resume(printRoutine)
    local downloadActive = coroutine.resume(downloadRoutine)
until not downloadActive and not printActive
print 'Both done!'

Running it produces this:

1
Downloading large file
FINISHED download (200, 5242880bytes)
2
3
4
5
6
7
8
9
10
FINISHED printing numbers
Both done!

As you can see, printRoutine is resumed first. It prints the number 1 and yields. The downloadRoutine is then resumed, which downloads the entire file, without yielding. Only then are the rest of the numbers printed.

I don't want to write my own socket library! What can I do?

Edit (later the same day): Some MUSH users have also noticed. They provide helpful ideas.

Anko - inactive in protest
  • 6,036
  • 5
  • 35
  • 47
  • 1
    Coroutines are not threads. You should not treat them like threads. If a process doesn't want to yield, then it's not going to, nor can you *force* it to yield. LuaSocket has some facilities for non-blocking IO, but I'm not very familiar with LuaSocket, so you'll have to investigate them. – Nicol Bolas Nov 11 '12 at 20:14
  • 1
    LuaSocket supports asynchronous (i.e. non-blocking) operations. RTFM before rewriting the library. – Mud Nov 12 '12 at 17:37
  • 3
    Mud: Yep, the raw `socket` does. But `socket.http` does not. ( See http://www.mail-archive.com/awesome@naquadah.org/msg04969.html .) – Anko - inactive in protest Nov 12 '12 at 22:11

2 Answers2

6

I don't see why you can't use PiL advice or copas library (this is almost the same answer as is given here).

Copas wraps the socket interface (not socket.http), but you can use low level interface to get what you need with something like this (not tested):

require("socket")
local conn = socket.tcp()
conn:connect("ipv4.download.thinkbroadband.com", 80)
conn:send("GET /5MB.zip HTTP/1.1\n\n")
local file, err = conn:receive()
print(err or file)
conn:close()

You can then use addthread from copas to give you a non-blocking socket and use step/loop functions to do receive while there is something to receive.

Using copas is less work, while using settimeout(0) directly gives you more control.

Community
  • 1
  • 1
Paul Kulchenko
  • 25,884
  • 3
  • 38
  • 56
  • Thanks you for the helpful alternatives! That solves my specific problem. My *question* is more general though: Many other libraries, such as LuaSec's `ssl.https` (non-trivial to re-implement) are still synchronous... – Anko - inactive in protest Nov 13 '12 at 12:17
  • Good point, although you may still be able to do the same thing with ssl.https. In fact, the example I gave is a simplified version of my example with ssl.https (http://notebook.kulchenko.com/programming/https-ssl-calls-with-lua-and-luasec). Having said that, I'm not sure if `ssl.https` works the same way with settimeout(0) as I haven't had a chance to test it. – Paul Kulchenko Nov 13 '12 at 18:02
2

Coroutines are not threads; they are cooperative, not simultaneous. When one coroutine yields to/from another, it gets blocked. You can't have two simultaneous execution pointers in vanilla Lua.

However, you can use external libraries to that effect. One of the most popular ones is Lua Lanes.

kikito
  • 51,734
  • 32
  • 149
  • 189