Get HTTPS page content with Lua

Question

I'm trying to access the contents of a webpage from my lua code. The following code works for non-HTTPS pages

local http=require("socket.http")

body,c,l,h = http.request("http://www.example.com:443")

print("status line",l)
print("body",body)

But on HTTPS pages, i get the following error.

Your browser sent a request that this server could not understand.
Reason: You're speaking plain HTTP to an SSL-enabled server port.
Instead use the HTTPS scheme to access this URL, please.

Now i did my research, some people recommends using Luasec, but i couldn't get it to work, no matter how much i tried. Also Luasec is a bit more complicated of a library than what i'm looking for. The page i'm trying to fetch only contains a json object as follows:

{
  "value" : "false",
  "timestamp" : "2017-03-06T14:40:40Z"
}

Not posting as an answer, but I struggled with 'ssl.https' too. Then I decided to write my own implementation of simple HTTP/HTTPS browser for lua. I've published it under MIT license. Check it out here: https://github.com/diovisgood/luaweb — Pavel Chernov, May 14 '18 at 12:30

score 8 · Answer 1 · answered Mar 06 '17 at 15:33

I have couple of luasec examples in my blog post; assuming you have luasec installed, something as simple as the following should work:

require("socket")
local https = require("ssl.https")
local body, code, headers, status = https.request("https://www.google.com")
print(status)

Sending http requests to port 443 (without using luasec) is not going to work as the http library is not aware of any handshake and encryption steps that need to happen.

If you have specific errors, you should describe what they are, but the above should work.

This works for me! it should be marked as the answer – Shrikanth Rajgopalan Apr 08 '21 at 05:52 — Shrikanth Rajgopalan, Apr 08 '21 at 05:52

Mike V. · Answer 2 · 2017-03-07T06:34:29.643

try this code:

local https = require('ssl.https')
https.TIMEOUT= 10 
local link = 'http://www.example.com'
local resp = {}
local body, code, headers = https.request{
                                url = link,
                                headers = { ['Connection'] = 'close' },        
                                sink = ltn12.sink.table(resp)
                                 }   
if code~=200 then 
    print("Error: ".. (code or '') ) 
    return 
end
print("Status:", body and "OK" or "FAILED")
print("HTTP code:", code)
print("Response headers:")
if type(headers) == "table" then
  for k, v in pairs(headers) do
    print(k, ":", v)        
  end
end
print( table.concat(resp) )

to get json file set MIME type in request table: content_type = 'application/json'

 body, code, headers= https.request{
    url = link,
    filename = file,
    disposition  = 'attachment',         -- if attach
    content_type = 'application/json',
    headers = { 
                ['Referer'] = link,
                ['Connection'] = 'keep-alive'
                    },         
    sink = ltn12.sink.table(resp)    
 }

Get HTTPS page content with Lua

2 Answers2