Rails how to parse text/event-stream?

Question

I have an API url that is a stream of data with the content type: text/event-stream.

How is it possible to listen to the stream? Like subsribe to each event to print the data? I have tried to use the ruby libary em-eventsource

My test.rb file:

require "em-eventsource"
EM.run do
  source = EventMachine::EventSource.new("my_api_url_goes_here")
  source.message do |message|
    puts "new message #{message}"
  end
  source.start 
end

When I visit my api url I can see the data updated each second. But when I run the ruby file in the terminal it does not print any data/messages.

Vasfed · Answer 1 · 2015-12-09T12:23:37.273

4

Set a timer to check source.ready_state, it seems like it does not connect to api for some reason

EDIT: it seems your problem is in https' SNI, which is not supported by current eventmachine release, so upon connecting eventsource tries to connect to default virtual host on api machine, not the one the api is on, thus the CONNECTING state

Try using eventmachine from master branch, it already states to have support for SNI, that is going to be released in 1.2.0:

gem 'eventmachine', github:'eventmachine/eventmachine'

edited Dec 09 '15 at 12:23

answered Dec 04 '15 at 11:19

Vasfed

18,013
10
47
53

`0` is `CONNECTING`, this means that eventsource failed or have not yet connected, by default it will retry in 3 seconds – Vasfed Dec 06 '15 at 12:12
Is api url a regular http or https? try visiting it with `curl -v` – Vasfed Dec 06 '15 at 12:13
It is `https` I don´t get any error messages. Just tried curl and I can see all of the streaming data. – Rails beginner Dec 06 '15 at 12:33
What version of EventMachine do you use? Api server may (and most probably does) use SNI, but support for it was not released in gem yet, only in master branch – Vasfed Dec 06 '15 at 12:47
`em-eventsource (0.2.1) em-http-request (1.1.2) em-socksify (0.3.1, 0.3.0) erubis (2.7.0) eventmachine (1.0.8, 1.0.7)` – Rails beginner Dec 06 '15 at 12:55
SNI support is going to be released in eventmachine 1.2.0, until that happens try using it from github - `gem 'eventmachine', github:'eventmachine/eventmachine'` – Vasfed Dec 06 '15 at 13:01
How do I check if a server is SNI? – Rails beginner Dec 06 '15 at 13:35
`SNI` aka Server Name Indication is quite common, used to distinguish virtual hosts, without it it is only possible to host a single https site on a ip addess. Try connecting replacing hostname with its ip in url and trying connecting manually - if it fails then SNI is required for that server – Vasfed Dec 06 '15 at 14:02

score 3 · Answer 2 · answered Dec 09 '15 at 12:12

require 'eventmachine'
require 'em-http'
require 'json'

http = EM::HttpRequest.new("api_url", :keepalive => true, :connect_timeout => 0, :inactivity_timeout => 0)

EventMachine.run do

  s = http.get({'accept' => 'application/json'})
  s.stream do |data|
    puts data
  end

end

I used EventMachine http libary.

Rails how to parse text/event-stream?

2 Answers2