hapi 18 eventsourcing not working without stream.end()

Question

Try to archive:

I try to use the HTML5 EventSourcing API https://developer.mozilla.org/de/docs/Web/API/EventSource to push events to my client application (javascript).

working example code with plain node http:

With a plain example node implementation it works perfectly and as expected. Example code: https://www.html5rocks.com/en/tutorials/eventsource/basics/

Problem:

When i try to integrate EventSourcing (or SSE) into my API endpoint which is based on hapi (currently using latest - 18.1.0) it does not work.

My route handler code mixed with some code i found:

const Stream = require('stream');

class ResponseStream extends Stream.PassThrough {
    setCompressor (compressor) {
        this._compressor = compressor;
    }
}

const stream = new ResponseStream();
let data = 0;

setInterval(() => {
    data++;
    stream.write('event: message\n');
    stream.write('data:' + data + '\n\n');
    console.log('write data...', data);
    // stream.end();        
}, 1000);

return h
    .response(stream)
    .type('text/event-stream')
    .header('Connection', 'keep-alive')
    .header('Cache-Control', 'no-cache')

Findings:

I already searched and it seems since hapi 17.x there they exposed the flush method for the compressor < https://github.com/hapijs/hapi/issues/3658 >, section features. But it still does not working.

They only way it sends a message is to uncomment the stream.end() line after sending the data. The problem obviously is that i cant send further data if i close the stream :/.

If i kill the server (with stream.end() line commented) the data gets transmitted to the client in a "single transmission". I think the problem is is still somewhere with the gzip buffering even when flushing the stream.

There are some code examples in the hapi github but i got none working with hapi 17 or 18 (all exmaples where hapi =< 16) :/

Someone know how to solve the problem or has a working EventSource example with latest hapi? I would kindly appreciate any help or suggestions.

Edit - Solution

The solution from the post below does work but i had also an nginx reverse proxy in front of my api endpoint it seems the main problem was not my code it was the nginx which had also buffered the eventsource messages. To avoid this sort of problem add in your hapi: X-Accel-Buffering: no; and it works flawless

score 2 · Accepted Answer · answered Jun 16 '19 at 22:23

2

Well I just tested with Hapi 18.1.0 and managed to create a working example.

This is my handler code:

handler: async (request, h) => {
    class ResponseStream extends Stream.PassThrough {
        setCompressor(compressor) {
            this._compressor = compressor;
        }
    }

    const stream = new ResponseStream();

    let data = 0;

    setInterval(() => {
        data++;
        stream.write('event: message\n');
        stream.write('data:' + data + '\n\n');
        console.log('write data...', data);
        stream._compressor.flush();
    }, 1000);

    return h.response(stream)
        .type('text/event-stream')
}

and this is client code just to test

var evtSource = new EventSource("http://localhost/");
evtSource.onmessage = function(e) {
    console.log("Data", + e.data);
};

evtSource.onerror = function(e) {
    console.log("EventSource failed.", e);
};

These are the resources that where I found my way to working example

https://github.com/hapijs/hapi/blob/70f777bd2fbe6e2462847f05ee10a7206571e280/test/transmit.js#L1816

https://github.com/hapijs/hapi/issues/3599#issuecomment-485190525

answered Jun 16 '19 at 22:23

metoikos

1,315
11
18

I still got the same error with your can as with mine. But as you said it worked with your code i thought there must something wrong elsewhere and it was the nginx reverse proxy. if i access the node server directly without nginx it workds flawless! need to take a look tomorrow what is wrong with the nginx config and will update my post here. big thanks! – user1966723 Jun 16 '19 at 23:06
Good, love to hear details about nginx. – metoikos Jun 17 '19 at 06:25
By the way have you checked this https://stackoverflow.com/a/13673298/12078 it's about nginx and event source – metoikos Jun 17 '19 at 06:39
Seems i have fixed it, just added 'X-Accel-Buffering': 'no' header. so many hours wasted for a single header. well, lessend lerned for next time: check direct without nginx first ;) – user1966723 Jun 17 '19 at 07:05
Ok, 1000 questions. The first one: can anyone explain me what this setCompressor method override and what the compressor.flush() method does. The second one, in hapi documentation, I find to trail for a "response stream". Does anyone have seen something like that ? – Ernest Jones Jan 18 '20 at 18:37