How to stream out logs via API on dropwizard?

Question

I would like to stream out logs via api endpoint /logs for dropwizard.

It is harder than what I thought. Dropwizard would read in the configuration via config.yml and keep those information as private. And I have no idea where would I be able to find the logger that logs everything? Am I missing something?

Is there another way to do this?

It is not really clear what it is you want to do? Can you give an example output? Do you want to keep a connection open over http and just keep streaming out logs without ever stopping? How does your client know when he is done? — pandaadb, May 16 '17 at 08:01
assuming I am logging: line1 line2 line3 line4 The moment I make a Get to /logs, I would like to see a continous stream of: line5 line6 line7 ... — user3019766, May 16 '17 at 23:04
The stream is not a problem really, but i do wonder if you are using this for the right thing? Could you elaborate on what you need to achieve in the end? — pandaadb, May 17 '17 at 07:52

score 0 · Answer 1 · edited May 23 '17 at 11:47

This is a streaming example, you can also read up on this here:

calling flush() on Jersey StreamingOutput has no effect

And

Example of using StreamingOutput as Response entity in Jersey

The code:

public class StreamingTest extends io.dropwizard.Application<Configuration> {

    @Override
    public void run(Configuration configuration, Environment environment) throws Exception {
        environment.jersey().property(ServerProperties.OUTBOUND_CONTENT_LENGTH_BUFFER, 0);
        environment.jersey().register(Streamer.class);
    }

    public static void main(String[] args) throws Exception {
        new StreamingTest().run("server", "/home/artur/dev/repo/sandbox/src/main/resources/config/test.yaml");
    }

    @Path("/log")
    public static class Streamer {

        @GET
        @Produces("application/octet-stream")
        public Response test() {
            return Response.ok(new StreamingOutput() {
                @Override

                public void write(OutputStream output) throws IOException, WebApplicationException {
                    while (true) {
                        output.write("Test \n".getBytes());
                        output.flush(); 

                        try {
                            Thread.sleep(1000); // simulate waiting for stream
                        } catch (InterruptedException e) {
                            e.printStackTrace();
                        }
                    }
                }
            }).build();
        }
    }
}

Which does what you want.

A few points The line evironment.jersey().property(ServerProperties.OUTBOUND_CONTENT_LENGTH_BUFFER, 0); is important. It tells the server not to buffer the output before returning it to the caller.

Additionally, you need to explicitly flush out the output by calling output.flush();

However, this seems to be the wrong way to ship logs. Have you ever looked into i.e. logstash? Or network based appenders that stream the logs directly to where you need them to be?

Hope that helps,

--artur

How to stream out logs via API on dropwizard?

1 Answers1