I'm trying to consume an HTTP streaming connection that never closes (it's the Gnip Compliance stream).
I want to work with HTTP Kit, but I can't get it to work. If I pass the :stream argument:
@(http/get "http://example.com/stream-connection" {:as :stream})
The response doesn't return until the request is closed (and it will never be completed).
Is there a way to do this?
I've tried the same thing with clj-http, no luck.
Have you tried making it async via callback:
(http/get "http://example.com/stream-connection" {:as :stream}
(fn [{:keys [status headers body error opts]}]
;; body is a java.io.InputStream
(with-open [inp body]
(let [rdr (clojure.java.io/reader inp)]
(doseq [i (cheshire.core/parsed-seq rdr)]
(println i))))))
In my understanding the connection is hanging until it reads the whole stream (which is neverending), so you shouldn't use deref here. Instead, you should use an asynchronous callback and iterate stream reader.
