I need a mature HTTP client library that is idiomatic to scala, concise in usage, simple semantics. I looked at the Apache HTTP and the Scala Dispatch and numerous new libraries that promise an idiomatic Scala wrapping. Apache HTTP client sure demands verbosity, while Dispatch was easily confusing.
What is a suitable HTTP client for Scala usage?
I did a comparison of most major HTTP client libraries available
Dispatch, and a few others libraries, are not maintained anymore. The only serious ones currently are spray-client and Play! WS.
spray-client is a bit arcane in its syntax. play-ws is quite easy to use :
(build.sbt)
libraryDependencies += "com.typesafe.play" %% "play-ws" % "2.4.3"
(basic usage)
val wsClient = NingWSClient()
wsClient
.url("http://wwww.something.com")
.get()
.map { wsResponse =>
// read the response
}
I've recently started using Dispatch, a bit arcane (great general intro, serious lack of detailed scenario/use-case based docs). Dispatch 0.9.1 is a Scala wrapper around Ning's Async Http Client; to fully understand what going on requires introducing one's self to that library. In practice, the only thing I really had to look at was the RequestBuilder - everything else falling nicely into my understanding of HTTP.
I give the 0.9 release a solid thumbs up (so far!) on getting the job done very simply.. once you get past that initial learning curve.
Dispatch's Http "builder" is immutable, and seems to work well in a threaded environment. Though I can't find anything in docs to state that it is thread-safe; general reading of source suggests that it is.
Do be aware that the RequestBuilder's are mutable, and therefore are NOT thread-safe.
Here are some additional links I've found helpful:
I can't find a ScalaDoc link for the 0.9.* release, so I browse the source code for the 0.9.* release;
ScalaDoc for the 0.8 release; a substantially different beast (today) than 0.9.
The "Periodic" Table of operators, also 0.8 related.
The older 0.8 "dispatch-classic" docs helped me understand how they used the url builders, and gave some hints on how things are tied together that did carry forward to 0.9.
A little late to the party here, but I've been impressed with spray-client.
It's got a nice DSL for building requests, supports both sync and async execution, as well as a variety of (un)marshalling types (JSON, XML, forms). It plays very nicely with Akka, too.
sttp is the Scala HTTP library we've all been waiting for!
It has a fluent DSL for forming and executing requests (code samples from their README):
val request = sttp
.cookie("session", "*!@#!@!$")
.body(file) // of type java.io.File
.put(uri"http://httpbin.org/put")
.auth.basic("me", "1234")
.header("Custom-Header", "Custom-Value")
.response(asByteArray)
It supports synchronous, asynchronous, and streaming calls via pluggable backends, including Akka-HTTP (formerly Spray) and the venerable AsyncHttpClient (Netty):
implicit val sttpHandler = AsyncHttpClientFutureHandler()
val futureFirstResponse: Future[Response[String]] = request.send()
It supports scala.concurrent.Future, scalaz.concurrent.Task, monix.eval.Task, and cats.effect.IO - all the major Scala IO monad libraries.
Plus it has a few additional tricks up its sleeve:
It has case class representations for both requests and responses (although it doesn't go as far as having e.g. strongly typed headers): https://github.com/softwaremill/sttp/blob/master/core/src/main/scala/com/softwaremill/sttp/RequestT.scala https://github.com/softwaremill/sttp/blob/master/core/src/main/scala/com/softwaremill/sttp/Response.scala
It provides a URI string interpolator:
val test = "chrabąszcz majowy"
val testUri: Uri = uri"http://httpbin.org/get?bug=$test"
import com.softwaremill.sttp.circe._
val response: Either[io.circe.Error, Response] =
sttp
.post(uri"...")
.body(requestPayload)
.response(asJson[Response])
.send()
Finally, it's maintained by the reliable folks at softwaremill and it's got great documentation.
Two Six years after originally responding to this post, I would have a different answer.
I've been using akka-http, a collaboration between the spray and akka teams. It's backed by Lightbend, tightly aligned with the akka async environment... it's the right tool for this job.
Having had some unhappy experiences with the Apache client, I set about writing my own. The built-in HttpURLConnection is widely asserted to be buggy. But that's not my experience of it. In fact, the reverse has been so, the Apache client having a somewhat problematic threading model. Since Java6 (or 5?), HttpURLConnection has provided efficient HTTP1.1 connections with essentials like keep-alive being built in, and it handles concurrent usage without fuss.
So, to compensate for the inconvenient API offered by HttpURLConnection, I set about writing a new API in Scala, as an open-source project. It's just a wrapper for HttpURLConnection, but unlike HttpURLConnection, it aims to be easy to use. Unlike Apache client, it should fit easily into an existing project. Unlike Dispatch, it should be easy to learn.
It's called Bee Client
My apologies for the shameless plug. :)
Besides Dispatch there is not much out there. scalaz had a attempt at building a functional http client. But it is outdated for a while an no version of it exists in the scalaz7 branch. Additionally there is a useful wrapper of the ning async-http-client within the playframework. There your can do calls like:
WS.url("http://example.com/feed").get()
WS.url("http://example.com/item").post("content")
You can use this API as inspiration if you don't use play! in your project and dislike the Dispatch API.
Spray
You really should consider using Spray. In my opinion it has a bit of tricky syntax, but it is still pretty usable if you aim to build a high-performance http client. The main advantage of using Spray is that it is based on the akka actor library, which is extremely scalable and powerful. You can scale out your http client to several machines by only changing conf files.
Moreover few month ago Spray join Typesafe, and as I understand it will become a part of the basic akka distribution. proof
Play2
Another option is the Play2 WS lib usage (doc). As far as I know it is still not separated from the Play distribution, but due to its extremely simplicity it is worth it to spend some time attaching the whole Play framework to get that part. There are some issues with providing configuration to it, so this is not great for drop-and-use cases. However, we have used it in few non Play-based projects and everything was fine.
ScalaJ-Http is a very simple synchronous http client
https://github.com/scalaj/scalaj-http
I'd recommend it if you need a no-ceremony barebones Scala client.
Surprised that no one mentioned finagle here. It is super simple to use:
import com.twitter.finagle.{Http, Service}
import com.twitter.finagle.http
import com.twitter.util.{Await, Future}
object Client extends App {
val client: Service[http.Request, http.Response] = Http.newService("www.scala-lang.org:80")
val request = http.Request(http.Method.Get, "/")
request.host = "www.scala-lang.org"
val response: Future[http.Response] = client(request)
Await.result(response.onSuccess { rep: http.Response =>
println("GET success: " + rep)
})
}
See quick start guid for more detail: https://twitter.github.io/finagle/guide/Quickstart.html
I've used Dispatch, Spray Client and the Play WS Client Library...None of them were simply to use or configure. So I created a simpler HTTP Client library which lets you perform all the classic HTTP requests in simple one-liners.
See an example:
import cirrus.clients.BasicHTTP.GET
import scala.concurrent.Await
import scala.concurrent.duration._
object MinimalExample extends App {
val html = Await.result(Cirrus(GET("https://www.google.co.uk")), 3 seconds)
println(html)
}
... produces ...
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en-GB">...</html>
The library is called Cirrus and is available via Maven Central
libraryDependencies += "com.github.godis" % "cirrus_2.11" % "1.4.1"
The documentation is available on GitHub
https://github.com/Godis/Cirrus
