How to set HTTP version with Haskell request

Question

I'm working through the first iteration of the Plutus Pioneer Lectures, and the code for lecture 6 that gets the exchange rate from coinmarketcap.com returns a 403 error:

getExchangeRate = runReq defaultHttpConfig $ do
    v <- req
        GET
        (https "coinmarketcap.com" /: "currencies" /: "cardano")
        NoReqBody
        bsResponse
        mempty
    let priceRegex      = "priceValue___11gHJ\">\\$([\\.0-9]*)" :: ByteString
        (_, _, _, [bs]) = responseBody v =~ priceRegex :: (ByteString, ByteString, ByteString, [ByteString])
        d               = read $ unpack bs :: Double
        x               = round $ 1_000_000 * d
    liftIO $ putStrLn $ "queried exchange rate: " ++ show d
    return x

if I try the same url in curl with the -L option to follow redirects, it works:

curl -i -L 'https://coinmarketcap.com/currencies/cardano'

The only difference I can see is that the Haskell code's request is HTTP/1.1, but the curl request is being sent with HTTP/2

I'd like to change the version of HTTP that the Haskell library is sending, in order to check if this is the problem. However, I'm a relative newbie to Haskell, so I haven't been able to figure out if there's a way with the Haskell Network.HTTP.Req module to set the HTTP version.

Is there a way to do this?

TIA

Using the `--http1.1` option of curl still yields the desired result, so I don't think the HTTP version is to blame. — Noughtmare, Sep 09 '21 at 08:35

Noughtmare · Accepted Answer · 2021-09-09T08:57:39.397

I believe coinmarketcap.com is blocking requests that don't have a User-Agent header specified. You can do that like this:

getExchangeRate = runReq defaultHttpConfig $ do
    v <- req
        GET
        (https "coinmarketcap.com" /: "currencies" /: "cardano")
        NoReqBody
        bsResponse
        (header "User-Agent" "my-app/0.1.0.0")
    let priceRegex      = "priceValue___11gHJ\">\\$([\\.0-9]*)" :: ByteString
        (_, _, _, [bs]) = responseBody v =~ priceRegex :: (ByteString, ByteString, ByteString, [ByteString])
        d               = read $ unpack bs :: Double
        x               = round $ 1_000_000 * d
    liftIO $ putStrLn $ "queried exchange rate: " ++ show d
    return x

With this code the request finishes correctly but I run into a pattern match error. It seems the priceValue___11gHJ regex doesn't work anymore.

This is exactly what I was looking for, thanks! The instructor, Lars Brünjes mentioned in the lecture that the regex would be very fragile. — marcel_g, Sep 13 '21 at 03:34

How to set HTTP version with Haskell request

1 Answers1