System.Net.WebException when porting from C# to F#

Question

I am trying to port some C# code over to F#.

The C# code has been taken from here (and slightly stripped back): https://github.com/joelpob/betfairng/blob/master/BetfairClient.cs

    public bool Login(string p12CertificateLocation, string p12CertificatePassword, string username, string password)
    {

        var appKey = "APPKEY";
        string postData = string.Format("username={0}&password={1}", username, password);
        X509Certificate2 x509certificate = new X509Certificate2(p12CertificateLocation, p12CertificatePassword);
        HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://identitysso.betfair.com/api/certlogin");
        request.UseDefaultCredentials = true;
        request.Method = "POST";
        request.ContentType = "application/x-www-form-urlencoded";
        request.Headers.Add("X-Application", appKey);
        request.ClientCertificates.Add(x509certificate);
        request.Accept = "*/*";


        using (Stream stream = request.GetRequestStream())
        using (StreamWriter writer = new StreamWriter(stream, Encoding.Default))

        writer.Write(postData);


        using (Stream stream = ((HttpWebResponse)request.GetResponse()).GetResponseStream())
        using (StreamReader reader = new StreamReader(stream, Encoding.Default))

The C# code above works great. However, when trying to run (what I think is) F# equivalent code, without any real alterations, I get an error message.

The code is being run from the same computer, same VS installation and with exactly the same 4 arguments.

The error message I get is on the second to last line:

member x.Login(username, password,p12CertificateLocation:string, p12CertificatePassword:string) = 
    let AppKey = "APPKEY"
    let  url = "https://identitysso.betfair.com/api/certlogin"
    let postData =  "username=" + username + "&password=" + password
    let x509certificate = new X509Certificate2(p12CertificateLocation, p12CertificatePassword)

    let req = HttpWebRequest.Create(url) :?> HttpWebRequest 
    req.ClientCertificates.Add(x509certificate)|>ignore
    req.UseDefaultCredentials <- true
    req.Method <- "POST"
    req.ContentType <- "application/x-www-form-urlencoded"
    req.Headers.Add("X-Application",AppKey)
    req.Accept <-"*/*" 

    use stream = req.GetRequestStream()
    use writer =new StreamWriter(stream,Encoding.Default)                      
    writer.Write(postData)

    // fails on this line:
    use stream = (req.GetResponse()  :?> HttpWebResponse ).GetResponseStream()
    // with System.Net.WebException: 'The remote server returned an error: (400) Bad Request.'
    use reader = new StreamReader(stream,Encoding.Default)

I'm a bit lost here, as to my mind the two code implementations should be identical?

You might have very specific reasons to port the C# code directly but there are also more idimatic clients that wrap HttpWebRequest, for example [Http.fs](https://github.com/haf/Http.fs) or the [Http Utilities](http://fsharp.github.io/FSharp.Data/library/Http.html) in FSharp.Data. — s952163, Jun 21 '17 at 23:32
The C# way is to use HttpClient, not HttpWebRequest directly. People don't chain calls to GetResponse, GetRresponseStream either - what's the *point* of reading a response if there is none, eg if the status code is an error? — Panagiotis Kanavos, Jun 22 '17 at 08:03
Finally, don't use a *random* encoding like you do here - Encoding.Default is the system's ANSI codepage. Almost all Web services use UTF8, except for some that may use UTF16. None use ANSI codepages. In any case, the encoding has to be specified in the approprieate headers — Panagiotis Kanavos, Jun 22 '17 at 08:06
In short, try to make a correct HTTP call, don't copy someone else's code. For example, [C# HTTP Request with POST](https://stackoverflow.com/questions/4015324/http-request-with-post) shows how easy it is to make a POST request - 5 lines — Panagiotis Kanavos, Jun 22 '17 at 08:09

ildjarn · Answer 1 · 2017-06-21T22:42:04.673

In this C# code:

using (Stream stream1 = request.GetRequestStream())
using (StreamWriter writer = new StreamWriter(stream1, Encoding.Default))
    writer.Write(postData);

using (Stream stream2 = ((HttpWebResponse)request.GetResponse()).GetResponseStream())
using (StreamReader reader = new StreamReader(stream2, Encoding.Default))

writer and stream1 are flushed and closed immediately after the writer.Write call is finished, before you call request.GetResponse(). (This fact is somewhat obscured due to the, uhh.. interesting formatting of your code.)

In this F# code:

use stream1 = req.GetRequestStream()
use writer = new StreamWriter(stream1, Encoding.Default)
writer.Write(postData)

use stream2 = (req.GetResponse() :?> HttpWebResponse).GetResponseStream()
use reader = new StreamReader(stream2, Encoding.Default)

writer and stream1 stay alive and remain unflushed and unclosed when req.GetResponse() is called; you need to put them in an artificial scope to get the same behavior as C#:

do  use stream1 = req.GetRequestStream()
    use writer = new StreamWriter(stream1, Encoding.Default)
    writer.Write(postData)

(* or

(use stream1 = req.GetRequestStream()
 use writer = new StreamWriter(stream1, Encoding.Default)
 writer.Write(postData))

*)

use stream2 = (req.GetResponse() :?> HttpWebResponse).GetResponseStream()
use reader = new StreamReader(stream2, Encoding.Default)

score 1 · Accepted Answer · edited Jun 22 '17 at 13:19

That's not "the C# way" to make an HTTP POST call. The typical way, in all supported .NET versions (ie 4.5.2 and later) is to use HttpClient. Even with HttpWebRequest, there are too many redundant or contradictory calls, like using default credentials (ie Windows authentication)

The C# way is this:

var client=new HttpClient("https://identitysso.betfair.com/api");
var values = new Dictionary<string, string>
{
   { "username", username },
   { "password", password }
};

var content = new FormUrlEncodedContent(values);
content.Headers.Add("X-Application",apiKey);

var response = await client.PostAsync("certlogin", content);
var responseString = await response.Content.ReadAsStringAsync();

In order to use a client certificate, you have to create the client instance using a custom HTTP Handler:

var handler = new WebRequestHandler();
var x509certificate = new X509Certificate2(certPath, certPassword);
handler.ClientCertificates.Add(certificate);
var client = new HttpClient(handler)
             {
                 BaseAddress = new Uri("https://identitysso.betfair.com/api")
             }

Writing the same code in F# is straight-forward:

let login username password (certPath:string) (certPassword:string) (apiKey:string) = 
    let handler = new WebRequestHandler()
    let certificate = new X509Certificate2(certPath, certPassword)
    handler.ClientCertificates.Add certificate |> ignore
    let client = new HttpClient(handler,BaseAddress = Uri("https://identitysso.betfair.com"))

    async {    
        let values = dict["username", username ; "password", password ] 
        let content = new FormUrlEncodedContent(values)
        content.Headers.Add( "X-Application" ,apiKey)    

        let! response = client.PostAsync("api/certlogin",content) |> Async.AwaitTask
        response.EnsureSuccessStatusCode() |> ignore
        let! responseString = response.Content.ReadAsStringAsync() |> Async.AwaitTask
        return responseString
    }

The client, handler are thread safe and can be reused so they can be stored in fields. Reusing the same client means that the OS doesn't have to create a new TCP/IP connection each time, leading to improved performance. It's better to create the client separately. :

let buildClient (certPath:string) (certPassword:string) =
    let handler = new WebRequestHandler()
    let certificate = new X509Certificate2(certPath, certPassword)
    handler.ClientCertificates.Add certificate |> ignore
    new HttpClient(handler,BaseAddress = Uri("https://identitysso.betfair.com"))


let login (client:HttpClient) username password  (apiKey:string) = 
    async {    
        let values = dict["username", username ; "password", password ] 
        let content = new FormUrlEncodedContent(values)
        content.Headers.Add( "X-Application" ,apiKey)    

        let! response = client.PostAsync("api/certlogin",content) |> Async.AwaitTask
        response.EnsureSuccessStatusCode() |> ignore
        let! responseString = response.Content.ReadAsStringAsync() |> Async.AwaitTask
        //Do whatever is needed here 
        return responseString
    }

System.Net.WebException when porting from C# to F#

2 Answers2