1

I am trying to perform a simple web request for a https webpage using mono and F# under Linux.

I've managed to get a solution working yesterday by disabling the certificate validation done by ServerCertificateValidationCallback, however now it suddenly doesn't work anymore.

The relevant parts of my script looks like this

#r "../../packages/FSharp.Data/lib/net40/FSharp.Data.dll"
#r "System.Xml.Linq.dll"
open FSharp.Data
open System
open System.IO
open System.Net
open System.Net.Security
open System.Threading
open System.Security.Cryptography.X509Certificates

// disabling certificate validation
ServicePointManager.ServerCertificateValidationCallback <-
    new RemoteCertificateValidationCallback(fun _ _ _ _ -> true)

type Either<'a,'b> = Left of 'a
                   | Right of 'b

let web_reader (url: string) =
    try
        Thread.Sleep 500
        let wr = WebRequest.CreateHttp url
        wr.UserAgent <- "test.bot" //I've also tried "safari" without luck
        let stream = wr.GetResponse().GetResponseStream()
        Right( (new StreamReader(stream)).ReadToEnd() )
    with exp -> Left (sprintf "ERROR IN 'web_reader' FOR URL %s\n\n%A" url exp)

web_reader "https://www.jobfinder.dk/"

and calling the web_reader function applied to the https url results in following exception

System.Net.WebException: Error: SendFailure (Error writing headers) ---> System.Net.WebException: Error writing headers ---> System.IO.IOException: The authentication or decryption has failed. ---> Mono.Security.Protocol.Tls.TlsException: The authentication or decryption has failed.
  at Mono.Security.Protocol.Tls.RecordProtocol.EndReceiveRecord (IAsyncResult asyncResult) <0x41d866e0 + 0x0010b> in <filename unknown>:0 
  at Mono.Security.Protocol.Tls.SslClientStream.SafeEndReceiveRecord (IAsyncResult ar, Boolean ignoreEmpty) <0x41d86620 + 0x0002b> in <filename unknown>:0 
  at Mono.Security.Protocol.Tls.SslClientStream.NegotiateAsyncWorker (IAsyncResult result) <0x41d838e0 + 0x00227> in <filename unknown>:0 
  --- End of inner exception stack trace ---
  at System.Net.WebConnection.EndWrite (System.Net.HttpWebRequest request, Boolean throwOnError, IAsyncResult result) <0x41d87f00 + 0x00207> in <filename unknown>:0 
  at System.Net.WebConnectionStream+<SetHeadersAsync>c__AnonStorey1.<>m__0 (IAsyncResult r) <0x41d87800 + 0x0013b> in <filename unknown>:0 
  --- End of inner exception stack trace ---
  --- End of inner exception stack trace ---
  at System.Net.HttpWebRequest.EndGetResponse (IAsyncResult asyncResult) <0x41d74070 + 0x0019f> in <filename unknown>:0 
  at System.Net.HttpWebRequest.GetResponse () <0x41d6ee90 + 0x00053> in <filename unknown>:0 
  at FSI_0001.web_reader (System.String url) <0x41d8a4a0 + 0x00073> in <filename unknown>:0

I really don't get what is wrong. What am I missing?

I am using:

  • F# 4.0
  • mono 4.4.1
  • Debian Jessie

I have tried to be very careful not to perform too many requests to that url and I've even added some delay with Thread.Sleep. I am still able to ping the server and access the given website, so I doubt that they blocked my IP or something like that. The exception is thrown at the very first attempt.

EDIT:

I just tested the code in Visual Studio with .Net, and there it seemed that the request stalled forever. So I debugged and investigated, and ended up finding a solution The request was aborted: Could not create SSL/TLS secure channel

adding following to my script:

ServicePointManager.Expect100Continue <- true;
ServicePointManager.SecurityProtocol <- SecurityProtocolType.Tls12

And then it works for .Net.

I also updated my version of mono from 4.2.1 to 4.4.1 which seems to have support for Tls12 as well, however I can still not get it working with mono.

Community
  • 1
  • 1
Michelrandahl
  • 3,365
  • 2
  • 26
  • 41
  • Could be the issue how mono handles certificates, have a look at this: https://github.com/mono/mono/pull/3188. – halcwb Jul 05 '16 at 09:24

0 Answers0