0

I simplified the code to show the problem

        Dim wc2 = New WebClient
        json = wc2.DownloadString("https://api.hitbtc.com/api/2/public/symbol") 'good result
        json = wc2.DownloadString("https://btc-alpha.com/api/v1/pairs/?format=json") 'good result
        'wc2.Headers.Add(System.Net.HttpRequestHeader.Pragma, "no-cache")
        wc2.Headers.Add(System.Net.HttpRequestHeader.AcceptEncoding, "deflate")
        json = wc2.DownloadString("https://api.hitbtc.com/api/2/public/symbol") 'bad result
        json = wc2.DownloadString("https://btc-alpha.com/api/v1/pairs/?format=json") 'good result

Why?

You can try this yourself. It seems that adding wc2.Headers.Add(System.Net.HttpRequestHeader.AcceptEncoding, "deflate") would "corrupt" results from https://api.hitbtc.com/api/2/public/symbol

why?

If we go to

https://api.hitbtc.com/api/2/public/symbol we have good json result

I used this code to the same effect

    Dim json = Await CookieAwareWebClient.downloadString1Async(url)

But then recently, without any changes, I got results that looks like this

xœÔ]¯Gn†ÿ‹¯É—¹ë¯™ÓšþRwÏœÑ ráØ ,â9À&Èk޵«‘ŽT'SÝoïYZèŠÍb±HÖ?ÿÏw~úˋ.Ÿ‹ïþôÝ¿þðŸï‹ß~ýõýÏ?þíYnÂ_~ûëÇ+éåþòÛ?üðñoõÏ?þúþßßÿüÑV^¿ze+?üø—éÿ7Á«?¿úûÿ^‡¥þò¾ùðËo~²ÿçøÃÇOæ{[ûÿú~Âåðý·÷_2üïŸ~ÇŸ[Ä7yþ«?¿öð»ãTü°øç?ʺwØïŠþþÏÑYïˆ~ËlÖ°š¦(Þäqô.|:ö‡ýŒì&߀æ«ùaF›·ÀŸ}£¿ðÂ?À]Ùç+£·_ÂðŸ´ô á¾y

I wonder why. What's going on? Why is the result garbled like that?

Other URL works fine.

I then play around the idea

    Dim url = generateURLFromCommand("symbol")
    Dim json = Await CookieAwareWebClient.downloadString1Async(url) 'bad result

    If url = "https://api.hitbtc.com/api/2/public/symbol" Then
        'json = Await CookieAwareWebClient.downloadString1Async("https://btc-alpha.com/api/v1/pairs/?format=json") good result
        Dim wc = New CookieAwareWebClient()
        Dim wc1 = New WebClient
        url = "https://api.hitbtc.com/api/2/public/symbol"

        json = wc.DownloadString(url) 'bad result
        json = wc1.DownloadString(url)
        json = Await wc.DownloadStringTaskAsync(url) 'bad result
        json = Await wc1.DownloadStringTaskAsync(url)

        wc1.Headers.Add(System.Net.HttpRequestHeader.Pragma, "no-cache")
        wc1.Headers.Add(System.Net.HttpRequestHeader.AcceptEncoding, "deflate")
        json = Await wc1.DownloadStringTaskAsync(url)
    End If

because json = Await wc.DownloadStringTaskAsync(url) 'bad result json = Await wc1.DownloadStringTaskAsync(url) 'good result

I suspect there is something in cookeawarewebclient that causes it

One of the difference is there are these 2 lines in cookieawarewebclient

        wc1.Headers.Add(System.Net.HttpRequestHeader.Pragma, "no-cache")
        wc1.Headers.Add(System.Net.HttpRequestHeader.AcceptEncoding, "deflate")
        json = Await wc1.DownloadStringTaskAsync(url)

So I tried

and the result is garbled again.

I wonder why. Why adding wc1.Headers.Add(System.Net.HttpRequestHeader.AcceptEncoding, "deflate")

destroy the results?

And why only happens on https://api.hitbtc.com/api/2/public/symbol

user4951
  • 32,206
  • 53
  • 172
  • 282
  • 1
    It looks like `CookieAwareWebClient` may not support DEFLATE compression? You might find https://stackoverflow.com/questions/2973208/automatically-decompress-gzip-response-via-webclient-downloaddata helpful. – pmcoltrane Feb 19 '20 at 17:44
  • I agree with @pmcoltrane. The content is a mess because "deflate" will (sometimes) get the content in a compressed format, but only if the server (remote) supports it. Some might compress and some won't, as you have noticed. – tgolisch Feb 19 '20 at 18:31

0 Answers0