3

I'm using MSXML2.XMLHTTP60 for http surfing in my VBA project. The issue is MSXML2.XMLHTTP60 is limited to four concurrent requests.

I'm trying to use WinHttp.WinHttpRequest.5.1 instead, and there is another issue. MSXML2.XMLHTTP60 parses gzip result automatically, but WinHttpRequest.responseText method fails with error:

No mapping for the unicode character exists in the target multi-byte code page.

How can I parse this result with standard Windows libraries?

Code examples:
MSXML2.XMLHTTP60 limitation:

Public req1 As MSXML2.XMLHTTP60
Public req2 As MSXML2.XMLHTTP60
Public req3 As MSXML2.XMLHTTP60
Public req4 As MSXML2.XMLHTTP60
Public req5 As MSXML2.XMLHTTP60

Private Const url As String = "http://speedtest.tele2.net/100MB.zip"


Public Sub ConcurrentIssue()
    Set req1 = New MSXML2.XMLHTTP60
    req1.Open "get", url, True

    Set req2 = New MSXML2.XMLHTTP60
    req2.Open "get", url, True

    Set req3 = New MSXML2.XMLHTTP60
    req3.Open "get", url, True

    Set req4 = New MSXML2.XMLHTTP60
    req4.Open "get", url, True

    Set req5 = New MSXML2.XMLHTTP60
    req5.Open "get", url, True

    req1.send
    req2.send
    req3.send
    req4.send

    'This query will be wait
    req5.send

End Sub

Problem is that WinHttp.WinHttpRequest.5.1 does not support decompression (proof link: https://msdn.microsoft.com/ru-ru/library/windows/desktop/hh227298(v=vs.85).aspx).
I need to decompress the response myself.

Decompression issue example:

Public Sub DecompressOk()
    Set req1 = New MSXML2.XMLHTTP60
    req1.Open "get", "http://www.google.ru", False
    req1.setRequestHeader "User-Agent", "Fiddler"
    req1.setRequestHeader "Accept-Encoding", "gzip, deflate"
    req1.send

    Debug.Print req1.responseText
End Sub

Public Sub WithoutDecompress()
    Dim req As WinHttp.WinHttpRequest
    Set req = New WinHttp.WinHttpRequest

    req.Open "get", "http://www.google.ru", False
    req.setRequestHeader "User-Agent", "Fiddler"
    req.setRequestHeader "Accept-Encoding", "gzip, deflate"
    req.send

    Debug.Print req.responseText
End Sub

I was trying to do this trick without success:

Public Sub DecompressIssue()
    Dim req As WinHttp.WinHttpRequest
    Set req = New WinHttp.WinHttpRequest

    req.Open "get", "http://www.google.ru", False
    req.setRequestHeader "User-Agent", "Fiddler"
    req.setRequestHeader "Accept-Encoding", "gzip, deflate"
    req.send

    SaveBinaryToFile req.responseBody, "C:\test.zip"

    Dim xmlReq As MSXML2.XMLHTTP60
    Set xmlReq = New MSXML2.XMLHTTP60

    xmlReq.Open "get", "C:\test.zip", False
    xmlReq.setRequestHeader "Accept-Encoding", "gzip, deflate"
    xmlReq.setRequestHeader "Content-Type", "text/html; charset=windows-1251"
    xmlReq.send

    Debug.Print xmlReq.responseBody
End Sub

Sub SaveBinaryToFile(arrBytes() As Byte, strPath As String)
    With CreateObject("ADODB.Stream")
        .Type = 1 ' adTypeBinary
        .Open
        .Write arrBytes
        .SaveToFile strPath, 2 ' adSaveCreateOverWrite
        .Close
    End With
End Sub
Community
  • 1
  • 1
Nikita Bannikov
  • 75
  • 3
  • 8
  • Please edit and add a minimal code. The question should be [complete](https://stackoverflow.com/help/mcve) so that others could reproduce the issues for both 4 concurrent requests `MSXML2.XMLHTTP60` limitation and "No mapping" error for `WinHttp.WinHttpRequest.5.1`. – omegastripes May 26 '17 at 06:55
  • Some webpages may contain unreadable characters, try to get binary content from `WinHttpRequest.responseBody`, and then convert it to the text via `ADODB.Sream` like in [this answer](https://stackoverflow.com/a/33484763/2165759). – omegastripes May 26 '17 at 12:58
  • Try to make XHR with `.setRequestHeader "Accept-Encoding", "identity"` to force the webserver to send uncompressed response. – omegastripes May 29 '17 at 06:05
  • I was trying to do it, but server freezes. I cannot influence on it. – Nikita Bannikov May 29 '17 at 09:33

1 Answers1

2

This answer confirms the comment made by omegastripes.

.setRequestHeader "Accept-Encoding", "identity" is the right answer!

I've searched for days for a way to decode the gzipped response, but wasn't aware we could tell the server to not compress the response.

Adrian Mole
  • 49,934
  • 160
  • 51
  • 83
codehead
  • 21
  • 2