How can I extract the uncompressed text from vcrpy binary response strings?

Question

I'm using vcrpy for automated http mock testing of my application. This is working great. However my mocking includes encoding 'gzip', and 'deflate', which means the vcrpy recording responses are in binary format. Here's an example:

interactions:
- request:
    body: null
    headers:
      Accept: ['*/*']
      Accept-Encoding: ['gzip, deflate']
      Authorization: [Basic Y2hlc3RlcjpiYWRnZXI=]
      Connection: [keep-alive]
      User-Agent: [python-requests/2.9.1]
    method: GET
    uri: http://localhost:8153/go/compare/DeployProduction/14/with/15
  response:
    body:
      string: !!binary |
        H4sIAAAAAAAAAO08bXPbNpPf8ytQ9q6yp+Y7KZGKrU7quG1mEscXO7276XQ0IAlKjCmCQ0KW9TxP
        /vstAJIiJUpxkkvS3tmTiSRisVjsG3YXIE6/e/76/Oa/ry7QbzevXk6enM7ZIkX3izQrz5Q5Y/lY
        11erlbayNVrMdNP3ff2ewygcluBo8gTB3ylLWErk93O6yHFB0FWSkzTJSImu8IwgFf1Kn5zqFeCT
        U2i6RREJlrMzJcZpSRQ0L0h8pugzquOyJKzUcZ6nSYhZQjPVCG0PE9saObHthJbjE2L79tCPhhY2
        ImxZlhEH2Hbt0PB8NyCBZ8RO6NsksoZDi1haWJYKWpAowWcKTlMFFSQ9U0q2Tkk5J4QpSJ88jKwc
        M0aKrEtf6EeBEYXG0ByFERmR2A9sO8CmPzR8LyAYEyOwwpHj48COseOaLgmsUehiHAxjOzYfRt+T
        0+9UFV3fPHtzM0Z5QRll65ygOaW3JYppgQALSrJ3JOREIVWtu1xcPn94hzIskpxtsaEswr3CCYwA
        pjUMh8Qf+iZIY2hGpu063iiyTOyDUPxRbFrRyPJHxHRBIiChKIij2LfMUWxo70plcqrLcUEK3/2R

(additional output omitted)

I have read about decompressing zlib, but it does not address decoding the binary yaml.

So that I can view the original text and verify the test results, how can I convert these binary strings into their original format?

score 4 · Accepted Answer · edited Jun 15 '21 at 16:38

4

The body string in the yaml file is base64 encoded, but when you load it, you'll get the raw bytestream. If you're curious, read how to decode a base64 string, though it shouldn't be necessary.

You know you have a gzip bytestream if the first three bytes are \x1f\x8b\x08. Read here if you want to decode this manually, but vcr has a wrapper method for you, which will also update some of the header information after decoding the body. This function is vcr.filters.decode_response().

To demonstrate this with an example, I'll use the file \tests\fixtures\wild\domain_redirect.yaml. There are two interactions, of which the second contains encoded data.

import yaml
from vcr.filters import decode_response 

with open('domain_redirect.yaml', 'r') as f:
    doc = yaml.load(f)

response = doc['interactions'][1]['response']
decoded = decode_response(response)
print(decoded['body']['string'])

edited Jun 15 '21 at 16:38

Wade Williams

3,943
1
26
35

answered Apr 01 '16 at 23:38

Reti43

9,656
3
28
44

[Github issue](https://github.com/kevin1024/vcrpy/issues/241) where they discuss that the body of requests in Python 3 is encoded. – Reti43 Apr 02 '16 at 00:57
Thanks @Reti43, I found that Github issue too, and added my own to the pile: https://github.com/kevin1024/vcrpy/issues/249 – EdwardTeach Apr 03 '16 at 15:42
@EdwardTeach I don't understand what difficulties you have decoding the body string. In the yaml file it's saved as a base64 string, but after `yaml.load()` it will be a bytestring. Didn't my example code work for you? – Reti43 Apr 03 '16 at 16:13
sorry for the misunderstanding: yes that should work, thank you! I've accepted your answer. – EdwardTeach Apr 03 '16 at 20:58
Here is a decoder script: https://gist.github.com/greenmoss/a400d6c20a52f0b3cd5415e833263b1f – EdwardTeach Apr 03 '16 at 22:31
2

I think the correct package name is `vcr.filters` not `filter`. See: https://github.com/kevin1024/vcrpy/blob/master/vcr/filters.py – Michael Rice Jan 25 '17 at 06:24

How can I extract the uncompressed text from vcrpy binary response strings?

1 Answers1