0

I'm making a simple Python script to communicate with a server to get an XML file containing player data like money, weapons owned, xp, etc.

It's too large of a file to send in one packet so it gets chunked. However, when using this while loop:

def parseChunk(chunk):
    if "\r\n" in chunk:
        return chunk.split("\r\n")[1]
    else:
        return chunk

while True:
    data = ssl_sock.recv(4096).decode()

    print(data)

    if "\r\n\r\n" in data and start_chunking is False:
        start_chunking = True
        temp = data.split("\r\n\r\n")[1]
        login_resp += temp.split("\r\n")[1]

    elif start_chunking:
        if data.startswith("0\r\n"):
            print("reached the end")
            break
        login_resp += parseChunk(data)

It prematurely print's "reached the end" and a good portion of the file is still missing. As far as I know when the server is done sending chunks it sends the "final chunk" which is 0\r\n which indicates no more data will be sent right? Apparently that's not so. The server also isn't sending a Content-Length header so I have no idea what amount of bytes to expect.

Any help or insight would be much appreciated. Thanks for your time. If more information is needed please let me know before downvoting. Thanks.

Edward Severinsen
  • 643
  • 9
  • 35
  • There is length information at the start of each chunk. See answers: https://stackoverflow.com/questions/7058827/what-is-the-maximum-chunk-size-in-http-response-with-transfer-encoding-chunked A parser needs to know that length. – VPfB Jun 09 '18 at 06:45
  • @VPfB Yes, there is. Regardless of that the server is still sending a 0 length chunk before the rest of the file is transmitted. – Edward Severinsen Jun 09 '18 at 06:50
  • But the string "0\r\n" can appear as data inside another chunk. A parser that ignores the chunk length is broken. – VPfB Jun 09 '18 at 07:05
  • @VPfB That's why I used `data.startswith("0\r\n")` because if the chunk I just received begins with that then it means it's a zero length chunk right? Maybe I'm missing something. – Edward Severinsen Jun 09 '18 at 07:12
  • 1
    But you cannot be sure where one chunk ends without knowing its length. The boundaries between chunks do not necessary correspond with blocks you are reading from network. – VPfB Jun 09 '18 at 07:20
  • @VPfB Oh, this makes more sense. I will check chunk lengths as well then, thanks. – Edward Severinsen Jun 09 '18 at 19:16

0 Answers0