how to handle when client sends more bytes than we can store and the buffer gets overwritten?

Question

We have a socket in python3 that receive x bytes from client, our problem lies when the client sends more bytes than x, when this happens our buffer gets overwritten and we lose previous bytes. We need a way to avoid lose the first bytes. We'll appreciate any help. Thanks!

class Connection(object):

def __init__(self, socket, directory):
    self.sock = socket
    self.directory = directory

def handle(self):

    while(True):
        data =  self.sock.recv(4096)
        if len(data) > 0:
        ...

we expect to stop the socket from receving or some way to avoid losing the bytes that we already have in the buffer

Answer 1

You could do the following:

def receivallData(sock, buffer_size=4096):
    buf = sock.recv(buffer_size)
    while buf:
        yield buf
        if len(buf) < buffer_size: break
        buf = sock.recv(buffer_size)

You can read more on this here:

Python Socket Receive Large Amount of Data

Answer 2

you can follow this logic:

• create a buffer in which you store all the receive data
• append the data you receive at each iteration of the loop to this buffer so you won't lose it
• check if you receive the full data in order to process it

The example below explains how to create the buffer and append the data to it (in the example I exit the loop if no more data is available on the socket or socket closed)

total_data = []
while True: 
    data = self.sock.recv(4096)
    if not data: 
        break
    total_data.append(data)
    # TODO: add processing on total_data

print "".join(total_data)