Saving input fragments into a list by index and merging them

Question

I am sending message from one PC to other PC over UDP with my own header. User can set size of fragment for the message what means that I have to divide message into fragments and send them over to other PC, what should work in my code (did not test because receiving is the problem).

Now, on the receiver side, I need to keep track of fragment index, total fragment number and some other things which are defined in my header to check for possible data loss like (CRC). But let's get back to the problem.

I am saving every received fragment pieces by it's index into a list. So let's say index of first fragment in header is 1, so I want to save first fragment on position 1. I keep doing this in a while cycle until I get last data fragment. But something doesn't work properly. At the end, I want to print out my received message in order as it was saved in the list, from index 1 until the end using ''.join(list).

Check out my code, it is either printing out the message divided into fragments or when I move out the print of while cycle, then it prints nothing, not even the Receive: message.

Sending:

def send_loop(self):
    global mType, fragSize, fragIndex, fragCount, crc
    mType=0
    fragSize=0
    fragIndex=0 
    fragCount=0 
    crc=0
    fragSize = int(input('Fragment size: ')) #max size of fragment
while True:
            message = input('Enter message: ')
    #sending text message
            if (message[:2] == '-m'):
                mType = 1 #text message
                if message.startswith('-m '):
                    message = message[3:] # remove "-m "
                fragCount = math.ceil(len(message) / fragSize)  #rounding up number of fragments to send

                while message!= '':
                    data = bytearray()
                    data.extend(message[:fragSize].encode('utf-8'))
                    fragIndex += 1
                    header = struct.pack('!hIIII', mType, fragSize, fragIndex, fragCount, crc)
                    self.sock.sendto(header + bytearray(data), (self.host, self.port))
                    message = message[fragSize:] #set start of message to the right by size of fragSize

Receiving:

def create_socket(self):
        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        sock.bind((self.host, self.port))
        rec_list = []
        while True:
            data, addr = sock.recvfrom(65535)
            header = data[:18]
            data = data[18:]
            (mType, fragSize, fragIndex, fragCount, crc) = struct.unpack('!hIIII', header)

        #type of message is text
            if mType == 1:
                if len(rec_list) < fragCount:
                    rec_list = ['None'] * fragCount   #empty list for messages of size fragCount
                rec_list[fragIndex] = data.decode('utf-8') 


                print(
                    '\nTyp: '       + str(mType) + 
                    '\nFragSize: '  + str(fragSize) + 
                    '\nFragIndex: ' + str(fragIndex) + 
                    '\nFragCount: ' + str(fragCount) + 
                    '\nCRC: '       + str(crc)
                    )
                msg = ''.join(rec_list)
                print('\nReceived: ' +  msg) 

Answer 1

So... Below you'll find some code that keeps listening (in the first while True) forever. What I added is the second while loop. When you receive something, you need to enter into "rebuild" mode (into the second loop) and continue being there until all the chunks that conform your message are received.

Since the total number of chunks that you need to rebuild the full message is sent in every package, you can use that information to know how many pieces you're going to need in order to rebuild the message. Every time you receive a package, that means you've received one chunk (and that information is kept in the received_chunks variable).

Once you have received the same number of chunks as the number of total chunks, you know that you have rebuild your message, and you can exit the inner while to continue listening for the next message.

Note that I'm also extending the rec_list if it's smaller than the total number of fragments that conform the message. You probably don't need that, because right before getting into the inner loop it's initialized to an empty list. In the inner loop, you could substitute this:

if len(rec_list) < fragCount:
    need_to_add = fragCount - len(rec_list)
    rec_list.extend([None] * need_to_add)

By this:

if not rec_list:
    rec_list = [None] * fragCount

I did it like this to avoid deviating too much from the code. There's more cleanup you can do. I'd recommend doing more print(s to see the status of the different variables and what's happening...

while True:
    received_chunks = 0
    rec_list = []
    # Let's start receiving chunks...
    while True:
        data, addr = sock.recvfrom(65535)
        header = data[:18]
        data = data[18:]
        (mType, fragSize, fragIndex, fragCount, crc) = struct.unpack('!hIIII', header)
        print(
            '\nTyp: ' + str(mType) +
            '\nFragSize: ' + str(fragSize) +
            '\nFragIndex: ' + str(fragIndex) +
            '\nFragCount: ' + str(fragCount) +
            '\nCRC: ' + str(crc)
        )
        # type of message is text
        if mType == 1:
            if len(rec_list) < fragCount:
                need_to_add = fragCount - len(rec_list)
                rec_list.extend([None] * need_to_add)  # empty list for messages of size fragCount
            rec_list[fragIndex - 1] = data.decode('utf-8')
        print("We have received %s chunks. We expect a total of %s."
              " We are awaiting for %s chunks"
              % (received_chunks, fragCount, fragCount - received_chunks))
        received_chunks += 1
        if received_chunks == fragCount:
            break  # Get out of the second loop because we have all the chunks that we need.

        print("rec_list so far=%s" % (rec_list))
    # This is where the second (inner) while loop ends.

    print("Yay!! We got a FULL message...")
    msg = ''.join(rec_list)
    print('\nReceived: ' + msg)