How to send files in "chunks" by socket?

Question

I'm trying to send a large file (.avi) over socket by sending the content of the file in chunks (a little bit like torrents). The problem is that the script doesn't send the file. I'm out of ideas here.

Any help or twerking of the script would be very appreciated.

Server:

import socket

HOST = ""
PORT = 8050
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((HOST, PORT))
sock.listen(1)
conn, addr = sock.accept() 
print("Connected by ", str(addr))

while 1:
    data = conn.recv(1024)

    if data.decode("utf-8") == 'GET':
        with open(downFile,'rb') as output:
            l = output.read(1024)
            while (l):
                conn.send(l)
                l = output.read(1024)
            output.close()

conn.close()

Client:

import socket

HOST = "localhost"
PORT = 8050
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((HOST,PORT))

while 1:

    message = input()
    sock.send(bytes(message,'UTF-8'))

    conn.send(str.encode('GET'))
    with open(downFile, 'wb+') as output:
        while True:
            rec = str(sock.recv(1024), "utf-8")
            if not rec:
                break
            output.write(rec)
        output.close()
    print('Success!')
sock.close()

_The problem is that the script doesn't send the file_ What does it do instead? Does it do anything? Do you get an error message? — John Gordon, Oct 01 '18 at 22:52
You have indentation issues with your code, plus you mix-match the names between two files. What is your intended behavior and what actually happens? If it doesn't fail as soon as you enter your message in the Client's prompt, you're not presenting your actual code here so we cannot tell you why your code doesn't work. — zwer, Oct 01 '18 at 22:52
Also, you seem to be unnecessarily turning the bytes received into a string at the client side. If the received data is not, in fact, a UTF-8 encoded string then this will result in data corruption. — President James K. Polk, Oct 01 '18 at 23:44

gammazero · Answer 1 · 2018-10-04T08:58:11.733

Here are a working client and server that should demonstrate transferring a file over a socket. I made some assumptions about what your code was supposed to do, for example, I assumed that the initial message the client sent to the server was supposed to be the name of the file to download.

The code also includes some additional functionality for the server to return an error message to the client. Before running the code, make sure the directory specified by DOWNLOAD_DIR exists.

Client:

import socket
import sys
import os

HOST = "localhost"
PORT = 8050
BUF_SIZE = 4096
DOWNLOAD_DIR = "downloads"

def download_file(s, down_file):
    s.send(str.encode("GET\n" + down_file))
    rec = s.recv(BUF_SIZE)
    if not rec:
        return "server closed connection"

    if rec[:2].decode("utf-8") != 'OK':
        return "server error: " + rec.decode("utf-8")

    rec = rec[:2]
    if DOWNLOAD_DIR:
        down_file = os.path.join(DOWNLOAD_DIR, down_file)
    with open(down_file, 'wb') as output:
        if rec:
            output.write(rec)
        while True:
            rec = s.recv(BUF_SIZE)
            if not rec:
                break
            output.write(rec)

    print('Success!')
    return None

if DOWNLOAD_DIR and not os.path.isdir(DOWNLOAD_DIR):
    print('no such directory "%s"' % (DOWNLOAD_DIR,), file=sys.stderr)
    sys.exit(1)

while 1:
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    try:
        sock.connect((HOST, PORT))
    except Exception as e:
        print("cannot connect to server:", e, file=sys.stderr)
        break

    file_name = input("\nFile to get: ")
    if not file_name:
        sock.close()
        break

    err = download_file(sock, file_name)
    if err:
        print(err, file=sys.stderr)
    sock.close()

Server:

import socket
import sys
import os

HOST = ""
PORT = 8050
BUF_SIZE = 4096

def recv_dl_file(conn):
    data = conn.recv(1024)
    if not data:
        print("Client finished")
        return None, None

    # Get command and filename
    try:
        cmd, down_file = data.decode("utf-8").split("\n")
    except:
        return None, "cannot parse client request"
    if cmd != 'GET':
        return None, "unknown command: " + cmd

    print(cmd, down_file)
    if not os.path.isfile(down_file):
        return None, 'no such file "%s"'%(down_file,)

    return down_file, None


def send_file(conn):
    down_file, err = recv_dl_file(conn)
    if err:
        print(err, file=sys.stderr)
        conn.send(bytes(err, 'utf-8'))
        return True

    if not down_file:
        return False # client all done

    # Tell client it is OK to receive file
    sent = conn.send(bytes('OK', 'utf-8'))

    total_sent = 0
    with open(down_file,'rb') as output:
        while True:
            data = output.read(BUF_SIZE)
            if not data:
                break
            conn.sendall(data)
            total_sent += len(data)

    print("finished sending", total_sent, "bytes")
    return True


sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((HOST, PORT))
sock.listen(1)
keep_going = 1
while keep_going:
    conn, addr = sock.accept()
    print("Connected by", str(addr))
    keep_going = send_file(conn)
    conn.close() # close clien connection
    print()

sock.close() # close listener

Thank you for the example, I have been experimenting the whole day. However, your server seems to contain an error where "totalt_sent" comes into play. If i remove this one including "data = data[sent:]" and "total_sent += sent" the transfer works, however it doesn't stop. — Lavonen, Oct 02 '18 at 17:25
The reason for `data = data[sent:]` is to handle the case where not all of the data could be sent, an to attempt delivery of the remaining data. Simpler to just replace the send look with `conn.sendall(data)`. — gammazero, Oct 04 '18 at 08:54
Updated to use `sendall()`. Also, buffer size should generally be larger in real code; some multiple of page size. — gammazero, Oct 04 '18 at 09:00

How to send files in "chunks" by socket?

1 Answers1

Linked