python socket file transfer

Question

I'm trying to write transfer files or chunks of data over a socket. I feel as if I'm reinventing the wheel, but my searches for a simple solution have failed (everything I find is either too simple or too complex). The server would run on a phone running python 2.5.4. The intended application would be to sync music files between the phone and a host computer.

This is the guts of what I have, which appears to work. I send and receive 'ok' to break up streams.

Is sending 'ok' back and forth essentially as stop bits to break up streams of data a reasonable technique?

Is there a standard way to do this?

Running any sort of library server (ftp, http) on the phone is not a useful solution given the limits of the phone's memory and processing power.

server:

import socket

c = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
c.bind(('', 1234))
c.listen(1)
s,a = c.accept()

while True:
    data = s.recv(1024)
    cmd = data[:data.find('\n')]

    if cmd == 'get':
        x, file_name, x = data.split('\n', 2)
        s.sendall('ok')
        with open(file_name, 'rb') as f:
            data = f.read()
        s.sendall('%16d' % len(data))
        s.sendall(data)
        s.recv(2)

    if cmd == 'end':
        s.close()
        c.close()
        break

client:

import socket

s = socket.socket()
s.connect(('192.168.1.2', 1234))

def get_file(s, file_name):
    cmd = 'get\n%s\n' % (file_name)
    s.sendall(cmd)
    r = s.recv(2)
    size = int(s.recv(16))
    recvd = ''
    while size > len(recvd):
        data = s.recv(1024)
        if not data: 
            break
        recvd += data
    s.sendall('ok')
    return recvd

print get_file(s, 'file1')
print get_file(s, 'file2')
s.sendall('end\n')

[Uhh...](http://docs.python.org/library/basehttpserver.html) — Ignacio Vazquez-Abrams, Jul 26 '11 at 21:32
Good thought, but http server runs rather badly on the phone. — foosion, Jul 26 '11 at 21:38

score 4 · Answer 1 · answered Jul 26 '11 at 21:57

Is sending 'ok' back and forth essentially as stop bits to break up streams of data a reasonable technique?

Most protocols use some terminator or another. Popular alternatives are '\r\n', '\r\n\r\n' or EOF (ctrl+d), but these are just arbitrarily chosen and no worse or better than your 'ok', as long as your client and server know how to handle it.

score 2 · Answer 2 · answered Dec 24 '12 at 18:00

Your code looks good.

You don't actually need to send across the size of the file. You can use while True, as the check if not data: break will stop the loop.

while True:
    data = s.recv(1024)

    if not data: print " Done "; break

    recvd += data

Also, why are you sending 'ok' is the other side doesn't check for it? You are just skipping 2 bytes at each side.

Don't you need to cater to multiple clients? No need for multi-threading?

HiroshiFuu · Answer 3 · 2015-05-06T10:40:51.283

U may look at this implementation. It also take care of if the file is in a sub-directory. Here is the link!

server

import socket
import os

print('Waiting for clinet to connect...')
c = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
c.bind(('', 1234))
c.listen(1)
s, a = c.accept()

print('Connected. Going to receive file.')
s.sendall('getfilename')
filename = s.recv(1024)
if '/' in filename:
    dir = os.path.dirname(filename)
    try:
        os.stat(dir)
    except:
        print('Directory does not exist. Creating directory.')
        os.mkdir(dir)
f = open(filename, 'wb')
print('Filename: ' + filename)

while True:
    s.sendall('getfile')
    size = int(s.recv(16))
    print('Total size: ' + str(size))
    recvd = ''
    while size > len(recvd):
        data = s.recv(1024)
        if not data: 
            break
        recvd += data
        f.write(data)
        #print(len(recvd))
    break
s.sendall('end')
print('File received.')

s.close()
c.close()
f.close()

client

import socket
import sys

if len(sys.argv) > 1 :
    print('Trying to connect...')
    s = socket.socket()
    s.connect(('127.0.0.1', 1234))

    print('Connected. Wating for command.')
    while True:
        cmd = s.recv(32)

        if cmd == 'getfilename':
            print('"getfilename" command received.')
            s.sendall(sys.argv[1])

        if cmd == 'getfile':
            print('"getfile" command received. Going to send file.')
            with open(sys.argv[1], 'rb') as f:
                data = f.read()
            s.sendall('%16d' % len(data))
            s.sendall(data)
            print('File transmission done.')

        if cmd == 'end':
            print('"end" command received. Teminate.')
            break

score 1 · Answer 4 · answered Jul 27 '11 at 10:03

1

Is there a standard way to do this?

Yes. http://www.faqs.org/rfcs/rfc959.html

Describes the standard way to do this.

Here is an implementation: http://docs.python.org/library/ftplib.html

answered Jul 27 '11 at 10:03

S.Lott

384,516
81
508
779

score 0 · Answer 5 · answered Jul 27 '11 at 07:09

rsync is the standard way to sync files between two computers. You could write it in Python like this http://code.activestate.com/recipes/577518-rsync-algorithm/ or you could wrap the C library like this http://freshmeat.net/projects/pysync/ with some tweaks like replacing MD4 with MD5.

Or, if you want to do this at the socket level, you really should be using asynchat with asyncore. Here is an FTP server written with asynchat http://pyftpdlib.googlecode.com/svn-history/r20/trunk/pyftpdlib/FTPServer.py but you should start by reading http://www.doughellmann.com/PyMOTW/asynchat/ Pay attention to the part about Message Terminators point 2. A lot of network protocols do odd stuff like this, i.e. sometimes they send and receive full line commands and responses, and sometimes they send and receive chunks of arbitrary data preceded by the count of how many bytes are in the chunk. You can handle this much more easily with asynchat, and your program will scale much better too.

python socket file transfer

5 Answers5

Linked