0

first of all thank you for your time. I have need to transfer images from one PC (Windows) to an other (Linux) and the other way arround. I used sockets and byte streams. How ever it takes 4 seconds to send one image which is insane amount of time. Am I missing some point here? Is there any library that supports fast file transfer on windows as well as on linux?

Server or sender

import numpy 
import socket
import threading
import os
from idlelib.IOBinding import encoding
import Communication
import multiprocessing
import time
import cv2
import WrapDatastructures
import sys
import ImageProcessorClient

Queue = multiprocessing.Queue()

def SendFileToClient(name, sock, image, image_number):
    #ImageRequest
    ImageRequest = sock.recv(1024)

    #Decode the Bytestring into ascii characters
    ImageRequestAsStr = ImageRequest.decode('ascii')
    print("Incoming Request String:",ImageRequestAsStr)

    if ImageRequestAsStr == "FILEFLAG":
        #Cascade to convert grayscale image to byte array and send it to Image Processor Client
        gray_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

        #Send size of the image to processor client
        memoryConsumption = sys.getsizeof(gray_image)
        print("Memory Consumtion",memoryConsumption)

        sendStr = "EXISTS" + str(memoryConsumption)

        #Convert the string to byte because otherwise it will not be send
        sock.send((sendStr.encode(encoding)))
        userResponse = sock.recv(1024)

        #the Responce will be received in byte and will be converted to a string to make it checkable
        userResponceStr = userResponse.decode('ascii')

        if userResponceStr[:2] == 'OK':
            #Get byte array
            print("Convert grayscale_image to bytedata")
            data = WrapDatastructures.singleChannelImgToByte(gray_image)
            #bytesToSend = data #send 1024 bytes
            #sock.send(bytesToSend)
            sendData = 0
            for i in range(0,memoryConsumption,1024):
                sock.send(data[i:i+1024])
        else:
            print("User response not known")
    else:
        sendStr = "ERR"
        sock.send(sendStr.encode(encoding))

    sock.close()

def Main():
    host = "127.0.0.1"
    port = 5000

    s = socket.socket()
    s.bind((host,port))
    s.listen(5)

    #init camera

    print("server started.")

    imageCounterPerContainer = 0
    while(True):
        #Take image and put it in Q

        #Call the controller to turn motor

        #look for client to get request
        if Queue.empty() is True:
            #create client
            print("create Client")

        while True:
            c, addr = s.accept()
            print("client connected ip:< " + str(addr) +">")
            imageCounterPerContainer +=1
            t = threading.Thread(target = SendFileToClient, args=("rtrThread",c,cv2.imread("image_33.jpg"),imageCounterPerContainer))
            t.start()
        s.close()
if __name__ == "__main__":
    Main()

Processor

import socket
from idlelib.IOBinding import encoding
import WrapDatastructures
import cv2
import time
def Main():
    host = "127.0.0.1"
    port = 5000
    start = time.time()
    s = socket.socket()
    s.connect((host,port))

    getfileRequest = "FILEFLAG" 

    if getfileRequest != "q":
        s.send(getfileRequest.encode())
        data = s.recv(1024)
        dataStr = data.decode('ascii')
        if dataStr[:6] == "EXISTS":
            filesize = int(dataStr[6:])
            print("Data is available size", filesize)
            sendStr = "OK"
            s.send(sendStr.encode(encoding))
            #create new file new_filename and 
            img = b""
            while True:
                data = s.recv(1024)
                if len(data) == 0:
                    break
                img = b"".join([img, data])
            print("Download Complete")
            transferedImg = WrapDatastructures.ByteToCV2(img, (2048,2448))
            cv2.imshow("transfered",transferedImg)
            end = time.time()
            print("Duration",end -start)
            cv2.waitKey(0)
            s.close()
        else:
            print("File does not exists")#
            s.close()        
    s.close()
if __name__ == "__main__":
    Main()
sjakobi
  • 3,546
  • 1
  • 25
  • 43
Max Krappmann
  • 490
  • 5
  • 19
  • Well, your protocol makes invalid assumptions about how TCP works and thus may not even be correct. TCP is a streaming protocol and does not guarantee to produce individual packets whose size is the same as your individual writes. The network stack may coalesce or split packets without telling you. Your protocol needs to provide its own boundaries, e.g. by prefixing individual elements by a 4-byte integer which is the length in bytes of the element to follow. You should perform all reads in a loop until the expected number of bytes have been read. You should also use `sendall()` for sending. – President James K. Polk Nov 30 '17 at 14:33
  • See [my answere here](https://stackoverflow.com/a/45860756/238704) for an example of a `recvall()` method that makes no assumptions about underlying packet sizes. – President James K. Polk Nov 30 '17 at 14:36
  • 1
    You can google for 'python library for file transfer' and check if any results are useful to you. In particular, check out [this answer](https://stackoverflow.com/a/7303734/238704) – President James K. Polk Nov 30 '17 at 14:58
  • I have read your answer and understand your point on robustness for transferring packages, but it seems that this approach wont provide any speed up or am I misleading? – Max Krappmann Nov 30 '17 at 14:59
  • 1
    It may not speed things up. Check out pyftpdlib, write a very simple 10-ish line client and server and transfer a large file just to see how fast your code *can* be. – President James K. Polk Nov 30 '17 at 15:03
  • Thanks I will look into it :) – Max Krappmann Nov 30 '17 at 15:10
  • Good luck, feel free to post a follow-up question as you experiment with this. – President James K. Polk Nov 30 '17 at 15:11

0 Answers0