2

I have a numpy ndarray that I'm trying to send via socket connection. When I try to load it on the server, using pickle.loads I get EOFError: ran out of input.

client.py

import numpy as np
import socket, pickle
import struct
HOST = "192.168.143.xxx"
PORT = 50007
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
centers = np.zeros((100, 43919))
packet = pickle.dumps(centers)
length = struct.pack('>I', len(packet))
packet = length + packet
s.send(packet)

data = s.recv(8192)
data_new = pickle.loads(data)
s.close()
print ('Received', data_new)

server.py

import socket, pickle, numpy as np
import struct
HOST = ''
PORT = 50007
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(2)

while 1:
    L = np.zeros((100, 43919))
    #wait to accept a connection - blocking call
    conn, addr = s.accept()
    print ('Connected with ', addr)
    buf = b''
    while len(buf) < 4:
        buf += conn.recv(4 - len(buf))

    length = struct.unpack('>I', buf)[0]
    print(length)
    data = conn.recv(length)
    if not data: break
    M = pickle.loads(data) # HERE IS AN ERROR, the same as np.loads(...)
    L += M
    data_out = pickle.dumps(L)
    conn.sendall(data_out)
    conn.close()

s.close()

I've tried reading this, this and this but nothing helps.

I'm using Python 3.4.

EDIT:

The exact error is:

File server.py, line 30, in <module>:
M = pickle.loads(data) #HERE IS AN ERROR
EOFError: Ran out of input.
Community
  • 1
  • 1
potockan
  • 3,998
  • 3
  • 25
  • 37
  • 1
    `recv(n)` returns **at most** `n` bytes. Also, you are using an hard-coded number (8192): how can you be sure that your object fits in 8192 bytes? – Andrea Corbellini Nov 12 '15 at 14:30
  • Probably I don't (I don't know yet how to change it) but my code fails before it gets to that point, so it doesn't matter. – potockan Nov 12 '15 at 14:31
  • Well, something is not adding up :) Your question is about `pickle.loads` and there is only one `pickle.loads(...)` line in your code. So why do you say that it's failing earlier? What error are you getting? – Andrea Corbellini Nov 12 '15 at 14:35
  • No, there are two `pickle.loads(...)` lines in my code. The error is in the server part of the code - it's marked by a comment. – potockan Nov 12 '15 at 14:42
  • The exact error is: `File server.py, line 30, in : M = pickle.loads(data) #HERE IS AN ERROR EOFError: Ran out of input.` – potockan Nov 12 '15 at 14:45
  • Oh, sorry. Not being familiar with numpy, I didn't know that `numpy.loads` is the same as `pickle.loads` -- anyhow, be sure to send all the data from the client (e.g. using `sendall`) and be sure to receive all of it (e.g. using a loop similar to `while len(buf) < 4:`) – Andrea Corbellini Nov 12 '15 at 14:47
  • Sorry, my mistake. It's the same with either `pickle.loads` or `np.loads`. Thanks for the other advices. – potockan Nov 12 '15 at 14:53
  • 1
    By the way: [Python pickle is not intended to be secure](https://blog.nelhage.com/2011/03/exploiting-pickle/): anyone who can connect to this socket can mess with your program. – poolie Nov 12 '15 at 15:13
  • And when I say, [pickle is not secure](https://blog.nelhage.com/2011/03/exploiting-pickle/), what I mean is: anyone who can connect to this socket can make your program run arbitrary commands including opening a remote shell. So, really seriously think twice before deploying this code. – poolie Nov 12 '15 at 17:21

1 Answers1

3

conn.recv(length) does not necessarily read length bytes, if there are less than that number available. You need to loop until you have enough.

See When does socket.recv(recv_size) return?

data = b''
l = length
while l > 0:
    d = sock.recv(l)
    l -= len(d)
    data += d

There is a similar issue in your code that uses .send(), although in that case there is an easy fix: just use .sendall() as you do in the other place.

Community
  • 1
  • 1
poolie
  • 9,289
  • 1
  • 47
  • 74