How do you send .txt over TCP/IP line by line with a certain delay?

Question

My code:

Client.py

import socket 
import pandas as pd 
import time
import pickle

# Load data
File = pd.read_csv('Data.txt', sep='\t', skipinitialspace=True, header=None, skiprows=1)

client_socket = socket.socket()
client_socket.connect(('localhost', 5555))  # Connect to server

client_socket.send(bytes("Sending file...", 'utf-8'))  # Note

Buffer = []

for x in range(File.shape[0]):
    count = 1
    Buffer = File.loc[count]
    client_socket.send(pickle.dumps(Buffer))
    count += 1
    time.sleep(2)  # Wait 2 sec

This is my server:

Server.py

import socket
import pickle 

server_socket = socket.socket()  
server_socket.bind(('localhost', 5555))  # local server-client

server_socket.listen(3) # Max. 3 connections
print('Waiting for connection')

Buffer = []
i = 0

while True:  
     client_socket, addr = server_socket.accept()  
     received_data = client_socket.recv(1024).decode()
     print('Client address', addr, received_data)

     client_socket.recv(pickle.loads(Buffer)).decode()
     print(Buffer[i])
     i =+ 1

     if(i == 10000):
         client_socket.close()  # Closing server socket
     else: continue

I am stuck with the following error:

   BrokenPipeError: [Errno 32] Broken pipe

The problem is with SIGPIPE, because the connection is interrupted based on what I found, in case I managed to implement the pickle correctly.

Server error:

Traceback (most recent call last):
  File "/Users/David/PycharmProjects/DP1/Server.py", line 19, in <module>
    client_socket.recv(pickle.loads(Buffer))
TypeError: a bytes-like object is required, not 'list'

I expected that decode() decodes the received pickle in bytes format to readable format on the side of the server. pickle.loads uses only one argument, so I cannot specify any encoding such as utf-8.

.txt file was exported from Excel (tab spacing):

Data.txt

Time    Speed   R_Ax    Activation  Delay   KP
11:11:37    13,1    124,45  100 2   4
11:11:39    13,08   124,26  100 2   4
11:11:41    13,15   124,925 100 2   4
11:11:43    13,08   124,26  100 2   4
11:11:45    13,11   124,545 100 2   4
11:11:47    13,13   124,735 100 2   4
11:11:49    13,13   124,735 100 2   4
11:11:51    13,05   123,975 100 2   4
11:11:53    13,07   124,165 100 2   4
11:11:55    13,11   124,545 0   2   999
11:11:57    13,1    124,45  0   2   999
11:11:59    13,06   124,07  0   2   999
11:12:01    13,07   124,165 0   2   999
11:12:03    12,99   123,405 0   2   999
11:12:05    13,03   123,785 0   2   999
11:12:07    13,05   123,975 0   2   999
11:12:09    13,11   124,545 0   2   999
11:12:11    13,04   123,88  0   2   999
11:12:13    13,04   123,88  0   2   999

Thank You.

I updated my post. I am not in the phase of reading the data in `server.py` yet, because I tried to successfully send Data.txt at first. Also I will have to somehow convert the time to seconds - starting from the first row at 0s while increasing by 2s (It is not always 2 sec, so I cannot substitute this column with 0, 2, 4, ... n seconds). — Josh E., Jun 05 '20 at 20:59
@PresidentJamesK.Polk I have updated the post, it is: `client_socket.send(bytes(Buffer))` because I cannot convert the series while some of the text is `str`. — Josh E., Jun 06 '20 at 05:48
Does this answer your question? [Sending string via socket (python)](https://stackoverflow.com/questions/21233340/sending-string-via-socket-python) — Ganesh Tata, Jun 06 '20 at 06:07
@GaneshTata Thanks for the suggestion, but @Roy2012 already helped me to solve it. Now I just need to extract Time values every `for` cycle and convert them into seconds, I hope I manage :). — Josh E., Jun 06 '20 at 17:01

score 1 · Accepted Answer · edited Jun 20 '20 at 09:12

1

Since you'd like to transfer a pandas series over the wire, one way to achieve this would be to use pickle. pickle can take a Series and convert it to a byte array, and then go in the other direction and convert a byte array to a series.

On the client-side, use:

client_socket.send(pickle.dumps(Buffer))

Then, on the server-side use: pickle.loads

Detailed answer

The actual code on both sides would look like that:

client:

import socket
import pandas as pd
import time
import pickle

# Load data
File = pd.read_csv('Data.txt', sep='\t', skipinitialspace=True, header=None, skiprows=1)

client_socket = socket.socket()
client_socket.connect(('localhost', 5555))  # Connect to server

client_socket.send(bytes("Sending file...", 'utf-8'))  # Note

Buffer = []

for inx in range(File.shape[0]):
    Buffer = File.loc[inx]
    print(f"Sending \n {Buffer}")
    client_socket.send(pickle.dumps(Buffer))
    time.sleep(2)  # Wait 2 sec

Server:

import socket
import pickle

server_socket = socket.socket()
server_socket.bind(('localhost', 5555))  # local server-client

server_socket.listen(3) # Max. 3 connections
print('Waiting for connection')

Buffer = []
i = 0

client_socket, addr = server_socket.accept()
received_data = client_socket.recv(1024)
print('Client address', addr, received_data)

while True:
     received_data = client_socket.recv(10000)

     Buffer.append(pickle.loads(received_data))
     print(Buffer[i])
     i =+ 1

client_socket.close()  # Closing server socket (we'll never get here)

edited Jun 20 '20 at 09:12

Community

1
1

answered Jun 06 '20 at 06:03

Roy2012

11,755
2
22
35

Thanks, now the I am getting `SIGPIPE` error which probably means that my App shuts down the socket and then send data to it which I tried to avoid adding a condition. I updated the post if you can briefly look at it, please. – Josh E. Jun 06 '20 at 07:01
See my details code. You had a couple of issues in your code that I fixed. The exception you got was because the server died because of these issues. – Roy2012 Jun 06 '20 at 07:16
Thanks! I am begging with Python, so my code is sometimes a bit messy. It works almost smoothly. However, after I received and printed the second row I get this error `BrokenPipeError: [Errno 32] Broken pipe` - I have tried to extend the Buffer but it did not help. Is there a way of fixing this without condition for `close()` that delays it? Would you be willing to give me a hint how to convert the first column into seconds starting from 0s? I tried using `datetime.timedelta` but it did not work. Is it better to "simply" multiply HH*3600, MM*60 and then make sum of HH+MM+SS? Thank You! – Josh E. Jun 06 '20 at 09:32
Hi Josh. First - it worked fine on my laptop, so I'm not sure what the issue is. Second, are you seeing any errors from the server script? A broken pipe on the client side probably indicates that the server crashed. – Roy2012 Jun 06 '20 at 09:35
Yes, there is following error on the server.py which is at line: `received_data = client_socket.recv(10000)` the error: `OSError: [Errno 9] Bad file descriptor`. I found someone suggesting creating a new socket each time, but it does not make much sense to me if the data has for example > 1000 rows. – Josh E. Jun 06 '20 at 10:54
how large is your data? do you see the error immediately, or just after a few rows? – Roy2012 Jun 06 '20 at 14:54
I have tried larger data with around 1000 rows (I can't share them because these are experimental data). I see this error at client.py site after two rows I get: `line 18, in client_socket.send(pickle.dumps(Buffer)) BrokenPipeError: [Errno 32] Broken pipe` from server.py I get this error after one row received and printed: `line 27, in received_data = client_socket.recv(10000) OSError: [Errno 9] Bad file descriptor` – Josh E. Jun 06 '20 at 15:10
My guess is that you're somehow closing the socket within the loop. make sure that on the server side, the line 'client_socket.close()' is outside the loop (or better - you can just remove it). – Roy2012 Jun 06 '20 at 15:27
That was easy, thank you :) It is just that I read that every socket should be closed at the end and that is why I wanted to include `client_socket.close()`. Putting it out of the loop results in unreachable code, because the while loop never ends. I may put some condition or timer here. I would like to ask you one more question, I have a simple method to convert time: HH:MM:SS into seconds and resetting it to 0 as a start point. Is it possible to choose just certain values (Time) every each `for` cycle and use this method to convert time into seconds? I am unable to reach the values I want. – Josh E. Jun 06 '20 at 16:59

How do you send .txt over TCP/IP line by line with a certain delay?

1 Answers1

Detailed answer