2

I've been following multiple tutorials to connect a java code to python through sockets.

Sending from java to python works great, with json array. However, I can't seem to receive things with java. I don't really understand how the listening should be done. Now I just listen in a 15 second while loop (the python should send as soon as it receives an input), but I feel like I am doing something substantially wrong. Maybe somebody has an idea?

client.py:

import socket
import sys
import numpy as np
import json

# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# Bind the socket to the port
server_address = ('localhost', 10004)
print >>sys.stderr, 'starting up on %s port %s' % server_address
sock.bind(server_address)

# Listen for incoming connections
sock.listen(1)

def mysum(x):
    return np.sum(x)

while True:
    # Wait for a connection
    print >>sys.stderr, 'waiting for a connection'
    connection, client_address = sock.accept()
    infile = sock.makefile();

    try:
        print >>sys.stderr, 'connection from', client_address

        # Receive the data in small chunks and retransmit it
        data = b''
        while True:
             new_data = connection.recv(16)
             if new_data:
                 # connection.sendall(data)
                 data += new_data
             else:
                 data += new_data[:]
                 print >>sys.stderr, 'no more data from', client_address
                 break
        data= data.strip('\n');
        print("data well received!: ")
        print(data,)
        print(np.array(json.loads(data)));

        #send a new array back

        sendArray =  np.array( [ (1.5,2,3), (4,5,6) ] );
        print("Preparing to send this:");
        print(sendArray);
        connection.send(json.dumps(sendArray.tolist()));

    except Exception as e:
        print(e)
        connection.close()
        print("closed");
    finally:
        # Clean up the connection
        connection.close()
        print("closed");

server.java:

import java.io.*;
import java.net.Socket;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PrintWriter;

import org.json.*;

import java.net.ServerSocket;

public class SocketTest {

    public static void main(String[] args) throws IOException {


        String hostName = "localhost";
        int portNumber = 10004;

        try (

                //open a socket
                Socket clientSocket = new Socket(hostName, portNumber);

                BufferedReader in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
                PrintWriter out = new PrintWriter(clientSocket.getOutputStream(), true);


        ) {

            System.out.println("Connected");
            Double[][] test2 = new Double[5][2];
            test2[1][1] = 0.1;
            test2[1][0] = 0.2;
            test2[2][1] = 0.2;
            test2[2][0] = 0.2;
            test2[3][1] = 0.1;
            test2[3][0] = 0.2;
            test2[4][1] = 0.2;
            test2[4][0] = 0.2;
            test2[0][1] = 0.2;
            test2[0][0] = 0.2;


            System.out.println("A");
            out.println(new JSONArray(test2).toString());
            System.out.println("B");


            long t = System.currentTimeMillis();
            long end = t + 15000;


            while (System.currentTimeMillis() < end) {
                String response;
                while ((response = in.readLine()) != null) {
                    System.out.println("receiving");
                    System.out.println( response );

                }


            }


            //listen for input continuously? 


            //clientSocket.close();
        } catch (JSONException e) {
            e.printStackTrace();
        }


    }


}

The output from python is:

data well received!: 
('[[0.2,0.2],[0.2,0.1],[0.2,0.2],[0.2,0.1],[0.2,0.2]]',)
[[ 0.2  0.2]
 [ 0.2  0.1]
 [ 0.2  0.2]
 [ 0.2  0.1]
 [ 0.2  0.2]]
Preparing to send this:
[[ 1.5  2.   3. ]
 [ 4.   5.   6. ]]
closed
waiting for a connection
connection from ('127.0.0.1', 40074)

Output from java:

A
B

The problem is that sendArray = np.array( [ (1.5,2,3), (4,5,6) ] ); is never received by java. I feel like I am missing something simple to make it listen... Thanks for any help.

dorien
  • 5,265
  • 10
  • 57
  • 116

2 Answers2

2

This is happening because your java code is blocking. See how B is not printed to your logs? That is because it does't execute since this part is waiting for a flush command: out.println(new JSONArray(test2).toString());. What you need to do is out.flush(); so it goes on.

limbo
  • 684
  • 9
  • 18
  • B is printed in the logs. So I think it's still something else. – dorien Jun 21 '16 at 00:12
  • Maybe you have a race condition. How about make the python code wait for 5 seconds to make sure your java code is listening. It would be funny if the python code somehow always gets executed faster and sends the data before the java code is listening. – limbo Jun 21 '16 at 00:33
1

You code has multiple issues.

1. Client.py and Server.java both waiting to receive data, which causes deadlock.

Client.py is blocked at while True, keeps waiting for new data.
Server.java has sent the JSONArray(test2).toString() whose length is 51. Then keeps waiting at in.readLine().

See How to identify end of InputStream in java. The same idea works in python. It's better to know how many bytes you want to read.

Change: Because JSONArray(test2).toString() has length 51, so you replace

    while True:
         new_data = connection.recv(16)
         if new_data:
             # connection.sendall(data)
             data += new_data
         else:
             data += new_data[:]
             print >>sys.stderr, 'no more data from', client_address
             break

With

 data = connection.recv(51)

2. Server.java calls in.readLine(). But Client.py never sending the '\n' and closed the socket too soon. It causes in.readLine() throw exception.

Change: send '\n' so Server.java can successfully read a line.

    connection.send(json.dumps(sendArray.tolist()));
    connection.send("\n");

3. Client.py close the socket, which causes the endless in.readLine() throws "Connection reset" exception.

Changes: Make sure both sides don't send/receive data anymore, then close the socket together.

Code after changes of 1&2. (It requires more efforts to solve 3 and it's not the point of this question):

public class Server {

  public static void main(String[] args) throws IOException {


    String hostName = "localhost";
    int portNumber = 10004;

    try (

        //open a socket
        Socket clientSocket = new Socket(hostName, portNumber);

        BufferedReader in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
        PrintWriter out = new PrintWriter(clientSocket.getOutputStream(), true);


    ) {

      System.out.println("Connected");
      Double[][] test2 = new Double[5][2];
      test2[1][1] = 0.1;
      test2[1][0] = 0.2;
      test2[2][1] = 0.2;
      test2[2][0] = 0.2;
      test2[3][1] = 0.1;
      test2[3][0] = 0.2;
      test2[4][1] = 0.2;
      test2[4][0] = 0.2;
      test2[0][1] = 0.2;
      test2[0][0] = 0.2;


      System.out.println("A");
      out.println(new JSONArray(test2).toString());
      System.out.println("B");


      long t = System.currentTimeMillis();
      long end = t + 15000;


      while (System.currentTimeMillis() < end) {
        String response;
        while ((response = in.readLine()) != null) {
          System.out.println("receiving");
          System.out.println( response );

        }
      }
      //listen for input continuously?
      //clientSocket.close();
    } catch (JSONException e) {
      e.printStackTrace();
    }
  }
}

Client.py output:

starting up on localhost port 10004
waiting for a connection
connection from ('127.0.0.1', 53388)
data well received!: 
('[[0.2,0.2],[0.2,0.1],[0.2,0.2],[0.2,0.1],[0.2,0.2]]',)
[[ 0.2  0.2]
 [ 0.2  0.1]
 [ 0.2  0.2]
 [ 0.2  0.1]
 [ 0.2  0.2]]
Preparing to send this:
[[ 1.5  2.   3. ]
 [ 4.   5.   6. ]]
closed

Server.java output:

Connected
A
B
receiving
[[1.5, 2.0, 3.0], [4.0, 5.0, 6.0]]
Exception in thread "main" java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:209)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
Community
  • 1
  • 1
waltersu
  • 1,191
  • 8
  • 20
  • Thanks for this. I don't get an error, so maybe I have removed too much code when posting it. I have a question though. I don't know the length or the array I need to send. Basically, java is going to send arrays (about 100 000 times), python will answer with a double. (like java is calling a remote function). – dorien Jun 21 '16 at 09:43
  • If Client knows the length of data, client sends the length first, then sends the data. So Server will know when the stream ends. If you don't know the length or you know the client possibly ends the stream earlier, One choice is to wrap your data into some packet. Then you can have a special packet to indicate the EOF. It's all about protocol. – waltersu Jun 21 '16 at 10:00
  • Actually, I think the receiving works fine. It just receives it in packets of length 16, and concatenates this until there is no more new data. It just seems to be the sending of data. – dorien Jun 21 '16 at 11:07