3

So, I'm trying to parse a website which uses websocket to send data to clients. From what I've read:

I understood that for creating a connection, I need to send a special http request to server and ot will retrieve some data which i use to access the channel.

What I've done:

  • created the http request

    `URL obj = new URL(url);
    HttpURLConnection con = (HttpURLConnection) obj.openConnection();
    
    // optional default is GET
    con.setRequestMethod("GET");
    
    //add request header
    con.setRequestProperty("Cache-Control", "no-cache");
    con.setRequestProperty("Host", "www.example.com");
    con.setRequestProperty("Upgrade", "websocket");
    con.setRequestProperty("Connection", "upgrade");
    con.setRequestProperty("Origin", "https://example.com");
    con.setRequestProperty("Sec-WebSocket-Extensions", "x-webkit-deflate-frame");
    con.setRequestProperty("Sec-WebSocket-Key", "QkWkChtTGpaZL5PkOMaRtg==");
    con.setRequestProperty("Sec-WebSocket-Version", "13");
    con.setRequestProperty("User-Agent", "Mozilla/..;
    

    `

and i read the response with code 200(not 101 which is supposed to be).

Questions:

  1. How do i generate Sec-Websocket-Key? (in example is taken from firebug, all headers actually)

1.1. What can be done more?

1.2. The same link from http must be used to access the websocket? only with wss instead of https?

  • after this I try to connect the client to server and receive messages in onMessage method.

    WebSocketContainer container = ContainerProvider.getWebSocketContainer(); Session session = container.connectToServer(ClientEndpoint.class, buildCEC(), new URI("wss://async.example.com/socket.io/1/?t=" + System.currentTimeMillis()));

  • I tried in buildCec() to insert headers to connection to server. ClientEndpointConfig config = ClientEndpointConfig.Builder.create().build(); Configurator conf = config.getConfigurator();

    `Map<String, List<String>> headers = new HashMap<String, List<String>>();
    
    headers.put("Connection", Arrays.asList("keep-alive", "Upgrade"));
    headers.put("Sec-Websocket-Extensions", Arrays.asList("x-webkit-deflate-frame"));
    headers.put("Sec-Websocket-key", Arrays.asList("cx36sHrtuW9HZAaB6kKa/Q=="));
    headers.put("Sec-Websocket-Version", Arrays.asList("13"));
    headers.put("Upgrade", Arrays.asList("websocket"));
    headers.put("User-Agent", Arrays.asList("Mozill..."));
    
    conf.beforeRequest(headers);
    
    return config;`
    

Questions

  1. Is my judgement good in case i work it out and receive 101 status code from http upgrade request?(i mean , what i do next with that details?)
  2. Please don't judge me, i'm at the beginning with websockets in java, and i don't know javascript too well, i want to do this client in pure java with jsr-356. I've done small examples with websockets, but i can't find anywhere a good client, with headers and a more complicated handshake.
  3. Is it possible to connect to an unknown server, not knowing all details?

I'm using a programatic endpoint and import javax.websocket.ContainerProvider; for dealing the websocket connection from jsr356.

VladLucian
  • 107
  • 1
  • 1
  • 9
  • I think it would be easier to use some client lib. Some googling: http://www.eclipse.org/jetty/documentation/current/jetty-websocket-client-api.html – Adrian Dec 16 '13 at 15:15
  • i want to understand the basics in which they created the concept of websockets, first, and after this use a library. I'll take a look anyway. – VladLucian Dec 16 '13 at 15:20
  • in the future, you might want to split up such many-topic questions into separate questions. – Joakim Erdfelt Dec 16 '13 at 17:05

2 Answers2

13

Lots of topics here.

  1. Don't use URLHttpConnection for WebSocket, it manages the output/input streams based on HTTP semantics, with no handling for Upgraded Connections. Use standard Socket.
  2. Read RFC-6455: The WebSocket Protocol
  3. The Sec-WebSocket-Key is mentioned many times in RFC6455, it has 1 overall purpose, for the client to prove that it received a valid WebSocket opening handshake from the server. There's rules for the client side, and rules for the server side on what it should do with it, and how it should respond to it. (in short, its a random value that the client picks, sends to the server, the server tacks on RFC specified GUID for this step, calculates an SHA-1 hash, and responds with this hash in its own Sec-WebSocket-Key, which the client validates to confirm that the server is really WebSocket capable)
  4. Read Section 5.2. Base Framing Protocol carefully, especially the parts around the 3 forms of payload length (you MUST implement support for all 3, as there are required rules for servers to validate this)' - see other answer about this - Java Websocket Text Frame Error at length >=128?
  5. As for javax.websocket.server.ServerEndpointConfig.Configurator, those methods cannot be called by you. Those are called by the JSR-356 websocket server implementation in the process of establishing a connection. See my other answer about this process - Accessing HttpSession from HttpServletRequest in a Web Socket @ServerEndpoint
  6. Be aware of the frame validation rules - How to parse and validate a WebSocket frame in Java?

There are a few JSR-356 standalone clients I'm aware of.

    <dependency>
      <groupId>org.eclipse.jetty.websocket</groupId>
      <artifactId>javax-websocket-client-impl</artifactId>
      <version>9.1.0.v20131115</version>
    </dependency>
    <dependency>
      <groupId>org.apache.tomcat</groupId>
      <artifactId>tomcat-websocket</artifactId>
      <version>8.0.0-RC5</version>
    </dependency>
    <dependency>
      <groupId>org.apache.tomcat</groupId>
      <artifactId>tomcat-coyote</artifactId>
      <version>8.0.0-RC5</version>
    </dependency>
    <dependency>
      <groupId>org.glassfish.grizzly</groupId>
      <artifactId>grizzly-websockets</artifactId>
      <version>2.3.8</version>
    </dependency>
    <dependency>
      <groupId>org.glassfish.tyrus.bundles</groupId>
      <artifactId>tyrus-standalone-client</artifactId>
      <version>1.3.3</version>
    </dependency>

If you do write your own WebSocket client, be sure you test it against the Autobahn WebSocket Protocol Testsuite (everyone who write an implementation tests against this testsuite, its kinda become the de facto protocol validation suite)

Community
  • 1
  • 1
Joakim Erdfelt
  • 46,896
  • 7
  • 86
  • 136
  • Thank you for such a detailed answer. I'll review what you told me and let you know. – VladLucian Dec 17 '13 at 06:38
  • 1
    Hello again, i read all what you gave me, i've implemented a little example, but now i want to parse the real site. I've used Jetty Client and everything seemed ok at first, the handshake gives 200 status code with all info. But when i try to make the connection it remains in pending. The site which i want to parse uses this : https://github.com/LearnBoost/socket.io-spec. I followed all steps in this documentation but no luck. When trying to make the real connection, it remains in pending. Are you aware of some sort of acceptance from the server? – VladLucian Dec 23 '13 at 09:56
  • It works for "ws://echo.websocket.org", to be more specific, when i say acceptance, i'm referring to some sort of header. – VladLucian Dec 23 '13 at 10:42
  • Your questions seem like a long chain of "... oh and another thing ...". Socket.IO is not websocket, its a complex layer on top of websocket. Your question is about websocket. Ask another question about this new problem. Don't glom onto an old question/answer. Stackoverflow is not a forum for ongoing conversations, its a question/answer site. – Joakim Erdfelt Dec 23 '13 at 16:50
2

I think here is what you´ll need:

 String secAccept = sha1base64(secKey + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11");

 private String sha1base64(String str) {
        MessageDigest md = null;
        try {
            md = MessageDigest.getInstance("SHA-1");
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
        }
        return Base64.encode(md.digest(str.getBytes()));
    }
Tiago Luz
  • 129
  • 6