WebSocket client in Common Lisp with usocket library

Question

I'm trying to upgrade a protocol, switching from HTTP 1.1 to WebSockets. I've tried to use the usocket. My code so far follows (and is available as a GitHub gist). After the handshake reading, functions return NIL or unexpected EOF error.

;; Define parameter sock for usocket stream
;; echo.websocket.org is a site for testing websockets
(defparameter sock (usocket:socket-connect "echo.websocket.org" 80))

;; Output confirms WebSocket protocol handshake as this implemented in browsers 
(format (usocket:socket-stream sock) "~A~%~A~%~A~%~A~%~A~A~%~A~%~A~%~%" 
        "GET /?encoding=text HTTP/1.1"
        "Connection: Upgrade"
        "Host: echo.websocket.org"
        "Origin: http://www.websocket.org"
        "Sec-WebSocket-Key: " (generate-websocket-key)
        "Sec-WebSocket-Version: 13"
        "Upgrade: websocket")

;; Write output to stream
(force-output (usocket:socket-stream sock))

;; Returns NIL
(do ((line                                                             
      (read-line (usocket:socket-stream sock) nil)                        
      (read-line (usocket:socket-stream sock) nil)))                      
    ((not line))                                                       
  (format t "~A" line))

;; Returns error: unexpected EOF
(read-line (usocket:socket-stream sock))

;; Returns NIL
(listen (usocket:socket-stream sock))

Is that gist a minimal example? For the sake of self-containedness, it'd be good to add that code here, too. Please include the actual, complete, error message too. If you can get a backtrace from the debugger, that might help too. — Joshua Taylor, Sep 01 '14 at 15:36
Thank you for your advises, I'll conform them as soon as become close to my desktop keyboard. — inkuzmin, Sep 01 '14 at 15:51
Are you sure that the result of `~%` is the sort of line terminator that you need for an HTTP connection? — Joshua Taylor, Sep 01 '14 at 16:07
Yes, I think so. Because when I send ordinary HTTP GET request with that line terminator - server response is fine and I can read it with read-lines. — inkuzmin, Sep 01 '14 at 16:17
Style note: instead of repeating the `read-line` form in the `do` loop, you could use `(loop :for line := (read-line in nil) :while line :do (#|...|#))`. — Svante, Sep 02 '14 at 12:25

Rainer Joswig · Accepted Answer · 2014-09-01T20:21:22.297

~% in a FORMAT statement is not portable for the purpose of HTTP protocols. It outputs the newline character #\newline. A newline depends on the platform the code runs on: cr (old Macs, Lisp Machines), crlf (Windows) or lf (Unix). Thus if you write a newline character to a stream, the output depends on the platform you are running on (or what the Lisp system thinks it should do). Windows has the same line end convention as HTTP.

Note:

on a Unix system usually #\newline is the same as #\linefeed. A single character.
on a Windows system often #\newline is the same as the sequence #\return #\linefeed. Two characters. Some Lisp systems might ignore this and use Unix conventions.

HTTP uses crlf. To reliably write crlf, you have to write the characters #\return and #\linefeed: (format stream "~a~a" #\return #\linefeed).

(defun write-crlf (stream)
  (write-char #\return stream)
  (write-char #\linefeed stream))

Some servers might be dumb enough to read input which does not follow the standard HTTP crlf conventions, though.

There might be ways to specify the line ending convention when opening a stream. usocket does not provide such a functionality, though.

I would also use finish-output (waits for completion) and not force-output (does NOT wait for completion of IO operations).

WebSocket client in Common Lisp with usocket library

1 Answers1