How to read a websocket response with cURL

Question

Writing a bash script to connect to GDAX's Websocket Feed at wss://ws-feed.gdax.com but curl doesn't seem to support this as I get

curl "wss://ws-feed.gdax.com"
curl: (1) Protocol "wss" not supported or disabled in libcurl

Answer 1

Assuming you have node installed, I would give wscat a shot; it is simple, intuitive, and powerful. Otherwise, @Pavel's answer has an abundance of venerable websocket client alternatives.

# install
npm install -g wscat

# use
wscat -c "wss://ws-feed.gdax.com"

Answer 2

Well, you can try to mimic the required headers to get some response using curl:

• https://gist.github.com/htp/fbce19069187ec1cc486b594104f01d0 or
• Linux Bash: How to open a websocket connection as client

Also, there are other ways to communicate with a WebSocket server, e.g.

• https://github.com/websockets/wscat
• https://github.com/bwasti/webpipe
• https://github.com/progrium/wssh

Answer 3

I'd like to add my own tool for this: websocat.

Example session with the service in question:

$ rlwrap websocat wss://ws-feed.gdax.com

# Now enter this line (without the #) for the required JSON request:
# {"type":"subscribe","channels": [{ "name": "heartbeat", "product_ids": ["BTC-USD"] }]}

{"type":"subscriptions","channels":[{"name":"heartbeat","product_ids":["BTC-USD"]}]}
{"type":"heartbeat","last_trade_id":46274575,"product_id":"BTC-USD","sequence":6312079752,"time":"2018-07-12T22:32:42.655000Z"}
{"type":"heartbeat","last_trade_id":46274575,"product_id":"BTC-USD","sequence":6312079800,"time":"2018-07-12T22:32:43.656000Z"}
{"type":"heartbeat","last_trade_id":46274575,"product_id":"BTC-USD","sequence":6312079834,"time":"2018-07-12T22:32:44.656000Z"}
{"type":"heartbeat","last_trade_id":46274575,"product_id":"BTC-USD","sequence":6312079945,"time":"2018-07-12T22:32:45.656000Z"}
{"type":"heartbeat","last_trade_id":46274575,"product_id":"BTC-USD","sequence":6312079990,"time":"2018-07-12T22:32:46.657000Z"}
{"type":"heartbeat","last_trade_id":46274575,"product_id":"BTC-USD","sequence":6312080042,"time":"2018-07-12T22:32:47.657000Z"}
{"type":"heartbeat","last_trade_id":46274576,"product_id":"BTC-USD","sequence":6312080169,"time":"2018-07-12T22:32:48.657000Z"}

# To stop the feed, type this line: 
{"type":"unsubscribe","channels": [{ "name": "heartbeat", "product_ids": ["BTC-USD"] }]}
{"type":"subscriptions","channels":[]}

Besides a websocket client, websocat supports WebSocket server and other modes and is aimed to integrate websockets into "UNIX" world in general.

Answer 4

ws start connection by http protocol, you have to change ws to http and some extra headers like this:

curl --include \
     --no-buffer \
     --header "Connection: Upgrade" \
     --header "Upgrade: websocket" \
     --header "Host: example.com:80" \
     --header "Origin: http://example.com:80" \
     --header "Sec-WebSocket-Key: SGVsbG8sIHdvcmxkIQ==" \
     --header "Sec-WebSocket-Version: 13" \
     "https://ws-feed.gdax.com"

https://gist.github.com/htp/fbce19069187ec1cc486b594104f01d0

Answer 5

You can connect to your server using Telnet also

telnet 120.22.37.128 6870

Once you connect you can send request in Json format

{"requestType":"hi"}     

Here 6870 is port port and to open port you need to run

sudo ufw allow 6870 

And to check pm2 logs you required to use below command

pm2 logs 0

Hope this hepls.

Answer 6

Considering there are curl answers here, although with curl you can establish a connection but can't send frames, I'll add mine.

The thing with curl answers I saw is that they give you some command, but don't explain it. That's the gap I'd like to fill in.

How does the websocket protocol work in a few words? A client connects to a server, sends a handshake request, receives "101 Switching Protocols" (a handshake response) after which they send frames back and forth.

The handshake looks along the following lines:

GET / HTTP/1.1
Host: ws.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw==
Sec-WebSocket-Version: 13
Origin: http://example.com

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk=

Upgrade makes it switch from HTTP(s) to the websocket protocol.

Connection specifies that Upgrade is a hop-by-hop header (headers that intermediaries should consume, not forward). But my experiments show that it works w/o this header. Or to be more precise, it might be optional if there's a reverse proxy in front of the server (e.g. nginx).

Sec-WebSocket-Key/Sec-WebSocket-Accept is a security measure described here, here and here.

Sec-WebSocket-Version specifies the websocket protocol version. According to RFC 6455 it should be equal 13.

Origin is needed when the client is a browser and the origin of the requesting page doesn't match the origin of the websocket server URL.

A detailed description of what should constitute a websocket handshake request can be found here.

That's how it goes with HTTP/1.1. HTTP/2 is a different story.

Knowing this to establish a websocket connection with curl:

$ curl -H 'Upgrade: websocket' \
       -H "Sec-WebSocket-Key: `openssl rand -base64 16`" \
       -H 'Sec-WebSocket-Version: 13' \
       --http1.1 \
       -sSv \
       https://ws.ifelse.io
...
> GET / HTTP/1.1
> Host: ws.ifelse.io
> Upgrade: websocket
> Sec-WebSocket-Key: e2dujvcbYbN747lapeH+WA==
> Sec-WebSocket-Version: 13
...
< HTTP/1.1 101 Switching Protocols
< Connection: upgrade
< upgrade: websocket
< sec-websocket-accept: 6wmMGMtN00aWw3loYd6P36EHKMI=

The other options are wscat, websocat.

Answer 7

Update in the middle of year 2023 :

curl 8 now support websocket protocole but you have to build it with websocket support:

get the sources :

git clone https://github.com/curl/curl.git
cd curl

after you have to configure the source with websocket support :

./configure --with-openssl --enable-ldaps --enable-ldap --disable-dependency-tracking --disable-symbol-hiding --enable-versioned-symbols --enable-threaded-resolver --with-lber-lib=lber --with-gssapi=/usr --with-nghttp2  --includedir=/usr/include/x86_64-linux-gnu --with-zsh-functions-dir=/usr/share/zsh/vendor-completions --without-libssh --with-libssh2 --enable-websockets

and to build it :

make
sudo make install

And now curl support "ws://" and "wss://" url type and websocket :

/usr/src/curl/src/curl -v -i -N -H "X-Fbx-App-Auth: $_SESSION_TOKEN" -H "Connection: Upgrade" -H "Upgrade: websocket" -H "Sec-WebSocket-Version: 13" -H "Sec-WebSocket-Key: SGVsbG8sIHdvcmxkIQ==" -H "Sec-WebSocket-Protocol: chat, superchat" -H "Host: fbx.fbx.lan" -H "Origin: fbx.fbx.lan" "wss://fbx.fbx.lan/api/latest/vm/10/console/" ; stty sane cooked    
* processing: wss://fbx.fbx.lan/api/latest/vm/10/console/
*   Trying 192.168.100.254:443...
* Connected to fbx.fbx.lan (192.168.100.254) port 443
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* Server certificate:
*  subject: C=FR; ST=PARIS; O=14RV; OU=14RV-NET; CN=*.fbx.lan; emailAddress=nba@fbx.lan
*  start date: Dec 19 08:20:33 2022 GMT
*  expire date: Dec 18 08:20:33 2024 GMT
*  issuer: C=FR; ST=PARIS; O=14RV; OU=14RV-NET; CN=14rv.lan; emailAddress=nba@14rv.lan
*  SSL certificate verify ok.
> GET /api/latest/vm/10/console/ HTTP/1.1
> Host: fbx.fbx.lan
> User-Agent: curl/8.2.1-DEV
> Accept: */*
> Connection: Upgrade
> Sec-WebSocket-Version: 13
> Sec-WebSocket-Key: Suj7OjwArmF2euYbI5wkkQ==
> X-Fbx-App-Auth: 81q7WBs/I6da+J+78wpkJHLby0Nn9uSk1DrMUZ74TxpIqFBrX6SZCNTCTQULJu5S
> Connection: Upgrade
> Upgrade: websocket
> Sec-WebSocket-Version: 13
> Sec-WebSocket-Key: SGVsbG8sIHdvcmxkIQ==
> Sec-WebSocket-Protocol: chat, superchat
> Origin: fbx.fbx.lan
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/1.1 101 Switching Protocols
< Server: nginx
< Date: Wed, 26 Jul 2023 09:24:16 GMT
< Content-Type: text/html; charset=utf-8
< Connection: upgrade
< Upgrade: websocket
< Sec-WebSocket-Accept: rPOYYXjztdK3tPXUEi8jMi0pL60=
< Sec-WebSocket-Protocol: binary
* Received 101, switch to WebSocket; mask 9adae327
< 

But for the moment, without using libcurl, I didn't succeed in having an interractive session over the websocket like I can have with 'Vitaly Shukela' tools websocat

It seems to be normal as the developpment of curl websocket support is quite recent, and at the time I'm writing (2023-07-26), here is what the WEBSOCKET.md file contents :

Command line tool WebSocket

The plan is to make curl do WebSocket similar to telnet/nc.

That part of the work has not been started

Ideas:

• Read stdin and send off as messages. Consider newline as end of fragment. (default to text? offer option to set binary)
• Respond to PINGs automatically
• Issue PINGs at some default interval (option to switch off/change interval?)
• Allow -d to specify (initial) data to send (should the format allow for multiple separate frames?)
• Exit after N messages received, where N can be zero.

Future work

• Verify the Sec-WebSocket-Accept response. It requires a sha-1 function.
• Verify Sec-WebSocket-Extensions and Sec-WebSocket-Protocol in the response
• Make WebSocket work with hyper
• Consider a curl_ws_poll()
• Make sure WebSocket code paths are fuzzed
• Add client-side PING interval
• Provide option to disable PING-PONG automation
• Support compression (CURLWS_COMPRESS)

As you can read , this part of the job had not been started but is in project at https://curl.se

So, for the moment, you can use websocket protocole with libcurl but we have to wait for the curl command line tool act like websocat tool

But you can now use curl to test a websocket server like echo.websocket.events