Issue with connecting to rethinkdb from Windows 11 via thinky (nodejs)

Question

I'm trying to onboard a new developer, that is using Windows 11 as the only one on our small team. I've guided him through installing WSL2 and Ubuntu 20.04.3 LTS (linux kernel: 5.10.93.2-microsoft-standard-WSL2).

We are 3 other developers who are using native Ubuntu, WSL2 Ubuntu 21.04 and macOS, respectively. We are all on nodejs 16.14 with the exact same package-lock.json file.

He is the only one getting an Error [ERR_STREAM_WRITE_AFTER_END]: write after end.

tldr;

Both errors are about writing to a Buffer.

Is anyone aware of any related issues to nodejs's Buffer implementation on Windows 11?

We are using thinky as outlined below:

'use strict';

const createThinky = require('thinky');

const { rethinkdbConfig } = require ('./utils/config.js');

const thinky = createThinky (rethinkdbConfig);

// Thinky is our ORM
var type = thinky.type;

// Creates the thinky DB model for payments - we save the amount and the stripe customerID for each transaction
// You can execute any rethinkdb query language on the model, eg Payment.count().execute()
var Payment = thinky.createModel("payments", {
    id: type.string(),
    amount: type.number(),
    customerID: type.string(),
    project: type.string(),
    projectName: type.string(),
    projectPercentage: type.number(),
    firefundPercentage: type.number(),
    type: type.string(),
    processor: type.string(),
    email: type.string(),
    charged: type.boolean(),
    recharged: type.boolean()
});

module.exports = {
  model: Payment,
};

Now, it doesn't matter if he connects to a local instance of rethinkdb (2.4.1~0focal), our staging or production rethinkdb (2.3.5~0trusty) on AWS.

I got him to try Netcat with nc -zv [url] 28015 to see if he could connect at all and he got connectıon successful. So I do not think it is a firewall issue.

Error stack trace

node ./bin/www
  firefund:www Listening on port 3000 +0ms

node:events:498
      throw er; // Unhandled 'error' event
      ^


Error [ERR_STREAM_WRITE_AFTER_END]: write after end                                                                         
  at new NodeError (node:internal/errors:371:5)
    at _write (node:internal/streams/writable:319:11)
    at Socket.Writable.write (node:internal/streams/writable:334:10)
    at Connection._sendProof (/home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/connection.js:294:19)
    at /home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/connection.js:248:12
    at Object.tryCatch (/home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/helper.js:170:3)
    at Connection._computeSaltedPassword (/home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/connection.js:247:12)
    at Socket.<anonymous> (/home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/connection.js:184:18)
    at Socket.emit (node:events:520:28)
    at Socket.emit (node:domain:475:12)
    at addChunk (node:internal/streams/readable:315:12)
    at readableAddChunk (node:internal/streams/readable:289:9)
    at Socket.Readable.push (node:internal/streams/readable:228:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
Emitted 'error' event on Connection instance at:
    at Socket.<anonymous> (/home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/connection.js:129:12)
    at Socket.emit (node:events:520:28)
    at Socket.emit (node:domain:475:12)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
        code: 'ERR_STREAM_WRITE_AFTER_END'
    }

The stack trace indicates that it is a connection error in node_modules/rethinkdbdash/lib/connection.js.

connection.js

connection.js: line 294 is this.connection.write(Buffer.concat([new Buffer(message.toString()), NULL_BUFFER])) at the bottom.

Connection.prototype._sendProof = function(authentication, randomNonce, saltedPassword) {
  var clientFinalMessageWithoutProof = "c=biws,r=" + randomNonce;
  var clientKey = crypto.createHmac("sha256", saltedPassword).update("Client Key").digest()
  var storedKey = crypto.createHash("sha256").update(clientKey).digest()

  var authMessage =
      "n=" + this.user + ",r=" + this.randomString + "," +
      authentication + "," +
      clientFinalMessageWithoutProof

  var clientSignature = crypto.createHmac("sha256", storedKey).update(authMessage).digest()
  var clientProof = helper.xorBuffer(clientKey, clientSignature)

  var serverKey = crypto.createHmac("sha256", saltedPassword).update("Server Key").digest()
  this.serverSignature = crypto.createHmac("sha256", serverKey).update(authMessage).digest()

  this.state = 2
  var message = JSON.stringify({
    authentication: clientFinalMessageWithoutProof + ",p=" + clientProof.toString("base64")
  })
  this.connection.write(Buffer.concat([new Buffer(message.toString()), NULL_BUFFER]))
}

Variant Error

He has also reported a variant of the same error as posted below:

node ./bin/www
  firefund:www Listening on port 3000 +0ms

node:events:498
      throw er; // Unhandled 'error' event
      ^


Error [ERR_STREAM_WRITE_AFTER_END]: write after end                                                                         
  at new NodeError (node:internal/errors:371:5)
    at _write (node:internal/streams/writable:319:11)
    at Socket.Writable.write (node:internal/streams/writable:334:10)
    at /home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/connection.js:143:23
    at Object.tryCatch (/home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/helper.js:170:3)
    at Socket.<anonymous> (/home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/connection.js:142:12)
    at Socket.emit (node:events:523:35)
    at Socket.emit (node:domain:475:12)
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1143:10)
Emitted 'error' event on Connection instance at:
    at Socket.<anonymous> (/home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/connection.js:129:12)
    at Socket.emit (node:events:520:28)
    at Socket.emit (node:domain:475:12)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
        code: 'ERR_STREAM_WRITE_AFTER_END'
    }

This variant suggest that the error is at line 143 in connection.js:

self.connection.write(Buffer.concat([versionBuffer, authBuffer, NULL_BUFFER]));

self.connection.on('connect', function() {
    self.connection.removeAllListeners('error');
    self.connection.on('error', function(error) {
      self.emit('error', error);
    });

    var versionBuffer = new Buffer(4)
    versionBuffer.writeUInt32LE(protodef.VersionDummy.Version.V1_0, 0)

    self.randomString = new Buffer(crypto.randomBytes(18)).toString('base64')
    var authBuffer = new Buffer(JSON.stringify({
      protocol_version: PROTOCOL_VERSION,
      authentication_method: AUTHENTIFICATION_METHOD,
      authentication: "n,,n=" + self.user + ",r=" + self.randomString
    }));

    helper.tryCatch(function() {
      self.connection.write(Buffer.concat([versionBuffer, authBuffer, NULL_BUFFER]));
    }, function(err) {
      // The TCP connection is open, but the ReQL connection wasn't established.
      // We can just abort the whole thing
      self.open = false;
      reject(new Err.ReqlDriverError('Failed to perform handshake with '+self.host+':'+self.port).setOperational());
    });
  });

If you read this far - THANK YOU!

Both errors are about writing to a Buffer.

Is anyone aware of any related issues to nodejs's Buffer implementation on Windows 11?

looks like a problem with `Buffer.concat` in nodejs windows 11.. pretty interesting :0 maybe try a customised concat `Buffer.concat` function? — The Bomb Squad, Mar 06 '22 at 17:33
oh, yea, isn't using `new Buffer` depreciated? maybe replace `new Buffer(size)` with `Buffer.alloc(size)` and `new Buffer(text)` with `Buffer.from(text)`.. doubt it would be a depreciation moment but it wouldn't hurt to try(not trying to be annoying and genuinely interested in this question) — The Bomb Squad, Mar 06 '22 at 17:39
Unfortunately, I can not make these kind of changes without writing an entire new ORM stack. We use `thinky`, which uses `rethinkdbdash`. Neither seems to be updated anymore. But the main point here is that this is working on nodejs 16 on our webserver and locally on native Ubuntu, WSL2 Ubuntu on Windows 10 and macOS. It is only an issue on Windows 11. — dotnetCarpenter, Mar 06 '22 at 18:03
@TheBombSquad do you have snippet that test `Buffer.concat` that I can send to our developer on Windows 11? Then he could run it in nodejs and see if it works on his machine. — dotnetCarpenter, Mar 06 '22 at 18:05

score 0 · Answer 1 · answered Mar 06 '22 at 18:56

0

Definitely not sure if this is THE answer, just spit-balling here that maybe you could make a custom Buffer.concat function to work in case it really is the nodejs Buffer.concat being the cause of the problem here ;-; hope it works I'm on windows 10 however

function BufferConcat(buffers){ //buffers is the array of buffers
  var bytes=Buffer.byteLength, i=0
  var length=buffers.reduce((b1,b2)=>bytes(b1)+bytes(b2))
  const BufferToReturn=Buffer.alloc(length)
  for(let buffer in buffers){
    let values=Object.values(buffer)
    for(let item of buffer){BufferToReturn[i++]=item}
  }
  return BufferToReturn
}

answered Mar 06 '22 at 18:56

The Bomb Squad

4,192
1
9
17

Hmm... I am not sold on the idea of monkey patching our dependencies.. I rather find the root cause. If it is node, then I can ask him to use a newer/older version than 16. I think what is clear to me, is that we need to move away from thinky. Perhaps https://github.com/rethinkdb/rethinkdb-ts is a way out for us. Still, if `Buffer.concat` does not work on Windows 11... :( – dotnetCarpenter Mar 06 '22 at 19:20
yea, even if this works, it's only a plaster @dotnetCarpenter :{ – The Bomb Squad Mar 06 '22 at 19:21

Issue with connecting to rethinkdb from Windows 11 via thinky (nodejs)

Error stack trace

connection.js

Variant Error

1 Answers1