1

So I'm using pg-promise to query my database. Since I'm using heroku postgres (free version), the maximum number of connections is 20.

To connect to the db, I use

pgp(process.env.DATABASE_URL + '?poolSize=10')
  .connect()
  .then( sco => {
    dbclient = sco;
  })
  .catch( err => {
    console.error(err);
  })

I'm using the dbclient variable to run queries, e.g.

dbclient
  .one('select ...')
  .then(() => ...)
  .catch( res.status(500).send);

Even though I'm setting the pool size to 10, the number of connections increases indefinitely and my app crashes.

How can I solve this? Do I have to release the client each time I run a query?

EDIT:

So I edited my code, this is exactly how I'm using it now, and I still have the same problem.

const pgp = require('pg-promise')();
pgp.pg.defaults.poolSize = 10;

router.get('/', (req, res) => {
  pgp(process.env.DATABASE_URL).any('select ...')
        .then((result) => res.status(200).send(result))
        .catch(err => res.status(500).send({err}));
});
justasking
  • 105
  • 3
  • 12

1 Answers1

4

First of all, do not use method connect at all. The database object can manage connections automatically. Method connect is there only for some very specific tasks, like setting up listeners, or to explicitely verify the connection.

You should execute single queries directly against the db object, and you should execute chains of queries inside tasks or transactions. See also Chaining Queries.

You can change the default pool size through the connection parameters, using max, like this: {max: 20}. See the Connection Syntax.

vitaly-t
  • 24,279
  • 15
  • 116
  • 138
  • Thank you for taking your time :) Ok, so now I'm using it like this (and still have the same problem ): const pgp = require('pg-promise')(); pgp.pg.defaults.poolSize = 10; const db = pgp(process.env.DATABASE_URL); db .any(...).then(...) .catch(...) – justasking Sep 19 '16 at 12:14
  • @justasking When the pool size reaches its maximum, the library won't crash. If your app crashes, it must be something else then. Try to debug and see what exactly goes wrong. And you should post the error stack here ;) – vitaly-t Sep 19 '16 at 12:36
  • The crashing is not the problem. The problem is that a new connection is established for each time I query something in the database, and the number of connections goes beyond the poolSize I set. It's possible I've misunderstood how pooling works, but the problem I want to solve here is to always have fewer than 20 connections to the database. – justasking Sep 19 '16 at 12:59
  • The connections will go to the maximum if you allocate them directly somewhere and never release. Are you sure you have removed all the calls to method `connect`? That's the only thing that would explain it. – vitaly-t Sep 19 '16 at 13:27
  • Hmm, I have edited my question. At the bottom you can see what my code looks like now. Do you see anything wrong there? – justasking Sep 19 '16 at 15:08
  • Besides that you create a whole new database object for each request - no, nothing else seems wrong. – vitaly-t Sep 19 '16 at 16:09
  • Ok, so I ran the tests from https://github.com/vitaly-t/pg-promise, and I get this stack trace: – justasking Sep 19 '16 at 17:13
  • Abnormal client.end() call, due to invalid code or failed server connection. at Object.genericPool.Pool.destroy (/Users/***/Downloads/pg-promise/node_modules/pg/lib/pool.js:65:18) at Pool.destroy (/Users/***/Downloads/pg-promise/node_modules/generic-pool/lib/generic-pool.js:193:17) at . (/Users/***/Downloads/pg-promise/node_modules/pg/lib/pool.js:55:22) at . (/Users/***/Downloads/pg-promise/node_modules/pg/lib/client.js:193:10) at Socket. (/Users/***/Downloads/pg-promise/node_modules/pg/lib/connection.js:66:10) – justasking Sep 19 '16 at 17:13
  • There, it tells you about invalid call into `client.end()` that you are doing somewhere... – vitaly-t Sep 19 '16 at 17:34
  • I'm not using client.end() at all, I'm only using the code I wrote in the bottom of my question. I got the stack trace when I ran the tests from github.com/vitaly-t/pg-promise – justasking Sep 19 '16 at 17:41
  • I have those tests running under every Node.js and every PostgreSQL version, never seen what you describe there. The are only two ways to run into that error: To call `client.end()` directly or to terminate the database connection right away. Maybe something is wrong with your database connectivity and it gets interrupted right away? I'm just guessing, because there is no other explanation that I can think of. Nobody complained about such things before, and the library has been unchanged and rock-solid for a long time. – vitaly-t Sep 19 '16 at 18:00
  • 1
    I found the bug! I actually had the connect method hidden somewhere in a middleware. Thank you very much for your time, it works perfectly now – justasking Sep 20 '16 at 12:02