0

Currently I'm using a timer with node-schedule to check for changes in the database, making queries every 1 minute and then comparing with the information already registered to see if there were changes since the last check.

With what I want to do I want to decrease the amount of unnecessary queries and be able to bring the data in real time, and this is interesting because I am using this timer to provide statistics on the amount of users, categories and articles.

I am using Knex as query-builder in my db.

This is my file with the timer (I am using MongoDB to store this information):

const schedule = require('node-schedule')

module.exports = app => {
    schedule.scheduleJob('*/1 * * * *', async function () {
        const usersCount = await app.db('users').count('id').first()
        const categoriesCount = await app.db('categories').count('id').first()
        const articlesCount = await app.db('articles').count('id').first()

        const { Stat } = app.api.stat

        const lastStat = await Stat.findOne({}, {},
            { sort: { 'createdAt': -1 } })
        
        const stat = new Stat({
            users: usersCount.count,
            categories: categoriesCount.count,
            articles: articlesCount.count,
            createdAt: new Date()
        })

        const hasChanged = !lastStat
            || stat.users !== lastStat.users
            || stat.categories !== lastStat.categories
            || stat.articles !== lastStat.articles;
        
        if (hasChanged) {
            stat.save().then(() => console.log('[STATS] Estatísticas atualizadas!'))
        }
    })
}

I tried this:

const config = require('../knexfile')
const knex = require('knex')(config)

knex.migrate.latest([config])

knex.raw(`
    CREATE or REPLACE FUNCTION public.notify_trigger() RETURNS trigger  $$
    DECLARE
        BEGIN
            PERFORM pg_notify( CAST('update_notification' AS text), row_to_json(NEW)::text );
            RETURN new;
        END;
    $$ LANGUAGE plpgsql;
    
    CREATE TRIGGER update_notification
    AFTER INSERT ON categories
    FOR EACH ROW
    EXECUTE FUNCTION public.notify_trigger();
`)

const { Client } = require('pg')

// The file contains the correct information
const client = new Client({
    // user: '**********',
    // host: 'localhost',
    // database: '*********',
    // password: '******************',
    // port: 5432,
    connectionString: 'postgres://*********:*********@localhost/**********'
})

client.connect((err, client, done) => {
    if (err) {
        const msg = 'Error in connecting database: ' + err
        console.log('\x1b[41m\x1b[37m ' + msg + ' \x1b[0m')
    } else {
        const msg = 'Success in connecting database'
        console.log('\x1b[42m\x1b[37m ' + msg + ' \x1b[0m')
        client.on('notification', (msg) => {
            console.log(msg.payload)
        })
        const query = client.query('LISTEN update_notification')
    }
})

module.exports = knex
Vanortton
  • 51
  • 11
  • 2
    Why did you tag "mongodb" and "postgresql"? – Wernfried Domscheit May 24 '23 at 13:40
  • Because I am using MongoDB to save the statistics. – Vanortton May 24 '23 at 13:41
  • 1
    [Related](https://stackoverflow.com/questions/23788530/detecting-column-changes-in-a-postgres-update-trigger) – Pointy May 24 '23 at 13:42
  • @Pointy Great, but how do I get this notification on my JS? – Vanortton May 24 '23 at 13:49
  • This doesn't seem like a Node issue, but rather a Postgresql feature question, like "is there away to maintain an open socket to a Postgresql server that recieves update notificates for a database or a set of tables?". I don't know the answer to that. – Pointy May 24 '23 at 14:05
  • You might be able to rig up something that leverages an OS-supplied file monitor API and watch the Postgresql log for particular messages. – Pointy May 24 '23 at 14:06

2 Answers2

2

Postgres supports LISTEN and NOTIFY commands that can be used to inform other applications of changes in the db. These can be used to avoid polling.

socket.io's postgres-adapter supports notifications over websockets.

This video may also be helpful.

Michel Floyd
  • 18,793
  • 4
  • 24
  • 39
  • Great, but my English is limited because I am not a native speaker, my native language is Brazilian Portuguese – Vanortton May 25 '23 at 12:33
  • In addition to that I have already tried to do something. I have edited my question adding what I tried to do, the problem is that I am not being notified when changes occur. – Vanortton May 25 '23 at 12:34
0

You can store the statistics in mongodb, and use watch to monitor changes.

https://www.mongodb.com/docs/drivers/node/current/usage-examples/changeStream/

PS: suggesting this because you mentioned you are using mongodb to store the statistics.

Someone Special
  • 12,479
  • 7
  • 45
  • 76