0

I’m trying

to create a mini Wireshark/debuging playground

I want to store all the requests & responses that pass through my API Back End so I can leverage them to debug what was going on.

The main goal is to generate the logs table on a web page with the ability to export as JSON format.

I have

an API written in Node JS using Express connecting to Postgres Database via Sequelize

I have a lot a requests come through my API.

Here are an example my requests

POST /api/login 
POST /api/getSessionTimeOut 
POST /api/checkIfGroupExist/25050-telenet 
POST /api/listUsersInGroup/25050-telenet 
POST /api/primary/createVxLan/ingress/103 
POST /api/primary/createVxLan/egress/103 
POST /api/primary/createSwitch/103 
POST /api/primary/createVxLan/ingress/104 
POST /api/primary/createVxLan/egress/104 
POST /api/primary/createSwitch/104 
POST /api/backup/createVxLan/ingress/103 
POST /api/backup/createVxLan/egress/103 
POST /api/backup/createSwitch/103 
POST /api/backup/createVxLan/ingress/104 
POST /api/backup/createVxLan/egress/104 
POST /api/backup/createSwitch/104 
POST /api/primary/installDevice 
POST /api/monitor/2724 
...
POST /api/monitor/2724 
POST /api/backup/installDevice 
POST /api/monitor/2725 
... 
POST /api/monitor/2725 
POST /api/createDynamicInterface/ingress/103 
POST /api/createDynamicInterface/egress/103 
POST /api/createDynamicInterface/ingress/104 
POST /api/createDynamicInterface/egress/104 
POST /api/createPolicyFirewall/v4/103/vpn 
POST /api/createPolicyFirewall/v4/104/inline 
POST /api/createPolicyFirewall/v4/103/inline 
POST /api/createPolicyFirewall/v4/103/inline

POST /api/createPolicyFirewall/v6/103/vpn 
POST /api/createPolicyFirewall/v6/103/inline 
POST /api/createPolicyFirewall/v6/104/inline 

POST /api/createPolicyFirewall/v6/103/inline

POST /api/installPackage/inline 

POST /api/monitor/2726 
... 
POST /api/monitor/2726 
POST /api/installPackage/vpn 
POST /api/monitor/2727 
... 
POST /api/monitor/2727

I would like to store each request into a logs table in my database.

I’ve tried

Migration

module.exports = {
    up: (queryInterface, Sequelize) =>
        queryInterface.sequelize.query('CREATE EXTENSION IF NOT EXISTS "uuid-ossp";')
            .then(() => {
                queryInterface.createTable('Logs', {
                    id: {
                        allowNull: false,
                        primaryKey: true,
                        type: Sequelize.Sequelize.UUID,
                        defaultValue: Sequelize.literal('uuid_generate_v4()')
                    },
                    user: {
                        type: Sequelize.STRING,
                        allowNull: true
                    },
                    accountId: {
                        type: Sequelize.STRING,
                        allowNull: true
                    },
                    cpeMac: {
                        type: Sequelize.STRING,
                        allowNull: false
                    },
                    pHnsId: {
                        type: Sequelize.STRING,
                        allowNull: true
                    },
                    gHnsId: {
                        type: Sequelize.STRING,
                        allowNull: true
                    },
                    serviceType: {
                        type: Sequelize.STRING,
                        allowNull: true
                    },
                    securityCluster: {
                        type: Sequelize.STRING,
                        allowNull: true
                    },
                    method: {
                        type: Sequelize.STRING,
                        allowNull: true
                    },
                    portalUrl: {
                        type: Sequelize.STRING,
                        allowNull: true
                    },
                    apiUrl: {
                        type: Sequelize.STRING,
                        allowNull: true
                    },
                    data: {
                        type: Sequelize.STRING,
                        allowNull: true
                    },
                    response: {
                        type: Sequelize.STRING,
                        allowNull: true
                    },
                    createdAt: {
                        type: Sequelize.DATE,
                        allowNull: false
                    },
                    updatedAt: {
                        type: Sequelize.DATE,
                        allowNull: false
                    },
                    deletedAt: {
                        type: Sequelize.DATE,
                        allowNull: true
                    }
                })
            }),
    down: (queryInterface) => queryInterface.dropTable('Logs')
};

Model

module.exports = (sequelize, DataTypes) => {
    const Log = sequelize.define('Log', {
        user: {
            type: DataTypes.STRING,
            allowNull: true
        },
        accountId: {
            type: DataTypes.STRING,
            allowNull: true
        },
        cpeMac: {
            type: DataTypes.STRING,
            allowNull: false
        },
        pHnsId: {
            type: DataTypes.STRING,
            allowNull: true
        },
        gHnsId: {
            type: DataTypes.STRING,
            allowNull: true
        },
        serviceType: {
            type: DataTypes.STRING,
            allowNull: true
        },
        securityCluster: {
            type: DataTypes.STRING,
            allowNull: true
        },
        method: {
            type: DataTypes.STRING,
            allowNull: true
        },
        portalUrl: {
            type: DataTypes.STRING,
            allowNull: true
        },
        apiUrl: {
            type: DataTypes.STRING,
            allowNull: true
        },
        data: {
            type: DataTypes.STRING,
            allowNull: true
        },
        response: {
            type: DataTypes.STRING,
            allowNull: true
        }
    });

    const schema = {
        user: "user",
        accountId: "accountId",
        cpeMac: "cpeMac",
        pHnsId: "pHnsId",
        gHnsId: "gHnsId",
        serviceType: "serviceType",
        securityCluster: "securityCluster",
        method: "method",
        portalUrl: "portalUrl",
        apiUrl: "apiUrl",
        data: "data",
        response: "response"
    };

    Log.list = (models) => new Transformer.List(models, schema).parse();
    Log.single = (model) => new Transformer.Single(model, schema).parse();

    return Log;
};

Controller

const Log = require('../models').Log;

module.exports = (config, jwtDecode, Op) => {
    let logs = {};

    /**
     * Create a Log
     *
     * @return {object} log
     */
    logs.create = async(req, res, next) => {
        try {

            let $body = {
                name: log.name,
                accountId: log.accountId,
                cpeMac: log.cpeMac,
                pHnsId: log.pHnsId,
                gHnsId: log.gHnsId,
                serviceType: log.serviceType,
                securityCluster: log.securityCluster,
                method: log.method,
                portalUrl: log.portalUrl,
                apiUrl: log.apiUrl,
                data: log.data,
                response: log.response
            };

            let response = await Log.create($body);
            res.status(200).send(JSON.parse(response));
        } catch (error) {
            next(error);
        }
    };

    return logs;
};

Service

module.exports = (config, request) => {
    let log = {};

    /*==============================
    =            create            =
    ==============================*/

    log.create = ($body) => {
        let $options = {
            method: "POST",
            uri: `/api/logs/create`,
            body: $body
        };

        return new Promise((resolve, reject) => {
            request($options)
                .then(data => resolve(JSON.stringify(data)))
                .catch(error => reject(error));
        });
    };

    return log;
};

route

app.post('/api/logs/create', controllers.logs.create);

Result

Now, that I have all of the pieces ready to go, but I am not sure how to connect all of these to be able to store all the requests/responses in the database?

Community
  • 1
  • 1
code-8
  • 54,650
  • 106
  • 352
  • 604
  • If you are wanting to log the requests your API handles, the log creation doesn't need to be an endpoint, it would probably be easier to have it as middleware. Then for every request, you can invoke you're db call to make that log, with your `req` having all the header/path/etc. information you need. – Joe Lissner May 07 '18 at 20:19
  • @JoeLissner : Thanks for your kind comments, I really like your suggestion a lot. Since I am a bit new to all of these NodeJS flow, can you please leave the answer with a bit more steps, I will use those to guide me further. – code-8 May 07 '18 at 20:52
  • I think what you need are hooks which can perform this logging everytime a request comes in. You can look at [this](https://stackoverflow.com/questions/20175806/before-and-after-hooks-for-a-request-in-express-to-be-executed-before-any-req-a) – AbhinavD May 08 '18 at 00:20
  • What you need to do is called logging. – Vinayk93 May 09 '18 at 17:18

1 Answers1

0

As a basic outline for creating this as middleware, you would do something like this:

App.js

/* You're imports and setup */

/*
    any time a call is made, it will hit this function
    app.use accepts a function, which will give the parameters
    req, res, next.
        i.e. app.use((req, res, next) => { ... })
    so we'll pass in you're logs.create function
    and the request will have all the information on what
    endpoint is being given.
*/
app.use(controllers.logs.create)

/* the rest of your routes */

If you provide your app.js file, I can give you a better version of what is above

Let me know if there is anything else I can do.

Joe Lissner
  • 2,181
  • 1
  • 15
  • 21