Azure Functions EventhubTrigger - database inserts

Question

i implemented some code in azure functions which is triggered by a even hub.
If triggered i want to insert the data to a azure sql database. I got my code running and the bulk insert is working but i often get a RequestError Timeout.

Could somebody give me some advice how to implement this use case the right way with azure functions. The function is actually triggered pretty often because of data which is send to event hub.

[2022-03-30T13:13:07.464Z] Executing 'Functions.hotDatatoSql' (Reason='(null)', Id=a329b220-6ef6-4d75-9c15-beed6d7375cb)
[2022-03-30T13:13:07.466Z] Trigger Details: PartionId: 0, Offset: 85929768072-85930225256, EnqueueTimeUtc: 2022-03-29T16:46:03.5990000Z-2022-03-29T16:47:49.0860000Z, SequenceNumber: 351947-352202, Count: 256
[2022-03-30T13:13:23.109Z] RequestError: Timeout: Request failed to complete in 15000ms
[2022-03-30T13:13:23.111Z] Executed 'Functions.hotDatatoSql' (Succeeded, Id=a329b220-6ef6-4d75-9c15-beed6d7375cb, Duration=15717ms)
[2022-03-30T13:13:23.113Z]     at BulkLoad.done [as callback] (C:\Home\Software\azure_functions\node_modules\mssql\lib\tedious\request.js:307:19)
[2022-03-30T13:13:23.142Z]     at Parser.<anonymous> (C:\Home\Software\azure_functions\node_modules\tedious\lib\connection.js:2910:26)
[2022-03-30T13:13:23.145Z]     at Object.onceWrapper (node:events:509:28)
[2022-03-30T13:13:23.148Z]     at Parser.emit (node:events:390:28)
[2022-03-30T13:13:23.150Z]     at Readable.<anonymous> (C:\Home\Software\azure_functions\node_modules\tedious\lib\token\token-stream-parser.js:32:12)
[2022-03-30T13:13:23.152Z]     at Readable.emit (node:events:390:28)
[2022-03-30T13:13:23.153Z]     at endReadableNT (node:internal/streams/readable:1343:12)
[2022-03-30T13:13:23.162Z]     at processTicksAndRejections (node:internal/process/task_queues:83:21) {
[2022-03-30T13:13:23.164Z]   code: 'ETIMEOUT',
[2022-03-30T13:13:23.167Z]   originalError: RequestError: Timeout: Request failed to complete in 15000ms

My Azure Function Code:

const mssql = require('mssql');
const { get } = require('./pool-manager')

const config = {
    user: "...",
    password: "...",
    server: '....',
    database: '...',
    pool: {
        max: 10,
        min: 0,
        idleTimeoutMillis: 30000
    }, 
    options: {
        encrypt: true,
        trustServerCertificate: true
    }
};

const table = new mssql.Table('dbo.testTable');
table.columns.add('row1', mssql.VarChar(512), {nullable: true});
table.columns.add('row2', mssql.DateTime2, {nullable: true});
table.columns.add('row3', mssql.NVarChar(mssql.MAX), {nullable: true});

module.exports = async function (context, eventHubMessages) {

    const pool = await get('default', config);

    eventHubMessages.forEach((message, index) => {

        table.rows.add(message.id, message.time, JSON.stringify(message.data));
    });

    const request = new mssql.Request(pool);

    try{
        let result = await request.bulk(table);
        //console.log(result);
    }
    catch(err){
        console.log(err);
    }
};

The db operation takes more than 15 seconds, which is the default timeout. Increase the timeout. See https://stackoverflow.com/questions/40190999/node-js-server-not-waiting-for-more-than-approx-15-sec — Peter Bons, Mar 30 '22 at 13:57

score 0 · Accepted Answer · answered Mar 31 '22 at 06:18

To achieve the above requirements, As suggested by @Peter Bons we need to increase the timeout as default timeout is 15 seconds .

For example to increase the idletimeout:

const config = {
    user: '...',
    password: '...',
    server: 'localhost',
    database: '...',
    pool: {
        max: 10,
        min: 0,
        idleTimeoutMillis: 300000
    }
}

For more information you can refer this SO THREAD as well.

Azure Functions EventhubTrigger - database inserts

1 Answers1