lambda function not able to send custom metric to datadog through dd-trace npm library

Question

I am trying to send custom metric from lambda functions to datadog. I have already configured lambda to send custom metric to lambda by following this this (Following custom installation). I have already:

1. Installed the Datadog Lambda library
2. Installed the Datadog Lambda Extension

This is how my lambda function code looks like

require("dotenv").config();
const tracer = require("dd-trace").init();
let response;

exports.lambdaHandler = tracer.wrap(
  "twilio.DDTraceFunction",
  async (event, context) => {
    try {
      tracer.dogstatsd.increment("example_metric.increment", 1, [
        "environment:dev",
      ]);
      tracer.dogstatsd.decrement("example_metric.decrement", 1, [
        "environment:dev",
      ]);

      response = {
        statusCode: 200,
        body: JSON.stringify({
          message: "hello world",
        }),
      };
    } catch (err) {
      console.log("err===============>", err);

      return {
        statusCode: 500,
        body: JSON.stringify("An error occurred"),
      };
    }

    return response;
  }
);

Currently I am trying to test it locally using aws sam cli. This is Error that I am encountering with

➜  sam-app sam local invoke DDTraceFunction                    
Invoking /opt/nodejs/node_modules/datadog-lambda-js/handler.handler (nodejs14.x)                                                          
arn:aws:lambda:ap-south-1:464622532012:layer:Datadog-Extension:10 is already cached. Skipping download                                    
arn:aws:lambda:ap-south-1:464622532012:layer:Datadog-Node14-x:62 is already cached. Skipping download                                     
Local image is up-to-date                                                                                                                 
Using local image: samcli/lambda-nodejs:14-x86_64-12d24702a7805bbe910f9c1a2.                                                              
                                                                                                                                          
Mounting /Users/global/Desktop/test/datadog/sam-app/lib-dd-trace as /var/task:ro,delegated, inside runtime container                  
START RequestId: a3b8482b-2d9d-4094-b7d7-dc82c747d66e Version: $LATEST
2023-08-18T04:03:24.243Z    a3b8482b-2d9d-4094-b7d7-dc82c747d66e    INFO    err===============> TypeError: Cannot read property 'incre    at step (/opt/nodejs/node_modules/datadog-lambda-js/index.js:44:23)ndex.js:168:62)acer.js:42:29).js:57:14)
END RequestId: a3b8482b-2d9d-4094-b7d7-dc82c747d66e
REPORT RequestId: a3b8482b-2d9d-4094-b7d7-dc82c747d66e  Init Duration: 0.55 ms  Duration: 3807.39 ms    Billed Duration: 3808 ms    Memory Size: 128 MB Max Memory Used: 128 MB 
{"statusCode": 500, "body": "\"An error occurred\""}%                       

This is how my function looks in template.yml looks

  DDTraceFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: lib-dd-trace/
      Handler: /opt/nodejs/node_modules/datadog-lambda-js/handler.handler
      Runtime: nodejs14.x
      Architectures:
        - x86_64
      Environment:
        Variables:
          DD_SERVERLESS_LOGS_ENABLED: true
          DD_TRACE_ENABLED: true
          DD_LAMBDA_HANDLER: app.lambdaHandler
      Layers:
        - arn:aws:lambda:ap-south-1:464622532012:layer:Datadog-Extension:10 
        - arn:aws:lambda:ap-south-1:464622532012:layer:Datadog-Node14-x:62

DD_API_KEY, DD_SITE, DD_SERVERLESS_LOGS_ENABLED, DD_TRACE_ENABLED, DD_LOGS_INJECTION env variable are already set template.yml global's section.

I am not able to access dogstatsd in tracer function. I believe dogstatsd is responsible for sending these metrics to datadog, but in my case tracer.dogstatsd is coming as undefined. In case dogstatsd is not available in tracer, which function of tracer shall I use to send custom metric count metric, rate metric , set metric etc to datadog. Any help is really appreciable.

Answer 1

Update:

A Lambda function may launch many concurrent execution environments when traffic increases. The function may submit count or gauge metric points that overwrite each other and cause undercounted results.

To avoid this problem, custom metrics generated by Lambda functions are submitted as distributions because distribution metric points are aggregated on the Datadog backend, and every metric point counts.

https://docs.datadoghq.com/serverless/custom_metrics/#understanding-distribution-metrics

Original Answer:

Since you already have the datadog-lambda-js, you can use that instead of dd-trace directly.

const { sendDistributionMetric } = require('datadog-lambda-js');

exports.lambdaHandler = async (event, context) => {
    try {
        sendDistributionMetric(
            'example_metric.increment',
            1,
            'environment:dev',
            'order:online'
        );

        return {
            statusCode: 200,
            body: JSON.stringify({
                message: "hello world",
            }),
        };
    } catch (err) {
        console.log("err===============>", err);

        return {
            statusCode: 500,
            body: JSON.stringify("An error occurred"),
        };
    }
}