"Invalid ELF header" using libxmljs on AWS Lambda

Question

I have a super basic AWS Lambda function using serverless, express, and libxmljs (which binds JavaScript to libxml):

Code

serverless.xml:

service: xmltest

provider:
  name: aws
  runtime: nodejs10.x
  stage: dev
  region: us-east-1

functions:
  app:
    handler: index.handler
    events:
      - http: ANY /
      - http: 'ANY {proxy+}'

plugins:
  - serverless-offline

index.js:

const serverless = require('serverless-http');
const express = require('express');
const libxml = require("libxmljs");

const app = express();

app.use(express.json());

app.post('/lookup', async function (req, res) {
  res.send({result: "hello world"});
});

module.exports.handler = serverless(app);

Works Locally But Not on AWS

When I run locally:

$ curl -X POST http://localhost:3000/lookup -d {"example":123}

{"result":"hello world"}

When I run on AWS:

$ curl -X POST https://REDACTED.execute-api.us-east-1.amazonaws.com/dev/lookup -d {"example":123}

{"message": "Internal server error"}

The CloudWatch logs say:

2019-06-05T12:46:43.280Z    undefined   ERROR   Uncaught Exception
{
    "errorType": "Error",
    "errorMessage": "/var/task/node_modules/libxmljs/build/Release/xmljs.node: invalid ELF header",
    "stack": [
        "Error: /var/task/node_modules/libxmljs/build/Release/xmljs.node: invalid ELF header",
        "    at Object.Module._extensions..node (internal/modules/cjs/loader.js:730:18)",
        "    at Module.load (internal/modules/cjs/loader.js:600:32)",
        "    at tryModuleLoad (internal/modules/cjs/loader.js:539:12)",
        "    at Function.Module._load (internal/modules/cjs/loader.js:531:3)",
        "    at Module.require (internal/modules/cjs/loader.js:637:17)",
        "    at require (internal/modules/cjs/helpers.js:22:18)",
        "    at bindings (/var/task/node_modules/bindings/bindings.js:84:48)",
        "    at Object.<anonymous> (/var/task/node_modules/libxmljs/lib/bindings.js:1:99)",
        "    at Module._compile (internal/modules/cjs/loader.js:701:30)",
        "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:712:10)"
    ]
}

Question

How can I import libxmljs on AWS?

I see some related questions like

So I’m guessing something in libxmljs is built for a different architecture (my local machine is macOS) than Amazon Linux. But I'm not sure how I can fix this?

"When I run locally" -- what OS and what processor does your localhost have? — Employed Russian, Jun 06 '19 at 02:25
"Mac OS ..." -- did you copy `node_modules/libxmljs/build/Release/xmljs.node` from local host to AWS? If so, your question is *exact* duplicate of https://stackoverflow.com/questions/29994411. You must build a different version for AWS, using a compiler that targets Linux (easiest to do on a Linux machine, possibly on a virtual one). — Employed Russian, Jun 06 '19 at 13:46
Possible duplicate of ["invalid ELF header" when using the nodejs "ref" module on AWS Lambda](https://stackoverflow.com/questions/29994411/invalid-elf-header-when-using-the-nodejs-ref-module-on-aws-lambda) — Employed Russian, Jun 06 '19 at 13:47
Yes, I guess my question is how to do that. I am deploying by running `sls deploy`. I don't know how either serverless or the AWS lambda deploy process works well enough to know exactly where the compile step is. All I did was add `libxmljs` to `package.json`, run `npm install`, and then run `sis deploy`. — Aaron Brager, Jun 06 '19 at 13:52
Is it as simple as running `npm install && sls deploy` from inside a docker image? Or is there some CI server that has easy setup for GitHub -> Serverless Deploy on merge? Or some other best practice? — Aaron Brager, Jun 06 '19 at 13:55

score 6 · Accepted Answer · answered Jun 08 '19 at 03:35

6

I solved the problem by running npm install on a Linux box instead of my Mac.

In my case I set up AWS CodePipeline which runs all the build scripts on an EC2 instance. Could also have solved this using other hosted CI pipeline, a Docker container, etc.

answered Jun 08 '19 at 03:35

Aaron Brager

65,323
19
161
287

score 0 · Answer 2 · answered Sep 15 '20 at 15:31

I had the same problem with Pusher. After reading @aaron-brager's answer, I realized the problem and try this. It worked for me.

#!/bin/bash

export PKG_DIR="python/lib/python3.6/site-packages/"

rm -rf ${PKG_DIR} && mkdir -p ${PKG_DIR}

docker run --rm -v $(pwd):/foo -w /foo lambci/lambda:build-python3.6 \
    pip3 install -r requirements.txt  -t ${PKG_DIR}

"Invalid ELF header" using libxmljs on AWS Lambda

Code

Works Locally But Not on AWS

Question

2 Answers2