-1

I want to take the GCC Compiler that is on my machine and all its dependencies and zip them up in a deployment package that I can send off to AWS Lambda (That way I can use a Lambda to compile C code). Is there an easy way to package the whole thing in one go so I can deploy and use it from AWS Lambda?

This is what I have right now

enter image description here

However when I invoke the function I get 

"gcc: error trying to exec 'cc1': execvp: No such file or directory\n"

as the response. Currently the way I compile gcc and those dependencies you see on the left panel was by spinning up a Amazon Linux docker container, installing gcc, and then zipping up gcc and its dependencies I found with the ldd command.

HashBr0wn
  • 387
  • 1
  • 11

3 Answers3

1

AWS Lambda runtime is described here. Basically, it's Amazon Linux. If I were you, I would try to grab the specified AMI and create a EC2 instance with it. Or just create an Amazon Linux 2 EC2 instance. Then I would log in to that instance and compile the binaries you need. Finally, I would export them in a ZIP file and ship with Lambda. This way chances are high that the binaries would work on Lambda.

madhead
  • 31,729
  • 16
  • 153
  • 201
  • I was more wondering about packaging gcc as a deployment package for Lambda? That way the compilation of a file can just be a Lambda function. Like the actual packaging of gcc is what I am struggling with. – HashBr0wn Jun 17 '20 at 20:32
  • Yeah, read about [custom runtimes](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html)? – madhead Jun 17 '20 at 21:16
  • This is not answering my question. I want to deploy a Lambda who's functionality is to use gcc to compile source code and then send that compiled code back or to maybe an S3 bucket. I am not trying to use a custom runtime or an EC2 instance – HashBr0wn Jun 18 '20 at 01:53
  • Still, it's better to install GCC (or compile it) on an Amazon Linux 2 (in EC2 or [Docker](https://hub.docker.com/_/amazonlinux) and then grab it from that instance (container) and bundle into your Lambda. This way, the build environment of your GCC will match the Lambda runtime. – madhead Jun 18 '20 at 09:06
1

Great question!

As you say, just packaging the binary doesn't help because you're missing the shared objects .so files, or some other dependencies. You can find your dependencies by running something like ldd, and this question helps. Projects like yumda, try to simplify this, and definitely worth a shot.

But under the hood, Lambda uses AmazonLinux and there's really no reason it can't be done. High level points we need are:

  • Build the binary in a AmazonLinux container
  • Determine the binary, and it's dependencies
  • Copy the binary and dependencies out of the container into a lambci container
  • Test out the lambci container (you'd typically need to set some env vars for this to work, e.g. $LD_LIBARY_PATH.
  • Once it runs, package that as a zip, and load it into your lambda, remembering to set the right env vars
  • As an option, I'd package gcc as a layer, so you can share it.

When i searched around, it looks like someone has done exactly this here. Hopefully it's exactly what you're looking for.

keithRozario
  • 376
  • 2
  • 4
  • Thanks for the straightforward answer. I used ldd to track down the dependencies and packaged those together to when I sent the deployment package. When I invoke the lambda however I get this error msg "gcc: error trying to exec 'cc1': execvp: No such file or directory". The gcc executable and its dependencies are just in the project folder of the Lambda so is this coming up because the location of stuff is different than expected? – HashBr0wn Jun 18 '20 at 17:46
  • I edited my original question to reflect the current situation I am in. – HashBr0wn Jun 18 '20 at 18:05
  • Have you tried setting your $PATH variables correctly? – keithRozario Jun 19 '20 at 01:13
  • Is it not being done correctly (reference code snapshot above in question)? – HashBr0wn Jun 19 '20 at 01:30
  • I think your path would be wrong, because you’re missing a ;. Try to print out the variables after setting them to check. – keithRozario Jun 19 '20 at 04:27
  • where am I missing a semicolon – HashBr0wn Jun 19 '20 at 15:51
  • Generally for $PATH we tend to do this `export PATH=$HOME/.local/bin:$PATH` , so that we add our path. Think of path as a list where colon is the separator. – keithRozario Jun 20 '20 at 04:15
0

Configure, build and install gcc into a specific directory, specified by --prefix option to configure.

After installing, change gcc spec file, so that it hardcodes -rpath into executables and shared libraries, so that you do not need to tinker with LD_LIBARY_PATH (which is a wrong solution most of the time) to make the executables find the right libstdc++.so, libgcc_s.so and its friends.

rsync the directory onto another machine into the same place in filesystem.

Or archive the install directory and unpack it on your target machine.

However, the target must should have the same libc and system libraries gcc was built with, otherwise that may not work as intended.

Alternatively, build locally and deploy your executables with all their dependencies using Exodus.

Maxim Egorushkin
  • 131,725
  • 17
  • 180
  • 271