1

I'm using this Node ImageMagick for validating images via converting them in AWS Lambda function. Locally I have no problem, but when I deploy my function I get the error no decode delegate for this image format on some images only:

{ Error: Command failed: convert: no decode delegate for this image format `/tmp/925bf249f8297827f51f0370642eb560.jpg' @ error/constitute.c/ReadImage/544.
convert: no images defined `/tmp/5d2baeb2-de13-4868-a970-ad919c609440.png' @ error/convert.c/ConvertImageCommand/3046.

at ChildProcess.<anonymous> 
(/var/task/node_modules/imagemagick/imagemagick.js:88:15)
at emitTwo (events.js:106:13)
at ChildProcess.emit (events.js:191:7)
at maybeClose (internal/child_process.js:886:16)
at Socket.<anonymous> (internal/child_process.js:342:11)
at emitOne (events.js:96:13)
at Socket.emit (events.js:188:7)
at Pipe._handle.close [as _onclose] (net.js:497:12) timedOut: false, killed: false, code: 1, signal: null }

This is after I failed to use ImageMagick that's built-in for AWS Lambda. Any idea how I can solve this?

Lee Maan
  • 699
  • 3
  • 10
  • 30
  • Can you please share your package.json file along with what you receive when you run node -v on your system? – Mavi Domates Dec 18 '18 at 13:39
  • @MaviDomates I updated the question with package.json. When it woks on my machine it just succeed and it saves the file in a new location, but in Lambda it's failing. – Lee Maan Dec 18 '18 at 13:46
  • please share your result for `node -v` (node version) command – Mavi Domates Dec 18 '18 at 13:51
  • So you are not using the nodejs package in your AWS / but using a different one? – Mavi Domates Dec 18 '18 at 13:53
  • 1
    Might be worth confirming the file is actually an image (maybe spawn `identify ` in your lambda and get the output); it's possible you are somehow saving it incorrectly. Also checkout related question https://stackoverflow.com/questions/44729088/how-can-i-install-graphicsmagick-or-imagemagick-on-aws-lambda. GL. – Elliot Nelson Dec 18 '18 at 13:55
  • @MaviDomates I'm using Node 6.10.3 (I can't upgrade due to other things). I am using my Nodejs package in AWS and I only wanted that. The file passed to Node in AWS is correct. I logged it, downloaded it and tested it locally and conversion worked fine. There is something wrong with ImageMagick when it runs in AWS Lambda. – Lee Maan Dec 18 '18 at 13:59
  • @LeeMoe Are you using a different package than what you are using locally for conversion, or are you using your imagemagick package on AWS as well? – Mavi Domates Dec 18 '18 at 14:01
  • @MaviDomates I'm not intending to use another package. As you can see ImageMagick is in my package.json, so I assume AWS Lambda is using it. Isn't the way it happens? – Lee Maan Dec 18 '18 at 14:05
  • @MaviDomates I heard somewhere else that I might need to "provide the input image as a buffer". Any idea what this mean how to do it? – Lee Maan Dec 18 '18 at 14:17
  • I don't think that's what you're having mate. I'm thinking, you only mentioned that this happens with *some* images. Can you confirm that this only happens with *.jpg extensions? – Mavi Domates Dec 18 '18 at 14:19
  • @MaviDomates Yes it happens only with some images, and they are in JPG. But what I know JPG can have many formats in terms of colorspace and other things. Not all JPG files fail though. Weird eh? – Lee Maan Dec 18 '18 at 14:44
  • So not all JPGs fail? Do you have any PNGs that are failing / or are you testing this with any other file formats? – Mavi Domates Dec 18 '18 at 14:46
  • @MaviDomates What confused the crap out of me why the error is `no decode delegate for this image format`. As if the decode delegate refused to download online while it exists in my machine. If so then how can I get it installed online? – Lee Maan Dec 18 '18 at 15:23
  • 1
    If all JPGs fail, then you might not have the jpg delegate installed in ImageMagick. Or it is an old libjpeg and may not work on your JPG files. Also if those JPG files are really JP2 (JPEG 2000), then you may not have the openjpeg 2.1 or higher delegate installed. Sorry I do not know AWS Lambda, but are you converting from JPG to PNG or PNG to JPG. Check that you also have libpng delegate installed. You can see by `convert -version` or with version numbers by `convert -list format`. – fmw42 Dec 18 '18 at 18:15
  • @fmw42 Thank you for the comment. I understand that by now. Yes the files are not all JPGs in reality. Some of my users simply rename their files to JPG believing they have done the job And yes I do have an image delegate problem also, but the issue is that it's so hard to get these delegates installed in an AWS Lambda environment. I'm working on it. – Lee Maan Dec 20 '18 at 08:37
  • 1
    When you figure out how to do that install, please let other know. It seems to be a common problem that others have questioned also. – fmw42 Dec 20 '18 at 21:19
  • @fmw42 Done! I posted an answer. – Lee Maan Jan 08 '19 at 00:00

1 Answers1

3

Finally I figured it out! There is no short way for this. This is what I ended up doing:

  • I ran Parallel on my Mac and I got the binaries of ImageMagick installed on a virtual CentOS machine. To install the binaries I followed the guides in the official website. I wrote a lot of commands in my virtual machine but I can summarise them in:

    yum install sudo -y
sudo yum -y install libpng-devel libjpeg-devel libtiff-devel gcc
sudo curl -O http://www.imagemagick.org/download/ImageMagick.tar.gz
sudo yum install tar -y
sudo tar zxvf ImageMagick.tar.gz
cd ImageMagick-7.0.8-22
sudo ./configure --prefix=/var/task/imagemagick --enable-shared=no --enable-static=yes
sudo yum install make -y
sudo yum install automake autoconf libtool -y
sudo sed -i 's|(gitversion|(./gitversion|' configure.ac
sudo yum install git -y
sudo autoreconf -i
sudo sed -i '$a LANG=en_US.utf-8' /etc/environment
sudo sed -i '$a LC_ALL=en_US.utf-8' /etc/environment
sudo make
sudo make install
tar zcvf ~/imagemagick.tgz /var/task/imagemagick/`

  • Then I copied the installation folder from the virtual machine into a folder I named /lib in my root directory of AWS Lambda repository.

  • Then in order to use the new binaries, I needed to copy ImageMagick folder from /lib into my node_modules on every deployment. This is because AWS Lambda didn't allow me to access any binary file from the /lib directly for unknown reason. In order to perform the copy on every deployment I added the following line to my deploy.sh file:

    `cp -R ./lib/imagemagick ./node_modules/imagemagick`

  • Now the binaries are ready to be used, but still AWS Lambda didn't give me permission to run any command directly from within node_modules. So every time I needed to run a particular command I needed to copy it (it is a binary file) into the /tmp folder and then I needed to change its mode chmod to be able to run it. This is the code for every needed command (in Node):

    const command = '/node_modules/imagemagick/[command path and file]';
execSync(`cp -a ${command} /tmp/`);
fs.chmodSync(command, 755);
execSync(`chmod +x ${command}`);

  • Now the command is ready to be used with child_process. For example:

    const argus = [originalImage, '-o', newImage];
child_process.execFile(command, argus, (err, stdout, stderr) (
    if (err) throw err;
    console.log('IMAGE CONVERTED');
)}

  • The above applies not only on ImageMagick, but on any other binary needed with AWS Lambda. I applied the same on Google's WebP library somewhere else in my function. I downloaded its Linux binaries from Google developers website.

This is it. My code works as expected. If you have any better idea on improving it I would appreciate a comment.

Lee Maan
  • 699
  • 3
  • 10
  • 30