1

I am new to JavaScript/Node.js and image processing and would like to take the output of puppeteer screenshot (preferably in memory) then be able to traverse colors of each pixel in the image, then stroke a rectangle border around certain coordinated of the image (x, y, width, height), then write save the image with the rectangle highlights to file using Node.js.

What I get from puppeteer is

returns: <Promise<string|Buffer>> Promise which resolves to buffer or a base64 string (depending on the value of encoding) with captured screenshot.

I read that I can create a canvas object using puppeteer to do this but I am not sure about this solution especially to retrieve pixel colors. I don't know if I should be using a package like PureImage for this or some other package.

mkrieger1
  • 19,194
  • 5
  • 54
  • 65
Ibrahim
  • 85
  • 5

2 Answers2

0

If you need to draw, the easiest way would be using canvas. First, you need to create canvas from your image and then draw desired shape. After that convert your canvas back to img format.

This shows you how to create canvas from image: 

https://www.w3schools.com/tags/tryit.asp?filename=tryhtml5_canvas_drawimage

This fiddle shows you how to extract each pixel color from canvas and turn it from rgb to hex: 

https://jsfiddle.net/ourcodeworld/8swevoxo/9/?utm_source=website&utm_medium=embed&utm_campaign=8swevoxo

And finally this fiddle shows you how to convert canvas back to image.

http://jsfiddle.net/robhawkes/9fKV9/

I hope this helps you enough to get you going.

aleksa_95
  • 251
  • 3
  • 7
  • Thanks, I do not know the height or width of the image from the buffer returned by Puppeteer... I assume it is necessary to specify the canvas size. I also wonder if there are risks in converting from/to images using canvas. I need it to be pixel perfect size match after conversion from image to canvas. – Ibrahim Feb 03 '20 at 16:01
0

Here is my code that....

  1. Handle a base64 screenshot taken by Puppeteer.
  2. Create an image element.
  3. Wait for the image onload event.
  4. set the src attribute of an image to the screenshot.
  5. Create canvas not attached to DOM and write image to it.
  6. Stroke two rectangles on the canvas.
  7. return the canvas data.

async function (rectangle1, rectangle2, screenshot) {
        let screenshotHighlighted = await driver.page.evaluate(async function (rectangle1, rectangle2, screenshot) {
            let image = new Image();
            let imgLoadPromise = async function(){ return new Promise((resolve, reject) => {
                image.onload = () => {console.log('loaded'); return resolve};
                image.onerror = reject;
            });}
            image.src = 'data:image/png;base64,' + screenshot;
            await imgLoadPromise;
            let canvas = document.createElement("CANVAS");
            let context = canvas.getContext("2d");
            canvas.width = image.naturalWidth;
            canvas.height = image.naturalHeight;
            context.drawImage(image, 0, 0);
            //solid black and dashed red
            context.strokeStyle = "black";
            context.strokeRect(rectangle1.minX, rectangle1.minY, rectangle1.width, rectangle1.height);
            context.setLineDash([5, 10]);
            context.strokeStyle = "red";
            context.strokeRect(rectangle1.minX, rectangle1.minY, rectangle1.width, rectangle1.height);
            context.setLineDash([]);
            //solid black and dashed yellow
            context.strokeStyle = "black";
            context.strokeRect(rectangle2.minX, rectangle2.minY, rectangle2.width, rectangle2.height);
            context.setLineDash([5, 10]);
            context.strokeStyle = "yellow";
            context.strokeRect(rectangle2.minX, rectangle2.minY, rectangle2.width, rectangle2.height);
            return canvas.toDataURL();
        }, rectangle1, rectangle2, screenshot);
        return screenshotHighlighted;
    }

To save the file in Node.JS

fs.writeFile(path.join(directory,imageFileName),highlightedScreenshot.split(',')[1], 'base64', 
  function(err) {
        if(err) {
                    console.log('ERROR IN SAVING IMAGE');
                    console.log(err);
                } 
            });

Traversing pixels of an HTML canvas can be found in the following answer

Get Pixels from HTML Canvas

Ibrahim
  • 85
  • 5