1

The page running inside puppeteer download images. When calling page.evaluate, some of the images must be written to disk before doing other operations on them.

What's the best way to do this? Be able to write those images from the browser running in puppeteer? Send buffers from puppeteer to node.js?

NOTE: some of these images might be result canvas operations, so they're not necessary the result of a request.

widgg
  • 1,358
  • 2
  • 16
  • 35
  • 1
    Should I run a server on the Node.js side and send them like this? – widgg Sep 16 '19 at 20:24
  • Hi widgg. How are you? Since there is so many possibilities and there is not a solution that works for every possibility, can you specify a little more what kind of images are you talking about? For example, are those images from the same origin? If not, the origin sets CORS headers accordinly ? You may also say what have you tried already, etc. – lcrespilho Sep 17 '19 at 11:55

1 Answers1

1

There is a few possible solutions:

1. If the page allows cors, you can send those images to your backend server. Use something like fetch
2. Another option is to get all images from page and pass them back to server. Here is also a few possible implementations:
2.1. You can get all image urls with something like this return JSON.stringify(Array.from(document.querySelectorAll('img'), i => i.src)) and then download them directly to your server.
2.2. Put image to canvas, and then use toDataUrl() to get base64 encoded data, which you could then pass and process on backend. More info here and here is about saving base64 data on disk


P.S. Just remember to JSON.stringify() your data when returning from puppeteer to backend, because puppeteer could not process the data.

Grynets
  • 2,477
  • 1
  • 17
  • 41
  • actually, for those that are not results for canvas operation, I can just send the URLs and redownload them on the node.js side. That solves this part of the problem and the canvases are more rare a bit. I'm also considering breaking buffers in chunks and use an exposed function to send them. It's slow, but it would work well. Good thing speed is not the most important part (yet) of this – widgg Sep 17 '19 at 13:27