5

I'm trying to run node apps that use the npm package Puppeteer to scrape websites, using VPSes that use root as the default user I login as.

It never works. I always have to spend half an hour googling to find the solution... well no more.

jspinella
  • 2,013
  • 3
  • 25
  • 39

3 Answers3

12

Do this:

0. Run sudo apt update

1. Install libx stuff

sudo apt-get install libpangocairo-1.0-0 libx11-xcb1 libxcomposite1 libxcursor1 libxdamage1 libxi6 libxtst6 libnss3 libcups2 libxss1 libxrandr2 libgconf-2-4 libasound2 libatk1.0-0 libgtk-3-0

2. Set --no-sandbox argument in the launch() part of your Puppeteer app (only required if you are running as root/root's cron)

const browser = await puppeteer.launch({headless: true, args: ['--no-sandbox']})

E: as Niko pointed out, --no-sandbox opens your server to a malicious attack from a website the scraper visits. It looks like there are some StackOverflow questions elsewhere that post alternatives that aren't awful to implement. I like David's answer here which seems to suggest that even in 2021 --no-sandbox is still a necessary evil in some cases.

E2: See answer below. It looks like in 2021 we also have to install libgbm-dev

Smiter
  • 1,069
  • 1
  • 14
  • 33
jspinella
  • 2,013
  • 3
  • 25
  • 39
5

Connecting to the server by SSH:

  1. Open your terminal
  2. Type: ssh root@[server ip]
  3. If it is the first time you connect to the server you will get a security prompt. Type 'yes'
  4. Fill in your password:  [password]
  5. You should now be connected to your server

Installing the app:

sudo apt-get update

sudo apt-get install -y libgbm-dev

sudo apt install -y gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget

sudo apt install curl

curl -sL https://deb.nodesource.com/setup_14.x | sudo -E bash -

sudo apt install nodejs

sudo apt install git

git config --global user.name "your name"

git config --global user.email "your email"

git clone "your project git url"

cd "your project folder"

npm i

npm start


If you need to run multiple apps on one VPS server - google tool "Screen"

Nik B
  • 617
  • 6
  • 11
0

After I installed libgbm, everything works fine for me! hope that help you out

    sudo apt-get update
    sudo apt-get install -y libgbm-dev
  • It looks like `libgbm-dev` is a new dependency for Puppeteer. I'll add it to my answer so it's current. Did you have to install anything else e.g. what I installed above? – jspinella Apr 11 '21 at 22:33