Firebase Webscraping Project

Question

I am doing the https://fireship.io/lessons/web-scraping-guide/ project and I am a beginner to webscraping and firebase. I have gotten to the 'npm run serve' part. But how do you pass in an object body like { "text": "https://fireship.io and https://www.instagram.com" } ?? Do I type it into my terminal window, do I put it in the index.js file, before running 'npm run serve' ? I am not using the 'Insomnia' software as mentioned in the video, nor do I really know what it is. I am on a mac. I don't understand how to actually use these 2 webscraping functions. Can anyone explain it simply? Attached is my error from my terminal window. thank you for any help!!

i  functions: Beginning execution of "scraper"
>  (node:26619) UnhandledPromiseRejectionWarning: SyntaxError: Unexpected token o in JSON at position 1
>      at JSON.parse (<anonymous>)
>      at /Users/stanleyjeong/Desktop/_CODING/web-scraper/fireship-webscraper/functions/index.js:121:27
>      at cors (/Users/stanleyjeong/Desktop/_CODING/web-scraper/fireship-webscraper/functions/node_modules/cors/lib/index.js:188:7)
>      at /Users/stanleyjeong/Desktop/_CODING/web-scraper/fireship-webscraper/functions/node_modules/cors/lib/index.js:224:17
>      at originCallback (/Users/stanleyjeong/Desktop/_CODING/web-scraper/fireship-webscraper/functions/node_modules/cors/lib/index.js:214:15)
>      at /Users/stanleyjeong/Desktop/_CODING/web-scraper/fireship-webscraper/functions/node_modules/cors/lib/index.js:219:13
>      at optionsCallback (/Users/stanleyjeong/Desktop/_CODING/web-scraper/fireship-webscraper/functions/node_modules/cors/lib/index.js:199:9)
>      at corsMiddleware (/Users/stanleyjeong/Desktop/_CODING/web-scraper/fireship-webscraper/functions/node_modules/cors/lib/index.js:204:7)
>      at /Users/stanleyjeong/Desktop/_CODING/web-scraper/fireship-webscraper/functions/index.js:118:5
>      at /usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:570:20
>  (node:26619) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
>  (node:26619) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
⚠  functions: Your function timed out after ~60s. To configure this timeout, see
      https://firebase.google.com/docs/functions/manage-functions#set_timeout_and_memory_allocation.
>  /usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:619
>                  throw new Error("Function timed out.");
>                  ^
>  
>  Error: Function timed out.
>      at Timeout._onTimeout (/usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:619:23)
>      at listOnTimeout (internal/timers.js:549:17)
>      at processTimers (internal/timers.js:492:7)

score 0 · Accepted Answer · answered Aug 19 '20 at 20:20

Specifically for the error: UnhandledPromiseRejectionWarning: SyntaxError: Unexpected token o in JSON:

Did some research, and found out the answer here: SyntaxError: Unexpected token o in JSON at position 1

Turns out, you don't need to use JSON.parse() in the index.js file:

exports.scraper = functions.https.onRequest((request, response) => {
    cors(request, response, async () => {


        const body = JSON.parse(request.body);
        // const data = await scrapeMetatags(body.text);

        const data = await scrapeImages(body.text);

        response.send(data)


    });
});

Firebase Webscraping Project

1 Answers1