-1

I have two axios functions set up in React. The first function scrapes a page to retrieve a list of links and randomly returns one of those links. The next function uses that link as an argument to scrape that webpage and returns the title and the first paragraph in an array. So the second function depends on the first.

When I try to implement this using the useEffect hook, I can get the first function to work, but the second function doesn't retrieve the link returned by the first one and gives an error. It's like the second one isn't waiting for the first one to be returned, despite me using await. Can anyone help me to see where I've gone wrong?

import getLink from "./getLink.js";
// Scrapes a page for an array of links,
// returns one of those links randomly (string)

import getPage from "./getPage.js";
// Scrapes the page from getLink,
// returns [h1 text, p text] (array)

import {useEffect,useState} from "react";

const Body = () => {
    const [text, setText] = useState([]);
    
    useEffect(() => {
        const getData = async () => {
            const link = await getLink();
            const data = await getPage(link);
            setText(data);
        }
        getData();
    }, []);
    
    return (
        <div>
            <h1>{text[0]}</h1>
            <div dangerouslySetInnerHTML={{__html: text[1]}}></div>
        </div>
    )
}

export default Body;

Error message: Access to XMLHttpRequest at 'link from getLink' from origin 'http://localhost:3000' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

Agent K
  • 1
  • 3
  • Does this answer your question? [No 'Access-Control-Allow-Origin' header is present on the requested resource—when trying to get data from a REST API](https://stackoverflow.com/questions/43871637/no-access-control-allow-origin-header-is-present-on-the-requested-resource-whe) – Azarro May 08 '22 at 12:12
  • This is a duplicate of other CORS related questions. Check out the answer here for a deeper understanding of CORS: https://stackoverflow.com/questions/10636611/how-does-access-control-allow-origin-header-work?rq=1. tl;dr: You cannot access another remote website's content from within your browser and your localhost (unless they specifically allow you to). – Azarro May 08 '22 at 12:13

2 Answers2

0

You need to make sure there is CORS at the server you are requesting the text from. You can include the right headers using middleware, in case you are using express.js, please have a look at the cors package docs.

You can check the response headers by running

curl -v http://localhost:3000

A temporary workaround would be to include { mode: 'no-cors' } in your request options.

Piotr Ostrowski
  • 520
  • 5
  • 8
0

The scraping works when I added "proxy": "[domain name]" in package.json and removed the domain names from the links I use in my project.

From: https://www.youtube.com/watch?v=hxyp_LkKDdk

Agent K
  • 1
  • 3
  • As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community May 09 '22 at 15:20