fetch data from multiple pages using axios

Question

I am trying to fetch 15 articles with a specific tag using Axios. However, on one page there are only 10 results with different tags, hence I have to map the data from multiple pages, following these examples:

How to fetch data over multiple pages?

Promise All with Axios

This is my code so far... obviously, it doesn't work. I believe that the mistake is the last getAllData(urls) because in the chrome dev tools I get an error :

export class Dashboard extends Component {
  state = {
    articles: [],
  };

  componentDidMount() {
    axios
      .get(
        "https://content.guardianapis.com/search?api-key=test"
      )
      .then((res) => {
        const urls = [];
        const pages = 5;
        //let pages = res.data.response.pages;
        for (let i = 1; i < pages; i++) {
          urls.push(
            "https://content.guardianapis.com/search?api-key=test&page=" +
              i
          );
        }

        const getAllData = (urls) => {
          return Promise.all(urls.map(fetchData));
        };

        const fetchData = (url) => {
          axios
            .get(url)
            .then((res) => {
              return {
                data: res.data,
              };
            })
            .catch((err) => {
              console.log(err);
            });
        };

//this is not working console.log(res) undefined ??

        getAllData(urls)
          .then((res) => {
            console.log(res);
            this.setState({
              articles: res.data.response.results.slice(0, 15),
            });
          })
          .catch((e) => {
            console.log(e);
          });
      });
  }

return code

 render() {
    const { articles } = this.state;
    const articleList = articles.length ? (
      articles.map((article) => {
        if (article.pillarName === "News")
          return (
            <div

            >
              <div >
                <a >
                  {article.webTitle}
                </a>
              </div>
              <div >{article.sectionName}</div>
            </div>
          );
      })
    ) : (
      <div>Loading...</div>
    )
 return (
      <div >
          {articleList}
      </div>
    );

I would really appreciate your help, I have been stuck for a while. Thanks

can you see if any network call is going or not in chrome debugger — Dlucidone, May 24 '20 at 12:50
I have updated my question, showing the error I get from the chrome dev tools. — Ade N, May 24 '20 at 12:59
can you put a console.log() in axios call response to see the response — Dlucidone, May 24 '20 at 13:00
i have put a console.log in axios ``` const fetchData = (url) => { axios .get(url) .then((res) => { console.log(res); return { data: res.data, }; }) .catch((err) => { console.log(err); }); }; ``` and i get the result of all 4 pages with all the data. So the information is there. i just dont know how to compile to get 20 articles from all 4 pages. — Ade N, May 24 '20 at 13:04

score 1 · Answer 1 · answered May 24 '20 at 13:12

1

Promise.all needs an array of promises and your fetchData function needs to return a promise.

const fetchData = (url) => {
          return axios //<-------- make sure to add return here
            .get(url)
            .then((res) => {
              return {
                data: res.data,
              };
            })
            .catch((err) => {
              console.log(err);
            });
        };

answered May 24 '20 at 13:12

gdh

13,114
2
16
28

yes, you are absolutely right if I add return i get the data not an array of undefined. Thank you... However, I still don't get the data. It still shows **TypeError: Cannot read property 'response' of undefined at Dashboard.js:47** . i guess the code below it does not collate the answers, as the console.log still shows an array of the 4 pages . `getAllData(urls) .then((res) => { console.log(res); this.setState({ articles: res.data.response.results.slice(0, 15), }); })` – Ade N May 24 '20 at 13:25
the `res` in `getAllData` function is an array(of resolved promises) ... so loop/map thru it and then setState – gdh May 24 '20 at 13:34

score 1 · Accepted Answer · answered May 24 '20 at 13:32

You need to return a promise in fetchData, then loop over result in when you call getAllData() to get the articles

 const fetchData = url => {
      return axios
        .get(url)
        .then(res => {
          return {
            data: res.data
          };
        })
        .catch(err => {
          console.log(err);
        });
    };

    //this is not working console.log(res) undefined ??

    getAllData(urls)
      .then(res => {
        console.log(res);

        let articles = [];

        for (let rs in res) {
          articles = [...articles, ...res[rs].data.response.results];
        }
        // console.log("++ articles: ", articles);
        this.setState({
          articles: articles
        });
      })
      .catch(e => {
        console.log(e);
      });

Thank you sooooooooooooooooo much. It's working. Amazing !!!!!!!!!! — Ade N, May 24 '20 at 13:36

fetch data from multiple pages using axios

2 Answers2