useState creates duplicates - infinite scrolling - Warning: Encountered two children with the same key

Question

To give some insight what I am trying to archive:

So basically the second useEffect renders a navlist, clicking on a navlist-item should render the correct genre movie-list, it worked until, I tried to implement infinite scrolling... now after clicking the navlist-item, it always adds the movies from the nav-item before, which creates duplicates.

And I get this Errors: Warning: Encountered two children with the same key. Also React tells me to add movies as dependency in the first useEffect. But if I do so, it triggers a infinite loop.

I think the core problem is: const newMovieList = [...movies, ...data.results] and the missing dependency.

Probably another problem is adding these many dependencies to a useEffect in the first place?

I tried for hours to fix it, but there always some weird side effects.. like one duplicate movie or some genre are working and others not.

Any advice or help, how to fix it, would be great.

import React, { useEffect, useState } from "react";
import { NavLink, useParams, useRouteMatch } from "react-router-dom";
import Movies from "./Movies";

export default function TestMovie(props) {
  const [loading, setLoading] = useState(false);
  const [genres, setGenres] = useState([]);
  const [movies, setMovies] = useState([]);
  const [pageNumber, setPageNumber] = useState(1);
  const params = useParams();
  const paramsId = params.id;
  const route = useRouteMatch();

  useEffect(() => {
    (async () => {
      setLoading(true);
      try {
        let response = "";

        if (route.path === "/") {
          response = await fetch(
            `https://api.themoviedb.org/3/movie/upcoming?api_key=${apiKey}&page=${pageNumber}`
          );
        } else if (paramsId === "23") {
          response = await fetch(
            `https://api.themoviedb.org/3/movie/popular?api_key=${apiKey}&page=${pageNumber}`
          );
        } else {
          response = await fetch(
            `https://api.themoviedb.org/3/discover/movie?api_key=${apiKey}&with_genres=${paramsId}&page=${pageNumber}`
          );
        }

        const data = await response.json();
        const newMovieList = [...movies, ...data.results];
        setMovies(newMovieList);
        console.log("PageNumber: " + pageNumber);

      } catch (error) {
        console.log(error);
      } finally {
        setLoading(false);
      }
    })();
  }, [setMovies, paramsId, route.path, pageNumber]);

  useEffect(() => {
    (async () => {
      try {
        const response = await fetch(
          `https://api.themoviedb.org/3/genre/movie/list?api_key=${apiKey}`
        );
        const data = await response.json();
        const newGenres = [{ id: 23, name: "Popular" }, ...data.genres];
        setGenres(newGenres);
      } catch (error) {
        console.log(error);
      }
    })();
  }, [setGenres]);


  return (
    <>
      <ul className="genre-list">
        <li className="genre-list__item">
          <NavLink exact activeClassName="active" to="/">
            Upcoming
          </NavLink>
        </li>
        {genres.map((genre) => (
          <li className="genre-list__item" key={genre.id}>
            <NavLink
              exact
              activeClassName="active"
              to={`/genre/${genre.id}-${genre.name}`}
            >
              {genre.name}
            </NavLink>
          </li>
        ))}
      </ul>
      <Movies
        setPageNumber={setPageNumber}
        movies={movies}
        loading={loading}
      ></Movies>
    </>
  );
}

The InfiniteScroll part:

export default function Movies(props) {
  const { movies, loading, setPageNumber } = props;

  function updatePageNumber() {
    setPageNumber((pageNumber) => pageNumber + 1);
  }

  return loading ? (
    <div className="loader">
      <Loader />
    </div>
  ) : (
    <>
      <InfiniteScroll
        dataLength={movies.length}
        next={updatePageNumber}
        hasMore={true}
      >
        <div className="movies-layout">
          {movies.map((movie) => (
            <Movie key={movie.id} movie={movie}></Movie>
          ))}
        </div>
      </InfiniteScroll>
    </>
  );
}

Answer 1

There might be different ways to do infinite scroll. Based on your way, the main issue is this line

  const newMovieList = [...movies, ...data.results];

You can't be sure the new result should be the new one, because user could hit old page (or change "rows per page" settings).

In order to avoid that, maybe the easiest way for you is to make a function to add item by item key.

  function getNewMovieList(current, data) {
    const movies = {}
    [...current, ...data].forEach(item => {
      movies[item.id] = item
    })
    return Object.values(movies)
  }

Literally you want to make sure it's still a new list without duplicated items. The error you are getting actually serve you good in this case.

Another way

Another way is to put fetch call inside a page component,

  const Page = ({ pageId }) => {
    const [data, setData] = useState([])
    useEffect(() => { go fetch data for this page ONLY })
  }

Now if you go to <Page pageId={5} />, you won't run into the data duplication issue at all.

But it probably depends on how you expect from the user, if you want to have smooth scroll behavior, maybe your approach is still the way to go, but if you can afford user to click any page, then second approach might be much safer and scalable.