How can we remove the duplicates in async function of java script

Question

I am taking the data from backend for that I am using the async function. But the data contains some duplicate elements so I need to remove them and get those data in DOM. Can we do that. If yes can you explain how to do that.

function Productpage(){
    const [data, setData] = useState([])
    let navigate = useNavigate()
    
    let getproducts = async () => {
        
        let res = await axios.get(`${env.apiurl}/users/products`)
        console.log(data)
        if (res.data.statusCode === 200) {
          setData(res.data.data)
        }
        else {
          alert(res.data.message)
        }
      }
      console.log(data)
    
      useEffect(() => {
        getproducts()
      }, [])

  return <>
  <div>
      {
      data.map((e,i)=>{
        return <div key={i} onClick={()=>navigate(`${e.producttype}`)} >
            <Card style={{width:"100%", height:"250px"}}>
                <CardImg
                alt="Card image cap"
                src={e.url}
                top
                width="50%"
                height="60%"
                />
                <CardBody>
                <CardTitle tag="h5">
                    {e.producttype}
                </CardTitle>              
                </CardBody>
            </Card>
            </div>
      })
      }
      </div>
    </>
    
}

Here data comes from backend and from that data only needed to print partcular values like url and product type in DOM only without any duplicate values for the product type.

As far as I can tell, your question has nothing to do with `async` or nodejs or react. It's just about getting rid of duplicates (in a list of strings). And, ironically, that summarizes this thread very well: https://stackoverflow.com/a/9229821/5767484 — NotX, Nov 09 '22 at 18:18

score 0 · Accepted Answer · answered Nov 09 '22 at 18:17

Remove duplicate elements by first creating a hashmap.

// Will have structure like {car: {}, paint: {}, toy: {}}
// Retains the LAST element in the list for each product type
const hashMap = res.data.data.reduce((acc, cur) => {
  return {...acc, [cur.producttype]: cur}; 
}, {});

setData(Object.values(hashMap));

However, probably the best way is to add support for this query on the api side. So you don't have to send a ton of data over the wire just to throw it out. Something like users/products?distinctKey=producttype

score 0 · Answer 2 · answered Nov 09 '22 at 18:17

Create a set of product types and ignore any items that are already in the set:

const dataWithoutDuplicates = [];
const encounteredProductTypes = new Set();
for(const item of data) {
  if(encounteredProductTypes.has(item.producttype) continue;
  dataWithoutDuplicates.push(item);
  encounteredProductTypes.add(item.producttype);
}

How can we remove the duplicates in async function of java script

2 Answers2