How to remove duplicated elements using jQuery unique?

Question

I am trying to make a dynamically generated content(images, text, videos) and sometimes there are some new contents which are duplicated.

How can I modify this jQuery call to remove all duplicate elements?

$(".parent").children(:not[unique]).remove();

For example after I click on the add more button I get Ajax content which some are duplicated and below is an example of duplicated content so how can I remove the duplicated ones dynamically :

<button class="btn">ADD MORE</button>
<div class="parent">
<div class="child">hello</div>
<div class="child">hello</div>
<img class="flower_image" src="/pictures/flower.jpeg">
<img class="flower_image" src="/pictures/flower.jpeg">
<video class="trailer_video" src="/videos/trailer.flv"></video>
<video class="trailer_video" src="/videos/trailer.flv"></video>
<span class="span_text">hello world></span>
<span class="span_text">hello world></span>
<a class="hyper_link" src="www.example.com"></a>
<a class="hyper_link" src="www.example.com"></a>
</div>

Refer this: https://stackoverflow.com/questions/2822962/jquery-remove-duplicate-elements — swapnil gandhi, Jul 23 '18 at 22:39
well I saw this earlier but the problem is I have multiple elements ` — Kenny J., Jul 23 '18 at 23:15

Louys Patrice Bessette · Answer 1 · 2018-07-23T23:21:31.330

If you refer to $.unique() (deprecated in favor of $uniqueSort() since 3.0...)

This is used to filter duplicate element from an array of elements... So if the same element appears more than once in the array, the duplicate is filtered out.

Notice that two different element, even if they are having the same content are not considered duplicates.

Example of an array containing duplicates:

var array=[];

$(".parent").children().each(function(){
  for(i=0;i<10;i++){
    array.push(this);
  }
});

console.log(array.length);

<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<ul class="parent">
  <li>Hello world</li>
  <li>Hello world</li>
  <li>Having fun?</li>
</ul>

So there is supposed to be 3 elements... But due to a weird scripting desing (intended here), there is 30.

Sometime you may way to gather all class1 element and then add the class2 element to it. What if some element have both classes? That is a way to filter that.

var array=[];

$(".parent").children().each(function(){
  for(i=0;i<10;i++){
    array.push(this);
  }
});

array = $.unique(array);

console.log(array.length);

<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<ul class="parent">
  <li>Hello world</li>
  <li>Hello world</li>
  <li>Having fun?</li>
</ul>

But NO WAY this will filter the duplicate «Hello world» of the examples... Since they are different elements.

so if it is impossible to filter the duplicates , what other way i can user ? — Kenny J., Jul 23 '18 at 23:39
and yes They all each have a class assigned to it so If there is a way to use this classname and remove the duplicate and leave only one class for each name — Kenny J., Jul 23 '18 at 23:40
Post a markup example where there are duplicates... And I may suggest something. — Louys Patrice Bessette, Jul 23 '18 at 23:43

score 0 · Accepted Answer · answered Jul 24 '18 at 05:46

I asked you for some markup where there are duplicates to look for a "patern" to exploit. And you provided some markup that really doesn't look like a real use-case markup. Example: A <video> usually has some <source> tags inside and a closing tag... And </img> is invalid. I was tempted not to answer at all... Because of this lack of serious.

But I then had an idea of something to do that might work after all...

So here is a code that works for the provided sample... It may work fine on your real markup and it may have plenty issues.

It's a starter without any waranty. You will have to improve it yourself from here. Look below for the explanations on how it works.

$(".clean").on("click",function(){

  var all_elements = $(".parent").children();

  all_elements.each(function(){
    var el_class = this.className;
    var el_src = this.src;
    var el_text = $(this).text();

    // data("verified") prevents the removal triggered by it's duplicate, if any.
    $(this).data("verified",true);

    all_elements.each(function(){
      if(el_class==this.className && el_src==this.src && el_text==$(this).text() && !$(this).data("verified")){
        $(this).remove();
      }
    });
  });
  
  // Turn all "surviving" element's data("verified") to false for future "clean".
  $(".parent").children().each(function(){
    $(this).data("verified",false);
  });
});

.parent>*{
  border:1px solid black;  /* Just to make empty element more obvious in this demo */
}

<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>

<button class="btn clean">CLEAN DUPLICATES</button><br>
<br>
<div class="parent">
  <div class="child">hello</div>
  <div class="child">hello</div>
  <img class="flower_image" src="/pictures/flower.jpeg">
  <img class="flower_image" src="/pictures/flower.jpeg">
  <video class="trailer_video" src="/videos/trailer.flv"></video>
  <video class="trailer_video" src="/videos/trailer.flv"></video>
  <span class="span_text">hello world></span>
  <span class="span_text">hello world></span>
  <a class="hyper_link" src="www.example.com"></a>
  <a class="hyper_link" src="www.example.com"></a>
</div>

There are two nested .each() loops working on all the elements that are child of .parent.

It compares two attributes:

the className (class list of the element)
the src

and the text it contains.

If something is empty or undefined, it doesn't matter because the duplicate it looks for should also be empty or undefined.

And finally, there is a data value used as a flag to prevent an element, which was the subject of a previous comparison, to be removed.

first of all I would like to thank you for your amazing answer truly I saw a great innovation and talent in your answer ... second I was completely serious when I was posting my markup but the catch is that video src does not matter as there will be plenty of videos this is just an example that represents the real issue and so as the rest of my mark ups lastly yes the img has no closing tag what was I thinking :) ... I would like to thank you again Mr @Louys Patrice Bessette and see you soon ... — Kenny J., Jul 24 '18 at 11:47

How to remove duplicated elements using jQuery unique?

2 Answers2