Try to remove item for a list created from text file

Question

I open two files and read the data from them into two lists

with open('file1.txt', 'r') as file:
     data_list1 = [line.strip() for line in file]
with open('file2.txt', 'r') as file:
     data_list2 = [line.strip() for line in file]

for each in list1:
   if each in list2:
      list1.remove(each)

After this I iterate over the first list and try to remove elements that also occur in the second list.

for each in data_list1:
    print(each)

And I got repeated results from the second list.

`remove` will only remove the first occurrence of the element. — Vishal Singh, Feb 17 '21 at 16:24
Don't remove elements from the list while iterating over it. That's a source of a lot of possible problems. — Yevhen Kuzmovych, Feb 17 '21 at 16:25
Relevant: https://stackoverflow.com/questions/1207406/how-to-remove-items-from-a-list-while-iterating — Brian61354270, Feb 17 '21 at 16:26

score 1 · Answer 1 · answered Feb 17 '21 at 16:30

You should never try to remove an item from the list you are iterating on. Here a possible way is to filter out elements from the first list when you read the second file:

with open('file1.txt', 'r') as file:
     data_list1 = [line.strip() for line in file]
with open('file2.txt', 'r') as file:
     data_list2 = list(filter(lambda x: x not in data_list1, line.strip() for line in file))

But if the lists are large, is could be more efficient to build a set from the first list because searching in a set is much faster that searching in a list:

with open('file1.txt', 'r') as file:
     data_list1 = [line.strip() for line in file]
data_set1 = set(data_list1)
with open('file2.txt', 'r') as file:
     data_list2 = list(filter(lambda x: x not in data_set1, line.strip() for line in file))

Timur Shtatland · Accepted Answer · 2021-02-17T16:45:06.467

Use list comprehension, instead of loops with remove():

lst1 = [1, 2, 3, 4]
lst2 = [2, 3]

lst1 = [x for x in lst1 if x not in lst2]
print(lst1)
# [1, 4]

Using remove() while iterating on the same list forward messes up the indexing. If you really want to delete elements from the list while iterating, then a good solution would be to iterate on the index of the list in reverse (starting from the end of the list), and use list.pop:

for i in reversed(range(len(lst1))):
    if lst1[i] in lst2:
        lst1.pop(i)

Try to remove item for a list created from text file

2 Answers2