unique write in to file using python

Question

i have a file with several person names and marks... the file contains several duplicates entry.

input

abs 88, abs 88, ddd 77, fff 62, ddd 77...

I need to remove all the duplicates entry and keep a unique list.

output

abs 88, ddd 77, fff 62 ...

The input file is dynamic and new data will be added frequently ( with duplicates) so i need to remove the duplicates and update the output files whenever there is changes..

So what I need is whenever a new entry comes in input file i need to check whether that entry is existing in output file... if yes reject else add that in the file.

how this can be done using Python ?

i tried " if 'name' not in array: add" but its seams slowing up when the data size increases.. — Sreejithc321, Oct 02 '14 at 06:14
Have a look at this Q/A: http://stackoverflow.com/questions/513882/python-list-vs-dict-for-look-up-table — Stefan Friedrich, Oct 02 '14 at 14:08

score 0 · Accepted Answer · answered Oct 02 '14 at 05:42

0

arrayYou can do it with an array like:

arr= []

if input_value in arr:
  # reject action
  ...
else:
  # add to array
  arr.append(input_value)

  # save to file 
  ...

answered Oct 02 '14 at 05:42

Stefan Friedrich

188
7

unique write in to file using python

input

output

1 Answers1