Find an incorrect line in a .txt file and delete it

Question

I have a .txt file with csv data like this :

1529074866;29.89;29.41;321;70;1;60;1003.05

1529074868;29.87;29.82;140;79;1;60;1003.52

I made this function to extract the data from the file:

 def __init__(self, file):
 
    self.data_ = {"Temps": [], "Temperature": [], "Humidite": [], "Luminosite": [], "Niveau sonore": [], "Radiations EM": [], "Rythme cardiaque": [], "Pression": [] }
    # data_ = {date : [t1, t2,...], temp : [temp1, temp2,...]...}. Cette disposition des données (par date, luminosité...) permet d'optimiser l affichage des graphiques ulterieurement.
    
    try:    
        
        for line in file:  # Cette commande permet de parcourir une à une toutes les lignes du fichier file. 
            
            line = line.rstrip()    # Cette commande permet de supprimer le caractère invisible de retour chariot en fin de ligne.
            line = line.rsplit(";") # Cette commande permet de transpormer la ligne en liste en coupant au niveau des ";".
            
            self.data_["Temps"].append( int(line[0]) )  # Ceci implique que la donnée correspondant à la date soit bien la 1ère donnée (rang 0) sur la ligne.
            self.data_["Temperature"].append( float(line[1]))   # Ceci implique que la donnée correspondant à la date soit bien la 2ème donnée (rang 1) sur la ligne.
            self.data_["Humidite"].append( float(line[2]) )  # ect
            self.data_["Luminosite"].append( float(line[3]) )
            self.data_["Niveau sonore"].append( float(line[4]) )
            self.data_["Radiations EM"].append( float(line[5]))
            self.data_["Rythme cardiaque"].append( float(line[6]) )
            self.data_["Pression"].append( float(line[7]) )
        
    except Exception as expt :  # Cette exception est executee si l'execution du bloc try si dessus a echoue.
        print("\n!!! Echec de l'importation - exception relevee !!!")
        print expt

I would like to create a function which extracts the first parameter of the line, which is the Unix time, and, if the time is not between [1514761200; 1546297200], deletes the line.

How can I do this?

Try https://stackoverflow.com/questions/4710067/deleting-a-specific-line-in-a-file-python — Bruno Ranieri, Jun 18 '18 at 08:20

score 2 · Answer 1 · answered Jun 18 '18 at 08:28

To delete a line from the file you're actually going to read it completely, and then to rewrite the filtered file.

One approach:

data_typo = "1529074866;29.89;29.41;321;70;1;60;1003.05\n"

with open("file.txt", "r") as f:
    lines = f.readlines() # Extract all the lines

data = [line.rstrip().split(";") for line in lines]

# data elements: ['1529074866', '29.89', '29.41', '321', '70', '1', '60', '1003.05']

At that point, a simple approach is to filter the list data.

def criterion(elt):
    if 1514761200 <= eval(elt[0]) and eval(elt[0]) <= 1546297200:
         return True
    else:
         return False

data_to_rewrite = list(filter(criterion, data)) # Keeps the elements where criterion returns True.

with open("new_file.txt", "w") as f:
    for elt in data_to_rewrite:
        line = ";".join(elt) + "\n"
        f.write(line)

Find an incorrect line in a .txt file and delete it

1 Answers1