how to filter value with python, like with grep

Question

I have a simple python script, which reads the serial input and saves it to a file (file.txt). The serial input is like this:{lat: 41.07494, lng:14.2742}, but sometimes there are errors or wrong values, e.g. {lat: 41.▒▒7494, lng:14.2742}, or something else, so the output has to be filtered and than stored to the file.

I tried to use grep with os, but this didnt work, and I dont think it is the proper way to do it in python.

#!/usr/bin/env python

import time
import serial
import os

a = 1
ser = serial.Serial(

 port='/dev/ttyUSB0',
 baudrate = 9600,
 parity=serial.PARITY_NONE,
 stopbits=serial.STOPBITS_ONE,
 bytesize=serial.EIGHTBITS,
 timeout=1
)
counter=0

while 1:
 val=ser.readline()
 print val
 f = open('/file.txt', 'a')
 f.write(val)
 f.close()

 os.system("grep '^{lat: [0-9][8-9]\.[0-9]\{5\}, lng:[0-9][0-9]\.[0-9]\{4\}},$' /file.txt > /filtered.txt")

So the only output should be like this: {lat: xx.xxxxx, lng:xx.xxxxx},

Answer 1

I would do something like this:

import re
f = open('/file.txt', 'a')

regex = re.compile('^{lat: [0-9][8-9]\.[0-9]\{5\}, lng:[0-9][0-9]\.[0-9]\{4\}},$')
lines_match = list(filter(lambda s: re.match(regex, s), f)

Answer 2

Using an external utility is inconvenient and wasteful anyway; Python has regular expressions in its re library.

You have a number of oddities and antipatterns in your code, but I guess you are looking for something like

import re

# ...
with open('/filtered.txt', 'w') as filtered:
    valid = re.compile(r'^\{lat: [0-9]{2}\.[0-9]{5}, lng:[0-9]{2}\.[0-9]{4}\},$')
    while True:
        val = ser.readline()
        if valid.match(val):
            filtered.write(val)

Notice how the BRE dialect of POSIX grep has the opposite behavior for braces (backslash makes it a metacharacter, without backslash is literal) from Python's re (without backslash is metacharacter, backslash to make it literal) and basically every sane modern dialect.

Your regex would curiously force the second digit of the latitude to be 8 or 9; I assume this was a mistake (your test data would not match). I would also review whether having five decimals for the lat but four for lng is correct (probably require at least four but permit more? [0-9]{4,})