Read Data into Python Line by Line as a List

Question

I would like to read in a series of coordinates with their accuracy into a triangulation function to provide the triangulated coordinates. I've been able to use python to create a .txt document that contains the list of coordinates for each triangulation i.e.

[(-1.2354798, 36.8959406, -22.0), (-1.245124, 36.9027361, -31.0), (-1.2387697, 36.897921, -12.0), (-1.3019762, 36.8923956, -4.0)]

[(-1.3103075, 36.8932163, -70.0), (-1.3017684, 36.8899228, -12.0)]

[(-1.3014139, 36.8899931, -34.0), (-1.2028006, 36.9180461, -54.0), (-1.1996497, 36.9286186, -67.0), (-1.2081047, 36.9239936, -22.0), (-1.2013893, 36.9066869, -11.0)]

Each of those would be one group of coordinates and accuracy to feed into the triangulation function. The text documents separate them by line.

This is the triangulation function I am trying to read the text file into:

def triangulate(points):
    """
    Given points in (x,y, signal) format, approximate the position (x,y).

    Reading:
    * http://stackoverflow.com/questions/10329877/how-to-properly-triangulate-gsm-cell-towers-to-get-a-location
    * http://www.neilson.co.za/?p=364
    * http://gis.stackexchange.com/questions/40660/trilateration-algorithm-for-n-amount-of-points
    * http://gis.stackexchange.com/questions/2850/what-algorithm-should-i-use-for-wifi-geolocation
    """
    # Weighted signal strength
    ws = sum(p[2] for p in points)
    points = tuple( (x,y,signal/ws) for (x,y,signal) in points )

    # Approximate
    return (
        sum(p[0]*p[2] for p in points), # x
        sum(p[1]*p[2] for p in points) # y
    )


print(triangulate([
    (14.2565389, 48.2248439, 80),
    (14.2637736, 48.2331576, 55),
    (14.2488966, 48.232513, 55),
    (14.2488163, 48.2277972, 55),
    (14.2647612, 48.2299558, 21),
]))

When I test the function with the above print statement it works. But when I try to load the data from the text file into the function as follows"

with open(filename, 'r') as file:
   for points in file:
       triangulation(points)

I get the error: IndexError: string index out of range. I understand that this is because it is not being read in as a list but as a string, but when I try to convert it to a list object points = list(points) it is also not recognized as a list of different coordinates. My question is how should I read the file into python in order for it to be translated to working within the triangulate function.

Your text file is read as-is, Python does not know about the structure of your lines. It does not see the list and tuples. A quick solution is to do something like `data = eval(points)`, although I strongly advise against it as `eval` is commonly discouraged. Instead, you should reformat your file to contain JSON, or any other machine-readable format. — FMCorz, Sep 25 '17 at 03:40

Kent Sommer · Answer 1 · 2017-09-25T05:25:43.890

As suggested by @FMCorz you should use JSON or some other machine-readable format.

Doing so is simple and just a matter of dumping your list of points to the text file in any machine-readable format and then later reading it back in.

Here is a minimal example (using JSON):

import json

def triangulate(points):
    """ Given points in (x,y, signal) format, approximate the position (x,y).

        Reading:
        * http://stackoverflow.com/questions/10329877/how-to-properly-triangulate-gsm-cell-towers-to-get-a-location
        * http://www.neilson.co.za/?p=364
        * http://gis.stackexchange.com/questions/40660/trilateration-algorithm-for-n-amount-of-points
        * http://gis.stackexchange.com/questions/2850/what-algorithm-should-i-use-for-wifi-geolocation
    """
    # Weighted signal strength
    ws = sum(p[2] for p in points)
    points = tuple( (x,y,signal/ws) for (x,y,signal) in points )

    # Approximate
    return (
        sum(p[0]*p[2] for p in points), # x
        sum(p[1]*p[2] for p in points) # y
    )

points = [(14.2565389, 48.2248439, 80),
            (14.2637736, 48.2331576, 55),
            (14.2488966, 48.232513, 55),
            (14.2488163, 48.2277972, 55),
            (14.2647612, 48.2299558, 21)]

with open("points.txt", 'w') as file:
    file.write(json.dumps(points))

with open("points.txt", 'r') as file:
    for line in file:
        points = json.loads(line)
        print(triangulate(points))

If you wanted to use a list of lists (a list containing lists of points), you could do something like this:

import json

def triangulate(points):
    """ Given points in (x,y, signal) format, approximate the position (x,y).

        Reading:
        * http://stackoverflow.com/questions/10329877/how-to-properly-triangulate-gsm-cell-towers-to-get-a-location
        * http://www.neilson.co.za/?p=364
        * http://gis.stackexchange.com/questions/40660/trilateration-algorithm-for-n-amount-of-points
        * http://gis.stackexchange.com/questions/2850/what-algorithm-should-i-use-for-wifi-geolocation
    """
    # Weighted signal strength
    ws = sum(p[2] for p in points)
    points = tuple( (x,y,signal/ws) for (x,y,signal) in points )

    # Approximate
    return (
        sum(p[0]*p[2] for p in points), # x
        sum(p[1]*p[2] for p in points) # y
    )


points_list = [[(-1.2354798, 36.8959406, -22.0), (-1.245124, 36.9027361, -31.0), (-1.2387697, 36.897921, -12.0), (-1.3019762, 36.8923956, -4.0)],
                [(-1.3103075, 36.8932163, -70.0), (-1.3017684, 36.8899228, -12.0)],
                [(-1.3014139, 36.8899931, -34.0), (-1.2028006, 36.9180461, -54.0), (-1.1996497, 36.9286186, -67.0), (-1.2081047, 36.9239936, -22.0), (-1.2013893, 36.9066869, -11.0)]]

with open("points.txt", 'w') as file:
    file.write(json.dumps(points_list))

with open("points.txt", 'r') as file:
    for line in file:
        points_list = json.loads(line)
        for points in points_list:
            print(triangulate(points))

Hey thanks for the response. I tried to recreate your work where I create the lists, and when I do the JSON dump and try to read it back into the function, I get errors like the following: in decode raise JSONDecodeError("Extra data", s, end) json.decoder.JSONDecodeError: Extra data: line 1 column 130 (char 129) — chaimocha, Sep 25 '17 at 05:11
Is it because the JSON file makes all of the paranthesis in the original file into brackets? — chaimocha, Sep 25 '17 at 05:16
How are you writing the list? If you have multiple point lists to write you should either do a loop write over the list of lists or write each list individually. I have edited my answer with another example. — Kent Sommer, Sep 25 '17 at 05:17

score 1 · Answer 2 · answered Sep 25 '17 at 03:56

What you get from the file is a string, but Python doesn't know anything about how that string should be interpreted. It could be a printed representation of a list of tuples, as in your case, but it could just as well be a part of a book, or it could be some compressed data, or so on. It's not the language's job to guess how to treat the string that gets read from the file. That's your job; you have to write some code to take those strings and parse them - that is, convert them into the data your program needs, using the reverse of the rules that were used to convert that data into strings in the first place.

Now, this is certainly a thing you could do, but it's probably better to just use something other than print(). That is, use a different set of rules for converting your data into strings, one where people have already written the code to reverse the process. A common format you could use is JSON, for which Python includes a library to do the conversions. Other formats that can work with numerical data include CSV (here's the Python module) and HDF5 (supported by an external library, probably overkill for your case). The point is, you need to choose some set of rules for converting between data and strings and use the corresponding code in both directions. In your original example, you were only using the rule for going from data to strings and expecting Python to guess the rule for going back.

If you want to read more about this, the process of converting data to strings (or, really, to something that can be put in a file) is called formatting or serialization, depending on context, and the reverse process of converting the strings back to the original data is called parsing or deserialization.

Read Data into Python Line by Line as a List

2 Answers2