Comparing two numpy arrays: (x1, y1) and (x2, y2) to check whether elements intersect

Question

I am having trouble trying to obtain a boolean array that indicates when an element in the second numpy array is also in the first array.

The challenging part is that each array is made up of Latitude/Longitude pairs and I want to make sure that each Lat/Lon in secondcoords is also in firstcoords. So, this is like an intersection

Here is what I have done thus far (with small example coordinates):

firstlat = [0, 1, 5, 5]
firstlon = [1, 0, 5, 4]

secondlat = [0, 2, 0, 5]
secondlon = [1, 2, 5, 5]

firstcoords = numpy.array((firstlat, firstlon))
firstcoords = numpy.transpose(firstcoords) # gets lat/lon pair


secondcoords = numpy.array((secondlat, secondlon))
secondcoords = numpy.transpose(secondcoords)

a = numpy.isin(secondcoords, firstcoords)

Wrong output:
[[ True  True]
[False False]
[ True  True]
[ True  True]]

Wanted output: [[True, False, False, True]]

Numpy isin flattens the arguments so although firstcoords[0] = [0 1], it seems to be improperly comparing it "element by element". However, as I saw, each element comprises of both the [lat lon]; and the purpose to transpose it was to get the lat / lons in tuple or tuple-like form for easier comparison. So, how do I fix my approach or what other approaches would be feasible for this problem?

Answer 1

If you zip the latitudes and longitudes up into tuples:

firstlat = [0, 1, 5, 5]
firstlon = [1, 0, 5, 4]

secondlat = [0, 2, 0, 5]
secondlon = [1, 2, 5, 5]

first_lat_lon = list(zip(firstlat,firstlon))
second_lat_lon = list(zip(secondlat,secondlon))

Then you can easily check which of the second list are in the first:

[x in first_lat_lon for x in second_lat_lon]

Which returns:

[True, False, False, True]

Answer 2

I don't know that the functionality you're looking for is possible in numpy. I recommend using the following:

in_second_and_first = set(zip(secondlat,secondlon)) & set(zip(firstlat,firstlon))

If you're using Python 2 (which I would strongly recommend against), use itertools.izip instead of the built-in zip.

Answer 3

One hack to make use of isin is to use structured arrays though your arrays not really 1-dimensional.

firstlat = [0, 1, 5, 5]
firstlon = [1, 0, 5, 4]

secondlat = [0, 2, 0, 5]
secondlon = [1, 2, 5, 5]

firstcoords = np.array(list(zip(firstlat, firstlon)), dtype=[("lat", int), ("lon", int)])
# array([(0, 1), (1, 0), (5, 5), (5, 4)], dtype=[('lat', '<i8'), ('lon', '<i8')])
secondcoords = np.array(list(zip(secondlat, secondlon)), dtype=[("lat", int), ("lon", int)])
# array([(0, 1), (2, 2), (0, 5), (5, 5)], dtype=[('lat', '<i8'), ('lon', '<i8')])

np.isin(secondcoords, firstcoords)
# array([ True, False, False,  True])

Reference

How to make Numpy treat each row/tensor as a value

Get intersecting rows across two 2D numpy arrays

Answer 4

Use a np.void view. This just looks at every row as a chunk of data instead of discrete values.

def vview(a):  #based on @jaime's answer: https://stackoverflow.com/a/16973510/4427777
    return np.ascontiguousarray(a).view(np.dtype((np.void, a.dtype.itemsize * a.shape[1])))

Then you can just use np.isin just as you wanted to

a = numpy.isin(vview(secondcoords), vview(firstcoords))

Beware this is a comparison on the data level, so there is no way to deal with floating point inaccuracies if your actual data is floats. On the other hand, it is extremely fast as it doesn't require restructuring or copying your data in any way.