10

I'm getting the following warning reading a geojson with geopanda's read_file():

...geodataframe.py:422: RuntimeWarning: Sequential read of iterator was interrupted. Resetting iterator. This can negatively impact the performance.
  for feature in features_lst:

Here's the code sample I used:

crime_gdf = gpd.read_file('datasets/crimes.geojson', bbox=bbox)

crimes.geojson is a file containing a large number of points, each with a 'Crime type'

bbox defines the boundaries

The code runs as expected, but I don't understand that warning.

EDIT

I converted the geojson to feather, and I get the same warning.

leosole
  • 331
  • 4
  • 13

3 Answers3

11

See my comment in Fiona's issue tracker: https://github.com/Toblerity/Fiona/issues/986

GDAL (the library Fiona uses to access the geodata) maintains an iterator over the features that are currently read. There a some operations that, for some drivers, can influence this iterator. Thus, after such operations we have to ensure that the iterator is set to the correct position that a continuous read of the data is ensured. Such operations include counting all features in a dataset, respectively calculating its extent.

There are different types of drivers in GDAL. Some drivers support random access, while some do not. For the drivers that do not support random access, the resetting of the iterator involves reading all features again up to the iterator position. As this is a possible costly operation, this RuntimeWarning is emitted, so that users are aware of this behavior.

rbuffat
  • 199
  • 8
  • Sorry for my ignorance in this issue, but is it possible for me to select a driver that support random access? – leosole Dec 02 '20 at 14:04
  • 1
    @leosole It depends on the file you are reading. There is one driver for GeoJSON, one for GPKG, etc. So you cannot use another driver for GeoJSON. – Stefan_EOX Dec 02 '20 at 15:34
  • I see. I can convert the file to something else. I tried converting to feather, but the warning persists – leosole Dec 02 '20 at 19:38
  • @leosole This is just a warning. If you do not see any performance-related issues you can safely ignore it. I suspect with geopandas it is the the same issue as with list(src). If this is the case the warning should probably not be raised anyway. In https://github.com/Toblerity/Fiona/issues/986 you will find more information about why list() causes this behavior. – rbuffat Dec 02 '20 at 21:12
5

I found this fiona PR that references the warning: https://github.com/Toblerity/Fiona/pull/965

I can reproduce the issue with fiona 1.8.18 when calling list(src) on a recently opened Collection, but not with 1.8.17. So I think this is a regression that was introduced in fiona 1.8.18, released on 2020-11-17.

Stefan_EOX
  • 1,279
  • 1
  • 16
  • 35
3

While it could cause problems ignoring warnings in some contexts, I am finding this one particularly annoying repeated in my code output.

I found the warning can be ignored using

import warnings
# filter out RuntimeWarnings, due to geopandas/fiona read file spam
# https://stackoverflow.com/questions/64995369/geopandas-warning-on-read-file
warnings.filterwarnings("ignore",category=RuntimeWarning)

I added in the comments referencing this post to explain why I am filtering out these warnings. When this warning is suppressed in a future geopandas / fiona release then this code snippet should probably be removed, as it may also suppress meaningful warnings and could be problematic in some contexts.

Carl Higgs
  • 301
  • 2
  • 7
  • 2
    See this answer for how to be more specific about which warnings to suppress: https://stackoverflow.com/a/9134842/4742780 – Erik Dec 01 '21 at 14:49