0

Python beginner here. I currently have some code that looks like

a=some_file_reading_function('filea')
b=some_file_reading_function('fileb')
# ...
if some_condition(a):
    do_complicated_stuff(b)
else:
    # nothing that involves b

What itches me is that the load of 'fileb' may not be necessary, and it has some performance penalty. Ideally, I would load it only if b is actually required later on. OTOH, b might be used multiple times, so if it is used once, it should load the file once for all. I do not know how to achieve that.

In the above pseudocode, one could trivially bring the loading of 'fileb' inside the conditional loop, but in reality there are more than two files and the conditional branching is quite complex. Also the code is still under heavy development and the conditional branching may change.

I looked a bit at either iterators or defining a class, but (probably due to my inexperience) could not make either work. The key problem I met was to load the file zero times if unneeded, and only once if needed. I found nothing on searching, because "how to load a file by chunks" pollutes the results for "file lazy loading" and similar queries.

If needed: Python 3.5 on Win7, and some_file_reading_function returns 1D- numpy.ndarray 's.

JM114516
  • 125
  • 2

2 Answers2

0
class LazyFile():
  def __init__(self, file):
    self.file = file
    self._data = None

  @property # so you can use .data instead of .data()
  def data(self):
    if self._data is None: # if not loaded
      self._data = some_file_reading_function(self.file) #load it

    return self._data

a = LazyFile('filea')
b = LazyFile('fileb')

if some_condition(a.data):
  do_complicated_stuff(b.data)
else:
  # other stuff
internet_user
  • 3,149
  • 1
  • 20
  • 29
0

Actually, just found a workaround with classes. Try/except inspired by How to know if an object has an attribute in Python. A bit ugly, but does the job:

class Filecontents:
    def __init__(self,filepath):
        self.fp = filepath
    def eval(self):
        try:
            self.val
        except AttributeError:
            self.val = some_file_reading_function(self.fp)
            print(self.txt)
        finally:
            return self.val
def some_file_reading_function(fp):
    # For demonstration purposes: say that you are loading something
    print('Loading '+fp)

    # Return value
    return 0


a=Filecontents('somefile')
print('Not used yet')
print('Use #1: value={}'.format(a.eval()))
print('Use #2: value={}'.format(a.eval()))

Not sure that is the "best" (prettiest, most Pythonic) solution though.

JM114516
  • 125
  • 2