1

joblib.dump doesn't seem to do anything for me whatsoever. Possibly I'm having some versioning conflicts or something. Any help is appreciated.

My joblib version: 0.13.2
Seems to affect 0.14.0 as well

To reproduce:

import joblib
import os

foo = open("bar", "w+b")
joblib.dump("test", foo)
print(os.stat("bar").st_size)

#prints 0... expect the size of a file containing the pickled string "test" > 0 bytes
Him
  • 5,257
  • 3
  • 26
  • 83

1 Answers1

1

The buffer has not been written to disk, so the file has been created but has no content. You have to flush the internal buffer then the content is written into the file:

>>> foo = open("bar", "w+b")
>>> joblib.dump("test", foo)
>>> foo.flush()
>>> print(os.stat("bar").st_size)
14

Or use a context manager which does that automatically:

>>> with open("bar", "w+b") as foo:
>>>     joblib.dump("test", foo)
>>> print(os.stat("bar").st_size)
14

Or you can disable buffering:

>>> foo = open("bar", "w+b", buffering=0)
>>> joblib.dump("test", foo)
>>> print(os.stat("bar").st_size)
14

Also have a look here maybe: How often does python flush to a file?

nicoring
  • 633
  • 5
  • 12