24

I'm taking all the filenames in a certain directory in a list, and would like to write this list in a pickle file. Here's the code I am using:

import _pickle as pickle
with open(filepath+'filenames.pkl', 'wb') as f:
    pickle.dump(filenames, f)

This gives me the following error:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-32-c59e6889d2fe> in <module>()
      1 import _pickle as pickle
----> 2 with open(dpath+'filenames.pkl', 'wb') as f:
      3     pickle.dump(filenames, f)

FileNotFoundError: [Errno 2] No such file or directory: '/data/train/filenames.pkl'

I am supposed to create this file. Why is this file expected already?

(I'm using Python 3.6.)

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Patthebug
  • 4,647
  • 11
  • 50
  • 91

8 Answers8

19

In your case most probably /data/train/ directory does not exist

I tried this code and got same error:

import pickle as pickle
filenames='sadasdas'
with open('/tmp/not_exist/filenames.pkl', 'wb') as f:
    pickle.dump(filenames, f)

output:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-7-6a22c2148796> in <module>()
      1 import pickle as pickle
      2 filenames='sadasdas'
----> 3 with open('/tmp/not_exist/filenames.pkl', 'wb') as f:
      4     pickle.dump(filenames, f)

FileNotFoundError: [Errno 2] No such file or directory: '/tmp/not_exist/filenames.pkl'

You can handle this problem by making sure the directory exists before writing to file: Programatic approach to do the same is this

import os
filename = "/tmp/not_exist/filenames.pkl"
os.makedirs(os.path.dirname(filename), exist_ok=True)
data = 'sadasdas'
with open('/tmp/not_exist/filenames.pkl', 'wb') as f:
    pickle.dump(data, f)

ref: https://stackoverflow.com/a/12517490/3027854

Community
  • 1
  • 1
Vikash Singh
  • 13,213
  • 8
  • 40
  • 70
7

I also had the same problem but none of the answers helped me. The directories were there and I used wb, but again and again the same problem.

I hope this helps some who have the same issue: I found out that the problem was that I had a lot of folders, also the name of the file was too long (due to the fact that I made up the filename as a string in which I was trying to indicate the characteristics of that file) and the folder that I was trying to dump into would raise "The file name(s) would be too long for the destination folder" when I tried to move that file manually.

After shortening the file name, the problem was resolved.

Mike
  • 369
  • 5
  • 21
  • 1
    Another issue besides file length I had was the fact that in my names I had a '/' character. Could be another item to look at. – Josh W Oct 25 '22 at 18:15
3

I had a similar problem while writing using a relative pathname (and a symlink). The directory defiantly existed but it was still raising the FileNotFoundError.

The problem disappeared when I switched to using the absolute path

abspath = pathlib.Path(filename).absolute()
with open(str(abspath), 'wb') as f:
    pickle.dump(thing_to_pickle, f)
pwinty
  • 31
  • 1
1

I had to dasable 'controlled folder access' from Windows defender on Windows 10.

Other antivirus/firewall may have the same effect.

https://support.microsoft.com/en-us/help/4046851/windows-10-controlled-folder-access-windows-defender-security-center

André VR
  • 11
  • 3
0

Make sure the filepath you are using is something that actually exists in your computer. Sometimes we copy and paste code from somewhere else and forget to change the drive names. You may be addressing an non existing drive.

0
  • Make sure the file path already exists.
  • If the path is very long, add the magic prefix \\?\ before your path: path = '\\\\?\\C:\\Users...
Be Chiller Too
  • 2,502
  • 2
  • 16
  • 42
0
import os
import sys
import pickle

filenamelst = ['a.txt','b.py','c.ipynb']
filenamelst_abspathname = os.path.abspath('filenamelst.pickle')
print(filenamelst_abspathname)
picklefile = open(str(filenamelst_abspathname),'wb')
pickle.dump(filenamelst, picklefile)
picklefile.close()

This works. On the side of your Google Colab Notebook the pickle file will be there in the session folder. You need to download from there.

Nisrin Dhoondia
  • 145
  • 1
  • 10
0

I don't know if this will help anyone else, but what happened for me was that somehow the filename I was trying to save was a reserved keyword which crashed the saving. Adding another character to the name was all that was needed to allow it to pickle correctly. Here's a function you can use to flag such errors ahead of time and avoid playing whack-a-mole.

https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.is_reserved

jbarak
  • 1
  • 1
  • 2
    Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Dec 05 '21 at 23:19