matplotlib savefig - No such file or directory

Question

I am having a very weird behavior for matplotlib's savefig function. I am doing as follows:

import os

# creating the output folder 
if not os.path.exists(output_folder):
   os.makedirs(output_folder)

# adding the name of the figure to the created output folder
plt.savefig(output_folder + 'forecasts_scatter_%s' % model_name)
# plt.savefig(os.path.join(output_folder, 'forecasts_scatter_%s.png' % model_name))
plt.close()

The errors:

Traceback (most recent call last):
  File "C:/Users/96171/Desktop/ministry_of_public_health/CodeUbrCorrected/bmwmlmcw.py", line 56, in <module>
    lm.cross_validation(model, hyperparameters[model_name], model_name)
  File "C:\Users\96171\Desktop\ministry_of_public_health\CodeUbrCorrected\cross_validation_smogn.py", line 554, in cross_validation
    self.cross_validation_grid(model_used, hyperparams, model_name)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\utils\testing.py", line 348, in wrapper
    return fn(*args, **kwargs)
  File "C:\Users\96171\Desktop\ministry_of_public_health\CodeUbrCorrected\cross_validation_smogn.py", line 1444, in cross_validation_grid
    'predicted')
  File "C:\Users\96171\Desktop\ministry_of_public_health\CodeUbrCorrected\cross_validation_smogn.py", line 1772, in plot_actual_vs_predicted_scatter_bisector
    plt.savefig(output_folder + 'forecasts_scatter_%s' % model_name)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\pyplot.py", line 689, in savefig
    res = fig.savefig(*args, **kwargs)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\figure.py", line 2094, in savefig
    self.canvas.print_figure(fname, **kwargs)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\backend_bases.py", line 2075, in print_figure
    **kwargs)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\backends\backend_agg.py", line 521, in print_png
    cbook.open_file_cm(filename_or_obj, "wb") as fh:
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\cbook\__init__.py", line 407, in open_file_cm
    fh, opened = to_filehandle(path_or_file, mode, True, encoding)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\cbook\__init__.py", line 392, in to_filehandle
    fh = open(fname, flag, encoding=encoding)
FileNotFoundError: [Errno 2] No such file or directory: '../output_ubr_shallow/all_columns_minus_weather_minus_lags_minus_civilians_balance/smogn_0.95_Manhattan_rel_mat1/grid_search/train_test_forecasts_scatterplot_bisector/forecasts_scatter_lasso.png'

The output I am specify is a relative path

'../output_ubr_shallow/all_columns_minus_weather_minus_lags_minus_civilians_balance/smogn_0.95_Manhattan_rel_mat1/grid_search/train_test_forecasts_scatterplot_bisector/'

and when I do os.path.exists(output_folder) it gives me True.

I know that we can specify relative paths in savefig. Otherwise what am I doing wrong ?

I also tried this:

# changing the .png thing and using os.path.join
plt.savefig(os.path.join(output_folder, 'forecasts_scatter_' + model_name + '.png'))

and this:

# changing the .png thing and using os.path.join
plt.savefig(os.path.join(output_folder, 'forecasts_scatter_%s.png' % model_name))

But still getting the same results.

Whats very weird I ran the same code from different script. I modified a bit the code to make it as follows:

 p = output_folder + 'forecasts_scatter_' + model_name + '.png'
 filename = os.path.split(p)[1]
 directory = os.path.split(p)[0]

 if not os.path.exists(directory):
    os.makedirs(directory)

 savepath = os.path.join(directory, filename)
 print('save path isss: {}'.format(savepath))
 plt.savefig(savepath)
 plt.close()

The first script has this as the output path:

save path isss: ../output_ubr_shallow/all_columns_balance/smogn_0.95_Manhattan_rel_mat1/grid_search/train_test_forecasts_scatterplot_bisector\forecasts_scatter_lasso.png

and the second script has this as the output path

save path isss: ../output_ubr_shallow/all_columns_minus_weather_minus_lags_minus_civilians_balance/smogn_0.95_Manhattan_rel_mat1/grid_search/train_test_forecasts_scatterplot_bisector\forecasts_scatter_lasso.png

The first is working and the second is not. Both scripts are found in the same directory!!!

Look at this: https://stackoverflow.com/questions/16333569/mixed-slashes-with-os-path-join-on-windows/18776536 — Raj, May 03 '20 at 06:24

Perl Del Rey · Accepted Answer · 2020-05-03T10:19:56.447

Wow after spending many hours on this I just replaced this:

this:

plt.savefig(os.path.join(output_folder, 'forecasts_scatter_%s.png' % model_name))

by this

plt.savefig(os.path.join(output_folder, 'scatter_%s.png' % model_name))

and it worked. I'm not sure if this is a possible bug in matplotlib. I think the length of the path in savefig has some limits.

I opened an issue on github https://github.com/matplotlib/matplotlib/issues/17313

score 0 · Answer 2 · answered Jul 20 '23 at 02:00

This issue is addressed in Pathname too long to open?

It is not related to matplotlib or Python, it's MS-DOS that has a maximum length to path strings of 260 characters. A workaround suggested in the above link is to use an extended path string, by prefixing your path with \\??\. I have not tested it, but in your case you should be able to do:

output_folder = r'\\??\..\output_ubr_shallow\all_columns_minus_weather_minus_lags_minus_civilians_balance\smogn_0.95_Manhattan_rel_mat1\grid_search\train_test_forecasts_scatterplot_bisector\'
plt.savefig(os.path.join(output_folder, 'forecasts_scatter_%s.png' % model_name))

matplotlib savefig - No such file or directory

2 Answers2