7

When I am removing files from Jupyter notebook environment, the disk space does not free up. I removed for about 40GB files and files disappeared from list, even from ls -a, however df -h shows that nothing happened. Also I killed all the processes using these files and even rebooted the system.

When I remove files using rm everything is fine. How can I free up space, or restore thos files to delete them using rm?

BatyaGG
  • 674
  • 1
  • 7
  • 19

2 Answers2

13

I also encountered the same issue, but later found out that files deleted in jupyter notebook are automatically moved to the trash, which is not permanently deleted instantly. This function was added very early ago: https://github.com/jupyter/notebook/pull/1968

Thus, to free up space, you need to go to your computer's trash folder to completely delete those files. In Linux, you can do rm -rf $HOME/.local/share/Trash/files to empty trash.

In Windows or MacOS, you just need to "Empty Trash" in the desktop folder

To restore those files, try to access them in your Trash folder, which is located in the .local folder in your home directory (in my system.).

This worked for me. I'm using jupyter lab with Amazon Linux 2 AMI.

Ref.

  1. recycle bin in linux: https://www.pcquest.com/recycle-bin-linux/#:~:text=Henceforth%20whenever%20you%20delete%20a,SYSTEM_ROOT%20of%20the%20Trash%20directory.
Dwa
  • 593
  • 4
  • 13
  • Why do you need to `export UNCOVER_DIRS`? – flow2k Jul 09 '21 at 05:03
  • 2
    @flow2k sorry, it was an error. I've reedited the answer :) It turns out I can safely empty the trash can without that `export UNCOVER_DIRS`, in which I didn't use the variable later on. – Dwa Aug 02 '21 at 13:12
3

I freed up the space and solved the issue when I was working with workbench(Google Cloud Vertex AI). Applicable to AI platform too. Open terminal from workbench File>New>Terminal

df -h -T /home/jupyter

Above code helps to know the free size before

Then this code below will delete the trash to free up the size

sudo rm -rf  /home/jupyter/.local/share/Trash/*

After deleting the trash after

Partha Sen
  • 2,759
  • 1
  • 18
  • 17