18

I created a batch file which copies a directory with xcopy for backup reasons. Then I use pkzip in order to zip the backup folder (and email it through a batch file which is working), but I am getting an error message for insufficient memory. I tried to increase the buffer size in command line properties but it didn't work. I also tried to increase the number of buffers but it didn't work either. Any thoughts/solutions?

The reason I am doing this backup routine is that I want these tasks to be automatically done through Windows scheduled tasks.

Pang
  • 9,564
  • 146
  • 81
  • 122
Panos Raul
  • 191
  • 1
  • 1
  • 6
  • 1
    So which command gives the error? Please [edit] your question to include your batch file and tell us which line gives the error. – DavidPostill Mar 28 '15 at 06:30
  • pkzip -spwd -x*.bat 2014 -rp -&, is the command and the error(35) is : Insufficient memory. But the strange thing is that i tested in 3 different computers and it worked like a charm in 2 of them, even for bigger files(>15MB). But in the third one it doesn't stabilize and sometimes it pops out the error but sometimes not. I guess that folders and long sub folders may be the problem due to the size of characters, but it's just an assumption. – Panos Raul Mar 28 '15 at 09:54
  • What version (and bitness) of Windows? What version of `pkzip`? If I can understand your `pkzip` statement: no operation defined (like `-add`); `-spwd`=scramble with password; `-x*.bat`=exclude batch files `2014`=archive name `-rp`=recurse subdirectories and store paths; `-&`=??? Maybe some kind of recursive definition imposed in it: archive `2004` file into itself? – JosefZ Mar 28 '15 at 16:59
  • Use powershell like here https://stackoverflow.com/questions/17842764/copy-item-files-in-folders-and-subfolders-in-the-same-directory-structure-of-sou – Domenico Jan 25 '19 at 19:17

3 Answers3

24

I too got this error : Insufficient memory. I found a relevant answer at this link : https://web.archive.org/web/20070214005534/http://www.terminally-incoherent.com/blog/2007/02/05/xcopy-insufficient-memory/

It appears that this message shows up when the fully qualified (ie. with path) name of the copied file is longer than 254 characters which seems to be Windows maximum path length.

In conclusion: xxcopy works, but robocopy works even better.

Finally my batch file looks like this

@echo off
ROBOCOPY "H:\Laptop-Backup" "E:\Laptop-Backup" /s

rem /e: Include directories and sub directories even if empty
REM /s Copy Subdirectories, but not empty ones.
Martin Capodici
  • 1,486
  • 23
  • 27
Thangasivam Gandhi
  • 349
  • 2
  • 6
  • 1
    ROBOCOPY is better than XCOPY! But RSYNC is better than ROBOCOPY and 4 years later we have windows subsystem linux so we can run it on windows now. Just FYI. – gunslingor Jul 10 '20 at 17:18
  • 1
    Commenting on @gunslingor's recommendation to use rsync within WSL Linux. It works, and you get everything that comes with rsync. But, I got a significant performance drop. Windows defender (antimalware) feels the need to scan every file being copied using rsync but not xcopy / robocopy. With my external hdd, xcopy gives me 180MBps, rsync drops to 15MBps. – Dr Phil Mar 26 '21 at 00:39
  • Finally I found a good reason to switch from using `xcopy` all the times to using `robocopy` all the times. Thank you for bringing this limitation to my attention. I would never guess the "insufficient memory" alone. – BsAxUbx5KoQDEpCAqSffwGy554PSah May 15 '23 at 19:11
3

The standard Windows command line tools like xcopy, rmdir cannot operate with paths longer than MAX_PATH(260 chars). If you want to remove the directory which contains such file which full path is longer than MAX_PATH, or if you want to copy recursively a directory to such a place where the full path to at least one of its files would be longer than MAX_PATH then these command line tools fail.

However you can bypass it with "move" command line tool, because in distinct to xcopy and rmdir(rd) it seems to not iterating the files in the directory one-by-one. This workaround may be not always convenient and requires additional actions, but for the automation/scripting purpose I don't see any other way if you cannot or don't want using special tools a.k.a robocopy. This way works with a directory, but not with a single file. Let me show an example.

Create a directory. Path length: 245 chars

E:\>mkdir E:\veryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryloooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooongdirectory

Thy to create a file inside it so that the file path length is bigger than MAX_PATH. Usual way it won't work.

E:\>echo "content" > E:\veryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryloooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooongdirectory\longlonglonglonglongfilename.txt
The system cannot find the path specified.

Workaround: Create a file(s) with a short path and move the directory containing the file(s) into the long path.

E:\>echo "content" > somedir\longlonglonglonglongfilename.txt

E:\>move somedir E:\veryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryloooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooongdirectory\
        1 dir(s) moved.

Check 1: Was the directory completely moved (with all files)? - Yes.

E:\>dir E:\veryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryloooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooongdirectory\somedir
 Volume in drive E is Workspace
 Volume Serial Number is C864-7C96

 Directory of E:\veryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryloooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooongdirectory\somedir

08/09/2019  11:52 AM    <DIR>          .
08/09/2019  11:52 AM    <DIR>          ..
08/09/2019  11:52 AM                12 longlonglonglonglongfilename.txt
               1 File(s)             12 bytes
               2 Dir(s)  130,574,221,312 bytes free

Check 2. The path is really longer than MAX_PATH, that's why it won't be accessible by the full path for the usual command line tools, so we did everything correct:

E:\>dir E:\veryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryloooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooongdirectory\somedir\longlonglonglonglongfilename.txt
 Volume in drive E is Workspace
 Volume Serial Number is C864-7C96

 Directory of E:\veryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryveryloooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooongdirectory\somedir

File Not Found

The same workaround is valid if you want to remove (with rmdir) the directory containing subfolders/files whose path is longer than MAX_PATH. Just move the upper directory to some short temporary location and there you can rmdir it.

NOTE: Windows command line tool "move" can move directories only within the same disk letter.

Alexander Samoylov
  • 2,358
  • 2
  • 25
  • 28
1

For me, the issue was that xcopy was trying to copy files with very long filenames inside a particular directory (call it /Documents/LongNames) containing lots of these types of names. Because the files with the long names all went together in my case, I zipped the Documents/LongNames sub-directory to a single zip file, since I do not use it often. The backup with xcopy of all of my files in /Documents works just fine now. In general a good way to debug a big batch xcopy is to have your batch file make a log of the files that it is copying. So when there is an issue, it is easier to find. This is how I found the issue to begin with.

JAGreen
  • 11
  • 1