powershell Compress-Archive OutOfMemoryException

Question

I need to compress a folder with power shell.

There is my code:

Get-ChildItem $YourDirToCompress -Directory  | 
           where { $_.Name -notin $DirToExclude} | 
              Compress-Archive -DestinationPath $ZipFileResult -Update

Move-Item -Path $ZipFileResult -Destination $ZipFileDest

I get:

Exception calling "Write" with "3" argument(s): "Exception of type 'System.OutOfMemoryException' was thrown."
At 
C:\Windows\system32\WindowsPowerShell\v1.0\Modules\Microsoft.PowerShell.Archive\Microsoft.PowerShell.Archive.psm1:820 
char:29
+ ...                     $destStream.Write($buffer, 0, $numberOfBytesRead)
+                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : OutOfMemoryException

I have set :

Set-Item WSMan:\localhost\Plugin\Microsoft.PowerShell\Quotas\MaxMemoryPerShellMB 8000
Set-Item WSMan:\localhost\Shell\MaxMemoryPerShellMB 8000
Restart-Service WinRM

The whole file is about 1.9 GiB and the compressed file is 500 MiB. I find it hard to believe it is really a memory problem.

Also, once or twice it succeeded on file creation (when MaxMemoryPerShellMB was set to 4000). But most times it fails.

What can I do?

What if you attempted this using the .NET class? First load the assembly by running `Add-Type -assembly "system.io.compression.filesystem"` at the top of the script. Then replace the `Compress-Archive` command line with `[io.compression.zipfile]::CreateFromDirectory($SourcePath, $DestinationPath)`. — AdminOfThings, Feb 26 '19 at 13:20
@AdminOfThings I need to exclude one sub directory - How can I do this with .net? — Hellpless, Feb 26 '19 at 13:29
You could copy to a temp directory. Compression isn't really a strength of either Powershell or .Net. If it says it's out of memory, then I guess it's exactly that. Consider using 3rd party tools like 7zip or DotNetZip which are far more efficient and have wider functionality. — marsze, Feb 26 '19 at 14:34
Getting the same error in 2022, on a folder I normally zip up every few days. Folder is over 11 GiB and zips down to around 4 GiB. Has been working for years, until today. I will post an answer if I can find what's going on. — Reg Edit, Jun 25 '22 at 12:47
You can find the PowerShell solution using `IO.Compression` [here](https://stackoverflow.com/a/72611161/15339544) — Santiago Squarzon, Jun 25 '22 at 13:52
That's a different issue, to do with a 2G limit on an individual file. The OP's file is under 2G, and works intermittently. — Reg Edit, Jun 25 '22 at 23:04

Reg Edit · Answer 1 · 2022-06-25T22:58:39.743

I was getting the exact same error in 2022:.

Exception calling "Write" with "3" argument(s): "Exception of type
'System.OutOfMemoryException' was thrown."
At C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules\Microsoft.PowerShell.Arch
ive\Microsoft.PowerShell.Archive.psm1:820 char:29
+ ...                     $destStream.Write($buffer, 0, $numberOfBytesRead)
+                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : OutOfMemoryException

OutOfMemoryException Microsoft.PowerShell.Archive.psm1

I was able to resolve it very simply after a bit of thinking.

I too doubted that it was really a memory problem, since I have 16 GiB of RAM and it was showing only around 9 GiB in use.

Microsoft Scripting Guy Ed Wilson has written an article Learn How to Configure PowerShell Memory, and for some this may provide the solution. In my case it did not, as PS appeared to be already configured to use the maximum available memory.

I was getting the issue on a folder I normally zip up every few days. This folder is over 11 GiB and zips down to around 4 GiB. This had previously been working for years.

However, thinking about it, to create a zip of this size I think it's likely that PS holds a significant amount of data in memory, more and more right up to the point when it commits it to the file system at the end. And I realized that although the computer has 16 GiB of RAM and was showing only around 9 GiB in use, 9 GiB is an unusually large amount of memory to be in use on that computer. Subtracting memory used by Windows, graphics etc still left several GiB free—but perhaps not enough for PS to create a large zipfile.

Sure enough, after a reboot to release the unusually large amount of RAM that had been in use, the process ran successfully and created my zipfile as usual.

In summary:

Check whether you can configure PS to use more RAM;
Check in Task Manager to see if you have less RAM free than usual;
Reboot to make maximum RAM available to PS.

score 0 · Answer 2 · answered Dec 21 '22 at 16:16

It seems that the built-in Zip module of PowerShell is quite limited and designed for simple straightforward tasks. One of those limitations is that the max file size is 2GB. Also, the way that it obtains memory seems not very efficient. For example, if you try to zip files by Winzip or WinRar using Windows UI, it will probably work. More info about the underlying library https://learn.microsoft.com/en-us/dotnet/api/system.io.compression.ziparchive

So, I recommend using 7Zip4Powershell library which worked perfectly for me with the same files/ Windows memory etc.

powershell Compress-Archive OutOfMemoryException

2 Answers2