Handle Convert.ToBase64String out of memory exception when converting to string

Question

When I try to do the following with .zip folder witch contains some videos I get out of memory exeption.

Byte[] bytes = File.ReadAllBytes(@"C:\folderWithVideos.zip");  
String base64File= Convert.ToBase64String(bytes);//<----- out of memory exception

How to handle this exception properly? I mean without try-catch, I have tried something like:

String base64File;
if (bytes.Length <= System.Int32.MaxValue)
   base64File = Convert.ToBase64String(bytes);

But it didn't helped, but bytes.Length <= 255 did helped, but I'm not sure that 255 is the right number.

I'm curious why you want to encode the file in the first place. — Crowcoder, Jul 22 '18 at 10:47
Possible duplicate of https://stackoverflow.com/questions/351126/convert-a-string-to-stream — DaveShaw, Jul 22 '18 at 10:49
@DaveShaw This link is about converting to stream, also it's VB, plz read my question cerfuly — IOException, Jul 22 '18 at 10:50
The only reason your code doesn't crash on ReadAllBytes() is because you run your program on a 64-bit operating system. Use Project > Properties > Build tab and untick "Prefer 32-bit" to avoid crashing on the ToBase64String() call. The practicality of this code is a low one, decent odds that whatever program needs to swallow that giant string is going to go down in flames the same way. — Hans Passant, Jul 22 '18 at 10:55
There is another problem with this code, the maximum size of the file is 805 megabytes. A larger file is going to exceed the maximum possible base64 string size. — Hans Passant, Jul 22 '18 at 11:00
@HansPassant Do you have any idea what should I do to avoid the exception without `try-catch` ? — IOException, Jul 22 '18 at 11:00
@IOException What you have to do is don't use `string`. Then *exactly* what you have to do depends on how you'll need to use the b64. You could have to send it to a site, save it to the disk, print it to a printer... — xanatos, Jul 22 '18 at 11:15
@xanatos I'm uploading file to some API, the requirement is file with base64 formt — IOException, Jul 22 '18 at 11:21
@IOException We are getting there then :-) You wrote a classical [XY question](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem). You don't want to convert a big file to b64. You want to upload a big file to an API that requires the b64 format. Now... Depending on how you call the API (a web service probably), there is the possibility of resolving this problem, that is a problem different from the one you asked. — xanatos, Jul 22 '18 at 11:27
Don't read and write it as a whole, but stream it. So create a file stream to open file, read it blockwise (as Base64 works by converting 3 bytes to 4 characters the blocksize must be divisible by 3), convert block to Base64, write Base64 string to your API network stream, proceed with next block and so on. This way your memory consumption is limited by the blocksize. — ckuri, Jul 22 '18 at 11:53

score 0 · Answer 1 · edited Jul 22 '18 at 11:20

Based on the code shown in the blog the following code works.

// using  System.Security.Cryptography
private void ConvertLargeFile()
{
    //encode 
    var filein = @"C:\Users\test\Desktop\my.zip";
    var fileout = @"C:\Users\test\Desktop\Base64Zip";
    using (FileStream fs = File.Open(fileout, FileMode.Create))
    using (var cs = new CryptoStream(fs, new ToBase64Transform(),
                                                CryptoStreamMode.Write))

    using (var fi = File.Open(filein, FileMode.Open))
    {
        fi.CopyTo(cs);
    }
    // the zip file is now stored in base64zip    
    // and decode
    using (FileStream f64 = File.Open(fileout, FileMode.Open))
    using (var cs = new CryptoStream(f64, new FromBase64Transform(),
                                                CryptoStreamMode.Read))
    using (var fo = File.Open(filein + ".orig", FileMode.Create))
    {
        cs.CopyTo(fo);
    }
    // the original file is in my.zip.orig
    // use the commandlinetool 
    //  fc my.zip my.zip.orig 
    // to verify that the start file and the encoded and decoded file 
    // are the same
}

he code uses standard classes found in System.Security.Cryptography namespace and uses a CryptoStream and the FromBase64Transform and its counterpart ToBase64Transform

Handle Convert.ToBase64String out of memory exception when converting to string

1 Answers1