How to send email with large attach? (OutOfMemoryError: java heap space)

Question

I met an OutOfMemoryError: Java Heap Space when I tried to attach a large attachment. When emails with some large files (say 50M) are sent, the error will be thrown.

The code is like this:

//add attaches
if (vo.getAttaches() != null) {
    InputStream iStream = null;
    ByteArrayDataSource bdSource = null;
    String filename = null;

    for (int i = 0; i < vo.getAttaches().length; i++) {
         iStream = new FileInputStream(vo.getAttaches()[i]);
         bdSource = new ByteArrayDataSource(iStream, null);
         filename = vo.getAttachesFileName()[i];
         email.attach(bdSource, MimeUtility.encodeText(filename), filename);
    }
}

It is bdSource = new ByteArrayDataSource(iStream, null) throws the exception above. I had read "out of memory using java mail" but i don't understand. How can I upload large attaches? If I define a buffer like byte[1024], then how to code email.attach()? Should all data in buffer use the same filename?

Update: Thanks for your kindness. I use FileDataSource instead of ByteArrayDataSource, it seems that there's no exception in my send() function. But I still can not send emails with large attach. The apache james got this error:

    java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2786)
    at java.io.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:133)
    at com.sun.mail.util.ASCIIUtility.getBytes(ASCIIUtility.java:261)
    at javax.mail.internet.MimeMessage.parse(MimeMessage.java:338)
    at org.apache.james.core.MimeMessageWrapper.parse(MimeMessageWrapper.java:477)
    at org.apache.james.core.MimeMessageWrapper.loadMessage(MimeMessageWrapper.java:205)
    at org.apache.james.core.MimeMessageWrapper.checkModifyHeaders(MimeMessageWrapper.java:414)
    at org.apache.james.core.MimeMessageWrapper.setHeader(MimeMessageWrapper.java:426)
    at org.apache.james.core.MimeMessageCopyOnWriteProxy.setHeader(MimeMessageCopyOnWriteProxy.java:652)
    at org.apache.james.transport.mailets.UsersRepositoryAliasingForwarding.service(UsersRepositoryAliasingForwarding.java:101)
    at org.apache.james.transport.mailets.LocalDelivery.service(LocalDelivery.java:64)
    at org.apache.james.transport.LinearProcessor.service(LinearProcessor.java:424)
    at org.apache.james.transport.JamesSpoolManager.process(JamesSpoolManager.java:405)
    at org.apache.james.transport.JamesSpoolManager.run(JamesSpoolManager.java:309)
    at java.lang.Thread.run(Thread.java:619)
03/07/12 13:08:33 ERROR spoolmanager: An error occurred processing Mail1341292071375-0 through transport
03/07/12 13:08:33 ERROR spoolmanager: Result was error
03/07/12 13:08:33 ERROR spoolmanager: Exception in processor <error>
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2786)
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
    at org.apache.james.core.MimeMessageUtil.copyStream(MimeMessageUtil.java:168)
    at org.apache.james.core.MimeMessageWrapper.writeTo(MimeMessageWrapper.java:276)
    at org.apache.james.core.MimeMessageUtil.writeTo(MimeMessageUtil.java:66)
    at org.apache.james.core.MimeMessageUtil.writeTo(MimeMessageUtil.java:50)
    at org.apache.james.mailrepository.MessageInputStream.writeStream(MessageInputStream.java:131)
    at org.apache.james.mailrepository.MessageInputStream.<init>(MessageInputStream.java:101)
    at org.apache.james.mailrepository.JDBCMailRepository.store(JDBCMailRepository.java:718)
    at org.apache.james.transport.mailets.ToRepository.service(ToRepository.java:98)
    at org.apache.james.transport.LinearProcessor.service(LinearProcessor.java:424)
    at org.apache.james.transport.JamesSpoolManager.process(JamesSpoolManager.java:405)
    at org.apache.james.transport.JamesSpoolManager.run(JamesSpoolManager.java:309)
    at java.lang.Thread.run(Thread.java:619)
03/07/12 13:08:33 ERROR spoolmanager: An error occurred processing Mail1341292071375-0 through error
03/07/12 13:08:33 ERROR spoolmanager: Result was ghost

Consider increasing the JVM heap space to 256 MB, probably its not enough as you are trying to upload large files. — 18bytes, Jul 02 '12 at 09:12

score 4 · Accepted Answer · answered Jul 02 '12 at 09:19

4

The problem is that you're trying to store the whole file in memory while composing the message, which actually isn't usually necessary. If you're attaching real files, then you are far better off using a javax.activation.FileDataSource instead of a javax.mail.util.ByteArrayDataSource (both implement the DataSource interface) as that can allow the data to be streamed rather than being held in memory.

answered Jul 02 '12 at 09:19

Donal Fellows

133,037
18
149
215

Thanks for your kindness. I use FileDataSource instead of ByteArrayDataSource, it seems that there's no exception in my send() function. But I still can not send emails with large attach. The apache james got this error: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2786) at java.io.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:133) …… – Joshua Jul 03 '12 at 05:16
Sounds like the mail server is also choking on a large attachment. This is not the fault of your client code, but rather a **bug** in Apache James. File a bug report. – Donal Fellows Jul 03 '12 at 13:03
Thank you. I think maybe I can store the large file separately. I'll try to store it into a "LargeAttach" table which have a blob, and just store a link string into James. Maybe this way can avoid the error. – Joshua Jul 04 '12 at 05:38

score 2 · Answer 2 · edited May 23 '17 at 12:29

For your update: It seems an old (?) variant of JamesSpoolManager reads the whole file into memory when transforming to "mail format (ascii)" in the same thread dated 30/Dec/10 the problem seems fixed.

ByteArrayDataSource will read the full input from the input stream provided, see javadoc:

Create a ByteArrayDataSource with data from the specified InputStream and with the specified MIME type. The InputStream is read completely and the data is stored in a byte array.

So if the file to be read is "larger" than your heap size (JVM limit) it will throw the OutOfMemoryException.

So, to answer your question, you have (at least) two options:

Use a FileDataSource like in this SO answer
Give your program more memory (probably not a good solution in this case..)

Thank you. FileDataSource can work good. But the Apache James throws Exception. And I will try to save large files into a separate table with a blob instead of store into Email. But in this situation, the receiver should link to my site to see the attach. — Joshua, Jul 04 '12 at 05:42

Rohit Batta · Answer 3 · 2015-07-22T07:47:13.140

-1

I was facing same issue. But, during multiple emails at a same time. Which result in two errors in logs.

1. OutOfMemoryError: Java Heap Space

2. Maximum number of connections exceeded

I have changed two parameters in wrapper.conf file.

#wrapper.java.initmemory=16
 wrapper.java.initmemory=32

#wrapper.java.maxmemory=64
wrapper.java.maxmemory=128

After restart of server, Error has gone and mail send-receive is working.

edited Jul 22 '15 at 07:47

answered Jul 22 '15 at 07:40

Rohit Batta

482
4
16

Cleb, Even i am not sure, why it is? I was facing this issue as my mails per minute count was more than 500/minute. So, i has increased HEAP SIZE – Rohit Batta Jul 22 '15 at 08:33

How to send email with large attach? (OutOfMemoryError: java heap space)

3 Answers3