0

I am trying to upload a big file (950MB) to Azure using the Azure SDK. I use the sample code avaliable here: http://azure.microsoft.com/en-us/documentation/articles/media-services-java-how-to-use/

Unfrotunately, this results in OutOfMemoryError. Presumably because the SDk is trying to load the whole file into memory. How can I avoid this?

Caused by: java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2271) ~[na:1.7.0_21]
    at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113) ~[na:1.7.0_21]
    at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93) ~[na:1.7.0_21]
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140) ~[na:1.7.0_21]
    at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:78) ~[na:1.7.0_21]
    at com.sun.jersey.api.client.CommittingOutputStream.write(CommittingOutputStream.java:90) ~[jersey-client-1.13.jar:1.13]
    at com.sun.jersey.core.util.ReaderWriter.writeTo(ReaderWriter.java:115) ~[jersey-core-1.13.jar:1.13]
    at com.sun.jersey.core.provider.AbstractMessageReaderWriterProvider.writeTo(AbstractMessageReaderWriterProvider.java:76) ~[jersey-core-1.13.jar:1.13]
    at com.sun.jersey.core.impl.provider.entity.InputStreamProvider.writeTo(InputStreamProvider.java:98) ~[jersey-core-1.13.jar:1.13]
    at com.sun.jersey.core.impl.provider.entity.InputStreamProvider.writeTo(InputStreamProvider.java:59) ~[jersey-core-1.13.jar:1.13]
    at com.sun.jersey.api.client.RequestWriter.writeRequestEntity(RequestWriter.java:300) ~[jersey-client-1.13.jar:1.13]
    at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:213) ~[jersey-client-1.13.jar:1.13]
    at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149) ~[jersey-client-1.13.jar:1.13]
    at com.microsoft.windowsazure.services.media.implementation.SASTokenFilter.doHandle(SASTokenFilter.java:60) ~[microsoft-windowsazure-api-0.4.5.jar:na]
    at com.microsoft.windowsazure.services.core.IdempotentClientFilter.handle(IdempotentClientFilter.java:41) ~[microsoft-windowsazure-api-0.4.5.jar:na]
    at com.sun.jersey.api.client.Client.handle(Client.java:648) ~[jersey-client-1.13.jar:1.13]
    at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680) ~[jersey-client-1.13.jar:1.13]
    at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) ~[jersey-client-1.13.jar:1.13]
    at com.sun.jersey.api.client.WebResource$Builder.put(WebResource.java:537) ~[jersey-client-1.13.jar:1.13]
    at com.microsoft.windowsazure.services.blob.implementation.BlobOperationRestProxy.createBlockBlob(BlobOperationRestProxy.java:544) ~[microsoft-windowsazure-api-0.4.5.jar:na]
    at com.microsoft.windowsazure.services.blob.implementation.BlobOperationRestProxy.createBlockBlob(BlobOperationRestProxy.java:529) ~[microsoft-windowsazure-api-0.4.5.jar:na]
    at com.microsoft.windowsazure.services.blob.implementation.BlobExceptionProcessor.createBlockBlob(BlobExceptionProcessor.java:419) ~[microsoft-windowsazure-api-0.4.5.jar:na]
    at com.microsoft.windowsazure.services.media.implementation.MediaBlobContainerWriter.createBlockBlob(MediaBlobContainerWriter.java:72) ~[microsoft-windowsazure-api-0.4.5.jar:na]
Klaus
  • 2,328
  • 5
  • 41
  • 62

2 Answers2

3

I believe the solution is to split the file and send it in chunks. There was an answer to this question that was deleted for unknown reasons but it pointed me to the right direction. I believe this should work: https://github.com/Azure/azure-sdk-for-java-samples/blob/master/MediaServicesLargeFileUpload/src/test/java/ext/microsoft/windowsazure/MediaServicesLargeFileUpload.java

Klaus
  • 2,328
  • 5
  • 41
  • 62
-1

Looks like your application server ran out of memory, noted by the message Java heap space. Increase the memory allocated for the JVM by using Xmx variable. For example:

java -Xmx1024M YourClass

Related:

Community
  • 1
  • 1
Luiggi Mendoza
  • 85,076
  • 16
  • 154
  • 332