Timeout in uploading a big file to google cloud storage

Question

I'm having trouble uploading large files to Google Cloud Storage. I successfully uploaded a 700MB file, but when I tried a 5GB text file, it threw the following exception. I was unable to find a solution with a Google search.

The problem is in the main method of a simple java class.

Exception in thread "main" java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out

Caused by: java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:150)
    at java.net.SocketInputStream.read(SocketInputStream.java:121)
    at sun.security.ssl.InputRecord.readFully(InputRecord.java:312)
    at sun.security.ssl.InputRecord.read(InputRecord.java:350)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:893)
    at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:850)
......

score 1 · Answer 1 · edited May 23 '17 at 10:31

1

Getting java.net.SocketTimeoutException: Connection timed out in android it looks like you may need to jump up your connection timeout setting. The link is for android, but the same thing applies, and it's implemented exactly the same.

edited May 23 '17 at 10:31

Community

1
1

answered Sep 06 '13 at 18:34

user2366842

1,231
14
23

In the java client library for google cloud storage, there is no where to set the connection timeout length. – user2755358 Sep 12 '13 at 00:46
Storage.Objects.Insert insertObject = storage.objects().insert(bucketName, objectMetadata, mediaContent); insertObject.execute(); It seems no where to set the timeout. – user2755358 Sep 12 '13 at 00:52

score 1 · Answer 2 · answered Oct 28 '13 at 22:39

1

With larger files, especially on mobile devices and wireless connections, you're much more likely to have your uploads interrupted by a broken connection. The solution to this is to make your upload resilient against broken connections. Google Cloud Storage handles this using a technique called Resumable Uploads. You'll need to make use of this technique so your application can recover from network issues.

answered Oct 28 '13 at 22:39

Benson

22,457
2
40
49

It used resumable upload. By default, it is resumable upload. – user2755358 Oct 28 '13 at 22:48

score 1 · Answer 3 · answered Jan 12 '21 at 13:20

In Java SDK we have option to change the connection timeout and retry.

HttpTransportOptions transportOptions = StorageOptions.getDefaultHttpTransportOptions();
            transportOptions = transportOptions.toBuilder().setConnectTimeout(60000).setReadTimeout(60000)
                    .build();
var storage = StorageOptions.newBuilder()
    .setRetrySettings(RetrySettings.newBuilder().setMaxAttempts(2).build())
    .setTransportOptions(transportOptions)
    .setProjectId("project_id").build().getService();

Timeout in uploading a big file to google cloud storage

3 Answers3