0

I am using HTTP Range requests to download files in part and merge them together. This works fine with HTTP but with HTTPS the number of bytes that's read by the inputStream is so small that it takes ages to complete. Also I don't want to remove the Tread.Sleep(100) because the CPU usages rises to 100%.

Current Source Code:

@Override
public Boolean call() throws Exception
{
    //Preparing Connection and setting ranges.
    HttpURLConnection urlConnection =
            (HttpURLConnection) fileLocation.openConnection();
    urlConnection.setRequestProperty(
    "Range", "Bytes=" + chunkStart + "-" + chunkEnd);

    InputStream inputStream = urlConnection.getInputStream();
    if (urlConnection.getResponseCode() != HttpURLConnection.HTTP_PARTIAL)
    {
        throw new Exception("Server returned status Code: " +
        urlConnection.getResponseCode());
    }

    ByteBuffer buffer = ByteBuffer.allocate((int)(chunkEnd - chunkStart));
    long bytesRead = 0;
    // Copying the inputStream to my byteBuffer
    ReadableByteChannel inputChannel = Channels.newChannel(inputStream);
    while ((bytesRead = inputChannel.read(buffer)) > 0)
    {
        System.out.println("Bytes read: " + bytesRead);
        Thread.sleep(100);
    }

    long writeOffset = chunkStart;
    buffer.flip();
    while (buffer.hasRemaining())
    {
      writeOffset += file.write(buffer, writeOffset);
    }
    buffer.clear();
  return true;
}

Downloading a 10mb file. Output with HTTP: (takes about 2 seconds)

 Bytes read: 197102
 Bytes read: 365844
 Bytes read: 218372
 Bytes read: 253512
 Bytes read: 336256
 ...

Output with HTTPS: (takes about 5 minutes)

Bytes read: 320
Bytes read: 1389
Bytes read: 1389
Bytes read: 1389
Bytes read: 1389
Bytes read: 1389
...

Removing the Thread.sleep gives this Output with HTTPS: (takes about 2 seconds but CPU usage is about 99%)

Bytes Read: 1389
Bytes Read: 1389
Bytes Read: 1389
Bytes Read: 1389
Bytes Read: 1389
...
Bytes Read: 16384
Bytes Read: 16384
Bytes Read: 16384
Bytes Read: 16384

I also tried to read the inputStream manually to a byteArray and then write it into the byteBuffer or using a BufferedInputStream but i get the same results.

I found a quite similar case on this topic but I'm not sure if that's really the right solution Java ssl performance issue in connection with downloads

Update: I looked into the SSLSocket debug info with System.setProperty("javax.net.debug","ssl:packet") and this is what I get

TLSv1.2 : TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
BufferSize: 16916
Packet Buffer Size: 16921
[Raw read]: length = 1413

0000: 00 00 00 00 00 00 00 01   CD 87 B9 AD B3 60 94 93  .............`..
0010: D5 13 0E 5A 3F D0 5D CE   C1 46 93 62 23 67 77 57  ...Z?.]..F.b#gwW
0020: 39 D4 07 46 96 89 72 12   2F 91 74 92 D8 C4 8F A4  9..F..r./.t.....
...more and more
0570: 04 E1 83 F6 7E 72 8A 38   5F EA 2B A0 F5 75 C3 57  .....r.8_.+..u.W
0580: 78 23 5E 01 AB                                     x#^..
pool-1-thread-1, READ: TLSv1.2 Application Data, length = 1413
Padded plaintext after DECRYPTION:  len = 1389

That explains why I can only read 1389 bytes per call, but how do I increase this size, after all the buffer size is much bigger and if I am running it without Thread.sleep it gets nearly filled.

Community
  • 1
  • 1
Tharon
  • 41
  • 1
  • 5

0 Answers0