Java/Android HttpURLConnection setChunkedStreamingMode not working with all PHP servers

Question

I've spent the last day or so trying to debug this issue and I'm out of ideas. Basically I have an Android app that is POSTing some data to a PHP/Apache web server. This code seems to work fine when I point it at my local test server. It also seems to work fine when I point it at my production server, but ONLY when I comment out the line conn.setChunkedStreamingMode(maxBufferSize);. Once that line is enabled, the post only works on my local test server, but when posting to the production server, the PHP $_FILES array is empty. I've tried passing numerous values to setChunkedStreamingMode (including 0 and 1024) but none of these seem to fix the problem.

At this point I'm assuming the issue has to do with the way the production server's PHP is configured, but as far as I can tell, all the important parameters on the server are the same as on my test instance. Additionally, they're both running the same version of Apache and PHP. My production server is run by Bluehost.

Here's the Java code I'm using to upload:

HttpURLConnection conn = null;
DataOutputStream dos = null;
DataInputStream inStream = null;
String lineEnd = "\r\n";
String twoHyphens = "--";
String boundary = "***************************************************";
int bytesRead, bytesAvailable, bufferSize;
byte[] buffer;
int maxBufferSize = 212144; // 1024*1024 = 1MB.  212144 is a quarter MB.
FileInputStream fileInputStream = null;
try
{
  // ------------------ CLIENT REQUEST
  fileInputStream = new FileInputStream(new File(existingFileWithFullPath));
  // open a URL connection to the Servlet
  URL url = new URL(BACKUP_POST_URL);
  // Open a HTTP connection to the URL
  conn = (HttpURLConnection) url.openConnection();
  // Allow Inputs
  conn.setDoInput(true);
  // Allow Outputs
  conn.setDoOutput(true);
  // Send in chunks (to avoid out of memory error)
  conn.setChunkedStreamingMode(maxBufferSize);
  // Don't use a cached copy.
  conn.setUseCaches(false);
  // Use a post method.
  conn.setRequestMethod("POST"); 
  conn.setRequestProperty("Connection", "Keep-Alive");
  conn.setRequestProperty("Content-Type", "multipart/form-data;boundary="
      + boundary);
  conn.setReadTimeout(200000); // 200 seconds...
  dos = new DataOutputStream(conn.getOutputStream());
  dos.writeBytes(twoHyphens + boundary + lineEnd);
  dos.writeBytes("Content-Disposition: form-data; name=\"uploadedfile\";filename=\""
      + fileName + "\"" + lineEnd);
  dos.writeBytes(lineEnd);
  // create a buffer of maximum size
  bytesAvailable = fileInputStream.available();
  bufferSize = Math.min(bytesAvailable, maxBufferSize);
  buffer = new byte[bufferSize];
  // read file and write it into form...
  bytesRead = fileInputStream.read(buffer, 0, bufferSize);
  while (bytesRead > 0)
  {
    try {
      dos.write(buffer, 0, bufferSize);          
    } catch (OutOfMemoryError oome) {
      Log.e(CommonStatic.LOG_NAME, "Out of memory error caught...");
      oome.printStackTrace();
      fileInputStream.close();
      throw new Exception("Out Of Memory!");
    }
    bytesAvailable = fileInputStream.available();
    bufferSize = Math.min(bytesAvailable, maxBufferSize);
    bytesRead = fileInputStream.read(buffer, 0, bufferSize);
  }
  // send multipart form data necesssary after file data...
  dos.writeBytes(lineEnd);
  dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
  fileInputStream.close();
  dos.flush();
  dos.close();

  // close streams
  Log.d(CommonStatic.LOG_NAME, "Backup file written to server successfully...");
}
catch (Exception ex)
{
  Log.e(CommonStatic.LOG_NAME, "Backup File Upload Error: " + ex.getMessage(), ex);
  throw new Exception (c.getString(R.string.SAVE_TO_CLOUD_ERROR));
}

And here's the PHP code I'm using on the other end to recieve:

$target = "userfiles/"; 
$target = $target . basename( $_FILES['uploadedfile']['name']);

if(move_uploaded_file($_FILES['uploadedfile']['tmp_name'], $target)) 
{ 
  echo "SUCCESS";
} 
else 
{ 
  echo "FAIL";
}

I stuck a print_r($_FILES); at the very beginning of the script to determine that $_FILES is empty on the production instance, but not on the test instance. Any ideas would be greatly appreciated.

something in your production configuration prevents this mode from working — njzk2, Sep 03 '12 at 14:54

score 2 · Answer 1 · answered Sep 18 '12 at 01:40

Unfortunately I wasn't able to find a direct solution to this problem, but I was able to find a workaround. Instead of using conn.setChunkedStreamingMode I used conn.setFixedLengthStreamingMode, which I had tried before with no success. The key to getting conn.setFixedLengthStreamingMode to work is to pass it the full length of the data you are attempting to send (in this case the file), plus the length of the headers. Fortunately header length is usually fixed in code such as this, so once you figure out how big your header is (keeping in mind things like the filename, which gets sent in the header under Content-Disposition is also variable) you can just put it in as a fixed number. To figure out my header length, I ran the code first without specifying any length for the header. The error message I received back gave me an expected and an actual value for the number of bytes sent, which allowed me to calculate the header length.

can you put the code you used to find out the headers from the conn? — desgraci, Jul 31 '13 at 14:27
@BlueScreenOfTOM: By using **conn.setFixedLengthStreamingMode** i am getting sockettimeout error for large video files. And i saw your code and it says 200 secs for time out. so on what basis did you set 200 sec? But by using **conn.setFixedLengthStreamingMode** i am able to eliminate both out of memory error and socket time out error also. — Mahantesh M Ambi, Jul 17 '14 at 11:44
by doing this ,you are telling whole file yo be uploaded as a chunk, which typically defeats the purpose of this method — Vihar, Oct 09 '15 at 07:58

Java/Android HttpURLConnection setChunkedStreamingMode not working with all PHP servers

1 Answers1

Linked