What I am trying to do is to download a gzipped file and uncompress it on the fly, from an https site. For that, I have the following code. While the code does download and unzip on the fly, it does not do it for the whole file. The condition (readLine = br.readLine()) != null
becomes false much before the end of the input is reached that is. What could be the problem? Any other way to do this, or solve it?
import java.net.*;
import java.io.*;
import javax.net.ssl.*;
import java.util.zip.GZIPInputStream;
import java.nio.charset.StandardCharsets;
......
BufferedReader br = null;
PrintWriter pw = null;
try
{
URL myurl = new URL(
"https://dnanexus-rnd.s3.amazonaws.com/NA12878-xten/reads/NA12878D_HiSeqX_R1.fastq.gz");
HttpsURLConnection con = (HttpsURLConnection)myurl.openConnection();
GZIPInputStream gzis = new GZIPInputStream(con.getInputStream());
br = new BufferedReader(new InputStreamReader(gzis, StandardCharsets.UTF_8));
pw = new PrintWriter(new File("NA12878D_HiSeqX_R1.fastq"));
String readLine;
while ((readLine = br.readLine()) != null)
pw.println(readLine);
}
finally
{
pw.close();
br.close();
}
Note that if I just download the gzipped file itself, that is, not also uncompress it, it downloads fine. So, it does not look like there is any problem with the server itself. Moreover, I don't get any exceptions.