0

Up till early this year the US Treasury web site posted monthly US Receipts and Outlays data in txt format. It was easy to write a program to read and store the info. All I use were:

URL url = new URL("https://www.fiscal.treasury.gov/fsreports/rpt/mthTreasStmt/mts1214.txt")
URLConnection connection.openConnection();
InputStream is = connection.getInputStream();

Then I just read the InputStream into a local file.

Now when I try same code, for May, I get an InputStream with nothing in it. Just clicking on "https://www.fiscal.treasury.gov/fsreports/rpt/mthTreasStmt/mts0415.xlsx" opens an excel worksheet (the download path has since changed).

Which is great if you don't mind clicking on each link separately ... saving the file somewhere ... opening it manually to enable editing ... then saving it again as a real .xlsx file (because they really hand you an .xls file.)

But when I create a URL from that link, and use it to get an InputStream, the is empty. I also tried url.openStream() directly. No different.

Can anyone see a way I can resume using a program to read the new format?

In case its of interest I now use this code to write the stream to the file bit by bit... but there are no bits, so I don't know if it works.

static void copyInputStreamToFile( InputStream in, File file ) {
    try {
        OutputStream out = new FileOutputStream(file);
        byte[] buf = new byte[1024];
        System.out.println("reading: " + in.read(buf));
        //This is what tells me it is empty, i.e. the loop below is ignored.
        int len;
        while((len=in.read(buf))>0){
            out.write(buf,0,len);
        }
        out.close();
        in.close();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

Any help is appreciated.

david
  • 2,435
  • 1
  • 21
  • 33
George
  • 509
  • 2
  • 9
  • 25
  • Grab the xls from the new uri pattern, and get the text with [apache poi](https://poi.apache.org/text-extraction.html) or [tika](http://tika.apache.org/)? – Elliott Frisch Jan 08 '16 at 04:52
  • Possible duplicate of [How to download and save a file from Internet using Java?](http://stackoverflow.com/questions/921262/how-to-download-and-save-a-file-from-internet-using-java) – dimo414 Jan 08 '16 at 05:03

0 Answers0