Comparing InputStream from Download to InputStream from File in FitNesse

Question

I'm trying to write a FitNesse test which compares an InputStream download against a reference XLSX file.

Now, my problem is, that the content I get from the InputStream download is obviously no valid Zip archive (which it should be, like the reference file), apart from the current file contents differing in general (but that's not the issue here).

The testing fixture's method I reference in the FitNesse test contains this code (updated):

File downloadedFile = writeDownloadedFile();
File downloadedAltFile = writeDownloadedFileAlt();
File expectedReferenceFile = new File(expectedReferenceFilePath.toUri());
InputStream webFixtureIS = getDialog().getInputStream();
InputStream expectedReferenceFileIS = Files.newInputStream(expectedReferenceFilePath);

byte[] testDownloadedFileBytes = Files.readAllBytes(downloadedFile.toPath());
byte[] testDownloadedAltFileBytes = Files.readAllBytes(downloadedAltFile.toPath());

// expectedReferenceFileIS = Files.newInputStream(expectedReferenceFilePath);
// compareEquals = ExcelCompare.workbookEqual(new XSSFWorkbook(OPCPackage.open(downloadedAltFile)), new XSSFWorkbook(OPCPackage.open(expectedReferenceFileIS)));
// expectedReferenceFileIS.close();

// Direct compare from webFixtureInputStream
webFixtureIS = getDialog().getInputStream();
expectedReferenceFileIS = Files.newInputStream(expectedReferenceFilePath);
compareEquals = ExcelCompare.workbookEqual(new XSSFWorkbook(OPCPackage.open(webFixtureIS)), new XSSFWorkbook(OPCPackage.open(expectedReferenceFileIS)));
webFixtureIS.close();
expectedReferenceFileIS.close();

ExcelCompare.workbookEqual() is based on the XLSX workbook comparing solution I applied for this purpose as a workaround here.

Basically, when comparing against webFixtureIS, OPCPackage.getParts() throws an Exception, because no PackageParts are availible in the download content (ZipArchive contains no entries):

org.apache.poi.openxml4j.exceptions.InvalidFormatException: Package should contain a content type part [M1.13]
at org.apache.poi.openxml4j.opc.ZipPackage.getPartsImpl(ZipPackage.java:197)
at org.apache.poi.openxml4j.opc.OPCPackage.getParts(OPCPackage.java:696)
at org.apache.poi.openxml4j.opc.OPCPackage.open(OPCPackage.java:280)

When comparing against downloadedAltFile instead (the commented-out code above), ZipPackage.getPartsImpl() -> ZipEntrySource.getEntries() will fail, since the internal ZipArchive is null.

Even so, writing the download InputStream into a file like this will result in a XLSX workbook, which can be opened with Excel and OpenOffice (updated):

private File writeDownloadedFileAlt() throws IOException, FileNotFoundException {

    InputStream webFixtureIS = getDialog().getInputStream();
    Path tmpFilePath = Files.createTempFile("jwebunitTestFileAlt", ".xlsx");
    byte[] webFixtureIsBytes = IOUtils.toByteArray(webFixtureIS);
    // Files.write(tmpFilePath, webFixtureIsBytes, StandardOpenOption.CREATE, StandardOpenOption.DELETE_ON_CLOSE);      
    Files.write(tmpFilePath, webFixtureIsBytes, StandardOpenOption.CREATE);     
    webFixtureIS.close();

    return new File(tmpFilePath.toUri());
}

private File writeDownloadedFile() throws IOException, FileNotFoundException {

    InputStream webFixtureIS = getDialog().getInputStream();
    File tmp = File.createTempFile("jwebunitTestFile.xlsx", null);

    int c=0;
    // InputStream in = getTestingEngine().getInputStream();
    tmp.createNewFile();
    tmp.deleteOnExit();
    OutputStream out = new BufferedOutputStream(new FileOutputStream(tmp));
    while ((c = webFixtureIS.read()) != -1) out.write(c);
    webFixtureIS.close();
    // out.close();

    return tmp;
}

writeDownloadedFileAlt() will result in a workbook, which can be opened by Excel without problems, being 0.3kB larger than 16.0kB file writeDownloadedFile() produces, which triggers Excel's repair mechanism (but can be opened still).

When saving the opened file again, Excel creates a file, sized 22.6kB - I still need to try out comparing the reference file against this one.

What am I missing?

Update: Opening and saving the generated "downloadedAltFile" with Excel will actually result in a comparable file with regular internal ZipArchive structure, but I need to know why or where this information is lost - and how to avoid that, of course. Any ideas?

Save the downloaded spreadsheet to a file, then run the Apache Tika app on it in `--detect` mode to see what it really is? — Gagravarr, Sep 20 '16 at 17:02
Thanks again for replying, Gagravar. Tika reveals "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", the same type as the reference file. — fozzybear, Sep 21 '16 at 10:16

score 1 · Answer 1 · answered Sep 21 '16 at 10:36

1

As far as I can tell in your example your closing webFixtureIS before doing anything with it. I would expect the line webFixtureIS.close(); to be below the assignment to compareEquals. Do you intend to copy this to a buffer like the expected file, or should it just be passed (while still open), or did you intend to store it on the local file system and pass it to the compare function, ...?

I would always recommend you to download the file, store it locally and then compare it. This also gives you the ability to review the actual content downloaded manually after the compare said it did not match...

answered Sep 21 '16 at 10:36

Fried Hoeben

3,247
16
14

Well, I'm writing the InputData to the BufferedOutputStream before closing the IS. Btw., I've removed the OS.close() code, as it was stated to be redundant. – fozzybear Sep 21 '16 at 12:00
Your code IS using the `webFixtureIS` after you closed it. And your code sample reads like your copying the `expectedReferenceFileIS` to the buffer, but nothing else. IF you are copying `webFixtureIS` to some buffer shouldn't you pass that buffer to `OPCPackage.open()`, to fill `downloaded`? – Fried Hoeben Sep 21 '16 at 12:15
Ok, seems I can place a comment - I've already updated my test code above, and using a closed InputStream should not apply anymore. Opening one twice in succession, as above, should pose not a problem, I think. For storing the download content via the writeDownloadFileAlt()-based method will not result in a file POI can open in the first place, due to the missing internal ZIP structure, as mentioned. This only works, after manually opening and saving the file again in Excel, which obviously rectifies the missing archive structure, but it will not help me with the test code. – fozzybear Sep 26 '16 at 10:28
Your current code does not seem to flush out. This can result in some missing bytes at the end of the tmp file. Try changing out's type to buffered output stream and calling flush on it. – Fried Hoeben Sep 27 '16 at 13:12

Comparing InputStream from Download to InputStream from File in FitNesse

1 Answers1