Java: How to perform dos2unix on multiple xml files in a zip file?

Question

I have a form to upload a zip file and then convert all the xml files inside the zip to unix format if they are in dos format. Right now I receive the input as an InputStream. How do I process the file in inputstream and perform (dos2unix) on it to convert it to unix format?

I tried to convert the stream to a file and then convert them but didn't work

public void uploadFile(UploadAuditConfig transaction,String fileType, InputStream in, String delimiter) {
    ZipInputStream zipInputStream = new ZipInputStream(in);
    ZipEntry entry = null;
    do{
                entry = zipInputStream.getNextEntry();
                //need to convert this entry to unix format if it is dos before I pass it to processFile method
                if(entry != null && !entry.isDirectory()) {
                    List<Map<String,String>> list =processFile(zipInputStream, delimiter);
                    zipInputStream.closeEntry();
                 }
    }while(entry!=null);
}


public List<Map<String, String>> processFile(InputStream in, String 
delimiter){
        List<Map<String,String>> acesList = new ArrayList<>();
        XMLInputFactory xif = XMLInputFactory.newInstance();
        XMLStreamReader xsr = xif.createXMLStreamReader(new InputStreamReader(in));
        while (xsr.nextTag() == XMLStreamConstants.START_ELEMENT) {
        File file = new File("/tmp/" + "out" + i + ".xml");
                FileWriter fw = new FileWriter(file);
                if (!file.exists())
                    file.createNewFile();
                FileOutputStream fos = new FileOutputStream(file, true);
                t.transform(new StAXSource(xsr), new StreamResult(fos));
                fos.close();

                if (i == 0) {
                    JSONObject xmlJSONObjHeader = XML.toJSONObject(content);
                    Object o = JsonPath.parse(xmlJSONObjHeader.toString()).read("$['Header']['BrandAAIAID']");
                    brandaaiaid = String.valueOf(o);
                    logger.info("Brand id: " + brandaaiaid);
                    file.delete();
                    fw.close();
                    i++;


                }
        }
        return acesList;
}

Expected: Unix formatted file from inputstream

What are you supposed to do with the result of the conversion? — Andreas, Aug 26 '19 at 22:54
Is this the complete code? `processFile()` has a return type of `List>` but doesn't have a `return` statement. — Kaan, Aug 26 '19 at 22:55
I didn't post the entire code for ProcessFile as it's very case specific. I just need to convert the entry in uploadFile method to unix format. — Dookoto_Sea, Aug 26 '19 at 22:59
Why? XML doesn't care, and therefore neither do any of its standard tools or APIs, including the ones you are using. You don't need to do this. — user207421, Aug 26 '19 at 23:21
@user207421 i need to process the XMLStreamReader as shown, and if it is in dos format it fails while reading the next tag. It is case specific and so I need the conversion. — Dookoto_Sea, Aug 26 '19 at 23:25
It fails how? This is the real problem. You're taking a hammer to crack a nut. It isn't clear that you even need an `XMLStreamReader` here at all. And get rid of the `exists()/createNewFile()` stuff. It is a complete waste of time and space, and you're doing it in the wrong place anyway. If it works you will get empty files. — user207421, Aug 26 '19 at 23:31
@user207421 there is some more processing that I am doing afterwards, I would really appreciate if you could help me solve the problem rather than saying it's not needed. When I am reading the next tag I am getting a XMLStreamException: ParseError: Content is not allowed in Prolog. If I use a unix formatted file it works fine, hence need the conversion — Dookoto_Sea, Aug 26 '19 at 23:34
I can only repeat. XML doesn't care. The error you cite is because you have already consumed the first element of the stream, so the SAX parser can't parse the rest of it. It has nothing to do with DOS2UNIX whatsoever. Your code is wrong, and as I already said, you don't need an `XMLStreamReader` here at all. Just create a `StreamSource` from the zip input stream. — user207421, Aug 26 '19 at 23:37
@user207421 I am trying to understand your explanation here, but it works completely fine if I am using a unix formatted file without any change to code. When I convert the dos file to unix externally and upload it I don't get any error and the file is processed. I debugged it and found out that the versionStr is NULL for dos files when processing. — Dookoto_Sea, Aug 27 '19 at 17:51
Possible duplicate of [How to convert files from Dos to Unix](https://stackoverflow.com/questions/9374991/how-to-convert-files-from-dos-to-unix) — Arnaud Claudel, Aug 27 '19 at 23:00

score 0 · Accepted Answer · answered Sep 04 '19 at 18:17

0

I was able to convert the inputstream to a file and then convert the file to unix using dos2unix and then again passing the file as inputstream.

    OutputStream outputStream = null;
    File file1 = new File("/tmp/output.txt");
    try
    {
        outputStream = new FileOutputStream(file1);

        int read = 0;
        byte[] bytes = new byte[1024];
        while ((read = in1.read(bytes)) != -1) {
            outputStream.write(bytes, 0, read);
        }
         } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } finally
        {
            if(outputStream != null)
            { 
            try {
                outputStream.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
    ExecuteShell.executeJavaCommand(file1.getAbsolutePath());

    logger.info("action=output.txt converted from dos to unix");

    InputStream in = null;
    try {
        in = new FileInputStream(file1);
    } catch (FileNotFoundException e) {
        e.printStackTrace();
    }

answered Sep 04 '19 at 18:17

Dookoto_Sea

521
1
5
16

I'd urge you to research the root cause of your problem, as XML doesn't normally care about line endings. The code you posted as an answer is more trouble than it's worth. It swallows exceptions. It's not thread-safe (may or may not be an issue, but if it is, have fun debugging). It's overly complex, writing and reading temp files plus the extra process to convert the line endings. It relies on an external tool to be present. Your answer also isn't a complete reusable example, so it's unlikely that others can benefit from it; refer [answer]. – Robert Sep 04 '19 at 18:40
I found the problem, so basically when I try to read the tags using xmlStreamReader it fails if the file is in dos format, when I used the code above to convert the inputstream to unix format it fixed the problem. Hence I posted the solution as I thought it would help someone who is facing the same problem – Dookoto_Sea Sep 04 '19 at 18:48
No. That is not a solution, that is a quick and dirty hack, a workaround. You did not find the real problem. Read the comments under your question. I'd not allow you to check that code in. It's up to you though. – Robert Sep 04 '19 at 20:55
@Robert I did find the real problem. The file had BOM formatting, after conversion it removed the first three bytes and it helped to read the tag. – Dookoto_Sea Sep 04 '19 at 22:43

Java: How to perform dos2unix on multiple xml files in a zip file?

1 Answers1