0

I have been asked to write a Java program that would take TSV files as an inpunt and generate a different TSV file (with quite some changes inside it that are very variable on the input and args) as output.

It is a pretty big program (took me 3 days to code but I'm not good) and it is finally working on inputs from 15k lines, generating 1500K lines output.

Writing the code, I had no idea that I would have to implement it in Talend afterwards, so it is a normal Java program that takes 4 args : Name of the input file, Name of the output file, int , int

I managed to put my Main as a routine with the needed extra packages (personal and openCSV).

My question is : Is it possible to implement it in Talend without changing it? Can't I just tell Talend, here is the file in input and those are the args? I had never heard of Talend before yesterday.

Here is the main if you are interested but i believe my question to be pretty generic.

public static void main(String[] args) throws ArgsExceptions {

    // Boolean set to true while everything is good
    Boolean everythingOk = true;

    String inputFile = null; // Name of the entry file to be transposed.
    String outputFile = null; // Name of the output file.
    int serieNb = 1 ; // Number of columns before the actual values in the input file. Can be columns describing the product as well as empty columns before the values.
    int linesToCopy = 0; // Number of lines composing the header of the file (those lines will be copy/pasted in the output)

    /*
     * Handling the arguments first. 
     */
    try {
        switch (args.length) {
        case 0:
            throw new EmptyArgsException();
        case 1:
            inputFile = args[0];
            String[] parts = inputFile.split("\\.");
            // If no outPutFile name is given, will add "Transposed" to the inputFile Name
            outputFile = parts[0] + "Transposed." + parts[1]; 
            break;
        case 2:
            inputFile = args[0];
            outputFile = args[1];
            break;
        case 3:
            inputFile = args[0];
            outputFile = args[1];
            serieNb = Integer.parseInt(args[2]);
            break;
        case 4:
            inputFile = args[0];
            outputFile = args[1];
            serieNb = Integer.parseInt(args[2]);
            linesToCopy = Integer.parseInt(args[3]);
            break;
        default:
            inputFile = args[0];
            outputFile = args[1];
            serieNb = Integer.parseInt(args[2]);
            linesToCopy = Integer.parseInt(args[3]);
            throw new OutOfBordersArgsException();

        }
    }
    catch (ArgsExceptions a) {
        a.notOk(everythingOk);
    }
    catch (NumberFormatException n) {
        System.out.println("Arguments 3 & 4 should be numbers."
                + " Number 3 is the Number of columns before the actual values in the input file. \n"
                + "(Can be columns describing the product as well as empty columns before the values. (1 by default)) \n"
                + "Number 4 is the number of lines to copy/pasta. (0 by default) \n"
                + "Please try again.");
        everythingOk = false;
    }
    // Creating an InputFile and an OutputFile
    InputFile ex1 = new InputFile(inputFile, linesToCopy); 
    OutputFile ex2 = new OutputFile(outputFile);

    if (everythingOk) {
        try (   FileReader fr = new FileReader(inputFile);
                CSVReader reader = new CSVReader(fr, '\t', '\'', 0);
                FileWriter fw = new FileWriter(outputFile);
                CSVWriter writer = new CSVWriter(fw, '\t', CSVWriter.NO_QUOTE_CHARACTER)) 
        {

            ex1.setReader(reader);
            ex2.setWriter(writer);
            // Reading the header of the file
            ex1.readHead();
            // Writing the header of the file (copy/pasta)
            ex2.write(ex1.getHeadFile());

            // Handling the line containing the columns names
            HeadOfValuesHandler handler = new HeadOfValuesHandler(ex1.readLine(), serieNb);
            ex2.writeLine(handler.createOutputHOV());

            // Each lien will be read and written (in multiple lines) one after the other.
            String[] row;
            CommonLine cl1; 
            // If the period is monthly
            if (handler.isMonthly()) { 

                while (!ex1.isAllDone()) { 

                    row = ex1.readLine();
                    if (!ex1.isAllDone()) {
                        cl1 = new CommonLine(row, handler.getYears(), handler.getMonths(), serieNb);

                        ex2.write(cl1.exportOutputLines());
                    }   
                }
            }
            // If the period is yearly
            else {

                while (!ex1.isAllDone()) { 

                    row = ex1.readLine();
                    if (!ex1.isAllDone()) {
                        cl1 = new CommonLine(row, handler.getYears(), serieNb);

                        ex2.write(cl1.exportOutputLines());     
                    }       
                }
            }       
        }
        catch (FileNotFoundException f) {
            System.out.println(inputFile + " can't be found. Cancelling...");
        }
        catch (IOException e) {
            System.out.println("Unknown exception raised.");
            e.printStackTrace();
        }

    }

}

Thanks for reading this far!

Fitz
  • 327
  • 1
  • 6
  • 19

2 Answers2

2

If it's necessary to keep it as it's, you can package your code as any runnable format, then invoke it through Talend tSystem component or tLibraryLoad component if it's a Jar file.

54l3d
  • 3,913
  • 4
  • 32
  • 58
  • Ah, hadn't thought of that. Would meet the requirement but would also be hard to maintain in my opinion. Also with this approach Talend wouldn't be needed in the first place. – tobi6 Jun 30 '16 at 11:50
  • 2
    In some cases, using existing code gives lower time-to-market, and Talend will be just for monitoring and logging purposes ! – 54l3d Jun 30 '16 at 11:53
  • Interesting approach. So in this case you would use additional log data stored somewhere, maybe with the admin console, and just use the code in a library for example? – tobi6 Jun 30 '16 at 11:56
  • @54l3d And would its user be able to change the args of my Main in a simple way? Meaning, in Talend itself? – Fitz Jun 30 '16 at 11:57
  • @tobi6 My "boss" is using Talend all day long for other purposes and retrieving those .tsv files he needs changed (which my program provides). At least, he would'nt have to retrieve them, exit talend, run my programm and then reopen Talend to put the new edited tsv files in it, right? – Fitz Jun 30 '16 at 11:58
  • Talend natively support logging into console, files, or databaes, there some component for that, see [here](https://www.talendforge.org/tutorials/tutorial.php?idTuto=33&nbrFields=23) – 54l3d Jun 30 '16 at 12:02
  • @54l3d I'll investigate this solution. Would be a blessing if it worked since i'm pretty sure it will take me less time than learning the whole Talend software and rewriting my code. Thank you! – Fitz Jun 30 '16 at 12:10
  • 1
    @Fitz If you mean invoke your code with different args each time, yes that's possible, you will need to prepare your args as global variables then prepare the invoking string, and that's all ! – 54l3d Jun 30 '16 at 12:12
  • 2
    @Fitz In fact, we have similar project, a Talend job that invoke an existing jar through `tSystem` using the command `java -jar myjar arg1 arg` – 54l3d Jun 30 '16 at 12:15
  • @54l3d Yes, this is exactly what I meant. And yes, that's my project too now it seems! I am currently going thru different tutorials to try to do this. I might ask for help in a new question in some time! – Fitz Jun 30 '16 at 12:21
  • @54l3d And I am looking at [this SO question](http://stackoverflow.com/questions/34305483/how-to-use-java-classes-in-talend) you answered to too, using tLibraryLoad to load my jar file. – Fitz Jun 30 '16 at 12:27
  • @54l3d I know about those components, was just wondering if you'd use them in another way than I know of since we use the administration console as well. Thanks for your feedback. – tobi6 Jun 30 '16 at 12:55
1

No, it is not possible to use the code as-is in Talend without changing it.

It is possible to recreate the workflow if well-defined, though.

EDIT

As 54l3d pointed out, it is possible to add external libraries and call them, which seems to be a good solution if only a low amount of additional work can be done. It would still be preferable to use "native" Talend for this if the environment would require it, e.g. if there were a lot of jobs to maintain.

tobi6
  • 8,033
  • 6
  • 26
  • 41