0

My boss asks me to develop a simple Java application to sort xls file containing a list of keywords. Everything works fine, but for big xls files my app take very, very long time to do her job. I think that my algorithms aren't really optimized but I've seen that the execution of my jar doesn't impact at all the performances of my computer (Windows 7 Pro 32 bits of my boss), and in fact it uses really nothing on the CPU and memoty. So, here's my question, how do I allow my program to take every thing he needs in power and memory to run and sort that xls like a Lamborghini ? Thanks !

EDIT: Here is my code, divides in two block: a main and a frame class:

public class Frame extends JFrame implements ActionListener
{
    /**
     * 
     */
    private static final long serialVersionUID = 1L;
    JTextField input; 
    JTextField output; 
    JTextField result;
    JTextArea gomin;
    JTextArea info;


    Frame()             
    {             
          JFrame jfrm = new JFrame("App CRI"); 
          jfrm.setLocation(100, 100); 
          jfrm.setLayout(new BorderLayout()); 
          jfrm.setSize(400, 170); 
          jfrm.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE); 
          jfrm.setResizable(false);
          input = new JTextField(10); 
          output = new JTextField(10); 
          result = new JTextField(10);
          gomin = new JTextArea();
          result.setEditable(false);
          info = new JTextArea("Merci de renseigner le nom des fichiers sans extension.\nExemple: \"10-09-2014\" pour le fichier \"10-09-2014.xls\".");
          info.setEditable(false);

          // set input fields 
          JPanel inFieldPane = new JPanel(); 
          inFieldPane.setLayout(new GridLayout(2,2)); 
          inFieldPane.add(new JLabel("Input file name:")); 
          inFieldPane.add(input); 
          input.addActionListener(this); 
          inFieldPane.add(new JLabel("Output file name:")); 
          inFieldPane.add(output); 
          output.addActionListener(this); 
          jfrm.add(inFieldPane,BorderLayout.NORTH); 

          // set submit button 
          JPanel submitPane = new JPanel(); 
          submitPane.setLayout(new FlowLayout());
          submitPane.add(info); 
          JButton submitButton = new JButton("Let's go !"); 
          submitButton.addActionListener(this); 
          submitPane.add(submitButton); 
          jfrm.add(submitPane,BorderLayout.CENTER); 

          // display results 
          JPanel outFieldPane= new JPanel();
          outFieldPane.setSize(350, 50);
          outFieldPane.add(gomin);
          gomin.setSize(350, 50);
          gomin.setEditable(false);
          jfrm.add(outFieldPane,BorderLayout.SOUTH);             

          jfrm.setVisible(true); 
    } 

    public void actionPerformed(ActionEvent e) 
    { 
          if(e.getActionCommand().equals("Let's go !")) 
          {  
                gomin.setText(Main.doIt(input.getText().trim(), output.getText().trim())); 
          } 
    } 
}

And here is my main class with all my methods, I haven't separated these methods in other classes even if I have too, and the quality isn't pretty bad I know:

public class Main {

    public static ArrayList<String> loadKeywords(String src)
    {
        ArrayList<String> arr = new ArrayList<String>();
        try
        {
            InputStream inp = new FileInputStream(src);
            Workbook wb = WorkbookFactory.create(inp);
            Sheet sheet = wb.getSheetAt(0);
            Row r;

            for(int i = 1; i <= sheet.getLastRowNum(); i++)
            {
                r = sheet.getRow(i);
                if(r.getCell(0) != null && r.getCell(0).getCellType() == Cell.CELL_TYPE_STRING)
                    arr.add(r.getCell(0).getRichStringCellValue().getString());
            }
        } catch (FileNotFoundException e)
        {
            System.out.println("Keywords file undetected !");
            e.printStackTrace();
        } catch (InvalidFormatException e)
        {
            System.out.println("Invalid format. Oups.");
            e.printStackTrace();
        } catch (IOException e)
        {
            System.out.println("Fuck ! IOException !");
            e.printStackTrace();
        }

        return arr;
    }

    public static boolean isValidCri(String inputString, ArrayList<String> arr)
    {
        for(int i =0; i < arr.size(); i++)
        {
            if(Pattern.compile(Pattern.quote(arr.get(i)), Pattern.CASE_INSENSITIVE).matcher(inputString).find())
            {
                return true;
            }
        }
        return false;
    }

    public static void process()
    {
        System.out.println("Process en cours ... ");

        ArrayList<String> arr = loadKeywords("C:\\Users\\9108250x\\Desktop\\app_cri\\keywords.xls");

        if(arr.isEmpty())
            System.out.println("rat�, erreur au chargement des keywords :(");
        else
        {
            try
            {
                InputStream inp = new FileInputStream("C:\\Users\\9108250x\\Desktop\\app_cri\\01-09-2014.xls");

                Workbook wb = WorkbookFactory.create(inp);

                Sheet itaSheet = wb.getSheetAt(0);

                Row r;
                int endy = itaSheet.getLastRowNum();
                for(int i = 1; i <= endy; i++)
                {
                    r = itaSheet.getRow(i);
                    if(r.getCell(27) != null && r.getCell(27).getCellType() == Cell.CELL_TYPE_STRING)
                    {
                        if(! isValidCri(r.getCell(27).getRichStringCellValue().getString(), arr))
                        {
                            itaSheet.shiftRows(i+1, itaSheet.getLastRowNum()+1, -1);
                            i--;
                            endy--;
                        }
                    }
                    else
                    {
                        itaSheet.shiftRows(i+1, itaSheet.getLastRowNum()+1, -1);
                        i--;
                        endy--;
                    }

                }

                FileOutputStream fileOut = new FileOutputStream("C:\\Users\\9108250x\\Desktop\\app_cri\\outPut.xls");
                wb.write(fileOut);
                fileOut.close();
            }
            catch (FileNotFoundException e)
            {
                System.out.println("File not found, bad url.");
                e.printStackTrace();
            }
            catch (InvalidFormatException e)
            {
                System.out.println("Invalid file format.");
                e.printStackTrace();
            }
            catch (IOException e)
            {
                System.out.println("Fuck, doesn't work !");
                e.printStackTrace();
            }
            System.out.println("et termin� avec succ�s !");
        }
    }

    public static String doIt(String input, String output)
    {
        String result = "succeed !";

        ArrayList<String> arr = loadKeywords(".\\keywords.xls");

        if(arr.isEmpty())
            result = "no keywords";
        else
        {
            try
            {
                InputStream inp = new FileInputStream(".\\"+ input +".xls");

                Workbook wb = WorkbookFactory.create(inp);

                Sheet itaSheet = wb.getSheetAt(0);

                Row r;
                int endy = itaSheet.getLastRowNum();

                for(int i = 1; i <= endy; i++)
                {
                    r = itaSheet.getRow(i);
                    if(r.getCell(27) != null && r.getCell(27).getCellType() == Cell.CELL_TYPE_STRING)
                    {
                        if(! isValidCri(r.getCell(27).getRichStringCellValue().getString(), arr))
                        {
                            itaSheet.shiftRows(i+1, itaSheet.getLastRowNum()+1, -1);
                            i--;
                            endy--;
                        }
                    }
                    else
                    {
                        itaSheet.shiftRows(i+1, itaSheet.getLastRowNum()+1, -1);
                        i--;
                        endy--;
                    }
                }

                FileOutputStream fileOut = new FileOutputStream(".\\"+ output +".xls");
                wb.write(fileOut);
                fileOut.close();
            }
            catch (FileNotFoundException e)
            {
                System.out.println("File not found, bad url.");
                result = "file not found";
                e.printStackTrace();
            }
            catch (InvalidFormatException e)
            {
                System.out.println("Invalid file format.");
                result = "invalid operation";
                e.printStackTrace();
            }
            catch (IOException e)
            {
                System.out.println("Doesn't work !");
                result = "IO exception";
                e.printStackTrace();
            }
            catch ( Exception e) 
            {
                result = "unknow exception";
            }
            System.out.println("et termin� avec succ�s !");
        }
        return result;
    }
    /**
     * @param args
     */
    public static void main(String[] args)
    {
        SwingUtilities.invokeLater(new Runnable() 
        { 
              public void run() 
              { 
                    new Frame(); 
              } 
        }); 
//      process();
    }

}
Aurel
  • 13
  • 4
  • Please post your code. – Kishore Oct 23 '14 at 10:10
  • 2
    If your code does not use a lot of CPU it is very unlikely that memory if the problem. Too little memory results in lots of garbage collection runs, which uses lots of CPU. – Thomas Stets Oct 23 '14 at 10:19
  • 2
    Why do you think lack of memory is your problem? If the performance decreases with larger inputs, my first thought is that your *algorithm* is inefficient... – Andrzej Doyle Oct 23 '14 at 10:22
  • Take a look at [this](http://stackoverflow.com/questions/2294268/how-can-i-increase-the-jvm-memory) – Pham Trung Oct 23 '14 at 10:29
  • Are you really using your own sort algorithm? Java provides pretty good quicksort out of the box - why is that not suitable? – BarrySW19 Oct 23 '14 at 10:45
  • I've added my code, I'm using a library to handle xls file and modify his content. Im sure this isn't the best way to do it, but can't more memory improve the speed of execution of these double for ? Don't have enough time to build a serious powerfull app. – Aurel Oct 23 '14 at 10:50

2 Answers2

2

Your algorithm is probably very inefficient with memory, but if you need to allocate more memory then use the folowing arguments:

-Xmx2G - This sets the maximal memory usage to 2GB. Change the 2G to whatever you want, 512M for 512MB and so on

-Xms1G - This sets the amount of ram the application will start with (Minimum). You can change this the same way as -Xmx.

Again, it is probably your algorithm but you could try changing the memory allocation.

EDIT: Just saw your code, it probably isn't your own algorithm, but the way WorkBook works. I would suggest increasing the allocated memory in this case.

berryh
  • 36
  • 4
  • I've tried it, and you're right I'm totally wrong in my demand. I think this come definitely from my algorithm and not from the memory allocated to the execution. But thank you for your help. – Aurel Oct 23 '14 at 11:04
0

Maybe you should take a look at this blog entry: Java – Reading a Large File Efficiently

Thomas Schmidt
  • 1,248
  • 2
  • 12
  • 37
  • You're right, maybe my problem comes from the reading time. Thanks for the link I'll read it. – Aurel Oct 23 '14 at 10:51