1

The following program takes in an excel file with product IDs (beginning with MT or MU) in the 1st column. It then goes through purchase orders (POs) that contain those product IDs in folders by supplier which have sub folders by year. The user enters the folder path and specifies the years it wants to search. The program then returns the most recent supplier, price and date in columns next to the product ID in the original spreadsheet specified by the user. The program takes about 3 to 4 hours to run for 10,000 product IDs and a years worth of POs (hundreds of POs). Sometimes it finishes (there's a message box that shows done when finished), and sometimes it doesn't given the same exact inputs. There are no errors, but CPU usage suddenly goes from about 40% to 1% Anyone know why?

    string filepath = "";
    string folderpath = "";

    private void button1_Click(object sender, EventArgs e)
    {
        //New Excel App
        Excel._Application oApp = new Excel.Application();
        oApp.Visible = true;


        //Opens Worksheet to be updated
        Excel.Workbook oWorkbook = oApp.Workbooks.Open(filepath);
        Excel.Worksheet oWorksheet = oWorkbook.Worksheets["Sheet1"];

        //Takes all MT/MU numbers from first column of worksheet needing updating and puts them into a string
        Excel.Range firstColumn = oWorksheet.UsedRange.Columns[1];
        System.Array myvalues = (System.Array)firstColumn.Cells.Value;
        string[] strArray = myvalues.OfType<object>().Select(o => o.ToString()).ToArray();

        //Specify what year to begin and end looking at POs
        int beginYear = Convert.ToInt32(textBox2.Text);
        beginYear = int.Parse(textBox2.Text);
        int endYear = Convert.ToInt32(textBox3.Text);
        endYear = int.Parse(textBox3.Text);
        int count = 0;
        List<string> yearList = new List<string>();
        while (count <= endYear-beginYear )
        {
            int addYear = beginYear + count;
            string addYearString = addYear.ToString();
            yearList.Add(addYearString);
            count++;

        }
        string[] years = yearList.ToArray();


       foreach (string year in years)
        {
            //Creates array of all excel files existing in path (including subdirectories) in folder chosen by user
            IEnumerable<string> files =
            from f in Directory.GetFiles(folderpath, "*.xls", SearchOption.AllDirectories)
            where Path.GetDirectoryName(f).Contains(year)
            select f;

            foreach (string file in files)
            {
                //Opens file
                Excel._Application oApp2 = new Excel.Application();
                oApp2.Visible = false;
                Excel.Workbook PO = oApp2.Workbooks.Open(file);
                Excel.Worksheet oWorksheet2 = PO.Worksheets["Sheet1"];

                foreach (string item in strArray)
                {
                    Excel.Range currentFind = null;
                    string newestDateSoFar = "01/01/1900";
                    DateTime newestDateTimeSoFar = Convert.ToDateTime(newestDateSoFar);

                    object misValue = System.Reflection.Missing.Value;

                    Excel.Range xlRange = oWorksheet2.get_Range("C1");

                    //Looks through column for MT/MU number
                    currentFind = xlRange.EntireColumn.Find(item,
                    misValue, Excel.XlFindLookIn.xlValues, Excel.XlLookAt.xlPart,
                    Excel.XlSearchOrder.xlByColumns, Excel.XlSearchDirection.xlNext,
                    true, misValue, misValue);

                    //If there is a match in the PO
                    if (currentFind != null)
                    {

                        //Get Date on PO
                        string currentDate = null;
                        DateTime currentDateTime = Convert.ToDateTime(currentDate);
                        if (oWorksheet2.Cells[6, 6].Value != null)
                        {
                            currentDate = oWorksheet2.Cells[6, 6].Value.ToString();
                            currentDateTime = Convert.ToDateTime(currentDate);
                        }

                        Excel.Range xlRange2 = oWorksheet.get_Range("A1");

                        //Looks through column of worksheet needing updating for MT/MU number, so can compare date
                        Excel.Range needsUpdatingFind = xlRange2.EntireColumn.Find(item,
                        misValue, Excel.XlFindLookIn.xlValues, Excel.XlLookAt.xlPart,
                        Excel.XlSearchOrder.xlByColumns, Excel.XlSearchDirection.xlNext,
                        true, misValue, misValue);

                        int dateColumn = Convert.ToInt32(textBox4.Text);
                        dateColumn = int.Parse(textBox4.Text);
                        if (needsUpdatingFind.get_Offset(0, dateColumn-1).Value != null)
                        {
                            newestDateSoFar = needsUpdatingFind.Offset[0, dateColumn-1].Value.ToString();
                            newestDateTimeSoFar = Convert.ToDateTime(newestDateSoFar);
                        }



                        //If Date on PO is most recent, get information that needs updating         
                        if (currentDateTime > newestDateTimeSoFar || needsUpdatingFind.get_Offset(0, dateColumn - 1).Value == null)
                        {

                            //Gets assosciated price and sets it in worksheet that needs updating
                            string price = null;
                            int priceColumn = Convert.ToInt32(textBox6.Text);
                            priceColumn = int.Parse(textBox6.Text);
                            if (currentFind.get_Offset(0, 2).Value != null)
                            {
                                price = currentFind.get_Offset(0, 2).Value.ToString();
                                needsUpdatingFind.Offset[0, priceColumn-1].Value = price;
                            }
                            else if (currentFind.get_Offset(-1, 2).Value != null)
                            {
                                price = currentFind.get_Offset(-1, 2).Value.ToString();
                                needsUpdatingFind.Offset[0, priceColumn-1].Value = price;
                            }
                            else if (currentFind.get_Offset(-2, 2).Value != null)
                            {
                                price = currentFind.get_Offset(-2, 2).Value.ToString();
                                needsUpdatingFind.Offset[0, priceColumn-1].Value = price;
                            }
                            else if (currentFind.get_Offset(-3, 2).Value != null)
                            {
                                price = currentFind.get_Offset(-3, 2).Value.ToString();
                                needsUpdatingFind.Offset[0, priceColumn-1].Value = price;
                            }
                            else if (currentFind.get_Offset(-4, 2).Value != null)
                            {
                                price = currentFind.get_Offset(-4, 2).Value.ToString();
                                needsUpdatingFind.Offset[0, priceColumn-1].Value = price;
                            }
                            else if (currentFind.get_Offset(-5, 2).Value != null)
                            {
                                price = currentFind.get_Offset(-5, 2).Value.ToString();
                                needsUpdatingFind.Offset[0, priceColumn-1].Value = price;
                            }

                            //Gets assosciated supplier and sets it in worksheet that needs updating
                            int supplierColumn = Convert.ToInt32(textBox5.Text);
                            supplierColumn = int.Parse(textBox5.Text);
                            string supplier = oWorksheet2.Cells[6, 2].Value.ToString();
                            needsUpdatingFind.Offset[0, supplierColumn-1].Value = supplier;

                            //Gets assosciated PO date and sets it in worksheet that needs updating

                            needsUpdatingFind.Offset[0, dateColumn-1].Value = currentDateTime.ToString();

                            //Resets current find
                            currentFind = null;
                            needsUpdatingFind = null;

                        }
                        while (Marshal.ReleaseComObject(xlRange2) != 0) { }


                        xlRange2 = null;


                        GC.Collect();
                        GC.WaitForPendingFinalizers();
                    }


                    while (Marshal.ReleaseComObject(xlRange) != 0) { }


                    xlRange = null;

                    GC.Collect();
                    GC.WaitForPendingFinalizers();
                }
                //Closes PO
                object missing = System.Reflection.Missing.Value;
                PO.Close(false, missing, missing);

                //Quits Excel
                oApp2.Quit();

                //Manual disposal because of COM
                while (Marshal.ReleaseComObject(oApp2) != 0) { }
                while (Marshal.ReleaseComObject(PO) != 0) { }
                while (Marshal.ReleaseComObject(oWorksheet2) != 0) { }


                oApp2 = null;
                PO = null;
                oWorksheet2 = null;


                GC.Collect();
                GC.WaitForPendingFinalizers();
            }

        }



        MessageBox.Show("Done");
    }

    private void button2_Click(object sender, EventArgs e)
    {

        OpenFileDialog openFileDialog1 = new OpenFileDialog();
        if (openFileDialog1.ShowDialog() == DialogResult.OK)
        {
            filepath = openFileDialog1.FileName;
            textBox1.Text = filepath;
        }
    }

    private void button3_Click(object sender, EventArgs e)
    {
        FolderBrowserDialog openFileDialog2 = new FolderBrowserDialog();
        if (openFileDialog2.ShowDialog() == DialogResult.OK)
        {
            folderpath = openFileDialog2.SelectedPath;
            textBox7.Text = folderpath;
        }
    }
}
}
SierraOscar
  • 17,507
  • 6
  • 40
  • 68
102263
  • 61
  • 7

1 Answers1

0

You should use a DataSet and OleDbDataAdapter to put the data into memory, then let C#/Linq do the work. It doesn't sound like you're talking about much data, so it would seem Interop is the root cause of the slowness. Here's an example/comparison: Open Excel, Parse Data? . In that post, note the comparison between ADO.Net and Interop. The chosen answer is much faster than the alternative, working answer. It seems you have encountered the same issue.

Community
  • 1
  • 1
jacoblambert
  • 787
  • 1
  • 11
  • 18
  • I would agree your method would make the program more efficient, and I would use that method if I was more familiar with it and had more time. But do you think the definitive reason it runs to completion sometimes but not others given the same exact input is because it runs slowly? I don't understand why that would be. The slowness is not a big problem for me as long as it does consistently finish. I have also run 100 product ID numbers against one year of POs and it finishes in about 10 minutes sometimes, and running at the same speed fails to finish other times. – 102263 Aug 21 '15 at 12:52
  • I'm guessing so - I'm guessing it uses a lot of memory and then crashes. – jacoblambert Aug 21 '15 at 12:53
  • you should copy and paste the code i chose as the answer - then just put replace your file location and MRN with product IDs in your where clause. I'm projecting a new object there - you can do that by replacing whatever columns you want in your spreadsheet and adding new lines as required. If you do that, you should see how much faster it is to open up and get started - and then you don't need excel after that. I think if you do you'll see it's not that hard and it's worth converting. – jacoblambert Aug 21 '15 at 13:01
  • I just tinker with this stuff, so correct me if I'm wrong..but looking at the Windows Task Manager, the Physical Memory never goes above 50% and I think I've corrected for memory. I think memory was a problem but once I closed and quit each Excel PO and did the ReleaseComObject stuff the memory issue improved as evidenced by the Windows Task Manager...does that make sense? – 102263 Aug 21 '15 at 13:08
  • And if memory was the issue, given the same exact task wouldn't it crash every time? – 102263 Aug 21 '15 at 13:09
  • just saw your last comment, I'll try it and let you know...thanks – 102263 Aug 21 '15 at 13:10
  • Interop objects are unmanaged - a cursory look shows you are disposing of them in at least some cases, but perhaps not all. Check this out: http://stackoverflow.com/questions/158706/how-to-properly-clean-up-excel-interop-objects – jacoblambert Aug 21 '15 at 13:53