1

I am trying to speed up the process of writing data to an excel file using python in my code. The PyExcelerate module has a good benchmark in terms of writing bulk data to files compared to other python modules.

  • Dimensions: Rows = 10000 Cols = 50

  • Times:

    • pyexcelerate : 10.11
    • xlwt : 15.67
    • xlsxwriter (optimised): 19.70
    • xlsxwriter : 23.50
    • openpyxl (optimised): 95.82
    • openpyxl : 95.90

Also, writing data in batches improves the time taken to write furthermore. See link below.

Speed up writing to files

Now, what I am trying to do is add data to a list as and when it is computed, and when the list size is equal to 500 or when 500 rows of data have been computed, write to the file.

wb = Workbook() ws = wb.new_sheet("test") ws.range("B2", "C3").value = [[1, 2], [3, 4]] wb.save("output.xlsx")

Is there a way that we could append the data for 500 rows in a batch?

Community
  • 1
  • 1
Vaulstein
  • 20,055
  • 8
  • 52
  • 73
  • 2
    If you install lxml then you'll get better results with openpyxl. As long as you're working with purely numerical data, PyExcelerate is probably the fastest module. Adding comfort usually comes with a performance cost but you might want to look at the `write_row()` method of xlsxwriter. – Charlie Clark Apr 10 '15 at 13:32
  • Thanks this is almost the solution that I required. I now just want to write the data in Batches of 500 using the `write_row()` – Vaulstein Apr 10 '15 at 14:25

1 Answers1

1

Use PyExcelerate its pretty fast then write_row from xlsxwriter and append from openpyxl .Refer https://pypi.python.org/pypi/PyExcelerate

Saurabh
  • 7,525
  • 4
  • 45
  • 46