1

Im trying to import the csv to google big query , but in the schema it does not allow header names with space. And my csv has several headers with spaces in between. The problem is, the csv file is 50gb. So I cannot open it in memory , rename and write it. Would be much obliged if I could know if there is a more convenient way to accomplish this in pandas.

Nirojan Selvanathan
  • 10,066
  • 5
  • 61
  • 82
  • and, [How can I strip the whitespace from Pandas DataFrame headers?](https://stackoverflow.com/questions/21606987/how-can-i-strip-the-whitespace-from-pandas-dataframe-headers) – Zero Sep 28 '17 at 08:18
  • 1
    The question marked as duplicate loads the whole file in memory. My problem is the file is of 50GB so cannot go for that approach. – Nirojan Selvanathan Sep 28 '17 at 08:22
  • Why not use bash for header cleaning? – Zero Sep 28 '17 at 08:30
  • Thanks, was able to do it with sed. https://stackoverflow.com/questions/13438095/replace-1st-line-in-a-text-file-by-a-string-shell-scripting – Nirojan Selvanathan Sep 28 '17 at 09:29
  • 1
    This is a great example of what people mean when they say that questions are closed prematurely on Stack Overflow. – NicholasM Jan 06 '20 at 00:12

0 Answers0