1

I am attempting to write a batch file which will append a series of multiple CSV files, but only carry over the first file’s header. That part works fine. I’m also trying to replace the spaces in the header line with underscores, but can’t make that work. Any suggestions?

SET Outputfolder=c:\Test

REM Merge all CSVs containing statistics to one and remove spaces in the header
SET first=1
>%OUTPUTfolder%\all_stats_merged.csv (
FOR %%I in (%OUTPUTfolder%\*_stats.csv)  DO (
IF defined first (
SET HeaderString0=%%I
SET HeaderString=%HeaderString0: =_%
TYPE "%HeaderString%" 
SET "first="
 ) else more +1 "%%I"
 )
 )
Jae
  • 143
  • 5
  • 2
    You need to enable delayed expansion. – Compo Mar 20 '18 at 21:40
  • 2
    Or just completely avoid using multi-line code blocks. `do call :Something` instead of `do (multiple lines here)`. – jwdonahue Mar 20 '18 at 21:46
  • How would I avoid the multi line code block? Don’t I need an outer “for” loop and the inner “if” statement to keep the header of only the first dataset? If there is a way to streamline the code, I’m certainly glad to explore it. – Jae Mar 20 '18 at 21:50
  • Or are you referring to the “two HeaderString” lines? – Jae Mar 20 '18 at 21:58
  • 2
    You could start by putting the header string into `HeaderString` instead of putting the filename in there. – LinuxDisciple Mar 20 '18 at 22:34
  • 1
    Your code will fail if the file has more than 65,536 lines. The more command has a limit. – Squashman Mar 21 '18 at 01:57

1 Answers1

1

First off, you're setting HeaderString to the name of the file, not the header string. It's difficult to trivially change your approach to get you what you want here.
Also you're trying to use the values of HeaderString and HeaderString0 in the IF-block where they're set. That's not going to work without delayed expansion, but that doesn't mean that you need delayed expansion; it just means you need to exit the IF block before you can use the values of those variables.
You can set values in an IF-block, then close the IF block, and then rely on them in a second IF-block, which gives you a cookie-cutter pattern for solving that issue.
That's still not going to help you here though because the approach is fundamentally flawed (no easy way to modify just the first line in this context).

A simpler approach would be to:

  1. get the header line
  2. do your underscores-for-spaces replacement of the header line
  3. add the modified header line to your output file
  4. add all the other content to your output file

With caveats mentioned in the link, this trick will efficiently get the first line (the header line) of one of the csv files into a variable.
Then it's trivial to do our spaces-for-underscores replacement.
Then just append the contents of the rest of the files.

@ECHO OFF
set Outputfolder=c:\Test

REM Get the header string out of one of the files
for %%I in (%outputFolder%\*_stats.csv)  do set /p HeaderString=< %%I

REM replace the spaces in that header string with underscores
SET HeaderString=%HeaderString: =_%

REM write that header as the first line of the output file
echo.%HeaderString%>%outputFolder%\all_stats_merged.csv

REM append the non-header lines from all the files
>>%outputFolder%\all_stats_merged.csv (
  for %%I in (%outputFolder%\*_stats.csv)  do more +1 "%%I"
)

I tested this on Windows 7 and 8.1

LinuxDisciple
  • 2,289
  • 16
  • 19
  • [Just one caveat with the MORE command](https://www.dostips.com/forum/viewtopic.php?t=2961#p13741) – Squashman Mar 21 '18 at 02:10
  • This solution worked, but revealed a new problem. My header string is long and gets truncated at 1024 characters. Keeping with the protocol, I’ll post a new question on the new topic. – Jae Mar 21 '18 at 12:22