Inconsistent "The syntax of the command is incorrect" error

Question

I guess I'm mad enough to use a batch file, in the first place, but this problem has me tearing my hair out! The below batch script usually works just fine. However, on first run it will give a "The syntax of the command is incorrect" error; and it will do this until I REM out everything, run it through, and then unREM it. Then it magically starts working again!

@echo off

REM Create processed data directory
set "PROCDIR=processed"
mkdir %PROCDIR% 2>nul

REM Loop through CSV files and process them
for %%f in (*.csv) do (
  set "in=%%~nf"
  set "input=%%~nxf"
  set "out_lang=%PROCDIR%\%in%_lang.csv"
  set "out_spr=%PROCDIR%\%in%_spr.csv"
  set "out_ajt=%PROCDIR%\%in%_ajt.csv"

  echo Processing %input%

  REM Split input file appropriately
  REM FIXME Hardcoded line ranges are brittle
  sed -n "1,2p"  %input% > %out_lang%
  sed -n "3,55p" %input% > %out_spr%.tmp
  sed -n "56,$p" %input% > %out_ajt%.tmp

  REM Unpack SPR data
  REM n.b., Needs gawk 4, or later
  gawk -f spr.awk %out_spr%.tmp > %out_spr%
  del %out_spr%.tmp

  REM Substitute AJT data values
  REM n.b., -i seems to be buggy in Windows
  sed -f ajt.sed %out_ajt%.tmp > %out_ajt%
  del %out_ajt%.tmp
)

If I REM out the @echo off, it appears it's failing because the variable substitution sometimes -- for unknown reasons -- doesn't work. Then, when it gets to sed, it just gives up.

I assume I'm missing something... Any ideas what it might be!? (Besides my marbles!)

Nothing funny, just alphanumeric with underscores and dashes (but it even baulks on `test.csv`!). I'll check for the BOM, but I'm pretty sure it's just straight up UTF-8; and I don't see anything in there that would make it indistinguishable from ASCII-7, if that also makes the difference... Thanks for the tip off :) — Xophmeister, Sep 11 '15 at 22:54
if a script behaves different the first time than in the following runs, it's usually a strong hint for the [delayed expansion trap](http://stackoverflow.com/a/30284028/2152082) (yes - I see, that's your Problem) — Stephan, Sep 12 '15 at 06:06

score 2 · Accepted Answer · answered Sep 11 '15 at 23:13

If I could give advice, use and abuse the PAUSE command, ECHO and escape characters ^ in it. Also Ctrl-C to stop the script during a PAUSE.

Here is a debugged version:

@echo off

SETLOCAL
REM Create processed data directory
set "PROCDIR=processed"
echo mkdir %PROCDIR% 2^>nul

REM Loop through CSV files and process them
for %%F in (*.csv) do (
  set "in=%%~nF"
  echo in: %in%
  set "input=%%~nxF"
  echo input: %input%
  set "out_lang=%PROCDIR%\%in%_lang.csv"
  set "out_spr=%PROCDIR%\%in%_spr.csv"
  set "out_ajt=%PROCDIR%\%in%_ajt.csv"

  echo Processing %input%
PAUSE

  REM Split input file appropriately
  REM FIXME Hardcoded line ranges are brittle
  echo sed -n "1,2p"  %input% ^> %out_lang%
  echo sed -n "3,55p" %input% ^> %out_spr%.tmp
  echo sed -n "56,$p" %input% ^> %out_ajt%.tmp
PAUSE

  REM Unpack SPR data
  REM n.b., Needs gawk 4, or later
  echo gawk -f spr.awk %out_spr%.tmp ^> %out_spr%
  echo del %out_spr%.tmp
PAUSE

  REM Substitute AJT data values
  REM n.b., -i seems to be buggy in Windows
  echo sed -f ajt.sed %out_ajt%.tmp ^> %out_ajt%
  echo del %out_ajt%.tmp
PAUSE

)
ENDLOCAL
GOTO :EOF
EXIT /B 0

I added setlocal enabledelayedexpansion and this seems to solve undeclared variable errors.

Here is a working version (you have to remove every redundant echo and carets ^):

@echo off
setlocal enabledelayedexpansion

REM Create processed data directory
set "PROCDIR=processed"
echo mkdir %PROCDIR% 2^>nul

REM Loop through CSV files and process them
for %%F in (*.csv) do (
  set "in=%%~nF"
  echo in: !in!
  set "input=%%~nxF"
  echo input: !input!
  set "out_lang=!PROCDIR!\!in!_lang.csv"
  set "out_spr=!PROCDIR!\!in!_spr.csv"
  set "out_ajt=!PROCDIR!\!in!_ajt.csv"

  echo Processing !input!
PAUSE

  REM Split input file appropriately
  REM FIXME Hardcoded line ranges are brittle
  echo sed -n "1,2p"  !input! ^> !out_lang!
  echo sed -n "3,55p" !input! ^> !out_spr!.tmp
  echo sed -n "56,$p" !input! ^> !out_ajt!.tmp
PAUSE

  REM Unpack SPR data
  REM n.b., Needs gawk 4, or later
  echo gawk -f spr.awk !out_spr!.tmp ^> !out_spr!
  echo del !out_spr!.tmp
PAUSE

  REM Substitute AJT data values
  REM n.b., -i seems to be buggy in Windows
  echo sed -f ajt.sed !out_ajt!.tmp ^> !out_ajt!
  echo del !out_ajt!.tmp
PAUSE

)
endlocal
GOTO :EOF
EXIT /B 0

score 1 · Answer 2 · edited May 23 '17 at 12:29

Stephan is right, batch code is not working because of not using delayed expansion within FOR loop.

@echo off
setlocal EnableDelayedExpansion
REM Create processed data directory
set "PROCDIR=processed"
mkdir "%PROCDIR%" 2>nul

REM Loop through CSV files and process them
for %%f in (*.csv) do (
  set "in=%%~nf"
  set "input=%%~nxf"
  set "out_lang=%PROCDIR%\!in!_lang.csv"
  set "out_spr=%PROCDIR%\!in!_spr.csv"
  set "out_ajt=%PROCDIR%\!in!_ajt.csv"

  echo Processing !input!

  REM Split input file appropriately
  REM FIXME Hardcoded line ranges are brittle
  sed -n "1,2p"  "!input!" >"!out_lang!"
  sed -n "3,55p" "!input!" >"!out_spr!.tmp"
  sed -n "56,$p" "!input!" >"!out_ajt!.tmp"

  REM Unpack SPR data
  REM n.b., Needs gawk 4, or later
  gawk -f spr.awk "!out_spr!.tmp" >"!out_spr!"
  del "!out_spr!.tmp"

  REM Substitute AJT data values
  REM n.b., -i seems to be buggy in Windows
  sed -f ajt.sed "!out_ajt!.tmp" >"!out_ajt!"
  del "!out_ajt!.tmp"
)
endlocal

Every environment variable referenced within a block defined by ( ... ) is expanded by command processor already on parsing the entire block. On one of the help pages output into a command prompt window on execution of if /? explains that in detail.

This could been seen on running the batch code in question without the first line from within a command prompt window. Running a batch file from within a command prompt window without echo off or with echo on should be used on debugging batch code not working as expected.

And in case of any CSV file in current directory contains 1 or more spaces in file name, it is necessary to enclose all file references in double quotes to have a working code also for such files.

Inconsistent "The syntax of the command is incorrect" error

2 Answers2