1

I was trying to use tokens in a for loop to separate a CSV file in my batch file. Here's the code snippet:

for /f "tokens=1-31* delims=," %%a in ("!revamped!") do (
  echo %%a
  echo %%b
  ...
  echo %%y
  echo %%z
      rem what parameter should I use next?
)

After using %%a all the way to %%z, I have no idea which parameter should I use next.

Question: What is the order of using parameters after using all %%a to %%z in a for loop?

  • 3
    [here](https://ss64.com/nt/for_cmd.html) is your answer. You can use 31 tokens maximum (or use another `for`) – Stephan Mar 31 '17 at 11:37
  • 2
    `*` is a token (in this case %%a) after which usage no other tokens can be used. After `z` comes `{` - check the ascii table. Though probably will be better to start with `%A` as upper case letters have smaller numbers in the ascii table. Mind that you are limited to `31` tokens. For more you'll need nested loop. – npocmaka Mar 31 '17 at 11:39
  • 2
    [I found a good example](http://stackoverflow.com/a/8520993/2152082) of nested `for` to use more than 31 tokens. (provided by dbenham) – Stephan Mar 31 '17 at 11:43
  • @Stephan I know the 31 tokens limit, but I think `%%a` to `%%z` is only 26 tokens –  Mar 31 '17 at 11:43
  • 2
    @SteveFest see the two links, I gave you. Copying it for your convenience: `? @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ \ ]` – Stephan Mar 31 '17 at 11:47
  • 2
    If you're evaluating CSV data, you could use just a plain `for` loop without any `/F` switch. `for %%a in (!revamped!) do echo %%~a`. `for` will split a string on unquoted commas, semicolons, spaces, and tabs, and there's no 31 token limit. – rojo Mar 31 '17 at 11:50
  • 2
    @SteveFest after z are `{` , `|` (which needs escaping) and `}` then is `` which cannot be used as token and is the last character in the not-extended ascii chars table. – npocmaka Mar 31 '17 at 11:50
  • @Stephan You need to make the "`for %%a` (CSV)" method as an answer as it worked like a charm for me! –  Mar 31 '17 at 11:55
  • 4
    We have a brand new thread over on Dostips.com that discusses this more in depth providing new options and concepts. http://www.dostips.com/forum/viewtopic.php?f=3&t=7703 – Squashman Mar 31 '17 at 12:52
  • 3
    @SteveFest: Note that your example does _not_ works as assumed. The `tokens=*` option specify _all the tokens_ returned in a single token-letter, so the `delims=` part don't matters in this case. Try: `for /f "tokens=* delims=," %%a in ("One,Two,Three") do echo a=%%a, b=%%b`; the output is: `a=One,Two,Three, b=%b`. To use _several tokens_ you always must specify _how many_ (maximum 31+1). For example: `for /f "tokens=1-31* delims=," %%? in ...` – Aacini Mar 31 '17 at 13:19

1 Answers1

2

As others already pointed out in comments, for /F is limited to 31 tokens. There are the following work-arounds:

  1. Use nested for /F loops, like this (50 tokens are used here):

    for /F "tokens=1-25* delims=," %%A in ("!revamped!") do (
        for /F "tokens=1-25* delims=," %%a in ("%%Z") do (
            echo token # 01: "%%A"
            echo token # 25: "%%Y"
            echo token # 26: "%%a"
            echo token # 50: "%%y"
            echo remaining tokens: "%%z"
        )
    )
    
  2. Use a standard for loop loop to gain a practically unlimited number of tokens:

    for /F "tokens=*" %%A in ("!revamped!") do (
        set /A "INDEX=0" & rem // (reset index number)
        set "LINE=%%A"
        rem /* In the following, every delimiter `,` is replaced by `" "`;
        rem    the entire line string is enclosed within `""`; hence
        rem    the result is a list of space-sepatated quoted tokens,
        rem    which can easily be handled by a standard `for` loop;
        rem    the surrounding quotes are later removed by the `~` modifier;
        rem    regard that the line string must not contain `*` or `?`: */
        for %%a in ("!LINE:,=" "!") do (
            set /A "INDEX+=1" & rem // (build index just for demonstration)
            echo token # !INDEX!: "%%~a"
        )
    )
    
aschipfl
  • 33,626
  • 12
  • 54
  • 99