1

I am trying to understand the C++ standard preprocessor requirement. A little bit tricky example I created has surprising results in GCC and VC++2010:

#define a(x,y) x##y
#define tzsW kka
a(t,zs  )W

GCC yields:

tzs W

Note the extra space added before W.

VC++2010 yields:

tzsW

Note there is no space added before W but the identifier is not further expanded. I scanned through the C++03 standard and cannot find anything saying we should prevent a new identifier (tzsW) from being created as in gcc. And nothing is about preventing this new identifier from further macro expanded (VC++ behaviour).

Why GCC and VC++2010 don't like the new identifier?

EDIT

if another macro invocation used, e.g.

a(t,zs  )[]

gcc yields:

tzs[]

Note no space added, showing gcc deliberately adds space to my previous case.

Brian Tompsett - 汤莱恩
  • 5,753
  • 72
  • 57
  • 129
JavaMan
  • 4,954
  • 4
  • 41
  • 69

2 Answers2

4

The output of the preprocessor is tokens, not plain text. Unless you use the token pasting operator, tokens are not normally combined in preprocessing.

When you view the output of the preprocessing step, you have to convert the tokens to text. gcc inserts the space so you are not mislead into thinking that tzsW is a single token. It doesn't need to do this in the tzs[ case as [ isn't a valid identifier character so there's no confusion.

Both compilers are correct in not treating tzsW as a single token to be re-expanded.

Note that the Visual C++ documentation acknowledges that compiling the output of the preprocessor may produce different and incorrect output compared with compiling the original source due to the fact that they don't insert whitespace to delimit tokens when converting the preprocessed output to text. This doesn't happen when the preprocessor output is passed straight to the next stage of the compiler in normal operation.

CB Bailey
  • 755,051
  • 104
  • 632
  • 656
  • I think that probably VS2010 is wrong in combining the `tzs` and `W` into a single identifier and GCC is correct in separating them. – Jonathan Leffler Apr 02 '14 at 10:50
  • @JonathanLeffler: But it's not combining them into a single identifier as indicated by the face the `tzsW` is not being replaced. It's just the lossy translation from tokens to text that's making them look like a single identifier if you siphon off the output from the middle of the compile. – CB Bailey Apr 02 '14 at 10:55
  • OK; it is sloppy at producing output that is comprehensible to humans. – Jonathan Leffler Apr 02 '14 at 11:08
1

  1. Space or no space?

    I did not found a reference in the standard, but this article discusses the issue. In short, it is not easy to decide if you want that space or not...

  2. Replacing tzsW with kka or not?

    See 16.3.4 in the standard:

    After all parameters in the replacement list have been substituted, the resulting preprocessing token sequence is rescanned with all subsequent preprocessing tokens of the source file for more macro names to replace.

    This indicates that the behaviour of VC is not correct here.

Danvil
  • 22,240
  • 19
  • 65
  • 88
  • The standard indicates that tzsW should be replaced with kka, which VC does not do. – Danvil Apr 02 '14 at 09:45
  • 1
    Um, only if it's a single token. In the example it isn't because `tzs` comes from an expansion of `a(t,zs )` and `W` is just `W`. Both compilers behave the same (they don't produce `kka`, just `tzs` and `W`) so I don't see how you can call out one as not correct and not the other. – CB Bailey Apr 02 '14 at 09:59
  • This is a bit ambiguous. GCC does not produce `tzsW` so it will of course not consider it for replacement. If you write for example `a(tzs,W)` GCC will produce `kka`. So, are you saying that `a(tzs,W)` and `a(tz,s)W` are different things (event both result in `tzsW` for VC) and VC would replace the first with `kka`, but not the second? – Danvil Apr 02 '14 at 10:22
  • 1
    Yes, `a(tz,s)W` and `a(tzs,W)` expand to different token sequences and they each expand to the same token sequence on both platforms. It's only writing the token sequence back to text which introduces the apparent ambiguity. – CB Bailey Apr 02 '14 at 10:59