Expand macros & retrieve macro values

Question

I am trying to use libclang python bindings to parse my c++ source files. I am unable to get the value of macro or expand a macro.
Here is my sample c++ code

#define FOO 6001
#define EXPAND_MACR \
        int \
        foo = 61
int main()
{
    EXPAND_MACR;
    cout <<  foo;
    return 0;
}

This is my python script

import sys
import clang.cindex

def visit(node):
    if node.kind in (clang.cindex.CursorKind.MACRO_INSTANTIATION,   clang.cindex.CursorKind.MACRO_DEFINITION):           
    print 'Found %s Type %s DATA %s Extent %s [line=%s, col=%s]' % (node.displayname, node.kind, node.data, node.extent, node.location.line, node.location.column)
for c in node.get_children():
    visit(c)

if __name__ == '__main__':
    index = clang.cindex.Index.create()
    tu = index.parse(sys.argv[1], options=clang.cindex.TranslationUnit.PARSE_DETAILED_PROCESSING_RECORD)
    print 'Translation unit:', tu.spelling
    visit(tu.cursor)

This is info I get back from clang:

Found FOO Type CursorKind.MACRO_DEFINITION DATA <clang.cindex.c_void_p_Array_3 object at 0x10b86d950> Extent <SourceRange start <SourceLocation file 'sample.cpp', line 4, column 9>, end <SourceLocation file 'sample.cpp', line 4, column 17>> [line=4, col=9]
Found EXPAND_MACR Type CursorKind.MACRO_DEFINITION DATA <clang.cindex.c_void_p_Array_3 object at 0x10b86d950> Extent <SourceRange start <SourceLocation file 'sample.cpp', line 6, column 9>, end <SourceLocation file 'sample.cpp', line 8, column 11>> [line=6, col=9]
Found EXPAND_MACR Type CursorKind.MACRO_INSTANTIATION DATA <clang.cindex.c_void_p_Array_3 object at 0x10b86d950> Extent <SourceRange start <SourceLocation file 'sample.cpp', line 12, column 2>, end <SourceLocation file 'sample.cpp', line 12, column 13>> [line=12, col=2]

If you observe my python script, node.data gives out

DATA <clang.cindex.c_void_p_Array_3 object at 0x10b86d950>

I can read Extent data returned by clang & then parse the file from start to end positions to get the value. I want to know if, there exists a better way to get macro values?
I want to get the value of macro(6001 in the sample) directly(without using Extent). How can I get that?
Additionally for EXPAND_MACR is want to get int foo = 61

I have already seen these post: Link-1 & Link-2.
Any help will be highly appreciated

score 5 · Accepted Answer · answered Jul 19 '16 at 09:41

No, line-by-line expansion using the extents seems to be the only way to extract the (expanded macros).

I suspect that the problem is that by the time libclang sees your code, the macros have already been removed by the preprocessor - the nodes you're seeing in the AST are more like annotations rather than true nodes.

#define FOO 6001
#define EXPAND_MACR \
        int \
        foo = 61
int main()
{
    EXPAND_MACR;
    return 0;
}

Is really the AST

TRANSLATION_UNIT sample.cpp
  +-- ...  *some nodes removed for brevity* 
  +--MACRO_DEFINITION FOO
  +--MACRO_DEFINITION EXPAND_MACR
  +--MACRO_INSTANTIATION EXPAND_MACR
  +--FUNCTION_DECL main
     +--COMPOUND_STMT 
        +--DECL_STMT 
        |  +--VAR_DECL foo
        |     +--INTEGER_LITERAL 
        +--RETURN_STMT 
           +--INTEGER_LITERAL

This is equivalent to running the preprocessor only (and telling it to give you a listing of all the preprocessor directives). You can see something similar by running:

clang -E -Wp,-dD src.cc

Which gives:

# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "src.cc" 2
#define FOO 6001
#define EXPAND_MACR int foo = 61


int main()
{
    int foo = 61;
    return 0;
}

score 0 · Answer 2 · answered Jul 21 '20 at 17:45

A workaround is to create another file with all macros

Example.hpp

#include "header_with_macro_ABC.h"

auto abc = ABC:

You can then parse this header file again to extract the variable declarations, and using the evaluate methods you can get the value.

Pros: No need to write your own expression parser. Cons: Need to create and parse another file.

This method may not be viable in many cases but for me it was when I had to parse all macros for ffigen (bindings generator for Dart).

Expand macros & retrieve macro values

2 Answers2