How to use Clang LibTooling multiple times

Question

Minimul source that use Clang LibTooling which is a very common way:

#include "pch.h"

#include "clang/Frontend/FrontendActions.h"
#include "clang/Tooling/CommonOptionsParser.h"
#include "clang/Tooling/Tooling.h"
#include "llvm/Support/CommandLine.h"
#include "clang/Driver/Options.h"
#include "clang/AST/AST.h"
#include "clang/AST/ASTContext.h"
#include "clang/AST/ASTConsumer.h"
#include "clang/AST/RecursiveASTVisitor.h"
#include "clang/Frontend/ASTConsumers.h"
#include "clang/Frontend/FrontendActions.h"
#include "clang/Frontend/CompilerInstance.h"
#include "clang/Tooling/CommonOptionsParser.h"
#include "clang/Tooling/Tooling.h"
#include "clang/Rewrite/Core/Rewriter.h"
#include <iostream>

using namespace std;
using namespace clang;
using namespace clang::driver;
using namespace clang::tooling;
using namespace llvm;

class ExampleVisitor : public RecursiveASTVisitor<ExampleVisitor> {
public:
    explicit ExampleVisitor(CompilerInstance *CI) {}
};

class ExampleASTConsumer : public ASTConsumer {
private:
    CompilerInstance *CI;
public: 
    explicit ExampleASTConsumer(CompilerInstance *CI) : CI(CI) {}   
    virtual void HandleTranslationUnit(ASTContext &Context) {
        ExampleVisitor(CI).TraverseDecl(Context.getTranslationUnitDecl());
    }
};

class ExampleFrontendAction : public ASTFrontendAction {
public:
    virtual std::unique_ptr<ASTConsumer> CreateASTConsumer(CompilerInstance &CI, StringRef file) {
        return  std::unique_ptr<ASTConsumer>(new ExampleASTConsumer(&CI)); 
    }
};

void run(int argc, const char **argv, llvm::cl::OptionCategory& tc) {
    CommonOptionsParser op(argc, argv, tc);
    ClangTool Tool(op.getCompilations(), op.getSourcePathList());
    std::cout <<"getSourcePathList.size="<< op.getSourcePathList().size()<<"\n";
    int result = Tool.run(newFrontendActionFactory<ExampleFrontendAction>().get()); 
}

int main(int argc, const char **argv) {
    llvm::cl::OptionCategory tc1("c1");
    llvm::cl::OptionCategory tc2("c2");
    llvm::cl::OptionCategory tc3("c3");
    run(argc, argv,tc1);
    run(argc, argv,tc2);
    run(argc, argv,tc3);
    std::cin.get();
    return 0;
}

the parameters to debug the application is:

"the_only_source_file_to_scan.cpp" --

which is fine.

The output is (from the method "run" above main()):

getSourcePathList.size=1
getSourcePathList.size=2
getSourcePathList.size=3

The problem is that main() calls run() 3 times with the same above parameter which contains only 1 source file to scan, but each time the size of the source-to-scan list stored in CommonOptionsParser increase by one (every item in the list is the same file input from argv), it just seems to append the source file to scan to the list each time.

Everything above is saved in newly created temporary variables in each run, then how and why does LibTooling keep states of the last run and how to "reset" these states?

Answer 1

use FixedCompilationDatabase could circumvent this problem, it can run multiple clangTool in one process

Answer 2

The above code use CommonOptionsParser, whose code is in

clang\lib\Tooling\CommonOptionsParser.cpp

in the method CommonOptionsParser::init there is:

  static cl::list<std::string> SourcePaths(...);

Each invocation will add its sources to this static variable. So it is this local static variable that cause the memory of the previous invocations. In each invocation this local static variable is modified by cl::ParseCommandLineOptions in some unkown way since it is not passed into cl::ParseCommandLineOptions at all. After SourcePaths is modified (i.e. sources of the current invocation are added to SourcePaths which may already contain sources of previous invocations), it is finally copied into a instance variable.

BTW @AbaoZhang's clue helps me find the location, indeed CommonOptionsParser::init use FixedCompilationDatabase interally but not for the above situation.

Answer 3

static cl::list<std::string> SourcePaths(...);

@jw_ Yes This code makes problem.

You can fix this.

for (auto iter = llvm::cl::AllSubCommands->OptionsMap.begin(); iter != llvm::cl::AllSubCommands->OptionsMap.end(); iter++)
{
    iter->getValue()->setDefault();
}
for (auto iter = llvm::cl::AllSubCommands->PositionalOpts.begin(); iter != llvm::cl::AllSubCommands->PositionalOpts.end(); iter++)
{
    (*iter)->setDefault();
}
for (auto iter = llvm::cl::AllSubCommands->SinkOpts.begin(); iter != llvm::cl::AllSubCommands->SinkOpts.end(); iter++)
{
    (*iter)->setDefault();
}