How to use Clang CompilerInstance for completion?

Question

I have a library which uses and exposes a clang::CompilerInstance. How can I use the CompilerInstance for getting code-completion- suggestions ?

Basically I'm looking to write a function with the following signature:

vector<string> completeSnippet(
  clang::CompilerInstance CI, 
  string codeSnippet,
  int completeAtIndex
);

Any ideas ?

Thanks in advance

Is it necessary that you use clang::CompilerInstance? Not other ways available from clang? — khrm, May 20 '16 at 06:48
Im using a library which only exposes a CompilerInstance. The AST and Sema objects are already created and I would like to reuse them. — Gaetano, May 20 '16 at 13:16
It's a custom closed source library build on clang 3.7. If there is a solution without using `CompilerInstance` it would be also helpful. — Gaetano, May 20 '16 at 16:25

Shangtong Zhang · Accepted Answer · 2016-05-22T09:57:08.377

To be honest I didn't figure out how to implement

vector<string> completeSnippet(
  clang::CompilerInstance CI, 
  string codeSnippet,
  int completeAtIndex
);

Instead, what I can provide is

vector<std::string> completeSnippet(clang::CompilerInstance& ci,
                                    const std::string& filename,
                                    unsigned Line /* start from 1 */,
                                    unsigned Column /* start from 1 */);

There is a workaround if you don't care much about performance, you can store the codeSnippet to a file and then pass the filename.

As I don't know the status of the ci, for example, whether it has a target, I just clarify the approach in main method, you can easily refactor it into a function.

int main()
{
    std::string Filename("test.cpp");
    unsigned Line = 7;
    unsigned Column = 5;
    clang::CompilerInstance ci;

    // Create diagnostics and target
    ci.createDiagnostics();
    std::shared_ptr<clang::TargetOptions> to = std::make_shared<clang::TargetOptions>();
    // clang -v
    //to->Triple = "x86_64-pc-win32";
    //to->Triple = "x86_64-apple-darwin";
    to->Triple = "x86_64-apple-darwin15";
    ci.setTarget(clang::TargetInfo::CreateTargetInfo(ci.getDiagnostics(), to));

    // Create language options
    clang::LangOptions &lo = ci.getLangOpts();
    lo.CPlusPlus = true;
    lo.CPlusPlus11 = true;

    // Create code complete options
    clang::CodeCompleteOptions cco;
    cco.IncludeMacros = 0;
    cco.IncludeCodePatterns = 1;
    cco.IncludeGlobals = 1;
    cco.IncludeBriefComments = 1;

    // Set up the callback, I will go back to this callback class later
    auto pCustomCodeCompleteConsumer = new CustomCodeCompleteConsumer(cco);
    ci.setCodeCompletionConsumer(pCustomCodeCompleteConsumer);

    // Set up code complete postions & file
    // Until now I didn't find a way to pass in a string rather than a file name
    clang::FrontendOptions& frontendOpts = ci.getFrontendOpts();
    frontendOpts.CodeCompletionAt.FileName = Filename;
    frontendOpts.CodeCompletionAt.Line = Line;
    frontendOpts.CodeCompletionAt.Column = Column;
    frontendOpts.Inputs.push_back(clang::FrontendInputFile(Filename, clang::InputKind::IK_CXX));

    // Execute
    clang::SyntaxOnlyAction Act;
    if (Act.BeginSourceFile(ci, ci.getFrontendOpts().Inputs[0])) {
        Act.Execute();
        Act.EndSourceFile();
    }
}

I copy an implementation of the callback class from PrintingCodeCompleteConsumer in Sema/CodeCompleteConsumer.cpp and Sema/CodeCompleteConsumer.h

class CustomCodeCompleteConsumer : public clang::CodeCompleteConsumer {
    clang::CodeCompletionTUInfo CCTUInfo;
public:
    CustomCodeCompleteConsumer(const clang::CodeCompleteOptions &CodeCompleteOpts)
    : clang::CodeCompleteConsumer(CodeCompleteOpts, false), CCTUInfo(new clang::GlobalCodeCompletionAllocator) {}

    void ProcessCodeCompleteResults(clang::Sema &SemaRef, clang::CodeCompletionContext Context,
                                    clang::CodeCompletionResult *Results,
                                    unsigned NumResults) override {
        std::stable_sort(Results, Results + NumResults, [](auto& lhs, auto& rhs) {
            return lhs.Priority > rhs.Priority;
        });
        using namespace clang;
        for (unsigned I = 0; I != NumResults; ++I) {
            std::string ccStr = "";
            llvm::raw_string_ostream OS(ccStr);
            OS << "COMPLETION: " << Results[I].Priority;
            switch (Results[I].Kind) {
                case CodeCompletionResult::RK_Declaration:
                    OS << "Decl : ";
                    OS << *Results[I].Declaration;
                    if (Results[I].Hidden)
                        OS << " (Hidden)";
                    if (CodeCompletionString *CCS
                        = Results[I].CreateCodeCompletionString(SemaRef, Context,
                                                                getAllocator(),
                                                                CCTUInfo,
                                                                includeBriefComments())) {
                            OS << " : " << CCS->getAsString();
                            if (const char *BriefComment = CCS->getBriefComment())
                                OS << " : " << BriefComment;
                        }

                    OS << '\n';
                    break;

                case CodeCompletionResult::RK_Keyword:
                    OS << "Keyword : ";
                    OS << Results[I].Keyword << '\n';
                    break;

                case CodeCompletionResult::RK_Macro: {
                    OS << "Macro : ";
                    OS << Results[I].Macro->getName();
                    if (CodeCompletionString *CCS
                        = Results[I].CreateCodeCompletionString(SemaRef, Context,
                                                                getAllocator(),
                                                                CCTUInfo,
                                                                includeBriefComments())) {
                            OS << " : " << CCS->getAsString();
                        }
                    OS << '\n';
                    break;
                }

                case CodeCompletionResult::RK_Pattern: {
                    OS << "Pattern : " 
                    << Results[I].Pattern->getAsString() << '\n';
                    break;
                }
            }
            OS.flush();
            std::cout << ccStr;
        }

    }

    void ProcessOverloadCandidates(clang::Sema &S, unsigned CurrentArg,
                                   clang::OverloadCandidate *Candidates,
                                   unsigned NumCandidates) {

    }

    clang::CodeCompletionAllocator &getAllocator() override {
        return CCTUInfo.getAllocator();
    }

    clang::CodeCompletionTUInfo &getCodeCompletionTUInfo() override { return CCTUInfo; }
};

I made a little change to original ProcessCodeCompleteResults to output info to console.

And my header files:

#define __STDC_CONSTANT_MACROS
#define __STDC_LIMIT_MACROS
#include <clang/Frontend/CompilerInstance.h>
#include <clang/Frontend/FrontendActions.h>
#include <clang/Lex/Preprocessor.h>
#include <clang/Basic/TargetOptions.h>
#include <clang/Basic/TargetInfo.h>
#include <clang/Basic/LangOptions.h>
#include <clang/Basic/SourceManager.h>
#include <clang/Basic/SourceLocation.h>
#include <clang/Basic/FileManager.h>
#include <clang/Sema/Sema.h>
#include <clang/Sema/CodeCompleteOptions.h>
#include <clang/Sema/CodeCompleteConsumer.h>
#include <clang/Parse/ParseAST.h>
#include <clang/AST/ASTContext.h>
#include <clang/AST/ASTConsumer.h>
#include <clang/AST/Decl.h>
#include <llvm/Support/raw_ostream.h>
#include <iostream>

Remaining problems:

I didn't find interface to pass in string rather than filename
You may need extra effort to process the returned completions.

Say we have two test files:

test1.cpp

void myFunc(int param) {

}

int main() {
    int count = 5;
    cou
    return 0;
}

test2.cpp

void myFunc(int param) {

}

int main() {
    int count = 5;
    myFunc
    return 0;
}

Both lang++ -cc1 -fsyntax-only -code-completion-at test2.cpp:7:4 test2.cpp and clang++ -cc1 -fsyntax-only -code-completion-at test1.cpp:7:4 test1.cpp output same things:

COMPLETION: __FUNCTION__
COMPLETION: __PRETTY_FUNCTION__
COMPLETION: _Nonnull
COMPLETION: _Null_unspecified
COMPLETION: _Nullable
COMPLETION: bool
COMPLETION: char
COMPLETION: class
COMPLETION: const
COMPLETION: Pattern : const_cast<<#type#>>(<#expression#>)
COMPLETION: count : [#int#]count
COMPLETION: Pattern : [#void#]delete <#expression#>
COMPLETION: Pattern : [#void#]delete [] <#expression#>
COMPLETION: double
COMPLETION: Pattern : dynamic_cast<<#type#>>(<#expression#>)
COMPLETION: enum
COMPLETION: extern
COMPLETION: Pattern : [#bool#]false
COMPLETION: float
COMPLETION: Pattern : goto <#label#>
COMPLETION: int
COMPLETION: long
COMPLETION: main : [#int#]main()
COMPLETION: myFunc : [#void#]myFunc(<#int param#>)
COMPLETION: Pattern : new <#type#>(<#expressions#>)
COMPLETION: Pattern : new <#type#>[<#size#>](<#expressions#>)
COMPLETION: operator
COMPLETION: Pattern : reinterpret_cast<<#type#>>(<#expression#>)
COMPLETION: Pattern : return <#expression#>
COMPLETION: short
COMPLETION: signed
COMPLETION: Pattern : [#size_t#]sizeof(<#expression-or-type#>)
COMPLETION: static
COMPLETION: Pattern : static_cast<<#type#>>(<#expression#>)
COMPLETION: struct
COMPLETION: Pattern : [#bool#]true
COMPLETION: Pattern : typedef <#type#> <#name#>
COMPLETION: Pattern : [#std::type_info#]typeid(<#expression-or-type#>)
COMPLETION: Pattern : typename <#qualifier#>::<#name#>
COMPLETION: Pattern : typeof <#expression#>
COMPLETION: Pattern : typeof(<#type#>)
COMPLETION: union
COMPLETION: unsigned
COMPLETION: Pattern : using namespace <#identifier#>
COMPLETION: void
COMPLETION: volatile
COMPLETION: wchar_t

Same with my implementation:

COMPLETION: 65 Keyword : __PRETTY_FUNCTION__
COMPLETION: 65 Keyword : __FUNCTION__
COMPLETION: 65 Keyword : __func__
COMPLETION: 50 Decl : myFunc : [#void#]myFunc(<#int param#>)
COMPLETION: 50 Decl : main : [#int#]main()
COMPLETION: 50 Keyword : short
COMPLETION: 50 Keyword : long
COMPLETION: 50 Keyword : signed
COMPLETION: 50 Keyword : unsigned
COMPLETION: 50 Keyword : void
COMPLETION: 50 Keyword : char
COMPLETION: 50 Keyword : int
COMPLETION: 50 Keyword : float
COMPLETION: 50 Keyword : double
COMPLETION: 50 Keyword : enum
COMPLETION: 50 Keyword : struct
COMPLETION: 50 Keyword : union
COMPLETION: 50 Keyword : const
COMPLETION: 50 Keyword : volatile
COMPLETION: 50 Keyword : bool
COMPLETION: 50 Keyword : class
COMPLETION: 50 Keyword : wchar_t
COMPLETION: 50 Keyword : auto
COMPLETION: 50 Keyword : char16_t
COMPLETION: 50 Keyword : char32_t
COMPLETION: 50 Keyword : _Nonnull
COMPLETION: 50 Keyword : _Null_unspecified
COMPLETION: 50 Keyword : _Nullable
COMPLETION: 40 Pattern : typedef <#type#> <#name#>
COMPLETION: 40 Pattern : if(<#condition#>){<#statements#>
}
COMPLETION: 40 Pattern : switch(<#condition#>){
}
COMPLETION: 40 Pattern : while(<#condition#>){<#statements#>
}
COMPLETION: 40 Pattern : do{<#statements#>
}while(<#expression#>)
COMPLETION: 40 Pattern : for(<#init-statement#>;<#condition#>;<#inc-expression#>){
<#statements#>
}
COMPLETION: 40 Pattern : return <#expression#>
COMPLETION: 40 Pattern : goto <#label#>
COMPLETION: 40 Pattern : using namespace <#identifier#>
COMPLETION: 40 Keyword : extern
COMPLETION: 40 Keyword : static
COMPLETION: 40 Pattern : [#bool#]true
COMPLETION: 40 Pattern : [#bool#]false
COMPLETION: 40 Pattern : dynamic_cast<<#type#>>(<#expression#>)
COMPLETION: 40 Pattern : static_cast<<#type#>>(<#expression#>)
COMPLETION: 40 Pattern : reinterpret_cast<<#type#>>(<#expression#>)
COMPLETION: 40 Pattern : const_cast<<#type#>>(<#expression#>)
COMPLETION: 40 Pattern : [#std::type_info#]typeid(<#expression-or-type#>)
COMPLETION: 40 Pattern : new <#type#>(<#expressions#>)
COMPLETION: 40 Pattern : new <#type#>[<#size#>](<#expressions#>)
COMPLETION: 40 Pattern : [#void#]delete <#expression#>
COMPLETION: 40 Pattern : [#void#]delete [] <#expression#>
COMPLETION: 40 Pattern : [#std::nullptr_t#]nullptr
COMPLETION: 40 Pattern : [#size_t#]alignof(<#type#>)
COMPLETION: 40 Pattern : [#bool#]noexcept(<#expression#>)
COMPLETION: 40 Pattern : [#size_t#]sizeof...(<#parameter-pack#>)
COMPLETION: 40 Pattern : [#size_t#]sizeof(<#expression-or-type#>)
COMPLETION: 40 Pattern : typename <#qualifier#>::<#name#>
COMPLETION: 40 Pattern : decltype(<#expression#>)
COMPLETION: 40 Pattern : typeof <#expression#>
COMPLETION: 40 Pattern : typeof(<#type#>)
COMPLETION: 40 Keyword : operator
COMPLETION: 34 Decl : count : [#int#]count

Though I have ordered them by priority

You can refer to ASTUnit::CodeComplete and AugmentedCodeCompleteConsumer::ProcessCodeCompleteResults in lib/Frontend/ASTUnit.cpp to learn more about the logic of clang_codeCompleteAt

How to use Clang CompilerInstance for completion?

1 Answers1

Linked