How to programmatically retrieve the package path where a given type is declared?

Question

I am looking for a way to retrieve the package(s) installed locally which contain the declaration for a given type and the default package name.

ie:

// FindPackagesForType returns the list of possible packages for a given type
func FindPackagesForType(typeName string) []string {
    return []string {} // TODO: implement
}

func TestFindPackagesForType(t *testing.T) {
    assert.Contains(t, FindPackagesForType("io.Reader"), "io")
    assert.Contains(
        t,
        FindPackagesForType("types.Timestamp"),
        "github.com/gogo/protobuf/types",
    )
    assert.Contains(
        t,
        FindPackagesForType("types.ContainerCreateConfig"),
        "github.com/docker/docker/api/types",
    )
}

I could try to retrieve all packages installed, and go through the AST in each looking for the declaration but if there is a solution which could do this more efficiently while also providing support for go modules I would like to use that.

The reason for this is to improve a code generation tool. The idea is to let the user provide the name of a type and let the tool identify the most likely candidate the same way goimports adds missing imports.

Take a look at https://github.com/golang/tools/tree/master/cmd/goimports — leaf bebop, Dec 18 '19 at 13:50
"io.Reader" can correspond to any path that ends with "/io". — Jonathan Hall, Dec 18 '19 at 14:27
Use output of `go list -f '{{.Name}} {{.ImportPath}}' all` to build `map[string][]string` where key is package name and value is slice of import paths. Split input string on "." to get package name and type name. For each import path with package name, use `go/build` to find files in package, parse with `go/parser`, walk AST and find type. — Charlie Tumahai, Dec 18 '19 at 14:34
I added more precision, @mh-cbon I am looking for all the declarations — Coyote, Dec 18 '19 at 15:25
@Coyote the answer i gave you does not satisfy your need ? I don't get it.. — , Dec 23 '19 at 10:58

score 1 · Answer 1 · answered Dec 18 '19 at 13:55

1

You can use reflect.TypeOf(any).PkgPath() to get the package path of certain type. However we need to pass an object with desired type (not string like you wanted).

package main

import (
    "bytes"
    "fmt"
    "reflect"
    "gopkg.in/mgo.v2/bson"
)

func main() {
    var a bytes.Buffer
    fmt.Println(FindPackagesForType(a)) // output: bytes

    var b bson.M
    fmt.Println(FindPackagesForType(b)) // output: gopkg.in/mgo.v2/bson
}

func FindPackagesForType(any interface{}) string {
    return reflect.TypeOf(any).PkgPath()
}

answered Dec 18 '19 at 13:55

novalagung

10,905
4
58
82

2

`reflect` package works at runtime. I believe OP wants `static` analysis, thus `ast` and other things described under the [go package](https://golang.org/pkg/go/). – Dec 18 '19 at 13:58
@mh-cbon do they? The question is unclear, the example code looks like they're trying to do it at runtime. – Adrian Dec 18 '19 at 14:20
I am unfortunately looking for static analysis of existing packages on the user's machine. I improved the question. Thank you. – Coyote Dec 18 '19 at 15:32

score 0 · Answer 2 · answered Dec 18 '19 at 20:31

below program lists uses and definitions of a given query type and given go package.

It is simple and straightforward to programmatically load a go program using the program loader package

package main

import (
    "flag"
    "fmt"
    "strings"

    "golang.org/x/tools/go/loader"
)

func main() {

    var query string
    var uses bool
    var defs bool
    flag.StringVar(&query, "query", "", "the fully qualified type path")
    flag.BoolVar(&uses, "uses", true, "capture uses")
    flag.BoolVar(&defs, "definitions", true, "capture definitions")
    flag.Parse()

    if query == "" {
        panic("query must not be empty")
    }

    var queryPkg string
    queryType := query
    if i := strings.LastIndex(query, "."); i > -1 {
        queryPkg = query[:i]
        queryType = query[i+1:]
    }

    var conf loader.Config
    _, err := conf.FromArgs(flag.Args(), false)
    if err != nil {
        panic(err)
    }
    prog, err := conf.Load()
    if err != nil {
        panic(err)
    }

    for pkgType, pkgInfo := range prog.AllPackages {
        if queryPkg != "" {
            if !strings.HasPrefix(pkgType.Path(), queryPkg) {
                continue
            }
        }
        if defs {
            for typeInfo, ident := range pkgInfo.Defs {
                if !strings.HasPrefix(typeInfo.Name, queryType) {
                    continue
                }
                f := prog.Fset.File(ident.Pos())
                fpos := f.Position(ident.Pos())
                fmt.Printf("def: %v %v.%v\n", fpos, pkgType.Path(), typeInfo.Name)
            }
        }

        if uses {
            for ident, oInfo := range pkgInfo.Uses {
                if !strings.Contains(oInfo.Type().String(), queryType) {
                    continue
                }
                f := prog.Fset.File(ident.Pos())
                fpos := f.Position(ident.Pos())
                fmt.Printf("use: %v %v\n", fpos, oInfo.Type().String())
            }
        }
        // -
    }
}

then you run it like this

$ go run main.go -query="io.Reader" io
def: /home/mh-cbon/.gvm/gos/go1.12.7/src/io/io.go:170:6 io.ReaderFrom
def: /home/mh-cbon/.gvm/gos/go1.12.7/src/io/io.go:77:6 io.Reader
def: /home/mh-cbon/.gvm/gos/go1.12.7/src/io/io.go:211:6 io.ReaderAt
use: /home/mh-cbon/.gvm/gos/go1.12.7/src/io/multi.go:20:13 []io.Reader
use: /home/mh-cbon/.gvm/gos/go1.12.7/src/io/multi.go:21:16 *io.multiReader
# a ton of output...
[mh-cbon@Host-001 ploader] $ go run main.go -query="Config" io
[mh-cbon@Host-001 ploader] $ go run main.go -query="io.Reader" -uses=false io
def: /home/mh-cbon/.gvm/gos/go1.12.7/src/io/io.go:170:6 io.ReaderFrom
def: /home/mh-cbon/.gvm/gos/go1.12.7/src/io/io.go:211:6 io.ReaderAt
def: /home/mh-cbon/.gvm/gos/go1.12.7/src/io/io.go:77:6 io.Reader

you probably got to improve the matcher engine to make it moresuitable.

This answer doesn't work as it requires the package to be provided. The question is how to retrieve the packages which could match a given type. — Coyote, Jan 10 '20 at 13:17
does not make sense `...as it requires the package to be provided`. — , Jan 10 '20 at 13:37
@Coyote this means you got to download and test for all and every packages available online. unrealistic. — , Jan 12 '20 at 17:49

How to programmatically retrieve the package path where a given type is declared?

2 Answers2