3

I've been dabbling with Go for about a month for a school project and I noticed the go/ast, go/token, go/parser, etc. packages in the src/pkg/go folder. However, the gc compiler was based on C files located in src/cmd/gc.

My question regards the new go command in Go1 that builds and runs programs: does this tool depend on the packages I referenced above? i.e. if I added a new token to /go/token/token.go, would it be recognized by the new go compiler?

calvin
  • 978
  • 7
  • 14

3 Answers3

5

The Go compiler is written in pure C and does not use the packages under go/. In the Go source tree, its lexer lives in src/cmd/gc/lex.c and its Bison grammar is src/cmd/gc/go.y.

The go/ packages are used in tools like godoc, gofmt, and various go tool subcommands. Maybe someday they can be used to write a Go compiler in Go as well, but no one's gotten very far on that path yet.

Evan Shaw
  • 23,839
  • 7
  • 70
  • 61
  • 1
    There is an immediate advantage in writing a compiler in the language it is supposed to compile: dog fooding allows you to feel your language. However it also brings a hurdle: portability. Having a C compiler for your language means that you can get the compiler working on any platform you feel like. Sure the runtime and codegen might still need work, but it's a lesser gap (though it can still be tough). – Matthieu M. Apr 22 '12 at 15:58
  • Actually there is llgo, which is a Go compiler written in Go using llvm as backend, is not yet complete but seems to be quite advanced: https://github.com/axw/llgo – uriel Jun 30 '12 at 18:36
2

Note (December 18th, 2013), there are plans to move the compiler from C to Go itself:

"Go 1.3+ Compiler Overhaul" (Russ Cox)

In that context packages like go/parser will be involved, and the "Phase 5" mentions:

Replace the front end with the latest (perhaps new) versions of go/parser and go/types.
Robert Griesemer has discussed the possibility of designing new go/parser and go/types APIs at some point, based on experience with the current ones (and under new names, to preserve Go 1 compatibility).
The work of connecting them to a compiler back end may help guide design of new APIs.

This is probably a testimony on how stable the language has become, since the old "A Tour of Go" (June 2012) mentioned before clearly stated:

The fact that Go wasn’t written in itself also made it much easier to make significant language changes.
Before the initial release we went through a handful of wholesale syntax upheavals, and I’m glad we didn’t have to worry about how we were going to rebootstrap the compiler or ensure some kind of backwards compatibility during those changes.

The question "Is there any plan to bootstrap Go in Go, to write the Go compiler in Go?" mentioned at the time (again, June 2012):

There’s no immediate plan. Go does ship with a Go program parser written in Go, so the first piece is already done, and there’s an experimental type checker in the works, but those are mainly for writing program analysis tools.

I’ve worked on bootstrapped languages in the past, and I found that bootstrapping is not necessarily a good fit for languages that are changing frequently. It reminded me of climbing a cliff and screwing hooks into the cliff once in a while to catch you if you fall.

Community
  • 1
  • 1
VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • Note to self: And that was my **9000th answer** on Stack Overflow in 63 months! Less than 6 months after the [8000th answer](http://stackoverflow.com/a/17569094/6309). Before that: [7000th answer](http://stackoverflow.com/a/14274272/6309), [6000th answer](http://stackoverflow.com/a/11644343/6309), [5000th answer](http://stackoverflow.com/a/7917396/6309), [4000th answer](http://stackoverflow.com/a/4953561/6309), [3000th answer](http://stackoverflow.com/a/3074849/6309), [2000th answer](http://stackoverflow.com/a/2027976/6309), and [1000th answer](http://stackoverflow.com/a/665252/6309). – VonC Dec 20 '13 at 09:24
1

does this tool depend on the packages I referenced above?

The 'go' tool does depend on those packages

if I added a new token to /go/token/token.go, would it be recognized by the new go compiler?

No.

zzzz
  • 87,403
  • 16
  • 175
  • 139
  • What if I added a new token, and a corresponding ast node and parse method? Or am I just completely on the wrong track – calvin Apr 21 '12 at 18:39
  • The go tool invokes the Go compiler but the Go compiler is a C project, it doesn't use the Go parser package. – zzzz Apr 21 '12 at 18:46