Will Go 1.21 include ability to host http via WebAssembly? How?

Question

I'd like to try an http server via WebAssembly on Go. I think that compiling go for webassembly outside the browser is not supported in go 1.20, and that the net/http libraries aren't included in tinygo.

I tried to do it with gotip after reading https://stackoverflow.com/a/76091829 (thanks @TachyonicBytes), but whenever I tried to start the server (or any blocking/waiting function), I got an error: fatal error: all goroutines are asleep - deadlock!. I tried moving things to a goroutine with wait functions and that either simply ended the function, or gave the same error. Here's how I ran it:

go install golang.org/dl/gotip@latest
gotip download
GOOS=wasip1 GOARCH=wasm gotip build -o server.wasm server.go && wasm3 server.wasm

Here's the example server.go:

package main

import (
    "fmt"
    "net/http"
    "sync"
)

func main() {
    s := http.Server{
        Addr: ":8080",
        Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            w.Write([]byte("Hello, World!"))
        }),
    }

    fmt.Println("about to serve")
    var wg sync.WaitGroup
    wg.Add(1)
    go func() {
        err := s.ListenAndServe()
        if err != nil {
            fmt.Printf("Unable to serve: %v\n", err)
        }
        wg.Done()
        fmt.Println("serving stopped")
    }()
    wg.Wait()
    fmt.Println("started up server")
}

So, is this just because go 1.21 is a WIP, because I'm failing to understand the proper way to start a blocking function, or because this sort of thing won't be supported in go 1.21?

I tried to start a go server in a server side webassembly runner wasm3 on an Intel Mac. I expected it to serve http, but found it either threw an error, or exited immediately.

Answer 1

Glad to have been of help!

Unfortunately no, it seems that wasm networking will not be a part of go 1.21. It's a bit complicated to implement networking in wasm. Running your code, I got this line:

    sdk/gotip/src/net/net_fake.go:229

Upon inspection, it has this disclaimer:

// Fake networking for js/wasm and wasip1/wasm.
// This file only exists to make the compiler happy.

The hard part of doing this is that WASI has only partial support for sockets, so no full blown Berkeley sockets for WASI, yet.

The good news is that you can actually do http, but in tinygo. Tinygo has partial support for the go net/http package, with it's drivers.

If you want to see some real-life usage of this, I am currently trying to port this project to wasm, using tinygo. If I recall correctly, I got it to work, but it has been a while, and I know for sure that I did not complete the conversion yet. Maybe it was impossible for the time being.

Another thing is that wasm3, despite having partial wasi implementation, may not have implemented the sockets part. I would suggest also playing with some other runtimes, like wasmtime, wasmer, wasmedge, or wazero, which @Gedw99 suggested. Wasmedge has great support for sockets, but in your case, the compiler is actually the problem.

Answer 2

I have managed to get this to work with 1.21 by passing an open file descriptor of a TCP socket to the guest module, and calling net.FileListener.

First part is achieved with the github.com/tetratelabs/wazero runtime using the submodule experimental/sock. Below is a simple demo.

host.go, run with gotip

package main

import (
    "context"
    _ "embed"
    "os"

    "github.com/tetratelabs/wazero"
    "github.com/tetratelabs/wazero/experimental/sock"
    "github.com/tetratelabs/wazero/imports/wasi_snapshot_preview1"
)

const socketDescriptor uint32 = 3

//go:embed module.wasm
var moduleData []byte

func main() {
    // The availability of networking is passed to the guest module via context.
    // AFAIK there is not yet any bespoke API to figure out
    // a) dynamic port allocation,
    // b) the file descriptor.
    // However, we can make an educated guess of the latter: since stdin,
    // stdout and stderr are the file descriptors 0-2, our socket SHOULD be 3.
    // Take note that this guess is for the perspective of the guest module.
    ctx := sock.WithConfig(
        context.Background(),
        sock.NewConfig().WithTCPListener("127.0.0.1", 8080),
    )

    // Runtime and WASI prep.
    r := wazero.NewRuntime(ctx)
    defer r.Close(ctx)
    wasi_snapshot_preview1.MustInstantiate(ctx, r)

    // Module configuration.
    cfg := wazero.NewModuleConfig().WithStdout(os.Stdout).WithStderr(os.Stderr)
    // stdout/stderr added for simple debugging: this breaks sandboxing.

    // Export a function for the guest to fetch the (guessed) fd.
    if _, err := r.NewHostModuleBuilder("env").NewFunctionBuilder().
        WithFunc(func() uint32 {
            return socketDescriptor
        }).Export("getSocketDescriptor").Instantiate(ctx); err != nil {
        panic(err)
    }
    // We also could provide the fd via an environment variable,
    // but working with strings can be annoying:
    // cfg = cfg.WithEnv("socketDescriptor", fmt.Sprint(socketDescriptor))

    // Compilation step
    compiled, err := r.CompileModule(ctx, moduleData)
    if err != nil {
        panic(err)
    }

    // Run the module
    if _, err := r.InstantiateModule(ctx, compiled, cfg); err != nil {
        panic(err)
    }
}

module.go, compiled with GOOS="wasip1" GOARCH="wasm" gotip build -o module.wasm module.go

package main

import (
    "net"
    "net/http"
    "os"
    "syscall"
)

//go:wasmimport env getSocketDescriptor
func getSocketDescriptor() uint32

func main() {
    // Receive the file descriptor of the open TCP socket from host.
    sd := getSocketDescriptor()

    // Blocking I/O is problematic due to the lack of threads.
    if err := syscall.SetNonblock(int(sd), true); err != nil {
        panic(err)
    }

    // The host SHOULD close the file descriptor when the context is done.
    // The file name is arbitrary.
    ln, err := net.FileListener(os.NewFile(uintptr(sd), "[socket]"))
    if err != nil {
        panic(err)
    }

    // HTTP server
    if err := http.Serve(ln, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte("Hello, World!\n"))
    })); err != nil {
        panic(err)
    }
}

Successfully tested with Ubuntu/WSL. Another way to find the socket fd from within the module is to iterate over positive integers until "bad file number" error or a syscall.Fstat() -> *syscall.Stat_t that implies a socket.

Update for clarity: After the two files are in place in the same directory run the following commands (people from the future should be able to replace gotip with just go) and visit http://127.0.0.1:8080 with your browser:

gotip mod init go-wasm-hello-world
gotip mod tidy
GOOS="wasip1" GOARCH="wasm" gotip build -o module.wasm module.go
gotip run host.go

Answer 3

"all goroutines are asleep - deadlock!" - its a common surprise when using golang with wasm.

The solution is that you would need to perform the http.Get in a separate goroutine. Don't use WaitGroup.

--

Also for running wasm outside the browser you can use wazero. https://github.com/tetratelabs/wazero