Why are Go's strings.Fields(str) and strings.Split(str, " ") so slow?

Question

I have been testing functions in Node and Go to compare their performance. In almost every test, Go is much faster than Node, except when strings.Fields() or strings.Split() is used, then Node is 2-3 times as fast.

Go (2.14s):

start := time.Now()
var newWords []string
str := "asd asjhfa lsjdhalsdjhfa dhfald hfaljdh faldhfasjdhfalsdh asd alsdh alksdh alksdh alksd alkjsd fadlkj dalkjdh asdhfef afa d6a 5a85dfa s5da5d ad a6sd58ad5a8sd5f 8as5f as5 a8s5 8as6d5 8asd65f8as6d58 a5sd 8a5ds8f7 a6s5d"

for j := 0; j < 1000000; j++ {
    words := strings.Split(str, " ")
    wordsLen := len(words)
    newWords = nil
    for i := 0; i < wordsLen; i++ {
        if words[i][:1] != "s" {
            newWords = append(newWords, words[i])
        }
    }
}
dur := time.Since(start)

fmt.Println(newWords)
fmt.Println(dur)

Node (847ms):

let start = new Date()
let newWords = []
let str = "asd asjhfa lsjdhalsdjhfa dhfald hfaljdh faldhfasjdhfalsdh asd alsdh alksdh alksdh alksd alkjsd fadlkj dalkjdh asdhfef afa d6a 5a85dfa s5da5d ad a6sd58ad5a8sd5f 8as5f as5 a8s5 8as6d5 8asd65f8as6d58 a5sd 8a5ds8f7 a6s5d"

for (let j = 0; j < 1000000; j++) {
    let words = str.split(' ')
    let wordsLen = words.length
    newWords = []
    for (let i = wordsLen - 1; i >= 0; i--) {
        if (words[i].substr(0, 1) !== 's') {
            newWords.push(words[i])
        }
    }
}

console.log((new Date() - start) + 'ms')

But `strings.Split(str, " ")` does work: [Go Playground](https://play.golang.org/p/E7VaBxY_iT) — icza, May 29 '17 at 14:23
This is not how you measure / benchmark execution times in Go. Check out this answer: [Order of the code and performance](https://stackoverflow.com/questions/41608578/order-of-the-code-and-performance/41608707#41608707) — icza, May 29 '17 at 14:46
I wouldn't use strings.Split for this kind of tasks, I could use a Scanner https://golang.org/pkg/bufio/#Scanner.Scan — Yandry Pozo, May 29 '17 at 20:39
@Travis using a Scanner with [ScanWords()](https://golang.org/pkg/bufio/#ScanWords), but it shouldn't improve the Big O time, after all who wants to scan the same strings over and over — Yandry Pozo, May 30 '17 at 04:45

peterSO · Answer 1 · 2017-05-29T17:14:53.810

First, let's run your tests.

$ go run travis.go
5.892531292s
$ node travis.js
4065ms
$

Go 5.89 seconds versus Node.js 4.07 seconds.

Second, let's run a Go benchmark and see what you are really doing. It's a lot more than the strings.Fields(str) or strings.Split(str, " ") that you said in your original post.

travis_test.go:

package main

import (
    "strings"
    "testing"
)

var str = "asd asjhfa lsjdhalsdjhfa dhfald hfaljdh faldhfasjdhfalsdh asd alsdh alksdh alksdh alksd alkjsd fadlkj dalkjdh asdhfef afa d6a 5a85dfa s5da5d ad a6sd58ad5a8sd5f 8as5f as5 a8s5 8as6d5 8asd65f8as6d58 a5sd 8a5ds8f7 a6s5d"

func Travis() {
    var newWords []string
    for j := 0; j < 1000000; j++ {
        words := strings.Split(str, " ")
        wordsLen := len(words)
        newWords = nil
        for i := 0; i < wordsLen; i++ {
            if words[i][:1] != "s" {
                newWords = append(newWords, words[i])
            }
        }
    }
}

func BenchmarkTravis(b *testing.B) {
    b.ReportAllocs()
    b.ResetTimer()
    for i := 0; i < b.N; i++ {
        Travis()
    }
    b.StopTimer()
}

Output:

$ go test -run=! -bench=Travis travis_test.go 
goos: linux
goarch: amd64
BenchmarkTravis-4   1   5832192616 ns/op   1488002816 B/op   7000015 allocs/op
PASS
ok      command-line-arguments  5.838s
$

Why are Go's strings.Fields(str) and strings.Split(str, " ") so slow?

1 Answers1