1

I need to make function that calculating hash of multiple large strings concatenation. How to do it efficient in go? To avoid unnecessary memory copies

I updated the question. Now the question is there a difference between make_hash and make_hash_ptr

func make_hash(v ...string) string {
    h := sha256.New()
    for _, s := range v {
        h.Write([]byte(s))
    }
    return hex.EncodeToString(h.Sum(nil))
}

func make_hash_ptr(v ...*string) string {
    h := sha256.New()
    for _, s := range v {
        h.Write([]byte(*s))
    }
    return hex.EncodeToString(h.Sum(nil))
}

func main() {
    s1 := "very large string...."
    s2 := "very large string...."
    s3 := "very large string...."
    hash := make_hash(s1, s2, s3)

    fmt.Println(hash)
}
user1374768
  • 288
  • 2
  • 11
  • 5
    Don't concatenate strings. Write them one-by-one into the hasher. The result is the same. – icza Apr 11 '22 at 11:00
  • Thank you! Does it matter in that case if the parameter is a string or a pointer to a string? – user1374768 Apr 11 '22 at 11:11
  • 1
    this seems relevant https://stackoverflow.com/questions/47156856/hashing-multiple-values-in-golang – blackgreen Apr 11 '22 at 11:18
  • 4
    Passing string pointers is unnecessary as strings are already headers pointing to the actual string data, and strings are immutable. – icza Apr 11 '22 at 11:41
  • If you are going to concatenate a big amount of string, you are penalizing yourself making it a slice in `make_hash(v ...string) string` –  Apr 11 '22 at 13:53
  • i dwonvot because this is too simplistic/convoluted. We dont know how the data is read, or arranged in the source. –  Apr 11 '22 at 13:55
  • If you MUST concatenate large strings, then the strings.Builder package is the way to go for efficiency. https://pkg.go.dev/strings#Builder – m0j0 Apr 11 '22 at 15:42
  • Are you concerned with `make_hash("ab", "c")` to produce the same result as `make_hash("a", "bc")`? If yes, you need to combine hashes by yourself, check my answer here: https://stackoverflow.com/a/76693851/1943849 – astef Jul 15 '23 at 13:05

0 Answers0