0

I'm trying to use for the first time the ulid package.

In their README they say:

Please note that rand.Rand from the math package is not safe for concurrent use. Instantiate one per long living go-routine or use a sync.Pool if you want to avoid the potential contention of a locked rand.Source as its been frequently observed in the package level functions.

Can you help me understand what does this mean and how to write SAFE code for concurrent use with libraries such ent or gqlgen?

Example: I'm using the below code in my app to generate new IDs (sometimes even many of them in the same millisecond which is fine for ulid).

import (
  "math/rand"
  "time"

  "github.com/oklog/ulid/v2"
)

var defaultEntropySource *ulid.MonotonicEntropy

func init() {
  defaultEntropySource = ulid.Monotonic(rand.New(rand.NewSource(time.Now().UnixNano())), 0)
}

func NewID() string {
  return ulid.MustNew(ulid.Timestamp(time.Now()), defaultEntropySource).String()
}

Is this a safe way to use the package?

Fred Hors
  • 3,258
  • 3
  • 25
  • 71

1 Answers1

2

Is this a safe way to use the package?

No, that sentence suggests that each rand.Source should be local to the goroutine, your defaultEntropySource rand.Source piece is potentially shared between multiple goroutines.

As documentated New function, you only need to make sure the entropy reader is safe for concurrent use, but Monotonic is not. Here is a two ways of implementing the documentation suggestion:

Create a single rand.Source per call o NewID(), allocates a new entropy for each call to NewID

func NewID() string {
    defaultEntropySource := ulid.Monotonic(rand.New(rand.NewSource(time.Now().UnixNano())), 0)
    return ulid.MustNew(ulid.Timestamp(time.Now()), defaultEntropySource).String()
}

Playground

Like above but using sync.Pool to possible reuse previously allocated rand.Sources

var entropyPool = sync.Pool{
    New: func() any {
        entropy := ulid.Monotonic(rand.New(rand.NewSource(time.Now().UnixNano())), 0)
        return entropy
    },
}

func NewID() string {
    e := entropyPool.Get().(*ulid.MonotonicEntropy)
    s := ulid.MustNew(ulid.Timestamp(time.Now()), e).String()
    entropyPool.Put(e)
    return s
}

Playground

Wagner Riffel
  • 302
  • 1
  • 3
  • Wow. Thanks! These are extremely useful examples! So the problem right now is to benchmark these to found the faster. – Fred Hors Jun 11 '22 at 15:40
  • I tried your suggestion about `crypto/rand` but I cannot use it concurrently as you can see here: https://go.dev/play/p/CkdK1K_H_Wt. Why? – Fred Hors Jun 11 '22 at 17:21
  • @FredHors Oh, I've read the wrong documentation, ULID is safe for concurrent access but MonotonicEntropy is not, ref: https://pkg.go.dev/github.com/oklog/ulid/v2#Monotonic, the panic is due likely a corruption in internal MonotonicEntropy state because of concurrent use – Wagner Riffel Jun 11 '22 at 18:37
  • Can you suggest some performant code? I found a method in this amazing post: https://qqq.ninja/blog/post/fast-threadsafe-randomness-in-go/#using-hashmaphash. Can we use it? – Fred Hors Jun 11 '22 at 18:42
  • The package suggests that you can use any io.Reader as entropy, so yes, you can use this but you will need to get ride of ulid.Monotonic, I'm not familiar with ulid to know the implication of that, in fact if you remove the wrapper around Monotonic and use crypto.Rand directly in MustNew, the race goes away; If you want to use the maphash approach, I suggest you read the implications of its concurrent use in https://pkg.go.dev/hash/maphash@go1.18.3#Hash – Wagner Riffel Jun 11 '22 at 18:51