0
model.compile(optimizer="adam", run_eagerly=False, metrics=[mm.RecallAt(10), mm.NDCGAt(10)])
model.fit(train, validation_data=valid, batch_size=4096, epochs=3)

ValueError                                Traceback (most recent call last)

C:\ProgramData\Anaconda3\lib\site-packages\merlin\models\loader\backend.py in _shuffle_indices(self)
    326     @annotate("_shuffle_indices", color="darkgreen", domain="nvt_python")
    327     def _shuffle_indices(self):
--> 328         generate_local_seed(self.global_rank, self.global_size)
    329         if self.seed_fn:
    330             new_seed = self.seed_fn()

C:\ProgramData\Anaconda3\lib\site-packages\merlin\core\dispatch.py in generate_local_seed(global_rank, global_size)
    647         cp.random.seed(seeds[global_rank].get())
    648     else:
--> 649         seeds = random_state.randint(0, 2**32, size=global_size)
    650     return seeds[global_rank]
    651 

mtrand.pyx in numpy.random.mtrand.RandomState.randint()

_bounded_integers.pyx in numpy.random._bounded_integers._rand_int32()

ValueError: high is out of bounds for int32

I was trying to train the model usng Nvidia Merlin

Expecting a model iteration step.

Progman
  • 16,827
  • 6
  • 33
  • 48
S sushanth
  • 1
  • Is this occurring on Windows? – Zois Tasoulas Nov 10 '22 at 01:11
  • Yes it is Window 11 machine, 64bit processor – S sushanth Nov 10 '22 at 07:45
  • I think the underlying issue is discussed here: https://stackoverflow.com/questions/36278590/numpy-array-dtype-is-coming-as-int32-by-default-in-a-windows-10-64-bit-machine . A possible solution would be passing dtype=np.int64 to the randint call. So that int64 numbers are used in all platforms. – Zois Tasoulas Nov 10 '22 at 21:10

1 Answers1

0

Hello here the solution which worked for me:

Go to your folder AppData\Roaming\Python\Python_version_here\site-packages\merlin\core

In the file dispatch.py change the function generate_local_seed as:

def generate_local_seed(global_rank, global_size):
    random_state = get_random_state()
    if cp:
        seeds = random_state.tomaxint(size=global_size)
        cp.random.seed(seeds[global_rank].get())
    else:
        seeds = random_state.randint(0, 2**31-1, size=global_size)
    return seeds[global_rank]

Here we declare the max value now as 2**31-1

Michele
  • 1
  • 1