My shuffle algorithm crashes when more trials than objects

Question

I am trying to make an experiment where a folder is scanned for images. For each trial, a target is shown and some (7) distractor images. Afterward, in half the trials people are shown the target image and in the other half, they are shown an image that wasn't in the previous display.

My current code sort of works, but only if there are fewer trials than objects:

repeats = 20

# Scan dir for images
jpgs = []

for path, dirs, files in os.walk(directory):
    for f in files:
        if f.endswith('.jpg'):
            jpgs.append(f)

# Shuffle up jpgs
np.random.shuffle(jpgs)

# Create list with target and probe object, Half random, half identical
display = []
question = []
sameobject = []
position = np.repeat([0,1,2,3,4,5,6,7], repeats)
for x in range(1,(repeats*8)+1):
    display.append(jpgs[x])
    if x % 2 == 0:
        question.append(jpgs[-x])
        sameobject.append(0)
    else:
        question.append(jpgs[x])
        sameobject.append(1)

# Concatonate objects together
together = np.c_[display,question,position,sameobject] 
np.random.shuffle(together)

for x in together:
        # Shuffle and set image
        np.random.shuffle(jpgs)
        myList = [i for i in jpgs if i != together[trial,0]]
        myList = [i for i in myList if i != together[trial,1]]

        # Set correct image for target
        myList[int(together[trial,2])] = together[trial,0]

First of all, I am aware that this is horrible code. But it gets the job done coarsely. With 200 jpgs and a repeat of 20, it works. If repeat is set to 30 it crashes.

Here is an example with repeat too high:

  File "H:\Code\Stims\BetaObjectPosition.py", line 214, in <module>
    display.append(jpgs[x])
IndexError: list index out of range

Is there a way to update my code in a way that allows more trials while all objects are used as evenly as possible (one object should not be displayed 3 times while another is displayed 0) over an entire experiment?

Full, reproducible example

Bonus points if anyone can see an obvious way to balance the way the 7 distractor images are selected too.

Thanks for taking your time to read this. I hope you can help me onwards.

Answer 1

The solution that changes your code the least should be to change each call of jpgs[x] to jpgs[x % len(jpgs)]¹. This should get rid of the IndexError; it basically wraps the list index "around the edges", making sure it's never to large. Although I'm not sure how it will interact with the jpgs[-x] call.

An alternative would be to implement a class that produces a longer sequence of objects from a shorter one. Example:

from random import shuffle


class InfiniteRepeatingSequence(object):
  def __init__(self, source_list):
      self._source = source_list
      self._current = []

  def next(self):
      if len(self._current) == 0:
          # copy the source
          self._current = self._source[:]

          shuffle(self._current)

      # get and remove an item from a list
      return self._current.pop()

This class repeats the list indefinitely. It makes sure to use each element once before re-using the list. It can easily be turned into an iterator (try changing next to __next__). But be careful since the class above produces an infinite sequence of elements.

---

¹ See "How does % work in Python?" for an explanation about the modulo operator.

Edit: Added link to modulo question.