0

I'm interested in migrating from psychtoolbox to shady for my stimulus presentation. I looked through the online docs, but it is not very clear to me how to replicate what I'm currently doing in matlab in shady.

What I do is actually very simple. For each trial,

  1. I load from disk a single image (I do luminance linearization off-line), which contains all the frames I plan to display in that trial (the stimulus is 1000x1000 px, and I present 25 frames, hence the image is 5000x5000px. I only use BW images, so I have a single int8 value per pixel).

  2. I transfer the entire image from the CPU to the GPU

  3. At some point (externally controlled) I copy the first frame to the video buffer and present it

  4. At some other point (externally controlled) I trigger the presentation of the remaining 24 frames (copying the relevant part of the image to video buffer for each video frame, and then calling flip()).

  5. The external control happens by having another machine communicate with the stimulus presentation code over TCP/IP. After the control PC sends a command to the presentation PC and this is executed, the presentation PC needs to send back an acknowledgement message to the control PC. I need to send three ACK messages, one when the first frame appears on screen, one when the 2nd frame appears on screen, and one when the 25th frame appears on screen (this way the control PC can easily verify if a frame has been dropped). In matlab I do this by calling the blocking method flip() to present a frame, and when it returns I send the ACK to the control PC.

That's it. How would I do that in shady? Is there an example that I should look at?

jez
  • 14,867
  • 5
  • 37
  • 64
cq70
  • 21
  • 6
  • 1
    The bit about sending ACKs is a really a separate (orthogonal) question. The short answer is to install an `AnimationCallback` as described at https://shady.readthedocs.io/en/release/auto/MakingPropertiesDynamic.html#the-animate-method and https://shady.readthedocs.io/en/release/source/Shady.html#Shady.Stimulus.SetAnimationCallback . The animation callback is evaluated immediately after flip, and you could implement your own stateful `w.framesCompleted`-dependent logic there. I can give more details/recommendations if you ask a separate question, but give it a first attempt before you do. – jez May 16 '19 at 17:26
  • Thanks jez, I'll give it a try. – cq70 May 16 '19 at 20:22

1 Answers1

0

The places to look for this information are the docstrings of Shady.Stimulus and Shady.Stimulus.LoadTexture, as well as the included example script animated-textures.py.

Like most things Python, there are multiple ways to do what you want. Here's how I would do it:

w = Shady.World()
s = w.Stimulus( [frame00, frame01, frame02, ...], multipage=True )

where each frameNN is a 1000x1000-pixel numpy array (either floating-point or uint8).

Alternatively you can ask Shady to load directly from disk:

s = w.Stimulus('trial01/*.png', multipage=True)

where directory trial01 contains twenty-five 1000x1000-pixel image files, named (say) 00.png through 24.png so that they get sorted correctly. Or you could supply an explicit list of filenames.

Either way, whether you loaded from memory or from disk, the frames are all transferred to the graphics card in that call. You can then (time-critically) switch between them with:

s.page = 0  # or any number up to 24 in your case

Note that, due to our use of the multipage option, we're using the "page" animation mechanism (create one OpenGL texture per frame) instead of the default "frame" mechanism (create one 1000x25000 OpenGL texture) because the latter would exceed the maximum allowable dimensions for a single texture on many graphics cards. The distinction between these mechanisms is discussed in the docstring for the Shady.Stimulus class as well as in the aforementioned interactive demo:

python -m Shady demo animated-textures

To prepare the next trial, you might use .LoadPages() (new in Shady version 1.8.7). This loops through the existing "pages" loading new textures into the previously-used graphics-card texture buffers, and adds further pages as necessary:

s.LoadPages('trial02/*.png')

Now, you mention that your established workflow is to concatenate the frames as a single 5000x5000-pixel image. My solutions above assume that you have done the work of cutting it up again into 1000x1000-pixel frames, presumably using numpy calls (sounds like you might be doing the equivalent in Matlab at the moment). If you're going to keep saving as 5000x5000, the best way of staying in control of things might indeed be to maintain your own code for cutting it up. But it's worth mentioning that you could take the entirely different strategy of transferring it all in one go:

s = w.Stimulus('trial01_5000x5000.png', size=1000)

This loads the entire pre-prepared 5000x5000 image from disk (or again from memory, if you want to pass a 5000x5000 numpy array instead of a filename) into a single texture in the graphics card's memory. However, because of the size specification, the Stimulus will only show the lower-left 1000x1000-pixel portion of the array. You can then switch "frames" by shifting the carrier relative to the envelope. For example, if you were to say:

s.carrierTranslation = [-1000, -2000]

then you would be looking at the frame located one "column" across and two "rows" up in your 5x5 array.

As a final note, remember that you could take advantage of Shady's on-the-fly gamma-correction and dithering–they're happening anyway unless you explicitly disable them, though of course they have no physical effect if you leave the stimulus .gamma at 1.0 and use integer pixel values. So you could generate your stimuli as separate 1000x1000 arrays, each containing unlinearized floating-point values in the range [0.0,1.0], and let Shady worry about everything beyond that.

jez
  • 14,867
  • 5
  • 37
  • 64