5

I'm building a dot probe task (people have to react quickly to a dot after an image disappears) and we need to measure reaction times and show stimuli for pre-defined intervals for usage in an online therapy study.

We already decided on Javascript and we're willing to impose some constraints on users, i.e. ruling out IE etc. We can't ban Windows completely though.

I read John Resig's post on the topic and according to this we'll have to ban every browser on Windows except Firefox and Chrome.

In addition this answer advises using console.time(); as best-practice for FF and Chrome.

I have some follow-up questions, considering that Resig's post is 4 years old now and that the question above is about measuring function execution time (that means the execution of code skewing the timer is good, not bad as in our case) and not about a reaction time study.

The following similar question just asked for the "best web language" and only received the blanket recommendation to use JS (which is what a couple of published studies have done, but they don't publish technical details).

  1. Can we somehow use the better accuracy of console.time() where available? I think not, because it only returns to the console, i.e. it can't be captured in a variable.
  2. Have there been any significant changes to timing accuracy in the last 4 years? I'm comfortable banning IE for many reasons, but maybe things have changed for Safari and Opera on Windows?
  3. Should I use a second method that ties into the execution process to get another set of times for comparison / cross-validation?
Community
  • 1
  • 1
Ruben
  • 3,452
  • 31
  • 47

2 Answers2

4

ObDisclaimer - I used to write software just like like this for a University research department.

You can use performance.now() (or performance.webkitNow(), depending on browser version) to get a more accurate timestamp than Date.now(). See here for more information.

One issue to take into account however is screen refresh. Assuming a 60 Hz refresh rate, there can be a 16 ms variance in when the image actually appears on screen, depending on:

  1. whether the software is synched to the screen refresh rate (not possible in JS, AFAIK), and
  2. where the image and dot probe are relative to the top of the screen - pixels drawn at the top of the screen are drawn before those below them.

You should also consider effects caused by things like keyboard scanning intervals. On one project the researcher found significant groupings around periods that were (from memory, this is 20 years ago) a multiple of 30ms or so, which appeared to be due to how frequently the keyboard was being scanned for key presses on the Psion PDA being used for the tests.

In this case I solved the problem by building a hardware "button box" using a PIC microcontroller which could send a serial byte at 9600 bps with no scanning latency, and <2ms to get the key press from the box to the PDA.

I was considering writing a paper about the issues around screen refresh. Then I got a real job ;-) I don't know if anyone else ever studied it.

Alnitak
  • 334,560
  • 70
  • 407
  • 495
  • Thanks! Every time I want to submit my comment, you add new good stuff :-) I didn't find performance.now() while googling for some reason, this helps greatly. I'll try to dig something up on the refresh rate and keyboard scanning. The top-to-bottom rendering doesn't worry me (that much), because top-bottom positions in the dot probe will be counter-balanced, so this should even out. One thing we'll have to do is simply ask users to give us a few parameters about their system (that we can't automatically obtain), to control for it as a covariate or to look at it without biased data. – Ruben Dec 20 '12 at 13:54
  • And yes, I think other people studied it, I'm in a different lab but most of the nerds in the cognitive and general psychology lab can go on about screen refresh rates for a while. They bristle at the idea of using flatscreens too. Unfortunately that'll be something we can't do much about, I guess. No button boxes for us either. – Ruben Dec 20 '12 at 13:57
  • @Ruben I'm not convinced that top-vs-bottom would actually even out, that might need a full statistical study. In any event, I'd be cautious about quoting any differences in reaction times you find to anything more than 10ms accuracy. My work was at psy.ox.ac.uk, BTW. – Alnitak Dec 20 '12 at 13:59
  • @Ruben this link looks interesting - it talks about CRT vs LCD, screen refresh, and also the keyboard issues I mentioned - http://www.psykinematix.com/documentation/PsykinematixHelp/Timing.html – Alnitak Dec 20 '12 at 14:04
  • Yes, it's only a hope, of course, but I don't see why would it be bias (not just random error) yet? Of course we're also expecting/hoping for effects far above 10ms. Since preceding studies used top-to-bottom, not left-to-right, I think we can't change that. Psykinematix looks interesting and I'll peruse their docs, but a closed-source software that has to be downloaded has been definitely ruled out. – Ruben Dec 20 '12 at 14:08
  • and here's another study - http://www.plosone.org/article/info:doi/10.1371/journal.pone.0012792 – Alnitak Dec 20 '12 at 14:36
3

There was a good article recently about this topic from Stian Reimers & Neil Stewart.

Presentation and response timing accuracy in Adobe Flash and HTML5/JavaScript Web experiments http://www.ncbi.nlm.nih.gov/pubmed/24903687

Markus Graf
  • 533
  • 3
  • 16
  • this should really just have been a comment as there is no substantive detail in the text. – Alnitak Oct 30 '14 at 00:21
  • @Alnitak The abstract is of corse not enough but there is a full text behind this article. You can it download as enrolled student or buy it. – Markus Graf Oct 30 '14 at 16:21
  • That's even worse - a link from a free site to a _paid_ article?! – Alnitak Oct 30 '14 at 17:02
  • @Alnitak Actually the article is open access, you can read it [here](http://download.springer.com/static/pdf/848/art%253A10.3758%252Fs13428-014-0471-1.pdf?auth66=1414753312_129f203a9384e1d4086a1827aac3af28&ext=.pdf). Upon closer inspection, the authors tested HTML4 timing accuracy though! They used jQuery `$.now()` and `$.delay()` and did not talk about reflows/repaints. Still, quite happy conclusions, bodes well for the study we did. I wrote to them, maybe they can repeat their test suite using newer methods. – Ruben Oct 31 '14 at 11:01