0

I would like to limit a group of MTURK HITs such that if a worker has taken any previous HIT included in the group, he or she cannot take any future hits included in the group.

I have seen two distinct methods of doing this described here on StackOverflow.

  1. Use qualifications. https://stackoverflow.com/a/16415447/364966

  2. Have the HIT check with a function on your server and if a worker has taken a previous HIT from the group, politely inform the worker that the hit won't be available. https://stackoverflow.com/a/3782847/364966 and https://stackoverflow.com/a/13851920/364966

One poster on StackOverflow commented that "I used to do this with qualifications, but found that it really limited participation." https://stackoverflow.com/a/3782847/364966

Is it better to use method #1 or method #2? Or, are both methods equally viable?

Community
  • 1
  • 1
VikR
  • 4,818
  • 8
  • 51
  • 96

1 Answers1

2

The easiest thing to do is just have one ExternalQuestion HIT that you host on your own server, wherein you serve up different content to each worker. In lieu of your own server, you can do this with javascript as well but the reliability might be reduced and you have to design it well to accommodate workers viewing the HIT in "preview" mode.

So, I would not do either of the suggested strategies you point to (and this is coming as the person who posted the answer #1 that you refer to in your question.)

Thomas
  • 43,637
  • 12
  • 109
  • 140
  • I've been looking into this and learning a bit more. It appears that when the external HIT is previewed by the MTURK worker, MTURK does not send the workerID to the external server. If you have multiple HITs operational, what do you show workers who are previewing the survey—i.e. how do you avoid previewing one survey, and then sending them to a different survey when they accept the hit? Thanks in advance for any thoughts/info. – VikR Jun 17 '15 at 01:06
  • You have to handle that somehow. You can use javascript to prevent them from doing anything until they accept. That's what I do. – Thomas Jun 17 '15 at 05:36
  • Do they get upset if they accept a hit and are then told they can't take it? – VikR Jun 17 '15 at 07:40
  • Yes, potentially. You can always ask for feedback from workers on http://turkernation.com/ before you launch something to see what they think would work best. – Thomas Jun 17 '15 at 07:57
  • I hadn't yet heard of turkernation.com. Thanks for telling me about it. – VikR Jun 17 '15 at 08:33