0

I am doing research using Adaptive Comparative Judgement where I present the user two things and let them choose which one is more difficult for them to understand. For now I am thinking of testing 10 things but it will be extended further.

I have been looking at ways to sort the data in relation to the difficulty of each item and cam across Bradley-Terry and Thurstone's models, but they are too mathematical for me to understand so I am wondering if anyone has done a similar thing and knows how to go about writing a code to do this type of sorting.

To give an example, on this website https://nomoremarking.com/demo1 there is a colour test and it asks the user to choose between two colours the one with the darker shade and automatically sorts the results. That is the sorting I am interested in.
Thanks!

Peter
  • 121
  • 1
  • 9
  • This is maybe a better question for cogsci.stackexchange.com or possibly statistics - your question is really more about the underlying science than how to program it, so may get better answers there. – Crashworks Mar 10 '16 at 23:07
  • Thanks, I will ask there as well. – Peter Mar 10 '16 at 23:21
  • Possible duplicate of [Sorting a set of objects by a user's preference](http://stackoverflow.com/questions/31298996/sorting-a-set-of-objects-by-a-users-preference) – m69's been on strike for years Mar 11 '16 at 00:57

1 Answers1

1

It seems to me you want to minimize the amount of comparisons the user has to make. As a baseline, you can use a insertion sort variant that decides the best place to insert a new item in O(log n). Here's an example in Python to sort integers:

import bisect, random

class InputCompare(object):
    def __init__(self, value):
        self.value = value
    def __lt__(self, other):
        print 'Is', other.value, 'greater than', self.value, '? (y/n)',
        return raw_input().lower() == 'y'

A = range(8)
random.shuffle(A)

print 'Sorting:', A

B = []
for x in A:
    bisect.insort(B, InputCompare(x))

print [x.value for x in B]

This program asks questions about pairs of numbers to determine their total order, like this:

Sorting: [4, 7, 2, 6, 1, 5, 0, 3]
Is 4 greater than 7 ? (y/n) n
Is 7 greater than 2 ? (y/n) y
Is 4 greater than 2 ? (y/n) y
Is 4 greater than 6 ? (y/n) n
Is 7 greater than 6 ? (y/n) y
Is 6 greater than 1 ? (y/n) y
Is 4 greater than 1 ? (y/n) y
Is 2 greater than 1 ? (y/n) y
Is 4 greater than 5 ? (y/n) n
Is 7 greater than 5 ? (y/n) y
Is 6 greater than 5 ? (y/n) y
Is 5 greater than 0 ? (y/n) y
Is 2 greater than 0 ? (y/n) y
Is 1 greater than 0 ? (y/n) y
Is 4 greater than 3 ? (y/n) y
Is 1 greater than 3 ? (y/n) n
Is 2 greater than 3 ? (y/n) n
[0, 1, 2, 3, 4, 5, 6, 7]
Juan Lopes
  • 10,143
  • 2
  • 25
  • 44