11

I need to create a poll that is to create a ranking list of items in order of how good they are. I intend to show each user two items together and make them choose one which they think is better, and repeat the process multiple times over. It is sort of similar to what you could see in the Social Network movie. How should I be ranking the items based on the received answers?

casperOne
  • 73,706
  • 19
  • 184
  • 253
ThePiachu
  • 8,695
  • 17
  • 65
  • 94
  • 1
    Similar? http://stackoverflow.com/questions/3937218/comparison-based-ranking-algorithm – Lior Kogan Oct 14 '11 at 14:08
  • 1
    Similar? http://stackoverflow.com/questions/7706573/how-is-this-comparison-ranking-algorithm-called – Lior Kogan Oct 14 '11 at 14:10
  • Are the items of similar type or not. For instance, are you comparing types of cars (Ferrari vs. Aston Martin) or are you intending to rank disparate objects (cars vs. GI Joe toys). If you're comparing items in the same "category" (transportation) you can just record # votes for each one and order by that as a super simple way of ranking. If you intend to compare disparate items, then it gets more complicated. – FloppyDisk Oct 14 '11 at 14:33
  • @FloppyDisk the items are of the same type (images), but the problem is that not all of them might be displayed the same amount of times, and the rating is highly subjective (which distorted image looks closer to the original). – ThePiachu Oct 14 '11 at 15:48
  • @ThePiachu I wrote a comment and it just got too long so I'm turning it into an answer. – FloppyDisk Oct 14 '11 at 16:01

3 Answers3

11

Look at the ELO chess rating system if you want something fancy.

hugomg
  • 68,213
  • 24
  • 160
  • 246
  • 1
    Not sure why this got a downvote. It's not *that* complicated - it can be done in C# with [50 lines of code](http://lukedurrant.com/2010/11/c-elo-rating-class-used-on-facemash-as-seen-in-the-social-network-movie/) - and it seems applicable to the OP's problem. – ladenedge Oct 14 '11 at 14:12
4

I think you can use the Elo Algorithm which was used to rank chess players and was created by Professor Arpad Elo. You can read more abot this algorithm on this Wikipedia Page

It was also used by Mark Zuckerburg in making Facemash. It was a site on which people can rate girls upon the bases of there hotness. If you have watched the picture "The Social Network" you would be knowing about this. If you want a practical use of this algorithm You can visit this page. If you know PHP you can easily change the look of the index.php (homepage). Yoou can replace the images of girls with those of what ever you want to rank

John Mayer
  • 51
  • 1
  • 6
1

My assumptions, based on your comment, proceed as follows.

  1. You have a collection of original images.
  2. You have x copies of a given original image where x >= 1.
  3. For a given picture, users will only see copies of that image and not images from another original picture.

For clarification, do you intend to set it up so user's see two random distorted images, or will you associate distorted images with specific original images?

If you intend to associate specific distorted images to specific original images, I think my idea will work.

  1. For each vote on a distorted image associated with an original item, add 1 to the total # of votes for that original.
  2. Add 1 to the # of yes votes on the distorted image chosen.
  3. Add 1 to the # of not votes on the distorted image not chosen.
  4. Add 1 to the the total # of votes on each distorted image.
  5. Normalize the "image rank" with image_rank = (# total_distorted_image_votes / # total votes for original image) * 100
  6. Normalize the "yes" rank with yes_rank = (#total_distorted_image_yes_votes / #total_distorted_image_votes)*100
  7. Sort by either image_rank or yes_rank. Using yes_rank will reward images with large percentages of yes votes whereas using image_rank will reward images that appeared a lot.

You can expand beyond this to start ranking original image "groups" as well if you have a counter for total overall votes. You just normalize those (image_votes / total_votes) * 100 and sort it. Then you'd get a "ranking" of which images appear(ed) the most.

FloppyDisk
  • 1,693
  • 16
  • 25
  • I intend to show the user the original image, and two distorted images. The user is to choose which of the two DISTORTED images looks closer to the image, which makes it a bit of a harder problem. – ThePiachu Oct 14 '11 at 19:13
  • If you only have one original image, then my idea works perfectly well. If you have multiple original images, the idea still works. You just need to create a reference between the original image and the distorted images displayed that tracks the # of yes votes and the total # of votes for each distorted image when it displays with an original. You could think of this like a graph and implement the idea that way. – FloppyDisk Oct 14 '11 at 19:36