For example, you might rate 'Christianity/Religion' as a 7, because Christianity is a very typical religion, 'Humanism/Religion' as a 3, because Humanism is a somewhat unlikely religion, and 'Pastafarianism/Religion' as a 1, because Pastafarianism isn't like other religions at all.This afternoon I recieved the following angry email:
Cheer.Greetings from Amazon Mechanical Turk, An Amazon Mechanical Turk user is contacting you from the Amazon Mechanical Turk website. Please review the message below and respond to the message as you see fit. Sincerely, Amazon Mechanical Turk http://requester.mturk.com 1200 12TH AVE South, SUITE 1200 SEATTLE, WA 98144-2734 USA Message from Susan XXXXXX (XXXXXXXXXXXX@msn.com) --------------------------------- Customer ID: A3VXXXXXXXXXXX Hello, I cannot in good conscience work on your hits, although I am one of the higher educated monkeys around here with a high acceptance %. Who are you to decide Pastafarianism rates a 1 on the religion scale? That is bordering on blasphemy, and showing extreme bias against His Noodly Appendage. May he forgive you. Ramen.
Yesterday I spent a good couple of hours trawling through my list of categories & exemplars, looking for spelling mistakes and capitalization errors. See, the plan is to use the word lists I got from that first experiment as input to the next two — giving Turkers words and asking them to label them with categories, or giving them word/category pairs and asking them to rate the typicality. Useful, hopefully, but very dependent on the quality of data I put in.
Anyway, manually going through and correcting the spelling mistakes and general weirdness of a couple hundred well-intentioned Turkers really opened my eyes to a whole new world of spelling. A test! Quick, how do you spell the seed on a “s_____ seed bun?”
Yeah, ‘sesame’. But are you sure you wouldn’t write it ‘seasame?’ Or perhaps ‘seaseme?’ Maybe even ‘seesamie?’
Anyway, enough carping. I actually learned a few things — that a ‘Cuba libre’ and a ‘Harvey Wallbanger’ are cocktails, for instance — and had to laugh out loud at general wise-assery a couple of times. To some anonymous turker ‘sex roleplaying’ is the first thing that comes to mind for ‘games’; for ‘craft’ someone wrote down ‘warcraft’ and ‘starcraft’’. Needless to say, I had to cull a few exemplars from the list.