So this morning I threw a new load of data collection experiments up on Mechanical Turk. Among other things, I'm giving people long lists of pairs of words (i.e. "apple/fruit") and asking them to rank how typical the first word is among members of the second. In my instructions I give an example, using religion:

For example, you might rate 'Christianity/Religion' as a 7, because Christianity is a very typical religion, 'Humanism/Religion' as a 3, because Humanism is a somewhat unlikely religion, and 'Pastafarianism/Religion' as a 1, because Pastafarianism isn't like other religions at all.

This afternoon I recieved the following angry email:

Greetings from Amazon Mechanical Turk,

An Amazon Mechanical Turk user is contacting you from the Amazon
Mechanical Turk website.  Please review the message below and respond to
the message as you see fit.

Sincerely,
Amazon Mechanical Turk
http://requester.mturk.com
1200 12TH AVE South, SUITE 1200
SEATTLE, WA 98144-2734 USA
Message from Susan XXXXXX (XXXXXXXXXXXX@msn.com)
---------------------------------
Customer ID: A3VXXXXXXXXXXX
Hello,
 I cannot in good conscience work on your hits, although I am one of the higher educated 
monkeys around here with a high acceptance %.

 Who are you to decide Pastafarianism rates a 1 on the religion scale? That is bordering 
on blasphemy, and showing extreme bias against His Noodly Appendage. May he forgive 
you. Ramen.

Cheer.

Yesterday I spent a good couple of hours trawling through my list of categories & exemplars, looking for spelling mistakes and capitalization errors. See, the plan is to use the word lists I got from that first experiment as input to the next two — giving Turkers words and asking them to label them with categories, or giving them word/category pairs and asking them to rate the typicality. Useful, hopefully, but very dependent on the quality of data I put in.

Anyway, manually going through and correcting the spelling mistakes and general weirdness of a couple hundred well-intentioned Turkers really opened my eyes to a whole new world of spelling. A test! Quick, how do you spell the seed on a “s_____ seed bun?”

.

..

…

Yeah, ‘sesame’. But are you sure you wouldn’t write it ‘seasame?’ Or perhaps ‘seaseme?’ Maybe even ‘seesamie?’

Anyway, enough carping. I actually learned a few things — that a ‘Cuba libre’ and a ‘Harvey Wallbanger’ are cocktails, for instance — and had to laugh out loud at general wise-assery a couple of times. To some anonymous turker ‘sex roleplaying’ is the first thing that comes to mind for ‘games’; for ‘craft’ someone wrote down ‘warcraft’ and ‘starcraft’’. Needless to say, I had to cull a few exemplars from the list.