Welcome to MTurk Crowd

Work on Amazon Mechanical Turk, learn from the best, and have fun doing it. Join the crowd today!

Sign Up

Rep Present Annotating reasons in German Amazon reviews

Discussion in 'Requesters' started by UKP-Lab, Dec 9, 2016.

  1. UKP-Lab

    UKP-Lab
    Expand Collapse
    New Member

    • Requester
    Joined:
    Dec 4, 2016
    Messages:
    7
    Likes:
    1
    Dear Turkers,

    we are the UKP-Lab of the Technical University Darmstadt in Germany and currently we run a study regarding Amazon reviews on Mechanical Turk.

    Participants need to label the general opinion an Amazon reviewer states in his or her review. The reviews are usually about 10 to 20 lines long, and they are all in German, so fluent knowledge of German is required for the task.

    We estimate one HIT to take one minute and pay 0.12$ per hit.

    You can find the hits at https://www.mturk.com/mturk/preview?groupId=39393HGTI5QMIY6JHIT1M5IIGWJEOF

    Thanks.
     
    clickhappier likes this.
  2. electrolyte

    electrolyte
    Expand Collapse
    The Ghost of MTurk Past

    • Administrator
    Joined:
    Jan 10, 2016
    Messages:
    18,844
    Likes:
    45,120
    Title: Meinungen in Produktbewertungen identifizieren. | PANDA
    Requester: Maria Sukhareva, TU Darmstadt [A3TC8ETMZEPR0K] (TO)
    TO Ratings:
    [​IMG]
    ☢☢☢☢ 1.25 Communicativity
    ☢☢☢☢☢ 2.90 Generosity
    ☢☢☢☢ 4.50 Fairness
    ☢☢☢☢☢ 2.00 Promptness
    Number of Reviews: 16
    (Submit a new TO rating for this requester)

    Description: Identifizieren Sie die allgemeine Meinung einer Produktbewertung. Fließendes Deutsch erforderlich! (Fluent German is required for this task!)
    Time: 15 minutes
    Hits Available: 4
    Reward: $0.12
    Qualifications: HIT approval rate (%) is not less than 95; Location is US
     
    Collapse Signature Expand Signature
  3. RippedWarrior

    RippedWarrior
    Expand Collapse
    New Member

    Joined:
    Jan 20, 2016
    Messages:
    7
    Likes:
    6
    I find it curious that you've excluded workers from Germany from completing this task. Anyway, good luck with your project!
     
  4. electrolyte

    electrolyte
    Expand Collapse
    The Ghost of MTurk Past

    • Administrator
    Joined:
    Jan 10, 2016
    Messages:
    18,844
    Likes:
    45,120
    Hmm, I wonder if that was accidental? @UKP-Lab@UKP-Lab, if you remove the "Location is US" qualification from your HIT, you will allow workers from any country to access your HIT. Is it mandatory for your project that you only use American workers?
     
    Collapse Signature Expand Signature
  5. UKP-Lab

    UKP-Lab
    Expand Collapse
    New Member

    • Requester
    Joined:
    Dec 4, 2016
    Messages:
    7
    Likes:
    1
    Thanks for the feedback. You both are right, including workers from more countries definitely makes sense for this task. We will rework the qualifications in the next few days to allow more workers to complete this task.
     
  6. UKP-Lab

    UKP-Lab
    Expand Collapse
    New Member

    • Requester
    Joined:
    Dec 4, 2016
    Messages:
    7
    Likes:
    1
  7. UKP-Lab

    UKP-Lab
    Expand Collapse
    New Member

    • Requester
    Joined:
    Dec 4, 2016
    Messages:
    7
    Likes:
    1
    The problem is that the command-line tools that Amazon provides to requesters do not permit the creation of HITs that are limited to workers in a set of countries—you can either set the HITs to be available to workers in any country, or set a single country. We decided to restrict our HITs by country because some workers were submitting work even though they obviously couldn't speak German. We chose the US because Amazon no longer allows sign-ups from Germany, whereas there are a lot of fluent German speakers living in the US.

    We have since worked around Amazon's one-country limitation by using a different tool to configure our HITs.
     
  8. UKP-Lab

    UKP-Lab
    Expand Collapse
    New Member

    • Requester
    Joined:
    Dec 4, 2016
    Messages:
    7
    Likes:
    1
  9. UKP-Lab

    UKP-Lab
    Expand Collapse
    New Member

    • Requester
    Joined:
    Dec 4, 2016
    Messages:
    7
    Likes:
    1
  10. electrolyte

    electrolyte
    Expand Collapse
    The Ghost of MTurk Past

    • Administrator
    Joined:
    Jan 10, 2016
    Messages:
    18,844
    Likes:
    45,120
    Collapse Signature Expand Signature
  11. UKP-Lab

    UKP-Lab
    Expand Collapse
    New Member

    • Requester
    Joined:
    Dec 4, 2016
    Messages:
    7
    Likes:
    1
  12. electrolyte

    electrolyte
    Expand Collapse
    The Ghost of MTurk Past

    • Administrator
    Joined:
    Jan 10, 2016
    Messages:
    18,844
    Likes:
    45,120
    Title: Meinungen in Rezensionen identifizieren | PANDA
    Worker: Preview | Accept | Requester
    Requester: Tristan Miller, TU Darmstadt [A3TC8ETMZEPR0K] (Contact)
    TO 1: [Pay: 3.09] [Fast: 2.00] [Comm: 1.25] [Fair: 4.56] [Reviews: 18] [ToS: 0]
    TO 2:
    Not Available
    Description: Identifizieren Sie Argumente für oder gegen die Meinung des Rezensenten über ein Produkt. Fließendes Deutsch erforderlich! (Fluent German is required for this task!)
    Time: 15 minutes
    HITs Available: 953
    Reward: $0.15
    Qualifications: Location is one of: AT, CH, DE, LI, LU, US; HIT approval rate (%) is not less than 95;
     
    Collapse Signature Expand Signature