Annotate QB Questions for Fun, Practice, Profit, and Science

Packet databases and other quizbowl sites, apps, or software should be discussed here.
Post Reply
User avatar
ezubaric
Rikku
Posts: 366
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Annotate QB Questions for Fun, Practice, Profit, and Science

Post by ezubaric »

Some of you might remember that a few years ago some students and I put out a web app that folks could use to practice questions. It proved very popular and helped spawn fuller-featured platforms like Protobowl.

We used that information for a research paper that you can read here:
http://www.umiacs.umd.edu/~jbg/docs/qb_emnlp_2012.pdf

We’re still working on helping computers understand quiz bowl questions, and we’re launching a new web app that will help us gather important data and will help *you* study quiz bowl questions by annotating when characters, authors, and works appear in quiz bowl questions:

http://qb.umiacs.umd.edu/tutorial.php

Our goal is to get annotations of when people, works, and characters are mentioned in quiz bowl questions. As before, we’re offering prizes to the people who participate the most. However, we hope that everyone will be able to benefit from reading existing quiz bowl questions carefully.

We’ve been testing it out here, and it’s helped me to remember plot details in works that I never knew or had long forgotten. Annotating these questions, I’ve found, is much more effective than simply reading the question, as you need to reconstruct how the author wrote the question.

I hope that you’ll enjoy it, especially as the annotations that you provide will help us create automated techniques to help computers understand natural language, in particular, quiz bowl questions.

Please try it out; we're offering Amazon gift cards to the three most prolific users as well as for additional random users.

http://qb.umiacs.umd.edu/tutorial.php
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
Gautam
Auron
Posts: 1413
Joined: Sun Feb 11, 2007 7:28 pm
Location: Zone of Avoidance
Contact:

Re: Annotate QB Questions for Fun, Practice, Profit, and Sci

Post by Gautam »

Awesome! Thanks for linking us to your paper.
Gautam - ACF
Currently tending to the 'quizbowl hobo' persuasion.

User avatar
Wynaut
Wakka
Posts: 172
Joined: Sun Jun 30, 2013 3:34 am
Location: Ann Arbor, MI

Re: Annotate QB Questions for Fun, Practice, Profit, and Sci

Post by Wynaut »

I really like this idea, but I have a quick question. What happens if, say, something like Three Sisters is mentioned in a Chekhov tossup or, in my case, Three Soldiers is mentioned in a Dos Passos tossup, and the question actually references part of a group of title characters? Does the rule in the FAQ about "Tinker, Tailor, Soldier, Spy" apply to these situations as well?
Kenji Shimizu
University of Michigan '18
Summit Academy North High School '13
Writer, NAQT
"...the best Seychelles player in quizbowl" -Auroni Gupta

User avatar
ezubaric
Rikku
Posts: 366
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Annotate QB Questions for Fun, Practice, Profit, and Sci

Post by ezubaric »

Cape Fear wrote:What happens if, say, something like Three Sisters is mentioned in a Chekhov tossup? ... Does the rule in the FAQ about "Tinker, Tailor, Soldier, Spy" apply to these situations as well?
Yes, the rule in the FAQ also applies there (including the caveat about how it's tricky). In general, we only want exact (or nearly exact) matches. So if "Olga, Maria, and Irina" appear in the question, you can mark those as a coreference, but if only one of the sisters appears, they should not be marked as a coreference.

Ideally, we'd like to do something better with this (e.g., to encode group membership), but it's very complicated, so we're just taking this approach for now.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
Santa Claus
Wakka
Posts: 180
Joined: Fri Aug 23, 2013 10:58 pm

Re: Annotate QB Questions for Fun, Practice, Profit, and Sci

Post by Santa Claus »

I think it would be a pretty cool thing if you could rearrange annotated terms in the side menu, so that if you wanted to change which group something/multiple somethings were in, you wouldn't have to reselect everyone of them; you could just drag them around.
Kevin Wang
Arcadia High School 2015
Amherst College 2019

2018 PACE NSC Champion
2019 PACE NSC Champion

Post Reply