Question Writing Comp. / Human vs. Computer Expo (@PACE NSC)

NAQT HSNCT, NAQT SSNCT, PACE NSC, HSAPQ NASAT, and NAC are discussed here.

Question Writing Comp. / Human vs. Computer Expo (@PACE NSC)

Postby ezubaric » Mon Apr 30, 2018 1:53 pm

We'll be running an exhibition game between a quiz bowl-playing artificial intelligence and a human team at PACE NSC 2018 in Reston, Virginia on June 2 (more details on venue and time as we get closer to the actual event). Unlike previous matches that used stock quiz bowl questions, we need your help to write questions that are difficult for computers.

Our research group has been working on building computers that can answer quiz bowl questions well. We’ve beat Ken Jennings, we narrowly beat a team of quiz bowl all-stars at HSNCT 2017, and Studio OUSIA soundly defeated the best possible quiz bowl team we could assemble at NIPS last year. You might think that computers are pretty smart, right?

You’d be wrong. Quiz bowl questions are supposed to be written pyramidally: harder clues at the start, easier clues at the end. The problem is that question writers don't write questions pyramidally for computers: they write questions for humans. If humans could write questions pyramidally for computers, computers would be in a whole heap of trouble, and look pretty dumb. Our goal is to offer a true test for how smart computers are.

Visit http://write.qanta.org to write questions that are difficult for computers and submit your questions to the competition!

Introduction video:
https://www.youtube.com/watch?v=DENelwdcqhk
Last edited by ezubaric on Mon Apr 30, 2018 2:41 pm, edited 1 time in total.
Jordan Boyd-Graber
UMD (College Park, MD)
UC Boulder, Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton '09
Caltech '04
Ark Math & Science '00

Write questions with the help of our question answering software:
http://aqwi-vip.umiacs.umd.edu/
User avatar
ezubaric
Rikku
 
Posts: 318
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD

Best Question Winners, Week 1

Postby ezubaric » Mon Apr 30, 2018 1:53 pm

Winners for Week #1 (04/23 - 04/30):

Here our the two winners for the first week! Both questions were able to completely trick QANTA. They use similar methods, both describe a real life action that is difficult for the model to solve by matching keywords from past questions. It is especially difficult to recognize phrases like "self-inflicted method of death", where suicide has only been previously described as "ending one's life". It is also difficult to know Cannibalism is taboo in modern society. Looking forward to more great questions in the future!

(We'll post all of the winners at http://qanta.org)


James V. McConnell encouraged flatworms to engage in this behavior in order to test whether doing this affected memory. An atypical morph of the tiger salamander is distinguished at the age of four weeks by physical changes to their jaws which increase their efficiency in this activity. The common name of the Latrodectus genus is a reference to their participation in this behavior as a mating practice, which is speculated as a way for female specimens to acquire additional nutrition. If you do this in Papua New Guinea, you acquire an elevated risk of contracting the prion disease kuru, which is spread through infected brain tissue. For 10 points, name this practice which is taboo in humans, the consumption of the flesh of one's own species.

Answer: Cannibalism




Lily Bart performs this action in The House of Mirth by Edith Wharton and Martha performs this action in The Children’s Hour by Lillian Hellman. The protagonist performs this action at the end of “A Perfect Day for Bananafish,” and Joe Keller and Willy Loman carry out this action in two separate Arthur Miller plays. Two members of the Monzano family perform this action with “ice-nine” in Cat’s Cradle, and Quentin Compson does this in the Charles River after fastening tailor’s weights to his feet. In real life, Sylvia Plath performed this action by placing her head in an oven. For the points, name this self-inflicted method of death.

Answer: Suicide
Last edited by ezubaric on Sat May 05, 2018 2:36 pm, edited 1 time in total.
Jordan Boyd-Graber
UMD (College Park, MD)
UC Boulder, Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton '09
Caltech '04
Ark Math & Science '00

Write questions with the help of our question answering software:
http://aqwi-vip.umiacs.umd.edu/
User avatar
ezubaric
Rikku
 
Posts: 318
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Postby ezubaric » Sat May 05, 2018 2:35 pm

This week's winner uses several phrases to evoke racism that computer systems cannot link to major US cities. While a human might be able to connect Gone with the Wind to Atlanta, this is much more difficult for a computer to figure out.

Opponents of this city’s public transit system have called it a scheme for “moving Africans rapidly through” this city. Oscar winner Hattie McDaniel was unable to attend the gala for the premiere of Gone with the Wind due to racial segregation laws in this city. Booker T. Washington proposed this city’s (*) namesake compromise in a speech given at the Cotton States and International Expo in, for ten points, what home of the world’s busiest airport and capital of Georgia?
Atlanta
Jordan Boyd-Graber
UMD (College Park, MD)
UC Boulder, Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton '09
Caltech '04
Ark Math & Science '00

Write questions with the help of our question answering software:
http://aqwi-vip.umiacs.umd.edu/
User avatar
ezubaric
Rikku
 
Posts: 318
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Postby ezubaric » Sun May 06, 2018 2:01 pm

We've been able to increase the cash value of our prizes. Please keep submitting your questions!

Prizes:
$10 for the best questions submitted every week
$300 for the person with the most selected questions
$200 for the person with the second most selected questions
$100 for the person with the third most selected questions
$10 for every question used in the exhibition match (only for those not eligible for the grand prize winners above)
Jordan Boyd-Graber
UMD (College Park, MD)
UC Boulder, Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton '09
Caltech '04
Ark Math & Science '00

Write questions with the help of our question answering software:
http://aqwi-vip.umiacs.umd.edu/
User avatar
ezubaric
Rikku
 
Posts: 318
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Postby ezubaric » Tue May 15, 2018 10:15 am

Congrats to last week's best question:
An aircraft carrier bearing this man's name hosted the "Mission Accomplished" speech. A city named after this man is the site of the only unicameral state legislature. A tunnel named after this man was the second tunnel built to cross the Hudson River. An organization named for this man fought against Francisco Franco's forces. A building dedicated to him was the site of the "I have a dream" speech. For 10 points, name this subject of the portrait on the penny.
Abraham Lincoln


For this week, we're raising the best question prize to $100. So get your question in before Saturday 16:00 UTC for your chance to win $100 (while still being eligible for the $500 grand prize). Tell your friends!

Note that questions don't have to be blind. You can rewrite existing questions that might have appeared elsewhere.
Jordan Boyd-Graber
UMD (College Park, MD)
UC Boulder, Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton '09
Caltech '04
Ark Math & Science '00

Write questions with the help of our question answering software:
http://aqwi-vip.umiacs.umd.edu/
User avatar
ezubaric
Rikku
 
Posts: 318
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Postby ezubaric » Fri May 18, 2018 11:40 am

We had some brief downtime this morning, but the site is back up. Chance for $100 prize for best question and $500 for best set ends in about 24 hours!
Jordan Boyd-Graber
UMD (College Park, MD)
UC Boulder, Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton '09
Caltech '04
Ark Math & Science '00

Write questions with the help of our question answering software:
http://aqwi-vip.umiacs.umd.edu/
User avatar
ezubaric
Rikku
 
Posts: 318
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD


Return to National Tournaments

Who is online

Users browsing this forum: No registered users and 1 guest