Question Writing Comp. / Human vs. Computer Expo (@PACE NSC)

NAQT HSNCT, NAQT SSNCT, PACE NSC, HSAPQ NASAT, and NAC are discussed here.
Post Reply
User avatar
ezubaric
Rikku
Posts: 357
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Question Writing Comp. / Human vs. Computer Expo (@PACE NSC)

Post by ezubaric » Mon Apr 30, 2018 1:53 pm

We'll be running an exhibition game between a quiz bowl-playing artificial intelligence and a human team at PACE NSC 2018 in Reston, Virginia on June 2 (more details on venue and time as we get closer to the actual event). Unlike previous matches that used stock quiz bowl questions, we need your help to write questions that are difficult for computers.

Our research group has been working on building computers that can answer quiz bowl questions well. We’ve beat Ken Jennings, we narrowly beat a team of quiz bowl all-stars at HSNCT 2017, and Studio OUSIA soundly defeated the best possible quiz bowl team we could assemble at NIPS last year. You might think that computers are pretty smart, right?

You’d be wrong. Quiz bowl questions are supposed to be written pyramidally: harder clues at the start, easier clues at the end. The problem is that question writers don't write questions pyramidally for computers: they write questions for humans. If humans could write questions pyramidally for computers, computers would be in a whole heap of trouble, and look pretty dumb. Our goal is to offer a true test for how smart computers are.

Visit http://write.qanta.org to write questions that are difficult for computers and submit your questions to the competition!

Introduction video:
https://www.youtube.com/watch?v=DENelwdcqhk
Last edited by ezubaric on Mon Apr 30, 2018 2:41 pm, edited 1 time in total.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
ezubaric
Rikku
Posts: 357
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Best Question Winners, Week 1

Post by ezubaric » Mon Apr 30, 2018 1:53 pm

Winners for Week #1 (04/23 - 04/30):

Here our the two winners for the first week! Both questions were able to completely trick QANTA. They use similar methods, both describe a real life action that is difficult for the model to solve by matching keywords from past questions. It is especially difficult to recognize phrases like "self-inflicted method of death", where suicide has only been previously described as "ending one's life". It is also difficult to know Cannibalism is taboo in modern society. Looking forward to more great questions in the future!

(We'll post all of the winners at http://qanta.org)

James V. McConnell encouraged flatworms to engage in this behavior in order to test whether doing this affected memory. An atypical morph of the tiger salamander is distinguished at the age of four weeks by physical changes to their jaws which increase their efficiency in this activity. The common name of the Latrodectus genus is a reference to their participation in this behavior as a mating practice, which is speculated as a way for female specimens to acquire additional nutrition. If you do this in Papua New Guinea, you acquire an elevated risk of contracting the prion disease kuru, which is spread through infected brain tissue. For 10 points, name this practice which is taboo in humans, the consumption of the flesh of one's own species.

Answer: Cannibalism

Lily Bart performs this action in The House of Mirth by Edith Wharton and Martha performs this action in The Children’s Hour by Lillian Hellman. The protagonist performs this action at the end of “A Perfect Day for Bananafish,” and Joe Keller and Willy Loman carry out this action in two separate Arthur Miller plays. Two members of the Monzano family perform this action with “ice-nine” in Cat’s Cradle, and Quentin Compson does this in the Charles River after fastening tailor’s weights to his feet. In real life, Sylvia Plath performed this action by placing her head in an oven. For the points, name this self-inflicted method of death.

Answer: Suicide
Last edited by ezubaric on Sat May 05, 2018 2:36 pm, edited 1 time in total.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
ezubaric
Rikku
Posts: 357
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric » Sat May 05, 2018 2:35 pm

This week's winner uses several phrases to evoke racism that computer systems cannot link to major US cities. While a human might be able to connect Gone with the Wind to Atlanta, this is much more difficult for a computer to figure out.
Opponents of this city’s public transit system have called it a scheme for “moving Africans rapidly through” this city. Oscar winner Hattie McDaniel was unable to attend the gala for the premiere of Gone with the Wind due to racial segregation laws in this city. Booker T. Washington proposed this city’s (*) namesake compromise in a speech given at the Cotton States and International Expo in, for ten points, what home of the world’s busiest airport and capital of Georgia?
Atlanta
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
ezubaric
Rikku
Posts: 357
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric » Sun May 06, 2018 2:01 pm

We've been able to increase the cash value of our prizes. Please keep submitting your questions!

Prizes:
$10 for the best questions submitted every week
$300 for the person with the most selected questions
$200 for the person with the second most selected questions
$100 for the person with the third most selected questions
$10 for every question used in the exhibition match (only for those not eligible for the grand prize winners above)
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
ezubaric
Rikku
Posts: 357
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric » Tue May 15, 2018 10:15 am

Congrats to last week's best question:
An aircraft carrier bearing this man's name hosted the "Mission Accomplished" speech. A city named after this man is the site of the only unicameral state legislature. A tunnel named after this man was the second tunnel built to cross the Hudson River. An organization named for this man fought against Francisco Franco's forces. A building dedicated to him was the site of the "I have a dream" speech. For 10 points, name this subject of the portrait on the penny.
Abraham Lincoln
For this week, we're raising the best question prize to $100. So get your question in before Saturday 16:00 UTC for your chance to win $100 (while still being eligible for the $500 grand prize). Tell your friends!

Note that questions don't have to be blind. You can rewrite existing questions that might have appeared elsewhere.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
ezubaric
Rikku
Posts: 357
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric » Fri May 18, 2018 11:40 am

We had some brief downtime this morning, but the site is back up. Chance for $100 prize for best question and $500 for best set ends in about 24 hours!
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
ezubaric
Rikku
Posts: 357
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric » Wed May 30, 2018 11:51 am

We're looking for players for the team to take on OUSIA on questions that are designed to be difficult for the computer:

https://docs.google.com/forms/d/e/1FAIp ... sp=sf_link
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
ezubaric
Rikku
Posts: 357
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric » Sat Jun 02, 2018 1:35 pm

We have a strong human team assembled, we have an outstanding set of computer-stumping questions, and we have our uplink with the computer system with the Japanese quiz bowl robot completed. Looking forward to seeing everyone in Regency Ballroom tonight at 8pm in Reston!
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
ezubaric
Rikku
Posts: 357
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric » Sat Jun 02, 2018 1:37 pm

We have a strong human team assembled, we have an outstanding set of computer-stumping questions, and we have our uplink with the computer system with the Japanese quiz bowl robot completed. Looking forward to seeing everyone in Regency Ballroom tonight at 8pm in Reston!
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
Galstaff, Sorceror of Light
Forums Staff: Moderator
Posts: 2174
Joined: Sat Aug 18, 2007 5:40 pm
Location: Richmond, VA

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by Galstaff, Sorceror of Light » Sat Jun 02, 2018 5:32 pm

This event is being moved to Grand Ballroom D. It will still take place at 8 PM.
Sarah Angelo,
Maggie L. Walker Governor's School 2010 / UVA 2014 / VCU School of Education 2016
Young Reader Literature and Middle School Literature, Pop Music, & Mythology Subject Editor, NAQT
2019 Tournament Director, PACE
Webmaster, ACF
Perpetually trying to retire from tournament directing.

User avatar
ezubaric
Rikku
Posts: 357
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric » Sat Jun 02, 2018 10:23 pm

Thanks for everyone who helped make tonights event a lot of fun. Not spoiling any results, as I'm hoping to upload video tomorrow morning.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
An Economic Ignoramus
Memerator
Posts: 522
Joined: Tue Aug 02, 2016 12:31 pm
Location: East Lansing, MI or Naperville, IL

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by An Economic Ignoramus » Wed Jun 13, 2018 11:13 am

Was video for this ever uploaded? If so, could you post a link?
Jakob Myers
MSU '21, Naperville North '17
"No one has ever organized a greater effort to get people interested in pretending to play quiz bowl"
-Ankit Aggarwal
Member, PACE
Memerator

User avatar
Thiccasso's Guernthicca
Lulu
Posts: 90
Joined: Sun Oct 15, 2017 10:01 pm
Location: Belmont, MA

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by Thiccasso's Guernthicca » Wed Jun 13, 2018 12:23 pm

An Economic Ignoramus wrote:Was video for this ever uploaded? If so, could you post a link?
This should be it, I think.

https://www.youtube.com/watch?v=oNXN1SlVevc
Wonyoung Jang
Belmont '18 // UChicago '22
Writer, NAQT

User avatar
ezubaric
Rikku
Posts: 357
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric » Sun Jun 17, 2018 5:11 pm

We now have a new video (better editing) here:
https://www.youtube.com/watch?v=wf5GHwDb6Ig
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

Post Reply