full detailsnaqt.com wrote:NAQT is pleased that its 2015 High School National Championship Tournament will showcase the first quiz bowl match in which a computer goes thumb-to-thumb against a team of humans.
On Saturday evening, a team of researchers will debut its computerized question-answering system (QANTA) in a tossup-only competition against four NAQT staffers. Unlike other question answering systems that are allowed to see the entire question at once (e.g., IBM Watson of Jeopardy! fame), this system decides when it has enough information to answer a question and is thus able to handle quiz bowl-style tossup questions.
Prior to the exhibition game, Jordan Boyd-Graber and his colleagues from the University of Colorado and the University of Maryland will describe the state-of-the-art machine learning research that went into the system and how interested students can get involved by using and improving the open-source system. Jordan is himself a former quiz bowl player for Caltech and Princeton.
The talk will start at 8:00 p.m. in the Grand Ballroom with the exhibition game to follow.
Human-computer quiz bowl match at the 2015 HSNCT
- Important Bird Area
- Forums Staff: Administrator
- Posts: 6136
- Joined: Thu Aug 28, 2003 3:33 pm
- Location: San Francisco Bay Area
- Contact:
Human-computer quiz bowl match at the 2015 HSNCT
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF
"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF
"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
- UlyssesInvictus
- Yuna
- Posts: 845
- Joined: Thu Feb 10, 2011 7:38 pm
Re: Human-computer quiz bowl match at the 2015 HSNCT
Ken Jennings wrote:I, for one, welcome our new computer overlords.
Raynor Kuang
quizdb.org
Harvard 2017, TJHSST 2013
I wrote GRAPHIC and FILM
quizdb.org
Harvard 2017, TJHSST 2013
I wrote GRAPHIC and FILM
- jonpin
- Auron
- Posts: 2266
- Joined: Wed Feb 04, 2004 6:45 pm
- Location: BCA NJ / WUSTL MO / Hackensack NJ
Re: Human-computer quiz bowl match at the 2015 HSNCT
Is there any chance the talk and/or the match will be streamed online or recorded for later broadcast?
Jon Pinyan
Coach, Bergen County Academies (NJ); former player for BCA (2000-03) and WUSTL (2003-07)
HSQB forum mod, PACE member
Stat director for: NSC '13-'15, '17; ACF '14, '17, '19; NHBB '13-'15; NASAT '11
"A [...] wizard who controls the weather" - Jerry Vinokurov
Coach, Bergen County Academies (NJ); former player for BCA (2000-03) and WUSTL (2003-07)
HSQB forum mod, PACE member
Stat director for: NSC '13-'15, '17; ACF '14, '17, '19; NHBB '13-'15; NASAT '11
"A [...] wizard who controls the weather" - Jerry Vinokurov
- Skepticism and Animal Feed
- Auron
- Posts: 3238
- Joined: Sat Oct 30, 2004 11:47 pm
- Location: Arlington, VA
Re: Human-computer quiz bowl match at the 2015 HSNCT
Hooray! One step closer to "Quiz Bowl Mogul"
Bruce
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source
Re: Human-computer quiz bowl match at the 2015 HSNCT
I will record it in the same fashion as all the podcasts (i.e., audio only), but I don't know enough about what to expect from the QANTA match to have any idea whether it will be interesting to listen to (e.g. how QANTA will be signaling or giving its answers). If the recording has any value and all participants approve, it will be posted.jonpin wrote:Is there any chance the talk and/or the match will be streamed online or recorded for later broadcast?
I have heard something about Jordan Boyd-Graber et al planning to video record the match, but I don't know if that's for their own purposes, or whether it will be posted publicly, etc.
Jonah Greenthal
National Academic Quiz Tournaments
National Academic Quiz Tournaments
- Mike Bentley
- Sin
- Posts: 6465
- Joined: Fri Mar 31, 2006 11:03 pm
- Location: Bellevue, WA
- Contact:
Re: Human-computer quiz bowl match at the 2015 HSNCT
Back when the Watson Jeopardy matches aired, IBM hosted a Q&A with some people who worked on it at the University of Washington. I asked them if they had ever considered trying Watson out on the quizbowl format (explaining a bit some of the ways it differed from Jeopardy). Sadly they had not. Glad to see someone's trying.
Mike Bentley
Treasurer, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
Treasurer, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
- Fado Alexandrino
- Yuna
- Posts: 834
- Joined: Sat Jun 12, 2010 8:46 pm
- Location: Farhaven, Ontario
Re: Human-computer quiz bowl match at the 2015 HSNCT
It would be cool to see, post finals, like the NSC All-Star game, where the HSNCT All-Stars (however defined) play against the robot.
Also, the robot should sign up for a consolation mini-tournament.
Also, the robot should sign up for a consolation mini-tournament.
Joe Su, OCT
Lisgar 2012, McGill 2015, McGill 2019, Queen's 2020
Lisgar 2012, McGill 2015, McGill 2019, Queen's 2020
Re: Human-computer quiz bowl match at the 2015 HSNCT
"QANTA would like to fit in one more game before flight leaves."The Last 21 Stanley Cup Winners wrote:Also, the robot should sign up for a consolation mini-tournament.
David Dennis
Middle School Choir Director
District Scholar Bowl Coach
Washington, MO
Murray State University, 2001
Breckinridge County High School, KY 1996
Middle School Choir Director
District Scholar Bowl Coach
Washington, MO
Murray State University, 2001
Breckinridge County High School, KY 1996
- Whiter Hydra
- Auron
- Posts: 1418
- Joined: Tue Dec 04, 2007 8:46 pm
- Location: Fairfax, VA
- Contact:
Re: Human-computer quiz bowl match at the 2015 HSNCT
I will be liveblogging the match at http://tinyurl.com/2015hsnctprelims.
Harry White
TJHSST '09, Virginia Tech '13
Owner of Tournament Database Search and Quizbowl Schedule Generator
Will run stats for food
TJHSST '09, Virginia Tech '13
Owner of Tournament Database Search and Quizbowl Schedule Generator
Will run stats for food
Re: Human-computer quiz bowl match at the 2015 HSNCT
And the end result after 40 tossups is 200 vs. 200. In a sign of poor planning, there was no contingency for a tie. I was expecting the humans to trounce QANTA.
We hope to make more appearances and to compete in a full tournament next year.
We'll be posting the full video on YouTube soon.
We hope to make more appearances and to compete in a full tournament next year.
We'll be posting the full video on YouTube soon.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998
Human-Computer Question Answering:
http://qanta.org/
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998
Human-Computer Question Answering:
http://qanta.org/
- the return of AHAN
- Auron
- Posts: 1988
- Joined: Thu Feb 01, 2007 10:40 pm
Re: Human-computer quiz bowl match at the 2015 HSNCT
ezubaric wrote: We hope to make more appearances and to compete in a full tournament next year.
future NAQT website release wrote: Congratulations to Kealing Middle School of Austin, TX for winning the 2016 MSNCT championship in a one-game final over QANTA, 450-315. This marks the 4th time a quizbowl machine has won the MSNCT in its 6-year history, and, coincidentally, the first time an actual computer has played in the final.
Jeff Price
Barrington High School Coach (2021 & 2023 HSNCT Champions, 2023 PACE Champions, 2023 Illinois Masonic Bowl Class 3A State Champions)
Barrington Station Middle School Coach (2013 MSNCT Champions, 2013 & 2017 Illinois Class AA State Champions)
------------------------------------------------------------------------------
Barrington High School Coach (2021 & 2023 HSNCT Champions, 2023 PACE Champions, 2023 Illinois Masonic Bowl Class 3A State Champions)
Barrington Station Middle School Coach (2013 MSNCT Champions, 2013 & 2017 Illinois Class AA State Champions)
------------------------------------------------------------------------------
Re: Human-computer quiz bowl match at the 2015 HSNCT
I created a new post about the potential rule sets for human / computer quiz bowl matches here:
http://www.hsquizbowl.org/forums/viewto ... 91#p303372
http://www.hsquizbowl.org/forums/viewto ... 91#p303372
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998
Human-Computer Question Answering:
http://qanta.org/
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998
Human-Computer Question Answering:
http://qanta.org/
- Stained Diviner
- Auron
- Posts: 5087
- Joined: Sun Jun 13, 2004 6:08 am
- Location: Chicagoland
- Contact:
Re: Human-computer quiz bowl match at the 2015 HSNCT
A robot cheering section would be pretty cool.
Re: Human-computer quiz bowl match at the 2015 HSNCT
Video available here:bird bird bird bird bird wrote:full detailsnaqt.com wrote:NAQT is pleased that its 2015 High School National Championship Tournament will showcase the first quiz bowl match in which a computer goes thumb-to-thumb against a team of humans.
https://www.youtube.com/watch?v=LqsUaprYMOw
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998
Human-Computer Question Answering:
http://qanta.org/
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998
Human-Computer Question Answering:
http://qanta.org/
- Fado Alexandrino
- Yuna
- Posts: 834
- Joined: Sat Jun 12, 2010 8:46 pm
- Location: Farhaven, Ontario
Re: Human-computer quiz bowl match at the 2015 HSNCT
Is there video of this?ezubaric wrote:
What we did: During our demo, we were showing what the computer was thinking; i.e., the the output from the guesser and the associated scores.
Joe Su, OCT
Lisgar 2012, McGill 2015, McGill 2019, Queen's 2020
Lisgar 2012, McGill 2015, McGill 2019, Queen's 2020
Re: Human-computer quiz bowl match at the 2015 HSNCT
We tried to use Quicktime to get a screen capture, but it flaked out while we were still giving our introduction. This is what it looked like on the final question.The Last 21 Stanley Cup Winners wrote:Is there video of this?ezubaric wrote:
What we did: During our demo, we were showing what the computer was thinking; i.e., the the output from the guesser and the associated scores.
Q: This national park is home to Artist's Drive, Zabriskie Point, and Dante's View,
from which one can see Telescope Peak. Its Racetrack Playa contains mysterious
"sailing stones," which can travel hundreds of
Vancouver -3.940807
Central Park -4.234285
Zion National Park -4.406702
Rio de Janeiro -4.421752
Mount Kosciuszko -4.737045
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998
Human-Computer Question Answering:
http://qanta.org/
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998
Human-Computer Question Answering:
http://qanta.org/
Re: Human-computer quiz bowl match at the 2015 HSNCT
Hugely impressive. I enjoyed reading about the science that went into this; great to see the video too.
Question: How much time did you invest in getting QANTA to sound more like Sinistar before committing to a live game?
Question: How much time did you invest in getting QANTA to sound more like Sinistar before committing to a live game?
Edmund Dickinson
UK Quizbowl
University of Oxford '11
UK Quizbowl
University of Oxford '11
Re: Human-computer quiz bowl match at the 2015 HSNCT
Not that this really matters, but I'm just curious: why were Jeopardy players chosen rather than top quiz bowl players?
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
- the return of AHAN
- Auron
- Posts: 1988
- Joined: Thu Feb 01, 2007 10:40 pm
Re: Human-computer quiz bowl match at the 2015 HSNCT
I'd guess that having Jeopardy champions gave the match a little more cachet than just having an All-Star team selected from LASA, DCC, Arcadia, etc, never mind the potential for jealousy over this guy being selected, but not that one.Stefan HSQBRankovich wrote:Not that this really matters, but I'm just curious: why were Jeopardy players chosen rather than top quiz bowl players?
To be sure, it was confided to me, 2nd hand, that at least one player was a bit perplexed about being selected, too, professing to be a poor quizbowler.
Jeff Price
Barrington High School Coach (2021 & 2023 HSNCT Champions, 2023 PACE Champions, 2023 Illinois Masonic Bowl Class 3A State Champions)
Barrington Station Middle School Coach (2013 MSNCT Champions, 2013 & 2017 Illinois Class AA State Champions)
------------------------------------------------------------------------------
Barrington High School Coach (2021 & 2023 HSNCT Champions, 2023 PACE Champions, 2023 Illinois Masonic Bowl Class 3A State Champions)
Barrington Station Middle School Coach (2013 MSNCT Champions, 2013 & 2017 Illinois Class AA State Champions)
------------------------------------------------------------------------------
Re: Human-computer quiz bowl match at the 2015 HSNCT
We chose a built in TTS system from OS X. This is "Tom".Edmund wrote:Question: How much time did you invest in getting QANTA to sound more like Sinistar before committing to a live game?
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998
Human-Computer Question Answering:
http://qanta.org/
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998
Human-Computer Question Answering:
http://qanta.org/
Re: Human-computer quiz bowl match at the 2015 HSNCT
NAQT did the choosing for us, and I think they optimized for something that would look good in a press release. I think they did a great job; it was a really fun match and neither side was dominant.Stefan HSQBRankovich wrote:Not that this really matters, but I'm just curious: why were Jeopardy players chosen rather than top quiz bowl players?
It would be great to play a team of Yaphe, Hoppes, Teitler, and Weiner, but what I've been telling people is that our system needs to be able to beat humans before it can take on super humans.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998
Human-Computer Question Answering:
http://qanta.org/
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998
Human-Computer Question Answering:
http://qanta.org/
-
- Lulu
- Posts: 23
- Joined: Fri Jul 09, 2010 9:41 pm
Re: Human-computer quiz bowl match at the 2015 HSNCT
A group presenting at the cs224d.stanford.edu poster session yesterday (the class is taught by Mohit's former boss Richard Socher, who mentioned QANTA in class) said they got 5% better performance than QANTA on both history and lit using Tree-LSTMs. So the race is on and the question dataset you guys released could even become a major test set for academic NLP/question answering...
James Bradbury, Stanford '16, TJHSST '12
Re: Human-computer quiz bowl match at the 2015 HSNCT
At least part of the reason we didn't have the computer play against our own members (Yaphe, Hoppes, Teitler, Hart? *shudders*) was question selection. We would have had to source some questions that those guys hadn't seen; it was a lot easier to use two rounds of the actual HSNCT set, which rules out anybody with a writing or editorial role. That also rules out a goodly number of the very good quiz bowl players who were on our staff, because we weren't going to tinker with set construction to make this work.
At some point one of us noticed that we had four recent, recognizable Jeopardy superchamps all on staff, and that none of them had a writing role, so that seemed the obvious thing to do for simplicity and also just for fun.
At some point one of us noticed that we had four recent, recognizable Jeopardy superchamps all on staff, and that none of them had a writing role, so that seemed the obvious thing to do for simplicity and also just for fun.
Nathan Murphy
NAQT Member
HSNCT/ICT Logistics Director
Gonzaga University 2007
NAQT Member
HSNCT/ICT Logistics Director
Gonzaga University 2007
Re: Human-computer quiz bowl match at the 2015 HSNCT
I get it. It would be interesting to see how it does against a top qb team when it's more developed.
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
Re: Human-computer quiz bowl match at the 2015 HSNCT
Agreed. We do intend to do more exhibition matches. We're planning one on the West Coast in early Fall (Bay Area or Seattle), Montreal in December, DC in early spring, and then a national tournament in late Spring.Stefan HSQBRankovich wrote:I get it. It would be interesting to see how it does against a top qb team when it's more developed.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998
Human-Computer Question Answering:
http://qanta.org/
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998
Human-Computer Question Answering:
http://qanta.org/
Re: Human-computer quiz bowl match at the 2015 HSNCT
Richard was also on the original QANTA paper. We also have improved quite a bit over the original QANTA in the leadup to the match, we need to recover from the final push and writeup the results.jekbradbury wrote:A group presenting at the cs224d.stanford.edu poster session yesterday (the class is taught by Mohit's former boss Richard Socher, who mentioned QANTA in class) said they got 5% better performance than QANTA on both history and lit using Tree-LSTMs.
We sure hope so! More citations, the better.jekbradbury wrote: So the race is on and the question dataset you guys released could even become a major test set for academic NLP/question answering...
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998
Human-Computer Question Answering:
http://qanta.org/
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998
Human-Computer Question Answering:
http://qanta.org/