ACFDB - a new tossup database

Old college threads.
User avatar
Matt Weiner
Sin
Posts: 8145
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Re: ACFDB - a new tossup database

Post by Matt Weiner »

leapfrog314 wrote:At any rate, this would not be difficult to do, especially if I'm not the one doing the categorizing (as actually getting them in the database requires work, too) but I'm not sure where the balance between "large 'n useful" and "too large to be useful" lies. Ideas? (Also, which mACF tournaments would people like to see in this?)
If you also allow tournaments to be rated by quality (eg, on a 1 to 5 scale), you could add a quality field to the search. That would let you add basically anything while still letting people limit their search to good questions.
Matt Weiner
Advisor to Quizbowl at Virginia Commonwealth University / Founder of hsquizbowl.org
User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: ACFDB - a new tossup database

Post by Mechanical Beasts »

Or even more qualitative quality descriptions--it's a little difficult to distinguish numerically between, say, CotKU and the 1998 Harvard T-Party and Penn Bowl 2006 and Chicago Opens and ACF Regionals 2001... but I'm sure that it's not unreasonable to have qualitative categories like Steinhice and Old mACF and Old ACF and mACF and ACF, is it? You'd just have to decide when 'old' is, for example, but it might be easier than having to decide whether a bad (like, no don't kill me I mean 1999 mercy please) ACF Regionals merits a 3 if Steinhice's best set does too...
Last edited by Mechanical Beasts on Tue Jul 08, 2008 10:41 pm, edited 1 time in total.
Andrew Watkins
User avatar
DumbJaques
Forums Staff: Administrator
Posts: 3109
Joined: Wed Apr 21, 2004 6:21 pm
Location: Columbus, OH

Re: ACFDB - a new tossup database

Post by DumbJaques »

but it might be easier than having to decide whether a bad (like, 2001) ACF Regionals merits a 3 if Steinhice's best set does too...
The issue of having a category on the database called "Steinhice" aside, it's pretty easy to rate packets. We're not really giving out awards here, we're just giving a guide for people to peruse the sets. Just use a criteria like: To what extent does this set accomplish being an ideal, mACF event given its nature (novice, very hard, whatever). I think the above proposal is going to needlessly subcategorize and just get in the way of finding good tournaments. No need to distinguish relative strengths of even things or compare packets across boundaries and time periods, just rate them by acceptability given present standards. Is ACF fall 2006 as good as the much-lauded 2007 version? Probably not, but for the purposes of our database, surely they can both be 5s. Likewise, if a very old ACF set is not as good as a very recent set by far less experienced writers, that's because we're in 2008 and things are different. The purpose of doing something like this is letting people get the most utility out of studying/finding packets, right? I think Matt's suggestion would best accomplish that while permitting many more packets to be added.
Chris Ray
OSU
University of Chicago, 2016
University of Maryland, 2014
ACF, PACE
User avatar
ezubaric
Rikku
Posts: 369
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: ACFDB - a new tossup database

Post by ezubaric »

leapfrog314 wrote:especially if I'm not the one doing the categorizing
Categorizing can be done quite easily automatically, especially with this much training data.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/
User avatar
Stained Diviner
Auron
Posts: 5085
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: ACFDB - a new tossup database

Post by Stained Diviner »

I don't see the need for rating tournaments or the concern for the database becoming too big. Think of ACF Fall, Regionals, and Nationals as three tournaments that are already in there. If you add Chicago Open, it's a fourth box that the user can check off if they want or leave blank if they don't want. The same could be done for NSC, Sun N Fun, Penn Bowl, CBI (if the questions are available), or whatever good and bad questions get added. The user goes to the website, checks off which sets s/he wishes to study from, and gets those questions. There might be a need to prioritize certain tournaments so that they can get added to the database in the next month or two as opposed to the next year or two, but that's a minor issue.
David Reinstein
Head Writer and Editor for Scobol Solo, Masonics, and IESA; TD for Scobol Solo and Reinstein Varsity; IHSSBCA Board Member; IHSSBCA Chair (2004-2014); PACE President (2016-2018)
User avatar
cvdwightw
Auron
Posts: 3291
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Re: ACFDB - a new tossup database

Post by cvdwightw »

The issue is that some tournaments evolve over time, and some don't, and then there are tournaments that widely vary from year to year depending on who's editing and who's submitting. Like, lumping "good Steinhice" with "bad Steinhice" or "good Penn Bowl" with "bad Penn Bowl" would be a bad idea.

I like the idea of a subjective quality scale: 5 - exemplars of today's best tournaments, 4 - several minor issues, or a single major issue, prevent it from being a 5, 3 - conforms to basic principles but has some serious flaws, 2 - some attempts were made to resemble good quizbowl, but ultimately failed, 1 - this set does not at all resemble good quizbowl practice. For reference, I'd put most recent ACF sets as 5, the best NAQT sets as 4 with most at 3, and stuff like Ghetto Warz I or the "half-edited" Steinhice "Penn Bowl" at 1.
Dwight Wynne
socalquizbowl.org
UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry
User avatar
Mike Bentley
Sin
Posts: 6461
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: ACFDB - a new tossup database

Post by Mike Bentley »

I dunno, I think a ratings system would just introduce unneeded drama and controversy, with people posting things like "How come my tournament is a 4 and not a 5?" or "this cabal / individual's preferences are totally skewed towards people they like", etc. I would hope people would be able to figure out for themselves if a question is good or bad, it's not very difficult.
Mike Bentley
Treasurer, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: ACFDB - a new tossup database

Post by Mechanical Beasts »

Bentley Like Beckham wrote:I dunno, I think a ratings system would just introduce unneeded drama and controversy, with people posting things like "How come my tournament is a 4 and not a 5?" or "this cabal / individual's preferences are totally skewed towards people they like", etc. I would hope people would be able to figure out for themselves if a question is good or bad, it's not very difficult.
And moreover, I think that people can sort things based on other kinds of criteria: if hypothetically this database contained both NAQT and ACF, it's not like we could call NAQT ICT a 4 and ACF Nationals a 5 without igniting innumerable arguments (and that's even supposing that that is justifiable in the first place).

What's wrong with qualitative labels? It's not like I thought mine through at all--but I think people mostly know what they'll get if they filter their results so that it doesn't contain anything written after the 1993 Illinois Remember the Lorax. Don't force them to represent quality, just different experiences. And I'm not saying that to equivocate, because there is quiz bowl that I find to be crap--I'm just saying that because a difficulty rubric just creates complications, where our acronyms and traditions give us approximations of quality ready-made.
Andrew Watkins
User avatar
Sima Guang Hater
Auron
Posts: 1957
Joined: Mon Feb 05, 2007 1:43 pm
Location: Nashville, TN

Re: ACFDB - a new tossup database

Post by Sima Guang Hater »

We could just have a tag for things that are in "goodpackets.zip"
Eric Mukherjee, MD PhD
Brown 2009, Penn Med 2018
Instructor/Attending Physician/Postdoctoral Fellow, Vanderbilt University Medical Center
Coach, University School of Nashville

“The next generation will always surpass the previous one. It’s one of the never-ending cycles in life.”
Support the Stevens-Johnson Syndrome Foundation
User avatar
Matt Weiner
Sin
Posts: 8145
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Re: ACFDB - a new tossup database

Post by Matt Weiner »

People who have vested interests in bad quizbowl will always try to start shit when moves to good quizbowl are made. That's not a good reason to stop making those moves.
Matt Weiner
Advisor to Quizbowl at Virginia Commonwealth University / Founder of hsquizbowl.org
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: ACFDB - a new tossup database

Post by grapesmoker »

You could implement a voting system that would allow people to rate packets.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
User avatar
Stained Diviner
Auron
Posts: 5085
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: ACFDB - a new tossup database

Post by Stained Diviner »

Another issue to keep in mind is that different packets are good for different people. I was the one who suggested to Carlo that he allow people to filter which questions they got based on Fall, Regionals, and Nationals, because as a high school coach I think it is more useful for my students to focus on the Fall questions, and I figured that elite players would rather focus on the Nationals questions. If this archive keeps expanding, it eventually will have Chicago Open questions on the one hand and some good novice or high school tournaments on the other hand, with several gradations in between. Some subset of each of those two sets will be of high quality, but different groups of people will be interested in studying different questions.

I'm not negating what Matt and Jerry are saying--it seems possible to have a rating system (by voting, panel, or dictator) and use it as a filter, and it might be helpful. However, it's also true that this database would be useful without a rating system even if it was expanded, and a rating system would be useful without this database.
David Reinstein
Head Writer and Editor for Scobol Solo, Masonics, and IESA; TD for Scobol Solo and Reinstein Varsity; IHSSBCA Board Member; IHSSBCA Chair (2004-2014); PACE President (2016-2018)
salamanca
Lulu
Posts: 65
Joined: Tue May 17, 2005 1:00 pm

Re: ACFDB - a new tossup database

Post by salamanca »

everyday847 wrote:Or even more qualitative quality descriptions--it's a little difficult to distinguish numerically between, say, CotKU and the 1998 Harvard T-Party and Penn Bowl 2006 and Chicago Opens and ACF Regionals 2001... but I'm sure that it's not unreasonable to have qualitative categories like Steinhice and Old mACF and Old ACF and mACF and ACF, is it? You'd just have to decide when 'old' is, for example, but it might be easier than having to decide whether a bad (like, 2001) ACF Regionals merits a 3 if Steinhice's best set does too...
Umm, I'd just like to point out that the 2001 edition of ACF Regionals is still an outstanding low to medium level set of questions. Read through those packets sometime and you will be surprised by how deep some of the clues go. I guess what I'm saying is newer tourneys aren't necessarily better than older ones by such eminent editors as Subash.

Peace,
Ezequiel
Ezequiel
ACF
User avatar
Matt Weiner
Sin
Posts: 8145
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Re: ACFDB - a new tossup database

Post by Matt Weiner »

salamanca wrote:Umm, I'd just like to point out that the 2001 edition of ACF Regionals is still an outstanding low to medium level set of questions. Read through those packets sometime and you will be surprised by how deep some of the clues go. I guess what I'm saying is newer tourneys aren't necessarily better than older ones by such eminent editors as Subash.
I have to agree with this; perhaps Andy meant the 1999 Regionals, which has not aged as well.
Matt Weiner
Advisor to Quizbowl at Virginia Commonwealth University / Founder of hsquizbowl.org
User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: ACFDB - a new tossup database

Post by Mechanical Beasts »

Matt Weiner wrote:
salamanca wrote:Umm, I'd just like to point out that the 2001 edition of ACF Regionals is still an outstanding low to medium level set of questions. Read through those packets sometime and you will be surprised by how deep some of the clues go. I guess what I'm saying is newer tourneys aren't necessarily better than older ones by such eminent editors as Subash.
I have to agree with this; perhaps Andy meant the 1999 Regionals, which has not aged as well.
This is, in fact, a case of me forgetting which Regionals I read and was surprised by. I mean absolutely no disrespect to the last generation of editors; I liked 2001 quite a bit.
Andrew Watkins
User avatar
theMoMA
Forums Staff: Administrator
Posts: 5993
Joined: Mon Oct 23, 2006 2:00 am

Re: ACFDB - a new tossup database

Post by theMoMA »

Carlo, I'm having trouble getting the database to randomize the order of questions. When I tell it to display questions randomly, it usually reverts to "Category Ascending." Am I doing something wrong, or if not, could you please look into this?
Andrew Hart
Minnesota alum
leapfrog314
Wakka
Posts: 248
Joined: Mon Sep 12, 2005 10:49 am
Contact:

Re: ACFDB - a new tossup database

Post by leapfrog314 »

theMoMA wrote:Carlo, I'm having trouble getting the database to randomize the order of questions.
Hey, thanks for pointing that out. I've fixed it.

Also, since you drew my attention to the ordering thing, I changed it so that category ascending/descending also sorts subcategories in order, i.e. Fine Arts - Art comes before Fine Arts - Music, which comes before Fine Arts - Other. Previously it only sorted the category, not the subcategories within them.
Carlo Angiuli, Indiana University
Director, Aegis Questions, Inc.
cdcarter
Yuna
Posts: 945
Joined: Thu Nov 15, 2007 12:06 am
Location: Minneapolis, MN
Contact:

Re: ACFDB - a new tossup database

Post by cdcarter »

Hey Carlo,
I am also having problems with randomization. There seem to be repeats over pages. Does it store the query, create a randomization, and then paginate that, or just pick 10 at random for each page?
Christian Carter
Minneapolis South High School '09 | Emerson College '13
PACE Member (retired)
User avatar
theMoMA
Forums Staff: Administrator
Posts: 5993
Joined: Mon Oct 23, 2006 2:00 am

Re: ACFDB - a new tossup database

Post by theMoMA »

cdcarter wrote:Hey Carlo,
I am also having problems with randomization. There seem to be repeats over pages. Does it store the query, create a randomization, and then paginate that, or just pick 10 at random for each page?
I think it does the latter, because I noticed that as well.
Andrew Hart
Minnesota alum
leapfrog314
Wakka
Posts: 248
Joined: Mon Sep 12, 2005 10:49 am
Contact:

Re: ACFDB - a new tossup database

Post by leapfrog314 »

cdcarter wrote:I am also having problems with randomization. There seem to be repeats over pages. Does it store the query, create a randomization, and then paginate that, or just pick 10 at random for each page?
Yeah, every page it just picks 10 random questions. If you want more random questions without repeats, choose a higher number to display on each page. Alternatively, if you want more than that, do a randomly-ordered search and download the entire results as a text file.
Carlo Angiuli, Indiana University
Director, Aegis Questions, Inc.
User avatar
Skepticism and Animal Feed
Auron
Posts: 3238
Joined: Sat Oct 30, 2004 11:47 pm
Location: Arlington, VA

Re: ACFDB - a new tossup database

Post by Skepticism and Animal Feed »

One thing I noticed: when I search for a small word, it brings up results for larger words that contain the first word. For instance, a search for "Odin" brings up a bunch of questions about Rodin. This happens even when I put Odin in quotes or put a space in front of him.
Bruce
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source
User avatar
Stained Diviner
Auron
Posts: 5085
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: ACFDB - a new tossup database

Post by Stained Diviner »

" odin " or " odin." or " odin,"
This will find pretty much all occurrences within questions without the longer words, but it won't list any of the times Odin is the answer. Obviously, it's a less than ideal solution.
David Reinstein
Head Writer and Editor for Scobol Solo, Masonics, and IESA; TD for Scobol Solo and Reinstein Varsity; IHSSBCA Board Member; IHSSBCA Chair (2004-2014); PACE President (2016-2018)
User avatar
Blackboard Monitor Vimes
Auron
Posts: 2362
Joined: Sat Aug 18, 2007 5:40 pm
Location: Richmond, VA

Re: ACFDB - a new tossup database

Post by Blackboard Monitor Vimes »

A slightly better solution: why not turn the category restriction on? That may not fix all occurrences of this, but it works for this particular case.
Sam L,
Maggie L. Walker Governor's School 2010 / UVA 2014 / VCU School of Education 2016
PACE
leapfrog314
Wakka
Posts: 248
Joined: Mon Sep 12, 2005 10:49 am
Contact:

Re: ACFDB - a new tossup database

Post by leapfrog314 »

Skepticism and Animal Feed wrote:a search for "Odin" brings up a bunch of questions about Rodin.
Reinstein and Sarah have good suggestions. If you just search for " odin" (with the quotes) you will get everything about Odin but nothing about Rodin, and I can't think of any word that starts with "odin" that isn't Odin. If I were to implement a "full words only" search, it would basically be like running a search for " odin" OR "odin ".

Also, as Sarah said, if you are looking for questions on Odin, just search within mythology. Likewise, a search on Adam Smith is more fruitful if you ignore fine arts, which turns up lots of questions on goldsmiths, the Smithsonian, etc.

Side note: I haven't really been keeping up with looking at the reported tossups...hopefully I'll get around to that soon.
Carlo Angiuli, Indiana University
Director, Aegis Questions, Inc.
User avatar
Golran
Auron
Posts: 1048
Joined: Tue Oct 16, 2007 1:23 am
Location: Southern California

Re: ACFDB - a new tossup database

Post by Golran »

I don't know what happened, but when I was reading through the science questions (I just did an empty search for science questions to get all of them) This came up:
1999 ACF Nationals - Packet by Vanderbilt - #20
Science -- Astronomy

Its presiding officer had to stand upon a lava promontory called Law Rock and recite all the laws passed by the body that year--a natural and effective check on legislative power and long-windedness. Located upon the plains of Thingvellir, it was founded in 930 AD, making it the oldest parliamentary body in the world. FTP, name the General Assembly of Iceland.

Answer: (binary) pulsar (prompt on neutron star)

Just thought you might want to know.
Drayer the Slayer
currently unaffiliated
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: ACFDB - a new tossup database

Post by grapesmoker »

I'm pretty sure that the Icelandic sagas tell of how their laws were passed down from the Hulse-Taylor binary. I think it's in Egil Skallagrimsson's story.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
leapfrog314
Wakka
Posts: 248
Joined: Mon Sep 12, 2005 10:49 am
Contact:

Re: ACFDB - a new tossup database

Post by leapfrog314 »

Thanks, Ian -- I fixed that tossup and the next one about pumice. For some reason my script didn't like the replacement tossups from that packet...

Also, I haven't really had a chance to look at any of the reported questions...I'm hoping to do that eventually, or maybe write another script to foist the fixin' on you guys.
Carlo Angiuli, Indiana University
Director, Aegis Questions, Inc.
jonah
Auron
Posts: 2383
Joined: Thu Jul 20, 2006 5:51 pm
Location: Chicago

Re: ACFDB - a new tossup database

Post by jonah »

Regionals just got added to the database, so there are 336 tossups to categorize. Please help out.
Jonah Greenthal
National Academic Quiz Tournaments
Javatron
Lulu
Posts: 28
Joined: Tue Feb 16, 2010 9:52 pm

Re: ACFDB - a new tossup database

Post by Javatron »

There is still that one tossup left uncategorized...XD Was that a mis-entry?
Sabrina Fritz

Villa Duchesne (MO) -- Class of 2011
Northwestern University (IL) -- Class of 2015
jonah
Auron
Posts: 2383
Joined: Thu Jul 20, 2006 5:51 pm
Location: Chicago

Re: ACFDB - a new tossup database

Post by jonah »

Carlo just added 2010 Nationals, 2010 Novice, and 2010 Fall, but those tossups are not yet categorized. If you can spare a few minutes to work on that, it would be great.
Jonah Greenthal
National Academic Quiz Tournaments
Locked