NAQT and distribution change

Dormant threads from the high school sections are preserved here.
Locked
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 6113
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

NAQT and distribution change

Post by Important Bird Area »

Oliver Ellsworth wrote:I'm fully aware that NAQT is simply doing what it does with its screwy distributions and strange answer selections. What I'm lamenting now is the fact that, in a position to touch an enormous number of quiz bowl teams, NAQT continually chooses to ignore the advice and criticism it receives beyond invidual questions or packets. Yes, I like that packets are retroactively edited so they're improved for future use, and I like that NAQT acknowledges when it makes dumb answer choices. But ultimately, NAQT sets still don't have what it takes to be esteemed at the level of HSAPQ and many housewritten sets. The differences are very obvious. Yes, NAQT is huge and it's growing, but as stated before, people often have no choice but to rely on it, so the numbers are a bit inflated. In fact, NAQT's growth is a simply a more urgent sign that things need to change. NAQT could be the single most powerful proponent of the activity it professes to advance, but instead it continues many of its silly ways. Wasted potential just makes me so sad....
So, as I mentioned over in the Illinois thread, here's a longer post about NAQT policy and distribution change.

Of course NAQT is well aware that many players who post on this forum would like us to change our distribution. If distribution change was just a straightforward matter of improving the quality of our quizbowl, I have no doubt that we would be happy to adopt many of the changes suggested here.

Assume for the moment (an assumption we don't necessarily agree to!) that proposed distribution changes would, in fact, improve the quality of NAQT's sets. Now I'd like to switch gears and talk about the cost of distribution change.

I hope everyone will agree that dead tossups are bad quizbowl. For example, the letter from the circuit asking for NAQT reform wrote: "we are entirely sure that what “average” teams want is easy questions, not bad questions." (Note that Zahed's comment above separates question quality and distribution issues: "I like that NAQT acknowledges when it makes dumb answer choices. But ultimately, NAQT sets still don't have what it takes to be esteemed at the level of HSAPQ and many housewritten sets" so I will assume for the moment that NAQT's questions meet a minimum standard of quality within any given category.) It logically follows that making a set more difficult makes it less attractive to average high school teams. (I'll exclude for the moment extremely easy sets aimed at middle schoolers.)

Now then: would distribution change make our sets more difficult? The answer is yes. Here are our categories, placed in order from "most often answered" to "least often answered." I've marked our own target for conversion rates, and the average conversion rates (including and excluding computational math). Sample size, by the way, is 19,546 IS tossups read between 2004 and 2009.

religious literature
mythology
---------------- (NAQT's stated conversion target)
miscellaneous
current events
general knowledge
geography
popular culture
children's literature
history
theology
literature (popular genres)
--------------------------- (average conversion rate, excluding comp)
sports
--------------------------- (average conversion rate, including comp)
science
literature (excluding myth, religion, kiddie/pop lit)
fine arts
social science
philosophy
computational math

Some notes:
Computational math, as always, is a conversion outlier. High school teams are surprisingly good at myth and religion. Apart from those exceptions, it is generally true that categories with below-average conversion rates are those our critics would like to see increased, and categories with above-average conversion rates are those our critics would like to see reduced:

below-average conversion: (literature, fine arts, social science, philosophy)
above-average conversion: (current events, geography, pop culture, miscellaneous/general knowledge)

Here's an experiment: assume for the moment that we at NAQT decided to switch from the existing IS set distribution and produce IS #94 on the ACF distribution. 31% of the set switches categories. What effect would this have on the difficulty of the set?* An ACF-distributed set would be 1.86% more difficult (3.63% more difficult exclusive of computational math).

I'm sure that's a tradeoff that many teams reading this would make in two seconds flat- sets that are 3-4% more difficult in exchange for improvement in the categories for 31% of the questions. But it's a much more difficult decision to make for a new team that's already struggling with the difficulty of our questions, and consequently for NAQT as a business. To sum up, I would like to suggest that Zahed's post has the causality wrong between our distribution and our market position: we don't believe we're wasting an opportunity to provide new teams with better-distributed questions. We believe that our existing distribution plays a small part in maintaining our outreach to teams that are new to good quizbowl.

*(Notes section:
-This argument assumes that the difficulty of the category remains constant as its share of the distribution expands or contracts; this may be a bad assumption. It might, for instance, be possible to improve our conversion numbers by preferentially selecting the strangest, most difficult tossup answers to be removed. I'm not at all sure we're capable of doing that, given the equally consistent criticism on this forum that our answer selection process is substandard.

-Alternatively, we could increase conversion even as we expand the distribution by repeating the same clues and answers more frequently. Since this is how middle clues become stock, I think this solution is a blind alley that would simply reward teams who study old packets and make quizbowl even more of a self-referential activity than it is now; I don't think it would necessarily help in reaching out to new teams. (Worth noting, by the way, that the general trend is that the harder-to-convert categories -lit, fine arts, phil- are *already* more strongly canonical than the rest of our sets. I'm not sure which direction that runs- whether this is because weak teams haven't learned the canon, or because our subject editors are aware of conversion problems and try to push the answer space toward the best-known subjects. A mix of both wouldn't surprise me.)

-All of the above should be distinguished from prescriptive arguments about the distribution (Of the form: "X% of popular culture questions is too much for academic quizbowl.") Obviously NAQT shares this argument, for certain values of X ("why don't you adopt the TRASHionals distribution, I bet novices would rather play that than actual literature and science?"), but where to place that line is not something we believe is a settled question.)
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
Susan
Forums Staff: Administrator
Posts: 1812
Joined: Fri Aug 15, 2003 12:43 am

Re: NAQT and distribution change

Post by Susan »

One thing I'm curious about, and that I don't know if you have the numbers to check easily:

I'd be interested to see the distribution of conversion rate across all topics in a subcategory. For example--and I'm totally making up numbers here--are literature tossups coming in at a conversion rate of, say, 70% because the majority of tossups are converted 60-80% of the time, or because there are a fair number of overly difficult questions that are being converted >30% of the time that skew the conversion rate downward? I'd just be curious to see what the spread is in some of these topics.
Susan
UChicago alum (AB 2003, PhD 2009)
Member emerita, ACF
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 6113
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: NAQT and distribution change

Post by Important Bird Area »

Susan, I'd be interested to see that as well, but don't have the numbers to check it easily. (I do know that R. was working on an interface for that last June, but it will be next June before we see anything like it.) I might try to put together something like that myself- maybe a set of histograms for the 2009 HSNCT?- but it will be after SCT is over.
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
User avatar
Wall of Ham
Rikku
Posts: 420
Joined: Thu Apr 27, 2006 9:28 am

Re: NAQT and distribution change

Post by Wall of Ham »

I think that Literature's low conversion rate indicates the way it's taught in high school. In English classes one may only read 10 books a year, as opposed to say history, which is far more comprehensive. So to a casual literature player or new player, they may only know certain books well, and never heard of the other books, whereas a similar history player could probably get nearly all IS tossups at the end.

Likewise, literature is not very "connected". If you assume most high schoolers learn literature through reading and not memorizing lists, a player who knows the ends and outs of Pride and Prejudice may not have even heard about Emma. In contrast, a science player who knows a lot about DNA has a large shot to get a tossup on something similar like RNA.

Fine arts, social science, and philosophy do not usually have a class dedicated to them in high schools, or if they did, it would be taken senior year.

The low conversion rate of computational math blows the "It's taught in high schools, so it should be there" argument out of the water. Since computational math has worse conversion rates than subjects not typically taught in HS (even with extra time as given by NAQT), clearly something is incompatible between computation math and quizbowl.
Barry
Cornell
User avatar
Stained Diviner
Auron
Posts: 5085
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: NAQT and distribution change

Post by Stained Diviner »

Thanks for starting this thread, Jeff. Though I disagree with NAQT's distribution, I get that there is a reason for it.

If I was going to change the NAQT distribution, my first priority would be increasing Fine Arts, my second priority would be decreasing general knowledge/pop culture/sports, and my third priority would be increasing the Big Three. I don't see the need to go to an ACF distribution, since I'm OK with high school quizbowl having less Philosophy and Social Science than collegiate quizbowl. I'm also OK with high school quizbowl having less Fine Arts than collegiate quizbowl, but it should have more than the current NAQT distribution.

Assuming that increasing the Big Three to 20% each and Arts to 10% would bring down your conversion rate by about 2% (and somebody could correct me if I'm wrong about it being 2%), my opinion is that:
A) Decreasing conversion by 2% probably is not going to make a big difference as to whether or not teams stick with the activity or not. I get the fact that it already is lower than you want it to be, so this is moving in the wrong direction, but it's moving a fairly small amount in the wrong direction.
B) My guess is that if you made it more of an institutional priority to write higher conversion questions, that you could increase conversion rates. I know that weak teams are often very weak in lit and arts, but easy questions in history and science often get answered by bad teams for reasons similar to what Barry stated above. Also, it's not difficult to write numerous questions in Geography and Pop Culture that have close to 100% conversion if you're looking for areas where your conversion rate could be higher than it currently is.
David Reinstein
Head Writer and Editor for Scobol Solo, Masonics, and IESA; TD for Scobol Solo and Reinstein Varsity; IHSSBCA Board Member; IHSSBCA Chair (2004-2014); PACE President (2016-2018)
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 6113
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: NAQT and distribution change

Post by Important Bird Area »

Shcool wrote:Assuming that increasing the Big Three to 20% each and Arts to 10% would bring down your conversion rate by about 2% (and somebody could correct me if I'm wrong about it being 2%), my opinion is that:
A) Decreasing conversion by 2% probably is not going to make a big difference as to whether or not teams stick with the activity or not. I get the fact that it already is lower than you want it to be, so this is moving in the wrong direction, but it's moving a fairly small amount in the wrong direction.
B) My guess is that if you made it more of an institutional priority to write higher conversion questions, that you could increase conversion rates. I know that weak teams are often very weak in lit and arts, but easy questions in history and science often get answered by bad teams for reasons similar to what Barry stated above. Also, it's not difficult to write numerous questions in Geography and Pop Culture that have close to 100% conversion if you're looking for areas where your conversion rate could be higher than it currently is.
That figure is about right (I calculate 1.89% more difficult, concentrated in the extra arts and especially the literature).

The question re: geography and pop culture is: how numerous are those 100% conversion questions? Enough to fill ten high school sets a year with tossups? Probably not, actually, since there are bunch of these already in our sets (they're the questions pushing the conversion rates in those categories above 80% the way it is now).

As an aside, writing convertible history questions is surprisingly difficult. There's a solid core of US history topics that get very good conversion numbers, but a lot of our world history goes dead more often than we would like.

Also, I owe you an email re: computational math that's been sitting in draft in my outbox for a week or so.
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
User avatar
Stained Diviner
Auron
Posts: 5085
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: NAQT and distribution change

Post by Stained Diviner »

I suppose now we're getting into two separate issues--how high could conversion be in various categories and what the distribution should be. I'll focus on the latter since it's the point of this thread.

I think we all want Fine Arts to be an important part of quizbowl, and I think you have to take it up to at least ten percent to accomplish that. Part of that is because it is actually two different big categories and a few small categories combined into one. Visual Art seems to deserve 1/1, Classical Music seems to deserve 1/1, and there is other stuff that gets thrown into Fine Arts such as Architecture, Jazz, Music Theory, and Musicals. At less than 1/1, a team's superiority in those categories becomes pretty meaningless, and teams never leave a tournament saying, "We should learn more about that stuff." There are a decent number of teams that have some knowledge in those areas, and that knowledge, which should be significant, is not worth much in an NAQT match.

At the last NAQT written tournament I attended, which used an A Set, there were 3 or 4 Visual Art tossups I heard out of 200 total tossups. (There were other Fine Arts tossups--I'm just counting painters and paintings here.) That's just not enough.
David Reinstein
Head Writer and Editor for Scobol Solo, Masonics, and IESA; TD for Scobol Solo and Reinstein Varsity; IHSSBCA Board Member; IHSSBCA Chair (2004-2014); PACE President (2016-2018)
User avatar
Jesus vs. Dragons
Tidus
Posts: 615
Joined: Wed Jan 28, 2009 10:04 pm

Re: NAQT and distribution change

Post by Jesus vs. Dragons »

While I am relatively new to good quizbowl, I have played in about 10 non-NAQT tournaments and, quite frankly, I do not have a problem with NAQT or their questions. While it is frustrating to go through rounds where there seems to be 5-6 geography tossups and 2 literature tossups, these are the exception, not the rule. This phenomenon has occurred at the past two NAQT tournaments I attended and I believe it is due to the fact that NAQT's distribution caters to the entire set and not individual rounds. If this were to be changed, I believe that a lot of the issues that people on these boards have with NAQT would end. When I first started playing quizbowl, I loved the sports tossups because it felt great to power things, but now that I have expanded my game and I now power academic things as well, the luster is slightly lost. Perhaps NAQT could devise a system that, as one goes up in difficulty, the distribution changes? I am not sure of NAQT's distribution for A-sets, IS sets, SCT, or ICT sets, but couldn't you change the distribution to better suit the field?
Ethan Hewett
UF 2013
Chipola College 2010
Sneads High School 2009
User avatar
Dresden_The_BIG_JERK
Tidus
Posts: 709
Joined: Tue Dec 09, 2008 10:56 am
Location: Lowell, IN

Re: NAQT and distribution change

Post by Dresden_The_BIG_JERK »

It kills me how low the conversion rate for math and science are. Aren't we, as a society, supposed to be moving toward emphasizing these topics more in the increasingly technological world? It's not a knock on NAQT, the science and math questions are usually fine...I just don't understand why these are things the players aren't getting.
BJ Houlding

Winnebago '04
Saint Joseph's College '08
IHSSBCA Certified Moderator
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 6113
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: NAQT and distribution change

Post by Important Bird Area »

TheLessFamousEthan wrote:This phenomenon has occurred at the past two NAQT tournaments I attended and I believe it is due to the fact that NAQT's distribution caters to the entire set and not individual rounds. If this were to be changed, I believe that a lot of the issues that people on these boards have with NAQT would end.
This isn't true; we balance the distribution both round-by-round and across the entire set. (See discussion here and most recently here, starting with Abid's 7:37. Andy's post at 7:50 is a better short explanation of this than anything I've yet written.)
TheLessFamousEthan wrote:Perhaps NAQT could devise a system that, as one goes up in difficulty, the distribution changes? I am not sure of NAQT's distribution for A-sets, IS sets, SCT, or ICT sets, but couldn't you change the distribution to better suit the field?
Yes, we've been doing that for quite a while now. Compare our high school distribution with the college distribution.
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
User avatar
Down and out in Quintana Roo
Auron
Posts: 2907
Joined: Wed Apr 09, 2008 7:25 am
Location: Camden, DE
Contact:

Re: NAQT and distribution change

Post by Down and out in Quintana Roo »

I like ACF distribution. I like NAQT distribution a lot less. I initially didn't mind NAQT and did well on it (both as a player, and as a coach of players) but i've come to dislike it more every time i see it. I would never advocate taking out trash questions completely... except in national tournaments/finals. But there is just too much of them for me to enjoy in NAQT packets. Losing to teams because we don't know enough about Rob Niedermeyer and Britney Spears (this literally happened last year) is frustrating. Losing to a team because we don't know enough about Tintoretto and Malinowski is more acceptable.

So i would certainly advocate for an increase in both Fine Arts and Social Science. These are important things. Why would we deprive teams of learning important people, works, and topics just because they have more fun on trash, trash lit, and identifying capitals of countries? Besides, there are plenty of art questions that are universally known, and once they got questions about Michelangelo or The Last Supper, they could go from there.

My opinions on computational math are already known, and i think it's abundantly clear that they have no place in good quizbowl tossups. I could be convinced to accept them in bonus form.
Mr. Andrew Chrzanowski
Caesar Rodney High School
Camden, Delaware
CRHS '97-'01
University of Delaware '01-'05
CRHS quizbowl coach '06-'12
http://crquizbowl.edublogs.org
User avatar
Sen. Estes Kefauver (D-TN)
Chairman of Anti-Music Mafia Committee
Posts: 5647
Joined: Wed Jul 26, 2006 11:46 pm

Re: NAQT and distribution change

Post by Sen. Estes Kefauver (D-TN) »

It's not a knock on NAQT, the science and math questions are usually fine...I just don't understand why these are things the players aren't getting.
I think the fact that the math questions have lots of computation explains that. It is extremely common for people who know the concepts behind the math to still have to let your tossups go dead due to the fact that there are computation time limits, poorly written questions and any other manner of problems.
Charlie Dees, North Kansas City HS '08
"I won't say more because I know some of you parse everything I say." - Jeremy Gibbs

"At one TJ tournament the neg prize was the Hampshire College ultimate frisbee team (nude) calender featuring one Evan Silberman. In retrospect that could have been a disaster." - Harry White
User avatar
Dresden_The_BIG_JERK
Tidus
Posts: 709
Joined: Tue Dec 09, 2008 10:56 am
Location: Lowell, IN

Re: NAQT and distribution change

Post by Dresden_The_BIG_JERK »

Jeremy Gibbs Free Energy wrote:
It's not a knock on NAQT, the science and math questions are usually fine...I just don't understand why these are things the players aren't getting.
I think the fact that the math questions have lots of computation explains that. It is extremely common for people who know the concepts behind the math to still have to let your tossups go dead due to the fact that there are computation time limits, poorly written questions and any other manner of problems.
It may a contributing factor, but I'm less than convinced it's the main culprit. I've seen ridiculously easy math go dead...stuff with basic arithmetic giveaways.
BJ Houlding

Winnebago '04
Saint Joseph's College '08
IHSSBCA Certified Moderator
Rococo A Go Go
Auron
Posts: 2248
Joined: Sat Jan 10, 2009 1:08 am
Location: Kentucky

Re: NAQT and distribution change

Post by Rococo A Go Go »

Social Science is definitely underrepresented in NAQT in my opinion, especially considering that it covers a wide range of important topics like Psychology, Economics, and Government. Also, having nearly 2/2 geography in a match is a little bit too much if you ask me, espeically considering that Geography is inherently present in other subjects (like History) in the set. The History distro is not too bad, although it almost seems like NAQT is trying to make up for it's neglect of Social Sciences by stuffing the History category with things like "American Gov't History" and "American Social History."
Nicholas C
KQBA member
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 6113
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: NAQT and distribution change

Post by Important Bird Area »

The "government history" vs. "social history" is just a quick mechanism to make sure that our US history isn't completely overrun by any particular type of answer (presidents, treaties, tariffs, labor unions, whatever). It's not intended to be stealth social science.

I'd sort of like to see that bad social science conversion rate broken down by subject, actually.

Not for NAQT: personally, I do think we have too much US history, and would like to see it cut back a bit (probably in favor of more non-Western history, although the answer space for that is already stretched thin at the IS level).

Edit: coherence
Last edited by Important Bird Area on Mon Jan 25, 2010 4:56 pm, edited 1 time in total.
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
Ringil
Rikku
Posts: 412
Joined: Sun Sep 21, 2008 12:46 am

Re: NAQT and distribution change

Post by Ringil »

I think there can be a bit more European history, as that is a very common topic in High School, at the cost of some American History (which I think is a bit overrepresented).
Libo
Washington '14, Michigan '13, Troy High School '09
User avatar
Matt Weiner
Sin
Posts: 8145
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Re: NAQT and distribution change

Post by Matt Weiner »

bt_green_warbler wrote:This isn't true; we balance the distribution both round-by-round and across the entire set.
Well; you do and you don't. The system, as I understand it, still can have anywhere from X to X+1 questions in a packet in any given category, which for smaller categories is a 100% difference between 0 and 1. Furthermore, the "round" may be considered 24/24 by the computer, but 90% of tournaments run on 20/20, so unless you have a distribution that guarantees a reasonable minimum and maximum for each category within the part of the game that actually determines the outcome, you're going to have even more variation.
Matt Weiner
Advisor to Quizbowl at Virginia Commonwealth University / Founder of hsquizbowl.org
User avatar
at your pleasure
Auron
Posts: 1723
Joined: Sun Aug 03, 2008 7:56 pm

Re: NAQT and distribution change

Post by at your pleasure »

soaringeagle22 wrote: The History distro is not too bad, although it almost seems like NAQT is trying to make up for it's neglect of Social Sciences by stuffing the History category with things like "American Gov't History" and "American Social History."
In addition to what Jeff said, social history is a lot of what academic historians study and there has been a lot of discussion at the college level about increasing social history.
Douglas Graebner, Walt Whitman HS 10, Uchicago 14
"... imagination acts upon man as really as does gravitation, and may kill him as certainly as a dose of prussic acid."-Sir James Frazer,The Golden Bough

http://avorticistking.wordpress.com/
User avatar
kayli
Auron
Posts: 1525
Joined: Sat Jun 21, 2008 10:15 pm
Location: Illinois

Re: NAQT and distribution change

Post by kayli »

What frustrates me about math comp is that some teams (good teams too) just refuse to do it even if it's something as simple as given radius r and height h find the surface area of this cylinder. I think this in an explanation as to why the math comp conversion is exceedingly low. The same idea sort of applies for science where after hearing a complex word like beta-lactam people just shut off. (Note: I'm not saying computational math is good; but the low conversions, I think, are partially motivated by odd factors such as jargon).

To be on topic, what bothers me a lot is the funk that goes into the NAQT distributions and sub-distributions. I think that fine arts deserves a higher place on the distribution, and I think that NAQT should get rid of all the trash in sets used for state tournaments, national tournament, and in finals packets (AT LEAST). Also, I think the distribution itself should be cleaned up a little. It's really weird trying to figure out what exactly is going into packets considering the plethora or distributions and sub-distributions occupying odd spaces, sub-spaces, and whatnot. The entire .3 this and .7 that makes sets seemingly random a lot of times.
Kay, Chicago.
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 6113
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: NAQT and distribution change

Post by Important Bird Area »

Arsonists Get All the Girls wrote:I think that NAQT should get rid of all the trash in sets used for state tournaments, national tournament, and in finals packets (AT LEAST).
Well, state and national championships we could do. ("Set popular culture to zero for IS #95 and HSNCT.") Removing it from all finals packets is not really viable. (Consider: how does one do that, assuming that IS #XX is played at twenty different tournaments, all of which may have differing numbers of teams in attendance?) And that's leaving aside the (in my opinion, dubious!) theory of quizbowl behind "it's ok to determine which teams make the finals on packets containing y quantity of subject x, but the finals themselves have to have zero."
Arsonists Get All the Girls wrote:Also, I think the distribution itself should be cleaned up a little. It's really weird trying to figure out what exactly is going into packets considering the plethora or distributions and sub-distributions occupying odd spaces, sub-spaces, and whatnot. The entire .3 this and .7 that makes sets seemingly random a lot of times.
It's less confusing than if we lumped all of this together into larger entities. I think it's straightforwardly better to say:

IS #95 contained 1/1 Norse myth, 0/1 Egyptian myth, 1/0 European myth, and 0/1 Mesopotamian myth

than just:
IS #95 contained 2/3 non-classical myth
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: NAQT and distribution change

Post by Mechanical Beasts »

Arsonists Get All the Girls wrote:I think that NAQT should get rid of all the trash in sets used for state tournaments
For what it's worth, state tournaments are often run on 20/20 out of NAQT packets; there's not more than 4/4 trash in pretty much any IS set packet, so it could be pretty easily eliminated by the TD if he chose.
Andrew Watkins
User avatar
cvdwightw
Auron
Posts: 3291
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Re: NAQT and distribution change

Post by cvdwightw »

Arsonists Get All the Girls wrote:What frustrates me about math comp is that some teams (good teams too) just refuse to do it even if it's something as simple as given radius r and height h find the surface area of this cylinder. I think this in an explanation as to why the math comp conversion is exceedingly low. The same idea sort of applies for science where after hearing a complex word like beta-lactam people just shut off. (Note: I'm not saying computational math is good; but the low conversions, I think, are partially motivated by odd factors such as jargon).
Correct me if I'm wrong, but most versions of competitive computational math allow one to read the problem before solving it; quizbowl mathcalc does not. I would suspect that another reason for the low mathcalc conversion is that if you miss or mishear a clue, it's over. If you hear "fourteen" instead of "forty" you can know immediately what to do, do all the math correctly, and still not get the points; in every other subject it's much harder to neg off a misheard clue unless you've pulled off a hilariously wrong reflex buzz. [This boils back down to the fact that mathcalc, especially as tossups, is inherently unlike any other form of quizbowl and has to be severely distorted to even resemble quizbowl]
Dwight Wynne
socalquizbowl.org
UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 6113
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: NAQT and distribution change

Post by Important Bird Area »

Matt Weiner wrote:Well; you do and you don't. The system, as I understand it, still can have anywhere from X to X+1 questions in a packet in any given category, which for smaller categories is a 100% difference between 0 and 1.
This comes down to "is it ever ok to designate 3/3 Egyptian myth? or must every tournament have categories that are at minimum an integral number of rounds?" I think our current structure is fine.
Matt Weiner wrote:Furthermore, the "round" may be considered 24/24 by the computer, but 90% of tournaments run on 20/20, so unless you have a distribution that guarantees a reasonable minimum and maximum for each category within the part of the game that actually determines the outcome, you're going to have even more variation.
This is, in theory, more of a concern. When we actually tested this, we found no difference in distribution between the first 20/20 and the last 4/4. However, what you're concerned about is not the aggregate across an entire tournament (or set of tournaments), but the possibility of variation within smaller numbers of rounds, right? (Such as "rounds 3, 4, and 5 of this IS set all had a physics tossup in the last 4.")
I'll ask R. what kind of change we might be able to adopt to reduce this sort of thing.
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
User avatar
kayli
Auron
Posts: 1525
Joined: Sat Jun 21, 2008 10:15 pm
Location: Illinois

Re: NAQT and distribution change

Post by kayli »

cvdwightw wrote:
Arsonists Get All the Girls wrote:What frustrates me about math comp is that some teams (good teams too) just refuse to do it even if it's something as simple as given radius r and height h find the surface area of this cylinder. I think this in an explanation as to why the math comp conversion is exceedingly low. The same idea sort of applies for science where after hearing a complex word like beta-lactam people just shut off. (Note: I'm not saying computational math is good; but the low conversions, I think, are partially motivated by odd factors such as jargon).
Correct me if I'm wrong, but most versions of competitive computational math allow one to read the problem before solving it; quizbowl mathcalc does not. I would suspect that another reason for the low mathcalc conversion is that if you miss or mishear a clue, it's over. If you hear "fourteen" instead of "forty" you can know immediately what to do, do all the math correctly, and still not get the points; in every other subject it's much harder to neg off a misheard clue unless you've pulled off a hilariously wrong reflex buzz. [This boils back down to the fact that mathcalc, especially as tossups, is inherently unlike any other form of quizbowl and has to be severely distorted to even resemble quizbowl]
This is true. My issue though is that math comp is often just sat through with no effort given. Quizbowl might just attract different types of people though. *shrug*
Kay, Chicago.
User avatar
Boeing X-20, Please!
Rikku
Posts: 406
Joined: Sun Jul 26, 2009 2:40 pm
Location: Evanston, IL

Re: NAQT and distribution change

Post by Boeing X-20, Please! »

This may be a bit off-topic from the main purpose of this thread, but I was wondering what are the principal factors for NAQT's decision to do timed halfs of 8 minutes and write packets of 24/24 instead of a fixed amount, like the standard 20/20?
Nolan Winkler
Loyola Academy '12
UChicago '16
User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: NAQT and distribution change

Post by Mechanical Beasts »

Nine (or ten in college DI), for what it's worth. Arguments that have been produced before include: more questions differentiates better; the tradeoff is that questions have to be shorter; NAQT believes that the tradeoff is a net benefit. That doesn't explain timing.
Andrew Watkins
nafitzgerald
Lulu
Posts: 21
Joined: Fri May 15, 2009 5:41 pm
Location: Eden Prairie, Minnesota

Re: NAQT and distribution change

Post by nafitzgerald »

This might be taking things in a slightly different direction, but why not reduce trash (or computational math) in favor of religion and mythology? Correct me if I'm wrong, but it looks like the high school distribution has 0.9/0.8 religion and mythology out of 24/24 per packet, significantly less than the ACF distribution. Moreover, religion and mythology are the two best-converted categories!

I'd also be curious to know the conversion statistics on subcategories of fine arts. I know there are quite a few band/orchestra people at my school, at least, who would love to see more classical music; I wouldn't be surprised if architects, sculptors, and painters were dragging the conversion rate down.
Neil Fitzgerald
Eden Prairie High School '10
MIT '14
User avatar
Dresden_The_BIG_JERK
Tidus
Posts: 709
Joined: Tue Dec 09, 2008 10:56 am
Location: Lowell, IN

Re: NAQT and distribution change

Post by Dresden_The_BIG_JERK »

nafitzgerald wrote:This might be taking things in a slightly different direction, but why not reduce trash (or computational math) in favor of religion and mythology? Correct me if I'm wrong, but it looks like the high school distribution has 0.9/0.8 religion and mythology out of 24/24 per packet, significantly less than the ACF distribution. Moreover, religion and mythology are the two best-converted categories!

I'd also be curious to know the conversion statistics on subcategories of fine arts. I know there are quite a few band/orchestra people at my school, at least, who would love to see more classical music; I wouldn't be surprised if architects, sculptors, and painters were dragging the conversion rate down.
This wouldn't surprise me either. Obviously the distribution shouldn't be changed because of it, but in my experience there are more "band geeks" in quizbowl than "visual art geeks"...it wouldn't surprise me at all if that was reflected in the conversion rates.
BJ Houlding

Winnebago '04
Saint Joseph's College '08
IHSSBCA Certified Moderator
User avatar
Theory Of The Leisure Flask
Yuna
Posts: 761
Joined: Fri Aug 29, 2003 11:04 am
Location: Brooklyn

Re: NAQT and distribution change

Post by Theory Of The Leisure Flask »

Dresden The Moderator wrote:
nafitzgerald wrote:This might be taking things in a slightly different direction, but why not reduce trash (or computational math) in favor of religion and mythology? Correct me if I'm wrong, but it looks like the high school distribution has 0.9/0.8 religion and mythology out of 24/24 per packet, significantly less than the ACF distribution. Moreover, religion and mythology are the two best-converted categories!

I'd also be curious to know the conversion statistics on subcategories of fine arts. I know there are quite a few band/orchestra people at my school, at least, who would love to see more classical music; I wouldn't be surprised if architects, sculptors, and painters were dragging the conversion rate down.
This wouldn't surprise me either. Obviously the distribution shouldn't be changed because of it, but in my experience there are more "band geeks" in quizbowl than "visual art geeks"...it wouldn't surprise me at all if that was reflected in the conversion rates.
I would posit that this is because there are vastly more "band geeks" than "visual art geeks" in the school-age population at large- and the gap is even wider when you consider singers and string players. Of course, just because you play an instrument doesn't mean you automatically have a good grasp of the music canon.
Chris White
Bloomfield HS (New Jersey) '01, Swarthmore College '05, University of Pennsylvania '10. Still writes questions occasionally.
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 6113
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: NAQT and distribution change

Post by Important Bird Area »

Crazy Andy Watkins wrote:Nine (or ten in college DI), for what it's worth.
That's ten-minute halves for all college play (including DII SCT).

More about the existence of the clock later, probably after SCT is over.
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
Locked