NAQT and the Dead Zone (ETC moved from Nationals)

Dormant threads from the high school sections are preserved here.
Locked
User avatar
Ike
Auron
Posts: 1063
Joined: Sat Jul 26, 2008 5:01 pm

NAQT and the Dead Zone (ETC moved from Nationals)

Post by Ike »

I was listening to some podcasts of the HSNCT for this year and noticed how tossups on Titian and oratorios went dead because the timer went out. I don't like fine arts going dead because of the timer, (or anything but pop culture really) but I thought two fine arts questions was a lot to lose from time and decided to look into the packets to investigate this matter.

I arbitrarily set up a net of tossups between 9-14 inclusive (6 tossups, this I dub the Dead Zone) and decided to count how many fine arts questions were in this region. I chose 9-14 because the slowest readers, or teams that don't insta-power many things will have a tossup go dead in the 9 to 10ish area, while the quickest readers and teams will usually have a tossup go dead in the 13ish 14ish area. (This is assuming a tossup gets interrupted by time in the first place.)

I found that 14 rounds had tossups on art / architecture / music land in the Dead Zone, and have a high potential of going dead mid-sentence. There are three controversies I left of this list...The tossup on The Freedom Tower (round 15)...I'm quite sure that's current events, Round 6's ivory...there seemed to be some mythological elements to it even though it was most certainly architecture and the tossup on Faneuil Hall (Round 2) which could be considered as history...although its likely fine arts.

Unless architecture isn't fine arts, (which I'm quite sure it is) there is definitely a disproportionately high amount of the usually one fine arts tossup per a round to land in the Dead Zone. I might do a statistical test later on just to make sure I'm not making a gut Type 1 error, but this bugs me, NAQT has a rather slim fine arts percentage for their distribution and it seems like they are specifically trying to have it burned...now if they really wanted to burn fine arts they could just put it in the rear of the packet...

So I'm quite sure NAQT tries to make their tournament distribution balance somewhat every round...but is controlling that even fair when stuff like this happens too? And has anyone else noticed this in other sets besides this year's HSNCT?
Ike
UIUC 13
User avatar
The Time Keeper
Auron
Posts: 1327
Joined: Sat Apr 19, 2003 9:26 pm
Location: Michigan

Re: NAQT and the Dead Zone

Post by The Time Keeper »

Ike wrote: So I'm quite sure NAQT tries to make their tournament distribution balance somewhat every round...
You'd be surprised.

Also, your John Nash-off-his-meds-like ability to find patterns everywhere impresses me.

If you're right about this and they are trying to burn up fine arts questions, then brother, we will take them down.
Pat Freeburn - No particular affiliation.
User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: NAQT and the Dead Zone

Post by Mechanical Beasts »

Dufoy, a saucy impertinent Frenchman wrote:
Ike wrote: So I'm quite sure NAQT tries to make their tournament distribution balance somewhat every round...
You'd be surprised.
Let the boy give the token benefit of the doubt. (Given that he's essentially asserting that NAQT must be trying an absolutely freakish strategy for burning fine arts, it can't hurt for him to.)

Ike, test this: do categories that receive twice as much distribution space have double the chance of falling into the dead zone?
Andrew Watkins
User avatar
Sir Thopas
Auron
Posts: 1330
Joined: Tue Mar 28, 2006 10:10 pm
Location: Hunter, NYC

Re: NAQT and the Dead Zone

Post by Sir Thopas »

everyday847 wrote:Ike, test this: do categories that receive twice as much distribution space have double the chance of falling into the dead zone?
Since I have far too much free time, I'm about to start working on this for the 2008 HSNCT.

EDIT: I'm doing this comprehensively, and I don't have the packets in front of me, so if anyone else wants to get out a quick answer, go ahead.
Guy Tabachnick
Hunter '09
Brown '13

http://memoryofthisimpertinence.blogspot.com/
User avatar
First Chairman
Auron
Posts: 3651
Joined: Sat Apr 19, 2003 8:21 pm
Location: Fairfax VA
Contact:

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by First Chairman »

I think a general categorization would be nice to have rather than be focused solely on whether fine arts questions are being spiked in that area. Granted, you may uncover something that is a glitch in the randomization process of compiling the questions for HSNCT...

I would also be interested in a concurrent analysis of any IS sets to see if there is a similar pattern in answer domains (as opposed to answer-space, which I consider a concept of difficulty rather than distribution). Anyone else up to doing an ICT analysis... feel free. :)
Emil Thomas Chuck, Ph.D.
Founder, PACE
Facebook junkie and unofficial advisor to aspiring health professionals in quiz bowl
---
Pimping Green Tea Ginger Ale (Canada Dry)
User avatar
at your pleasure
Auron
Posts: 1723
Joined: Sun Aug 03, 2008 7:56 pm

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by at your pleasure »

So that's where the architecture tossups have been going.
Mabye they try to distribute evenly in the dead zone(so no one subject has more than a 1-2 in the dead zone) and art history/fine arts is hit harder because there are fewer to begin with. Did you see if there were any other patterns in the dead zone?
I would also be interested in a concurrent analysis of any IS sets to see if there is a similar pattern in answer domains (as opposed to answer-space, which I consider a concept of difficulty rather than distribution). Anyone else up to doing an ICT analysis... feel free. :)
This sounds like a good idea, but since so many regular season tournaments run untimed rounds, distribution within packets might not matter as much.\
On a related note, if certain subjects are more hurt by the dead zone than others, this is yet another reason to ditch the clock.
Douglas Graebner, Walt Whitman HS 10, Uchicago 14
"... imagination acts upon man as really as does gravitation, and may kill him as certainly as a dose of prussic acid."-Sir James Frazer,The Golden Bough

http://avorticistking.wordpress.com/
User avatar
First Chairman
Auron
Posts: 3651
Joined: Sat Apr 19, 2003 8:21 pm
Location: Fairfax VA
Contact:

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by First Chairman »

I understand that many invitationals run without a clock on NAQT packets, but it would interest me if the sorting algorithm changed for those few tournaments that run with a clock. I would like to expect that the packets in the invitational series emulate the national set to some extent.

Heck, if I knew that a math calc tossup happened around questions 3-6 every game (which they did to a very annoying extent from past invitational packets I had read), it would affect strategy if that calc tossup happened at #9 to #14 at the HSCT when it had a better chance to be spiked.
Emil Thomas Chuck, Ph.D.
Founder, PACE
Facebook junkie and unofficial advisor to aspiring health professionals in quiz bowl
---
Pimping Green Tea Ginger Ale (Canada Dry)
User avatar
Gautam
Auron
Posts: 1413
Joined: Sun Feb 11, 2007 7:28 pm
Location: Zone of Avoidance
Contact:

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Gautam »

First Chairman wrote: sorting algorithm
There are sorting algorithms involed, now? I thought it was some bored guy arbitrarily placing stuff.
Gautam - ACF
Currently tending to the 'quizbowl hobo' persuasion.
User avatar
Stained Diviner
Auron
Posts: 5085
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Stained Diviner »

First they hire Mormons, and now this!
David Reinstein
Head Writer and Editor for Scobol Solo, Masonics, and IESA; TD for Scobol Solo and Reinstein Varsity; IHSSBCA Board Member; IHSSBCA Chair (2004-2014); PACE President (2016-2018)
User avatar
Ike
Auron
Posts: 1063
Joined: Sat Jul 26, 2008 5:01 pm

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Ike »

I went through again and did some calculations just to make sure its not my gut.

I went through each tossup in the Dead Zone from each round (08 HSNCT), and put them by category according to distribution.

Geography: 11
Literature (includes religion and myth, like they say they do): 26
Sci: 23
History: 27
Math: 13
Fine Art: 17 (I've counted Freedom Tower as architecture because all of the clues head that way, and same with ivory...one round even had two tossups in that cluster of 6)
Soc: 3
Pop Culture / Sports: 12
Current Events: 9
General Knowledge / Cross: 2
Philosophy: 4
Stuff I can't classify: 3
Total 150

The stuff I can't classify includes have answers of Pravda (newspaper) Thomas Friedman, and space elevator...they're a bit weird to figure into...IF you guys have those tossups and can tell, that would really help.

So if the Dead Zone has not been tampered with we should get a semi-accurate distribution of the tournament as a whole, minus random error.

Geo: 7.3%
Lit: 17.3%
Sci: 15.3%
Hist: 18%
Math: 8.6%
Fine Art: 11.3%
Soc: 2%
Pop culture / Sports: 7.3%
Current Events: 6%
General Knowledge: 1.3%
Philosophy: 2.6%
Stuff I can't classify: 2%
Total: 99.4% (Round off error, I always rounded .3 repeating and .6 repeating to .3 and .6 respectively)

I don't know what the actual high school distribution is, but I swear I see more doses of pop culture / sports then less then 10%, it might be what hogwash they actually let fly was literature...but I'm quite sure they have a higher pop culture and sports distibution then this. If anyone has the actual distributions for HS, that would be nice.

That being said, is it me, or are they trying to keep their pop culture and sports out of the Dead Zone? I will probably check out some IS sets for more on this, but this is what i have now.
Ike
UIUC 13
User avatar
Stained Diviner
Auron
Posts: 5085
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Stained Diviner »

Here is the NAQT distribution:
Current Events 7.5%
Fine Arts 7.0%
Foreign Language 0.5%
Geography 7.0%
General Knowledge / Mixed 5.5%
History 18.5%
Literature / Mythology 18.5%
Popular Culture 7.5%
Philosophy 2.0%
Science 18.5%
Sports 4.0%
Social Science 3.5%

I'm not sure what statistical test you want to use to determine whether the differences are significant. You also have the problem that your classifications may not be the same as theirs. I'm also not sure why the last two or three questions are not considered dead zone, since those are the questions least likely to be used in an actual match.
David Reinstein
Head Writer and Editor for Scobol Solo, Masonics, and IESA; TD for Scobol Solo and Reinstein Varsity; IHSSBCA Board Member; IHSSBCA Chair (2004-2014); PACE President (2016-2018)
User avatar
Sen. Estes Kefauver (D-TN)
Chairman of Anti-Music Mafia Committee
Posts: 5647
Joined: Wed Jul 26, 2006 11:46 pm

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Sen. Estes Kefauver (D-TN) »

How often do tossups get interrupted by the clock anyway? It seemed to me like over half the time you were at some point in a bonus or about to start up the next tossup when the clock goes off.
Charlie Dees, North Kansas City HS '08
"I won't say more because I know some of you parse everything I say." - Jeremy Gibbs

"At one TJ tournament the neg prize was the Hampshire College ultimate frisbee team (nude) calender featuring one Evan Silberman. In retrospect that could have been a disaster." - Harry White
User avatar
Blackboard Monitor Vimes
Auron
Posts: 2362
Joined: Sat Aug 18, 2007 5:40 pm
Location: Richmond, VA

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Blackboard Monitor Vimes »

Mr. Rakehell wrote:How often do tossups get interrupted by the clock anyway? It seemed to me like over half the time you were at some point in a bonus or about to start up the next tossup when the clock goes off.
It happened to us at least four or five times at HSNCT, IIRC, and could have been the reason for one of our losses at TJ's NAQT tournament. Most of those were only on the third clue at most, which was really frustrating.
Sam L,
Maggie L. Walker Governor's School 2010 / UVA 2014 / VCU School of Education 2016
PACE
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 6113
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Important Bird Area »

Ike wrote:The stuff I can't classify includes have answers of Pravda (newspaper) Thomas Friedman, and space elevator... ...IF you guys have those tossups and can tell, that would really help.
Respectively: history, literature (non-fiction), science. (The second seems odd to me; I would have guessed current events.)
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
User avatar
at your pleasure
Auron
Posts: 1723
Joined: Sun Aug 03, 2008 7:56 pm

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by at your pleasure »

Yep, it looks like they're putting a disproportionate chunck of fine arts in the dead zone.
Anybody want to fire off an email to R. Hentzel or someone else in NAQT?
Douglas Graebner, Walt Whitman HS 10, Uchicago 14
"... imagination acts upon man as really as does gravitation, and may kill him as certainly as a dose of prussic acid."-Sir James Frazer,The Golden Bough

http://avorticistking.wordpress.com/
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 6113
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Important Bird Area »

dg61 wrote:Anybody want to fire off an email to R. Hentzel or someone else in NAQT?
I've already brought this thread to R.'s attention.
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
User avatar
Ike
Auron
Posts: 1063
Joined: Sat Jul 26, 2008 5:01 pm

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Ike »

Yeah, that's the college distribution I'm assuming, but I am curious as to the HS one with math involved, I would like to know what they sap away for it.

An appropriate statistical test might be chi square, again I can't do anything unless I get the math distribution also. And hopefully all of the expected counts are above five. It would be nice if Mr. Hentzel or NAQT did do their own tests...as they can classify things a lot better then I know, (although I'm quite sure, I'm quite sure I got most of it)

Another thing to consider is that NAQT is deliberately placing the fine arts not in the last 6ish tossups of the packet so they actually get read...if that happens, then they are just trying to have more fine arts (the opposite what i was thinking) and the reason of the higher count is a byproduct of a "good thing" they are trying to do.

I might do a recount with the last two or three tossups, I originally just noticed a game with tossups getting cut short on art, and that's why I went with that area there. However whatever non-random process they are using, I hope they are receptive to the last math tossup fracas from a while back, and whatever else.

I also point out that if this is deliberate, there could be a potential "question-counting problem." If at the HSNCT this happens again next year, where 2/3 of the rounds have this dead zone fine arts question, after tossup ten or 11, and a fine arts questions hasn't been read yet... during the half, a coach might choose to sub in a fine arts player, have him power something then sub him back out for whatever else...now its not like this can be ultra-exploited, but sure, it is a very neat trick and can get you something you may really need.
Ike
UIUC 13
rhentzel
Rikku
Posts: 260
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by rhentzel »

Ike wrote:Unless architecture isn't fine arts, (which I'm quite sure it is) there is definitely a disproportionately high amount of the usually one fine arts tossup per a round to land in the Dead Zone. I might do a statistical test later on just to make sure I'm not making a gut Type 1 error, but this bugs me, NAQT has a rather slim fine arts percentage for their distribution and it seems like they are specifically trying to have it burned...now if they really wanted to burn fine arts they could just put it in the rear of the packet...
I was going to start off with a bunch of statistics, but, upon consideration, I decided to note that this strikes me as an unbelievably strange motivation to attribute to NAQT. If we want fewer fine arts questions in our packets, we'll simply put fewer fine arts in them and weather the complaints. We're not going to engage in Machiavellian attempts to deprive people who enjoy fine arts of their questions by taking the time to track them down and put them in the dead zone for timed tournaments. I'm almost at a loss for words to understand why anybody would think we would consciously design our packet structure in this way.

Now, on to statistics.

The 2008 HSNCT had 40 fine arts tossups, 17 of which fell into the range of 9 to 14. Here's the complete breakdown:

1 2
3 4
5 1
7 2
8 2
9 4
10 6
12 1
13 3
14 3
16 2
17 1
18 1
20 1
21 4
22 2
23 1

Since 6 tossups is 23% of the packet, we'd only expect 9.23 fine arts tossups to fall in that region. I must say that I'm a little surprised that 17 did, but I'm not immediately convinced that this a large enough sample to indicate that there is a bug in our packet allocation software (or to accuse us of some sort of anti-art malevolence).

Speaking of that software, questions are ordered randomly within a packet without regard to their content or subject matter with one exception: Questions from the same general category (history, fine arts, science, etc.) cannot occur closer than three apart. So if the first tossup is literature, the next literature tossup can't be until number four.

At the same time, I somewhat question the definition of the dead zone as starting with tossup 10; after all, if it were restricted to 11 to 14, we would find only four fine arts in the dead zone (versus an expected 6.15). What is the appropriate definition of the dead zone?

Here are the breakdowns of the number of tossups used in each game of the HSNCT:

16 4
17 1
18 12
19 28
20 74
21 133
22 129
23 156
24 146
25 108
26 184
27 2
29 1

If we assume, for the sake of argument and the lack of more precise data, that a game with N questions had a 50% chance to burn either of the two median questions (e.g., 10 and 11 for a 20-question game and 11 and 12 for a 23-question game), then we find these numbers for the frequency with which each tossup was burned:

8 2.5 0%
9 8.5 1%
10 117.5 16%
11 168 23%
12 205 28%
13 147 20%
14 93 13%

Which suggests to me that including tossup 9, at least, in the dead zone is really untenable. That changes the imbalance to 13 observed tossups versus 7.7 predicted, with the bulk of the issue found in the six fine arts tossups that just happened to be #10. I don't have a better explanation for that than mere chance, but, really, if NAQT for some inexplicable reason wanted to burn fine arts tossups, we certainly knew prior to the 2008 HSNCT that we should put them at #11 and #12 (not #10) since we had the tossups-per-game stats for 2007 as well.

For our Invitational Series as a whole, distributed using the same system and the same algorithm, we come up with these fine arts tossup counts:

9 9
10 6
11 13
12 4
13 10
14 6

That covers 187 fine arts tossups, so we have 48 in the dead zone and an expected value of 187 x 6/20 = 56.1 so, on the whole, Invitational Series have fewer than expected number of fine arts tossups between numbers 9 and 14.

It's just chance. The 2007-2008 Invitational Series had fewer, the 2008 HSNCT had relatively more.
rhentzel
Rikku
Posts: 260
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Re: NAQT and the Dead Zone

Post by rhentzel »

Dufoy, a saucy impertinent Frenchman wrote:
Ike wrote: So I'm quite sure NAQT tries to make their tournament distribution balance somewhat every round...
You'd be surprised.
NAQT goes to fairly significant lengths to ensure that its tournaments and packets are balanced; assuming that I'm interpreting your comment correctly as saying that we don't go far enough, what would you like to see done differently?
rhentzel
Rikku
Posts: 260
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by rhentzel »

dg61 wrote:So that's where the architecture tossups have been going.
Mabye they try to distribute evenly in the dead zone(so no one subject has more than a 1-2 in the dead zone) and art history/fine arts is hit harder because there are fewer to begin with. Did you see if there were any other patterns in the dead zone?
NAQT does not currently make any programmatic (or manual) attempt to reorder questions to avoid the "dead zone," either for specific questions (which we might believe to be especially good) or for specific categories.

This is out of a basic belief that the inherent randomness in the packet ordering process will balance out.
rhentzel
Rikku
Posts: 260
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by rhentzel »

Ike wrote:I don't know what the actual high school distribution is, but I swear I see more doses of pop culture / sports then less then 10%, it might be what hogwash they actually let fly was literature...but I'm quite sure they have a higher pop culture and sports distibution then this. If anyone has the actual distributions for HS, that would be nice.
Sports and popular culture comprised 7.2% of the 2008 HSNCT.
User avatar
The Time Keeper
Auron
Posts: 1327
Joined: Sat Apr 19, 2003 9:26 pm
Location: Michigan

Re: NAQT and the Dead Zone

Post by The Time Keeper »

rhentzel wrote: NAQT goes to fairly significant lengths to ensure that its tournaments and packets are balanced; assuming that I'm interpreting your comment correctly as saying that we don't go far enough, what would you like to see done differently?
I've frequently heard people mention that the individual packets do not (or did not, this may have been changed) maintain a strict distribution among the first x tossups where x could either be 20 (in an ideal, non-clock world) or perhaps some derived value based on the average number of tossups likely to be read in a typical timed game (variance between readers and teams can understandably make this figure difficult to find). People have complained in the past that this has lead to skewed category selection in the portions of packets that are read during games which seems pretty undesirable to everyone and also fairly easily remedied. Again, I'm basing this off of what I've heard others say and a couple packet sets that I've been able to look over so I don't know how widespread this issue is or if it has already been addressed this past year.

I believe I have also seen posts here in the past where NAQT members have stated that NAQT aims for an overall "per tournament set" distribution but not a set per packet distribution. I believe this post was made in regard to criticisms of one of the high school sets and I'll try to find the exact post so I can make sure I'm not misrepresenting what was said.

I apologize if any of my points are either outdated or blatantly incorrect.
Pat Freeburn - No particular affiliation.
User avatar
Matt Weiner
Sin
Posts: 8145
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Matt Weiner »

The accusation that certain subjects (usually, art) were "buried" in the portion of the questions less likely to be read has in fact been made a few times in the past, including by a person who now works for NAQT. However, I always found it hard to believe, and NAQT has now explicitly denied it twice ( at viewtopic.php?f=21&t=5016&p=69621#p69621 and above in this thread). Absent some better statistical evidence than a 15-packet sample, I see no reason to doubt the truth of those denials.

With that said, I've also heard no good argument against going to a fixed per-packet distribution instead of the current tournament-level approach. I believe R. stated, in his interview on Fred's blog, that avoiding the per-packet distribution allows for the sort of fine-tuning ("there will be exactly 3 questions on Indian religions in this tournament" or whatever) that NAQT's computerized packet assembling process makes possible. I didn't quite get that, since it seems to me that you could continue to take advantage of the NAQT question production system to make fine-grained subdistributions for every subject across the whole tournament, while also making sure that there are always 4 literature tossups in each round instead of 3 in some and 5 in others (or whatever the range actually is).
Matt Weiner
Advisor to Quizbowl at Virginia Commonwealth University / Founder of hsquizbowl.org
User avatar
Sir Thopas
Auron
Posts: 1330
Joined: Tue Mar 28, 2006 10:10 pm
Location: Hunter, NYC

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Sir Thopas »

For me, at least, the real problem is that NAQT's distribution accounts for pitifully little art. I have a feeling that Ike is frustrated by the same issue. The quoted figure of 7.0% even seems suspect to me—going through the first two packets on the podcast, in which 21 and 23 tossups were read in the two games. Geography and arts are both supposed to be equal, but I found:

5/6 geography (Mt. Hood, Lake Chad, Okinawa [which probably went into the history distribution], Turin, Natchez Trace, and UAE)
2 arts (Monet, Primavera). I'm not putting Faneuil Hall into arts, because it's not.

Even accounting for the (unnecessary) fluctuations between packets, it's ridiculous that two distributions which are supposed to be equal actually differ—in whatever, even somewhat negligible, sample size—by more than 100%. Not to mention that there is absolutely no classical music, sculpture, and so on.

One more note about the distribution: in packet 1, there's no real literature tossup until tossup 15. Leander was tossup 2; The Golden Bough, tossup 3, but still, these aren't really literature. Seriously, tossup 15! Think about that for a second! You take a huge category in quizbowl, and then just decide to leave it out for the first three-fifths or three-quarters of the game. Whatever randomizing computer program is used, I think it needs a change.


EDIT: OK, that Leander tossup is really neither literature nor myth, although I'm sure it was classified as such. My transcription:

This mythological character shares his first name with the leader of the Jameson Raid that contributed to the Second Boer War. He was a youth from Abydos whose principal exploit was emulated by Lord Byron. He loved a European priestess of Aphrodite and crossed the Hellespont nightly to see her. FTP, name this title character of a Christopher Marlowe poem, who drowns himself swimming to see Hero.

It's sort of all over the place. Calling it literature would a bit weird, because a lot of it isn't, while calling it myth makes the fact that this was the only myth tossup in the first two rounds all the more outrageous.

EDIT, pt. 2: Whoops, so I was going off the distribution for college. I doubt there'd be that much of a difference but for the math computation. Surely, in any case, geography is not given 150% more weight than arts in HS per the distribution.
Last edited by Sir Thopas on Wed Aug 13, 2008 8:46 pm, edited 2 times in total.
Guy Tabachnick
Hunter '09
Brown '13

http://memoryofthisimpertinence.blogspot.com/
User avatar
Blackboard Monitor Vimes
Auron
Posts: 2362
Joined: Sat Aug 18, 2007 5:40 pm
Location: Richmond, VA

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Blackboard Monitor Vimes »

A Servant wrote:For me, at least, the real problem is that NAQT's distribution accounts for pitifully little art. I have a feeling that Ike is frustrated by the same issue. The quoted figure of 7.0% even seems suspect to me—going through the first two packets on the podcast, in which 21 and 23 tossups were read in the two games. Geography and arts are both supposed to be equal, but I found:

5/6 geography (Mt. Hood, Lake Chad, Okinawa [which probably went into the history distribution], Turin, Natchez Trace, and UAE)
2 arts (Monet, Primavera). I'm not putting Faneuil Hall into arts, because it's not.

Even accounting for the (unnecessary) fluctuations between packets, it's ridiculous that two distributions which are supposed to be equal actually differ—in whatever, even somewhat negligible, sample size—by more than 100%. Not to mention that there is absolutely no classical music, sculpture, and so on.

One more note about the distribution: in packet 1, there's no real literature tossup until tossup 15. Leander was tossup 2; The Golden Bough, tossup 3, but still, these aren't really literature. Seriously, tossup 15! Think about that for a second! You take a huge category in quizbowl, and then just decide to leave it out for the first three-fifths or three-quarters of the game. Whatever randomizing computer program is used, I think it needs a change.
The art is annoying, but what I find even worse is that sports is considered separate from pop culture but lit is lumped with RMP (sort of what Guy is getting at in his last paragraph). I don't understand the reasoning there. I also don't understand the reasoning for a tournament-based rather than round-based distribution. Say randomization leaves a round with 2 "lit" and 6 science toss-ups. The better team doesn't necessarily win; the team with the better science player does. That may not be the best example, but hopefully everyone gets the basic point. It doesn't seem fair to have rounds with different compositions.
Sam L,
Maggie L. Walker Governor's School 2010 / UVA 2014 / VCU School of Education 2016
PACE
User avatar
Sir Thopas
Auron
Posts: 1330
Joined: Tue Mar 28, 2006 10:10 pm
Location: Hunter, NYC

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Sir Thopas »

MLWGS-Gir wrote:The art is annoying, but what I find even worse is that sports is considered separate from pop culture but lit is lumped with RMP (sort of what Guy is getting at in his last paragraph). I don't understand the reasoning there. I also don't understand the reasoning for a tournament-based rather than round-based distribution. Say randomization leaves a round with 2 "lit" and 6 science toss-ups. The better team doesn't necessarily win; the team with the better science player does. That may not be the best example, but hopefully everyone gets the basic point. It doesn't seem fair to have rounds with different compositions.
For what it's worth, here's what I counted of the big three in packet 1 (21 tossups heard):
3 lit (Plautus, Invisible Man, Hemingway)
3 sci (Gay-Lussac's Law, Bohr, blood transfusion. I'm not counting a tossup on "wasabi" as science)
3 hist (Pu Yi, Salmon Chase, Balfour Declaration)

And packet 2 (23 tossups heard):
4/5 lit (The Lottery, Maxim Gorky, The Waste Land, Art Spiegelman (?), C.S. Lewis)
6 sci (Lagrange points, Flores, urea, lambda, zero point energy, parasympathetic nervous system. The first four were in the first 7 tossups)
3-5 hist (Okinawa (?), Faneuil Hall (?), Kerensky, Sitting Bull, Leonidas)

This wild variability is supremely frustrating to play on, and just plain unfair.
Guy Tabachnick
Hunter '09
Brown '13

http://memoryofthisimpertinence.blogspot.com/
User avatar
grashid
Lulu
Posts: 70
Joined: Fri Jun 06, 2008 5:34 pm

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by grashid »

rhentzel wrote: The 2008 HSNCT had 40 fine arts tossups, 17 of which fell into the range of 9 to 14. Here's the complete breakdown:

1 2
3 4
5 1
7 2
8 2
9 4
10 6
12 1
13 3
14 3
16 2
17 1
18 1
20 1
21 4
22 2
23 1
It seems frustrating to me (as I interpret this data table as art questions per round) that round 10 had SIX fine arts question while the Last THREE rounds has 7 total.
g. rashid Walton 09
"After a few minutes of conversation, Ehrenfest remarked "I think I like your papers better than you," to which Pauli shot back "I think I like you better than your papers." The two became very good friends from then on."
Susan
Forums Staff: Administrator
Posts: 1812
Joined: Fri Aug 15, 2003 12:43 am

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Susan »

grashid wrote:It seems frustrating to me (as I interpret this data table as art questions per round) that round 10 had SIX fine arts question while the Last THREE rounds has 7 total.
It's not art questions per round, it's the number of times that an art question was the first (second, third...twenty-third) question in the packet.
Susan
UChicago alum (AB 2003, PhD 2009)
Member emerita, ACF
User avatar
Sir Thopas
Auron
Posts: 1330
Joined: Tue Mar 28, 2006 10:10 pm
Location: Hunter, NYC

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Sir Thopas »

grashid wrote:It seems frustrating to me (as I interpret this data table as art questions per round) that round 10 had SIX fine arts question while the Last THREE rounds has 7 total.
The first number is the question number within each round, not the round number.
Guy Tabachnick
Hunter '09
Brown '13

http://memoryofthisimpertinence.blogspot.com/
User avatar
Ike
Auron
Posts: 1063
Joined: Sat Jul 26, 2008 5:01 pm

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Ike »

At the same time, I somewhat question the definition of the dead zone as starting with tossup 10; after all, if it were restricted to 11 to 14, we would find only four fine arts in the dead zone (versus an expected 6.15). What is the appropriate definition of the dead zone?
It was completely arbitrary. I know sometimes some bad readers get through only 9 or 10 questions, while Mr. Hentzel gets through a lot more, the definition was selected while thinking about it at midnight, and your data provides for a much better definition. But thanks for the data, saved me / us tons of work.
I'm almost at a loss for words to understand why anybody would think we would consciously design our packet structure in this way....to accuse us of some sort of anti-art malevolence
It just seems very odd, and I don't like tossups getting burned because of the timer. Especially if its a category that has a 1 or 2 frequency per a packet. I know its nearly impossible to control what gets burned and what doesn't, but when that tossup on Degas slips through your fingers...and you won't be seeing it until the next round, that bites.
I have a feeling that Ike is frustrated by the same issue.
At a rate of 1-2 fine arts tossups per a packet, you won't hear that much of a complaint, there are a lot of other things for me to complain about, which go beyond the scope of this thread.
For what it's worth, here's what I counted of the big three in packet 1 (21 tossups heard):
3 lit (Plautus, Invisible Man, Hemingway)
3 sci (Gay-Lussac's Law, Bohr, blood transfusion. I'm not counting a tossup on "wasabi" as science)
3 hist (Pu Yi, Salmon Chase, Balfour Declaration)

And packet 2 (23 tossups heard):
4/5 lit (The Lottery, Maxim Gorky, The Waste Land, Art Spiegelman (?), C.S. Lewis)
6 sci (Lagrange points, Flores, urea, lambda, zero point energy, parasympathetic nervous system. The first four were in the first 7 tossups)
3-5 hist (Okinawa (?), Faneuil Hall (?), Kerensky, Sitting Bull, Leonidas)
Art Spiegelman is lit, whether its legitimate is open for debate, but I'm willing to say it is on the backing of a few people I've talked to. Flores is social science (anthropology). Wasabi I think is general knowledge. Okinawa is history, as most of the clues dont mention geographical elements.
Ike
UIUC 13
User avatar
Sen. Estes Kefauver (D-TN)
Chairman of Anti-Music Mafia Committee
Posts: 5647
Joined: Wed Jul 26, 2006 11:46 pm

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Sen. Estes Kefauver (D-TN) »

Ike, this whole thing is absurd. NAQT has it's problems, but R. is absolutely right that this is just crazy paranoia where you are digging for problems to complain about, and there is no way in hell that NAQT is actively stuffing one category in a space of 3 tossups you randomly decided were a "dead zone."
Charlie Dees, North Kansas City HS '08
"I won't say more because I know some of you parse everything I say." - Jeremy Gibbs

"At one TJ tournament the neg prize was the Hampshire College ultimate frisbee team (nude) calender featuring one Evan Silberman. In retrospect that could have been a disaster." - Harry White
User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Mechanical Beasts »

Yeah, I'm on Charlie's side here. NAQT has absolutely no rationale to do what you accuse them of doing, or to do it in the way they accuse: a more reliable way of being devious would be to make fine arts disproportionately after tossup twenty.

I think that NAQT merits criticism, certainly, as do all institutions, for some of its choices, but they're no evil empire, and they shouldn't be treated with the paranoia that might accompany one.

I need to rip the comma key off my keyboard.
Andrew Watkins
User avatar
AKKOLADE
Sin
Posts: 15782
Joined: Thu Apr 24, 2003 8:08 am

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by AKKOLADE »

For real guys. There's plenty of valid NAQT criticisms, and to depend upon reaches when you can go with the ones that do exist is unnecessary.
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
The Atom Strikes!
Tidus
Posts: 612
Joined: Wed Jun 13, 2007 7:05 pm
Location: Houston, Texas

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by The Atom Strikes! »

I think that this whole problem would just be solved if in NAQT rounds, the moderators would simply finish whatever tossup they were currently on when the timer went off.
Henry Gorman, Wilmington Charter '09, Rice '13, PhD History Vanderbilt '1X
User avatar
Eärendil
Wakka
Posts: 138
Joined: Fri Feb 08, 2008 8:56 am
Location: Queenstown, New Zealand

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Eärendil »

Aww, but that would take all the fun out of negging on purpose at the last second!
Arnav // Stanford University
User avatar
Ike
Auron
Posts: 1063
Joined: Sat Jul 26, 2008 5:01 pm

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Ike »

Ike, this whole thing is absurd. NAQT has it's problems, but R. is absolutely right that this is just crazy paranoia where you are digging for problems to complain about, and there is no way in hell that NAQT is actively stuffing one category in a space of 3 tossups you randomly decided were a "dead zone."
I didn't mean for it to be played off as paranoia, but you have to admit that some of the placement of questions is off, and at the very least is annoying. I certainly didn't randomly decide this, and it is statistically odd.
NAQT has absolutely no rationale to do what you accuse them of doing,
(I'm in the middle of something right now, so I will check my posts, later) I said that there is an impression of them doing something, not that they are, which is something I think aretwo different things. Now ok, I fully concede it may be a bad thing to suggest that in a first post, but I didn't mean for my posting to be interpreted like that and am fully aware of this in the future.
they're no evil empire, and they shouldn't be treated with the paranoia that might accompany one.
I didn't paint any pictures like this, and I enjoy NAQT. I dont have that paranoia and I find it pretty funny that I would have this image in my head.
There's plenty of valid NAQT criticisms, and to depend upon reaches when you can go with the ones that do exist is unnecessary.
Maybe so but a mathematical basis for criticism is one good way to get certain things changed. Now whether or not my initial post has any merit at this point, it brought up some points which Guy noticed...that probably need to be addressed. If I though posting here wouldn't spark anything , I would have just fired this data to Mr. Hentzel myself.

Just to make myself clear, I dont think NAQT deliberately said / thought "thrash fine arts" or an equivalent, but there was something anomalous. I again concede full responsibility for suggesting that in the first place, and it was a mis-guided attempt to add gravitas to my point. Is it minor to NAQT's problems? You guys are right. And quite frankly I dont think I swatted a fly with a nuclear missile. I just posted what I found. You don't see me making extreme charges of contempt, nor do I intend to against NAQT. All the beefs I have are personal preference, and there's no changing that. But this is something a bit more solid.
Ike
UIUC 13
User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Mechanical Beasts »

Sure, it led to Guy bringing up concerns; that's good. My question, though, is whether the best way to have Guy start bringing up concerns is to present data on a different issue and say "something that can't possibly be all random is going on here" when in fact it'd be highly unusual for anything else to be the case.

It would have been equally good if Guy ended up saying his piece without your argument, since I don't think your argument is too likely to hold water. I'm generally wary of anything that could cause NAQT to think "oh, well, now they're just making stuff up;" if our claims are even close to baseless, we lose a little legitimacy in our other objections.
Andrew Watkins
User avatar
at your pleasure
Auron
Posts: 1723
Joined: Sun Aug 03, 2008 7:56 pm

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by at your pleasure »

I'll concur with Guy and Sarah, and I will add that the reason that Fine arts and the other lesser academic categories(ie, RMP and whatever I'm forgetting) are rarer and more sensitve to quirks of distribution is proabably the trash and less academic categories getting too much space.
And Hentzel, I don't think anyone (possibly excepting Ike) thought there was an intentional effort to burn fine arts.
Douglas Graebner, Walt Whitman HS 10, Uchicago 14
"... imagination acts upon man as really as does gravitation, and may kill him as certainly as a dose of prussic acid."-Sir James Frazer,The Golden Bough

http://avorticistking.wordpress.com/
User avatar
Blackboard Monitor Vimes
Auron
Posts: 2362
Joined: Sat Aug 18, 2007 5:40 pm
Location: Richmond, VA

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by Blackboard Monitor Vimes »

dg61 wrote:I'll concur with Guy and Sarah, and I will add that the reason that Fine arts and the other lesser academic categories(ie, RMP and whatever I'm forgetting) are rarer and more sensitve to quirks of distribution is proabably the trash and less academic categories getting too much space.
And Hentzel, I don't think anyone (possibly excepting Ike) thought there was an intentional effort to burn fine arts.
That, and no one's really asking you to change your established distribution. I think we're all just interested in the logic behind it, and, if by exploring that logic for what could possibly be the first time in quite a while, you decide that some small changes could be worth considering further, I'm sure most of us would love to contribute to that discussion in order to move from Ike's suspicions to a productive dialogue.
Sam L,
Maggie L. Walker Governor's School 2010 / UVA 2014 / VCU School of Education 2016
PACE
User avatar
First Chairman
Auron
Posts: 3651
Joined: Sat Apr 19, 2003 8:21 pm
Location: Fairfax VA
Contact:

Re: NAQT and the Dead Zone (ETC moved from Nationals)

Post by First Chairman »

I also want to voice in to say, the presence of a pattern does not imply intent for that pattern. However, it should make us more sensitive to these issues. Nothing more than that.
Emil Thomas Chuck, Ph.D.
Founder, PACE
Facebook junkie and unofficial advisor to aspiring health professionals in quiz bowl
---
Pimping Green Tea Ginger Ale (Canada Dry)
Locked