Side-by-side stats for 2010-11 sets

Dormant threads from the high school sections are preserved here.
Locked
User avatar
AKKOLADE
Sin
Posts: 15782
Joined: Thu Apr 24, 2003 8:08 am

Side-by-side stats for 2010-11 sets

Post by AKKOLADE »

I put together an Excel sheet that puts the average, median, 3rd quartile, 1st quartile, standard deviation, max, and min PPB values of the various high school quiz bowl sets from last year side-by-side. This includes every stat line that I could find for these tournaments.

The sets included are:

BATE
BHSAT
DAFT
DCCAT
Delta Burke
FKT
GSAC
HFT
HT15
HT16
HT17
IMSANITY
IS96
IS97A
IS98
IS99A
IS100
IS101A
IS102
IS103A
IS104
IS105
LIST
MD Spring
Prison Bowl
PSATTPOT
RTO
SCOP
VCUVandy
WUHSAC

The spreadsheet should default to the overall view.

I excluded stats that did not include PPB for obvious reason. I also excluded cases like "team gets 5 ppb, only has one bonus, 30s it" to avoid skewing the data; there was only one or two cases of this.

I might have/probably missed some data, particularly tournaments without teams that would make my "best teams watch list" for hsqbrank and JV tournaments. I did not include middle school only events/divisions, because that doesn't quite seem to be the point.
Attachments
all sets.xlsx
(171.07 KiB) Downloaded 533 times
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
User avatar
Down and out in Quintana Roo
Auron
Posts: 2907
Joined: Wed Apr 09, 2008 7:25 am
Location: Camden, DE
Contact:

Re: Side-by-side stats for 2010-11 sets

Post by Down and out in Quintana Roo »

This is pretty ridiculously awesome stuff.

One question, and i guess this is best perhaps addressed to Jeff, is regarding Fred's "5 point spread pie" chart in the last tab of the sheet. When it comes to NAQT IS sets, is it NAQT's goal to achieve as much of a perfect "bell curve" on PPB as possible?

For example, Fred's data shows that
  • PPB Range #
    0.00-4.99 51
    5.00-9.99 381
    10.00-14.99 479
    15.00-19.99 334
    20.00-24.99 114
    25.00-30.00 2
Obviously NAQT would want the higher two categories to rise in number, but is there a specific goal that they're looking for in their sets? Certainly, to me, there's a problem with only 2 teams, in the entire country, being able to reach over 25ppb on all the IS sets combined, yet more than 50 teams could not even reach 5ppb. Or, to take that even further, it's still a problem to me that less than 115 teams, in the entire country, could break 20ppb, but that almost 400 teams could not reach 10ppb.
Mr. Andrew Chrzanowski
Caesar Rodney High School
Camden, Delaware
CRHS '97-'01
University of Delaware '01-'05
CRHS quizbowl coach '06-'12
http://crquizbowl.edublogs.org
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 6113
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: Side-by-side stats for 2010-11 sets

Post by Important Bird Area »

This is awesome work, Fred.

As for theory-of-bonus-conversion:

0. Obviously it's a good idea to make our bonuses easier; of course we would like to see the number of teams in those highest categories increase and improve the overall averages by a couple of points.

That being said, I don't think it's possible to generate a true bell curve with even tails on both sides. Some explanations:

1. Teams on the far left end will always be more numerous than teams on the far right, because "let's find four interested freshmen who've never played before so we can field an E team" is easier than "let's build a national championship-quality team that can put up 26 ppb."

2. The ceiling on high bonus conversions is more limiting than the floor on low bonus conversions. Even awesome teams will miss bonus parts sometimes because they mishear the moderator, disagree while conferring, or just plain get a bonus in some subcategory the team doesn't know well enough to earn all 30 points. Furthermore, a team putting up very high bonus conversions can always leave and go play something more challenging (up to and including "let's go to CO where no team in history has ever broken 25 ppb!"). We don't know what full-strength State College A would have converted on an IS set in 2011; probably something pretty ridiculous, given the 66/34/36 and 24.5 ppb stat line they assembled with only three players. By contrast, it's easy to find brand-new teams that put up 3 to 4 ppb on introductory sets. Those teams don't have anywhere else to go because the alternative to playing novice sets is not playing. (And beyond a certain bound, editors run into problems making the easy parts of bonuses that much easier, as anyone who has ever written the last few bonuses of a middle school set can attest.)

I would be interested, by the way, to see what the highest all-time bonus conversion figure is for any team at any level.
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
User avatar
AKKOLADE
Sin
Posts: 15782
Joined: Thu Apr 24, 2003 8:08 am

Re: Side-by-side stats for 2010-11 sets

Post by AKKOLADE »

It should be noted that I completely forgot that 5 point split pie chart tab thing was on the sheet before I uploaded it (I think it's a remnant from the "all IS stats" sheet). So it has no new info.
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
User avatar
Stained Diviner
Auron
Posts: 5085
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: Side-by-side stats for 2010-11 sets

Post by Stained Diviner »

Fred is even more awesome than I thought.

I wonder how difficult it would be to perform Excelsior rankings on this data to correct for field quality. Basically, you would take all the bonus conversions for a single team during the year and pair them up into matches. If a team appears four times in the spreadsheet, then you would get a total of six matches between different combinations of the sets they played on. The result might be a good measure of which set was the easiest. It wouldn't be a perfect comparison because team composition varies from week to week, but given the amount of data that sort of noise might not be so bad.

One of the problems with just using the front page numbers is that some sets were OK at some sites and not at others. For example, the overall numbers for BATE are bad, but if you look at the raw data you see that it played well at two or three sites and poorly at two or three sites. So, the conclusion shouldn't be that BATE is bad, it should be that it's more appropriate for stronger fields, assuming that this year's set is similar to last year's. I think this data is useful to tournaments picking sets, though that job is still complicated by the fact that front page data is a little misleading because it is very dependent on last year's field strengths.

It seems like a goal would be to have a median around 15 and the quartiles somewhere around 11 and 19, which would lead to a standard deviation around 6. Several sets were around those numbers, though several were not. It's difficult to hit those numbers, in part because often a set is written without knowing what field will play on it, especially the fields at mirrors. It's also important to keep in mind that whether or not a set hits the right numbers is not the best measure of whether or not a set was good or not--it has to have interesting clues about interesting topics, pyramidal tossups with accessible answers, and bonuses of consistent difficulty, all of which are more important than appropriate bonus conversion but more difficult to measure.

One interesting comparison is that IS104 and IS105 had better numbers than the earlier IS sets. I'm wondering if that's just random noise, a difference in field strength, or a reflection of NAQT looking at some of the low earlier numbers and trying to make their IS Sets a little easier as the year went on.
David Reinstein
Head Writer and Editor for Scobol Solo, Masonics, and IESA; TD for Scobol Solo and Reinstein Varsity; IHSSBCA Board Member; IHSSBCA Chair (2004-2014); PACE President (2016-2018)
User avatar
Excelsior (smack)
Rikku
Posts: 386
Joined: Sun Jan 25, 2009 12:20 am
Location: Madison, WI

Re: Side-by-side stats for 2010-11 sets

Post by Excelsior (smack) »

I wonder how difficult it would be to perform Excelsior rankings on this data to correct for field quality. Basically, you would take all the bonus conversions for a single team during the year and pair them up into matches. If a team appears four times in the spreadsheet, then you would get a total of six matches between different combinations of the sets they played on. The result might be a good measure of which set was the easiest. It wouldn't be a perfect comparison because team composition varies from week to week, but given the amount of data that sort of noise might not be so bad.
Let me see if I'm understanding this. Let's say that school X played on the following sets with the following PPBs:
IS-103A: 22
HSAPQ-18: 19
IS-105
HFT: 12

Then, we would create matches between each of these sets using bonus conversion as "match scores", so that IS-103A has three wins, HSAPQ-18 has two wins, IS-105 has one win, and HFT has zero wins. We do this for all the data that Fred has, and then we run this through KRACH. This would give us a ranking in which the top-ranked set is the easiest one of the year, while the bottom-ranked one is the hardest. Does this sound right? If so, I can implement this (though this seems to me like it would be more of a historical curiosity than anything else).
Ashvin Srivatsa
Corporate drone '?? | Yale University '14 | Sycamore High School (OH) '10
User avatar
Excelsior (smack)
Rikku
Posts: 386
Joined: Sun Jan 25, 2009 12:20 am
Location: Madison, WI

Re: Side-by-side stats for 2010-11 sets

Post by Excelsior (smack) »

Ranking of 2010-11 sets by difficulty

I did this. In addition to noise from changing lineups and such, there is also noise due to school-naming shenanigans (umpteen schools named University, etc). I went through and made a bit of an effort to rename teams when there were obvious changes to be made (DCD <-> Detroit Country Day, etc), but I was obviously not able to do this completely.
Ashvin Srivatsa
Corporate drone '?? | Yale University '14 | Sycamore High School (OH) '10
User avatar
Mewto55555
Tidus
Posts: 709
Joined: Sat Mar 13, 2010 9:27 pm

Re: Side-by-side stats for 2010-11 sets

Post by Mewto55555 »

It seems like there are disproportionately many sets from later in the year near the top (perhaps due to teams improving as the year went on?)
Max
formerly of Ladue, Chicago
User avatar
Stained Diviner
Auron
Posts: 5085
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: Side-by-side stats for 2010-11 sets

Post by Stained Diviner »

Thanks, Ashvin. That's pretty cool.

While tournaments near the end of the year do have a natural advantage due to teams getting better, I think this also had something to do with people deciding during last year that sets were too hard in general and then purposely making sets like LIST and VCUBilt more accessible.

Did RTO use the same scoring system as everybody else? If those numbers are out of 30 points, then they are extremely low.
David Reinstein
Head Writer and Editor for Scobol Solo, Masonics, and IESA; TD for Scobol Solo and Reinstein Varsity; IHSSBCA Board Member; IHSSBCA Chair (2004-2014); PACE President (2016-2018)
User avatar
AKKOLADE
Sin
Posts: 15782
Joined: Thu Apr 24, 2003 8:08 am

Re: Side-by-side stats for 2010-11 sets

Post by AKKOLADE »

Leucippe and Clitophon wrote:One of the problems with just using the front page numbers is that some sets were OK at some sites and not at others. For example, the overall numbers for BATE are bad, but if you look at the raw data you see that it played well at two or three sites and poorly at two or three sites. So, the conclusion shouldn't be that BATE is bad, it should be that it's more appropriate for stronger fields, assuming that this year's set is similar to last year's. I think this data is useful to tournaments picking sets, though that job is still complicated by the fact that front page data is a little misleading because it is very dependent on last year's field strengths.
My impression of BATE from the data alone is that it was somewhat difficult, but not ridiculously so, for top teams (21st out of 30 sets in 1st quartile), but for teams below that tier, it was too hard (29th out of 30 in both median and 3rd quartile ppb).

However, I certainly wouldn't advise looking at the data in a vacuum.
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
No Electricity Required
Wakka
Posts: 198
Joined: Wed May 20, 2009 10:09 pm
Location: West-Central Illinois

Re: Side-by-side stats for 2010-11 sets

Post by No Electricity Required »

Excelsior (smack) wrote:Ranking of 2010-11 sets by difficulty

I did this. In addition to noise from changing lineups and such, there is also noise due to school-naming shenanigans (umpteen schools named University, etc). I went through and made a bit of an effort to rename teams when there were obvious changes to be made (DCD <-> Detroit Country Day, etc), but I was obviously not able to do this completely.
Is there a reason GSAC is not in this list?
tristan willey
formerly: macomb, uiuc
writer: naqt
he/him or they/them
User avatar
Excelsior (smack)
Rikku
Posts: 386
Joined: Sun Jan 25, 2009 12:20 am
Location: Madison, WI

Re: Side-by-side stats for 2010-11 sets

Post by Excelsior (smack) »

No Electricity Required wrote:Is there a reason GSAC is not in this list?
Oops. Fixed.
Ashvin Srivatsa
Corporate drone '?? | Yale University '14 | Sycamore High School (OH) '10
User avatar
Francis the Talking France
Wakka
Posts: 164
Joined: Sun Oct 03, 2010 4:01 am
Location: Denver, CO
Contact:

Re: Side-by-side stats for 2010-11 sets

Post by Francis the Talking France »

Leucippe and Clitophon wrote:Thanks, Ashvin. That's pretty cool.

While tournaments near the end of the year do have a natural advantage due to teams getting better, I think this also had something to do with people deciding during last year that sets were too hard in general and then purposely making sets like LIST and VCUBilt more accessible.

Did RTO use the same scoring system as everybody else? If those numbers are out of 30 points, then they are extremely low.
RTO was actually that hard I think..Even the top teams we had there were struggling mightily with the bonuses. The tossups were fine though, I think.
Matt Duchan
Chapel Hill High School '12
University of Denver '16
Director of the 2013 Colorado State History Bowl
User avatar
rchschem
Yuna
Posts: 762
Joined: Thu Apr 24, 2003 7:36 am
Location: Durham, NC

Re: Side-by-side stats for 2010-11 sets

Post by rchschem »

MattNC wrote:
Leucippe and Clitophon wrote:
Did RTO use the same scoring system as everybody else? If those numbers are out of 30 points, then they are extremely low.
RTO was actually that hard I think..Even the top teams we had there were struggling mightily with the bonuses. The tossups were fine though, I think.
I'm gonna go with weak field on that one. I would not feel good about comparing our questions on the basis of points put up at the tournament. No offense to the teams involved, it's just where we are right now.

Eric
Eric Grunden, Research Triangle High School/NCATA
User avatar
Francis the Talking France
Wakka
Posts: 164
Joined: Sun Oct 03, 2010 4:01 am
Location: Denver, CO
Contact:

Re: Side-by-side stats for 2010-11 sets

Post by Francis the Talking France »

Every team aside from TJ Classical that attended the HSNCT had a lower PPB at RTO than at the HSNCT though. I think it's a combination of a weak field and a tough set.
Matt Duchan
Chapel Hill High School '12
University of Denver '16
Director of the 2013 Colorado State History Bowl
User avatar
Broad-tailed Grassbird
Tidus
Posts: 712
Joined: Sun Nov 23, 2008 10:27 pm
Location: Somewhere nice.

Re: Side-by-side stats for 2010-11 sets

Post by Broad-tailed Grassbird »

List of villages in West Virginia wrote:This is pretty ridiculously awesome stuff.

One question, and i guess this is best perhaps addressed to Jeff, is regarding Fred's "5 point spread pie" chart in the last tab of the sheet. When it comes to NAQT IS sets, is it NAQT's goal to achieve as much of a perfect "bell curve" on PPB as possible?

For example, Fred's data shows that
  • PPB Range #
    0.00-4.99 51
    5.00-9.99 381
    10.00-14.99 479
    15.00-19.99 334
    20.00-24.99 114
    25.00-30.00 2
Obviously NAQT would want the higher two categories to rise in number, but is there a specific goal that they're looking for in their sets? Certainly, to me, there's a problem with only 2 teams, in the entire country, being able to reach over 25ppb on all the IS sets combined, yet more than 50 teams could not even reach 5ppb. Or, to take that even further, it's still a problem to me that less than 115 teams, in the entire country, could break 20ppb, but that almost 400 teams could not reach 10ppb.

I get what you are saying Andrew, but it's really hard to make the 0-5 group get smaller without truly giving away points. I checked from the past MSU tournaments, and the teams that actually pulled this off were all getting less than 15 tossups in the whole day. That's truly miserable, but often times it's 2 freshmen who have never played before. Their knowledge base is smaller than the average quiz bowler to begin with, and with the diverse subjects quiz bowl has, these freshmen probably don't anything about the bonuses they actually get.

As for getting above 25 points in a tournament. That is a truly remarkable feat. That means you average less than one wrong bonus part for every two tossups you get. Over the course of a tournament a team with a ~25 PPB is going to get tons and tons and tons of bonuses. High School quizbowlers, even the top ones do suffer from fatigue. And eventually even the best of teams will zero a bonus on occasion. You can only get so many 10s in a tournament before it knocks you down to a 22-23 PPB. It would be nice to make a true bell curve, but for competition purposes, the current level of bonus difficulty at NAQT tournaments is ideal for the top 60-70% of competition. The bottom 30% are probably a little in over their heads, but the best way to get better is to play questions a little above your level, especially at that age.
Nalin
Scranton Middle School (2000-2003)
Brighton High School (2003-2007)
Michigan State University (2007-2011)
Semi-Retired (2012-present)
Tower Monarch
Rikku
Posts: 360
Joined: Sun Oct 07, 2007 6:23 pm
Location: Richmond, VA

Re: Side-by-side stats for 2010-11 sets

Post by Tower Monarch »

Pszczew wrote: I get what you are saying Andrew, but it's really hard to make the 0-5 group get smaller without truly giving away points. I checked from the past MSU tournaments, and the teams that actually pulled this off were all getting less than 15 tossups in the whole day. That's truly miserable, but often times it's 2 freshmen who have never played before. Their knowledge base is smaller than the average quiz bowler to begin with, and with the diverse subjects quiz bowl has, these freshmen probably don't anything about the bonuses they actually get.

As for getting above 25 points in a tournament. That is a truly remarkable feat. That means you average less than one wrong bonus part for every two tossups you get. Over the course of a tournament a team with a ~25 PPB is going to get tons and tons and tons of bonuses. High School quizbowlers, even the top ones do suffer from fatigue. And eventually even the best of teams will zero a bonus on occasion. You can only get so many 10s in a tournament before it knocks you down to a 22-23 PPB. It would be nice to make a true bell curve, but for competition purposes, the current level of bonus difficulty at NAQT tournaments is ideal for the top 60-70% of competition. The bottom 30% are probably a little in over their heads, but the best way to get better is to play questions a little above your level, especially at that age.
I agree with what you're saying overall, but I think it's a completely reasonable goal to ensure that teams with 20+PBB never 0 a bonus, so making the easy part a notch closer to "truly giving away points" would allow for a slight increase in the top and bottom group while shifting the plurality from 10-15 to 15-20, which to me is preferable.
Cameron Orth - Freelance Writer/Moderator, PACE member
College: JTCC 2011, Dartmouth College '09-'10, '11-'14
Mathematics, Computer Science and Film/Media Studies
High School: Home Schooled/Cosby High '08-'09, MLWGSGIS A-E '06-'08
User avatar
Broad-tailed Grassbird
Tidus
Posts: 712
Joined: Sun Nov 23, 2008 10:27 pm
Location: Somewhere nice.

Re: Side-by-side stats for 2010-11 sets

Post by Broad-tailed Grassbird »

Tower Monarch wrote:
Pszczew wrote: I get what you are saying Andrew, but it's really hard to make the 0-5 group get smaller without truly giving away points. I checked from the past MSU tournaments, and the teams that actually pulled this off were all getting less than 15 tossups in the whole day. That's truly miserable, but often times it's 2 freshmen who have never played before. Their knowledge base is smaller than the average quiz bowler to begin with, and with the diverse subjects quiz bowl has, these freshmen probably don't anything about the bonuses they actually get.

As for getting above 25 points in a tournament. That is a truly remarkable feat. That means you average less than one wrong bonus part for every two tossups you get. Over the course of a tournament a team with a ~25 PPB is going to get tons and tons and tons of bonuses. High School quizbowlers, even the top ones do suffer from fatigue. And eventually even the best of teams will zero a bonus on occasion. You can only get so many 10s in a tournament before it knocks you down to a 22-23 PPB. It would be nice to make a true bell curve, but for competition purposes, the current level of bonus difficulty at NAQT tournaments is ideal for the top 60-70% of competition. The bottom 30% are probably a little in over their heads, but the best way to get better is to play questions a little above your level, especially at that age.
I agree with what you're saying overall, but I think it's a completely reasonable goal to ensure that teams with 20+PBB never 0 a bonus, so making the easy part a notch closer to "truly giving away points" would allow for a slight increase in the top and bottom group while shifting the plurality from 10-15 to 15-20, which to me is preferable.
This is true, but we're moving away from this because the "stock bonus" is frowned upon. I think everyone would rather see a nationals level team zero a bonus every now and then instead of having more stock bonuses.
Nalin
Scranton Middle School (2000-2003)
Brighton High School (2003-2007)
Michigan State University (2007-2011)
Semi-Retired (2012-present)
User avatar
AKKOLADE
Sin
Posts: 15782
Joined: Thu Apr 24, 2003 8:08 am

Re: Side-by-side stats for 2010-11 sets

Post by AKKOLADE »

Pszczew wrote:I think everyone would rather see a nationals level team zero a bonus every now and then instead of having more stock bonuses.
I doubt anyone who is putting together a "regular difficulty" or easier set wants this, since if there are bonuses present that top teams are going to 0, then the hundreds of teams in lower tiers are going to be 0'ing lots more bonuses.
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
User avatar
Stained Diviner
Auron
Posts: 5085
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: Side-by-side stats for 2010-11 sets

Post by Stained Diviner »

I can see an elite team zeroing an occasional Pop Culture or Computational bonus if you use them, because it's possible for elite teams to be bad at those things and those questions can be quirky. However, if a legitimately very good team zeroes a Lit, Science, History, Fine Arts, or Social Science bonus, then it's pretty close to a certainty (though not a complete certainty) that the bonus is very bad.
David Reinstein
Head Writer and Editor for Scobol Solo, Masonics, and IESA; TD for Scobol Solo and Reinstein Varsity; IHSSBCA Board Member; IHSSBCA Chair (2004-2014); PACE President (2016-2018)
User avatar
Irreligion in Bangladesh
Auron
Posts: 2123
Joined: Thu Jul 08, 2004 1:18 am
Location: Winnebago, IL

Re: Side-by-side stats for 2010-11 sets

Post by Irreligion in Bangladesh »

I feel like more people need to post to the effect of "Nalin's opinion is wrong and dangerous," so that up and coming question writers don't stumble here and get the idea that an impossible bonus is something "everyone would rather see."

In the interest of adding to the discussion-- What exactly do you mean by "stock bonus," in what way(s) is it being frowned upon, and by whom?
Brad Fischer
Head Editor, IHSA State Series
IHSSBCA Chair

Winnebago HS ('06)
Northern Illinois University ('10)
Assistant Coach, IMSA (2010-12)
Coach, Keith Country Day School (2012-16)
User avatar
Sen. Estes Kefauver (D-TN)
Chairman of Anti-Music Mafia Committee
Posts: 5647
Joined: Wed Jul 26, 2006 11:46 pm

Re: Side-by-side stats for 2010-11 sets

Post by Sen. Estes Kefauver (D-TN) »

Nalin, where on earth do you come up with your awful opinions?
Charlie Dees, North Kansas City HS '08
"I won't say more because I know some of you parse everything I say." - Jeremy Gibbs

"At one TJ tournament the neg prize was the Hampshire College ultimate frisbee team (nude) calender featuring one Evan Silberman. In retrospect that could have been a disaster." - Harry White
User avatar
AKKOLADE
Sin
Posts: 15782
Joined: Thu Apr 24, 2003 8:08 am

Re: Side-by-side stats for 2010-11 sets

Post by AKKOLADE »

I feel pretty confident saying this: any idea that we need to make anything at the high school level that aims to appeal to more than just the top 40 or so teams* more difficult is absolutely wrong, and there's data that supports that.

So, yes, Nalin's post is wrong. To say the least.

*so, basically, no tournaments, unless you want to include some of the regular season sets, because even nationals want to appeal to more than just those teams
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 6113
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: Side-by-side stats for 2010-11 sets

Post by Important Bird Area »

Pszczew wrote:This is true, but we're moving away from this because the "stock bonus" is frowned upon. I think everyone would rather see a nationals level team zero a bonus every now and then instead of having more stock bonuses.
Let's assume for the moment that there is concern (somewhere) about bonuses becoming "stock" because they repeat the same three parts over and over again.

What would be the right way to fix this problem? The correct solution is "write about different subjects for the hard part of the bonus." "Deliberately writing a bonus that every team in the field will zero" is a terrible idea. (And both of the above situations are to be distinguished from "this is an otherwise very good team that earns zero on a particular bonus because they don't know economics/African literature/earth science.")
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
User avatar
Stained Diviner
Auron
Posts: 5085
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: Side-by-side stats for 2010-11 sets

Post by Stained Diviner »

This point may be pointless, but I'm going to make it anyways.

When we look at this data, we are analyzing it per team as opposed to analyzing it per question, and that lowers the statistics.

As a simple example as to how this works, let's say that State College played a match against New Trier B and that State College answered 18 TU and converted 25 PPB, while New Trier B answered 2 TU and converted 5 PPB. (I don't know how New Trier B would get those 2 TU, but bear with me.) The way we normally look at stats, and the way this spreadsheet does it, we would say that the average bonus conversion was 15 PPB. However, during the match there were 60 bonus parts asked and 46 answered correctly, which corresponds to 23 PPB.

For an actual example, I looked at the Greenville Comet last year, which used HSAPQ. If you average the PPBs for each team, which is the stat that shows up in this spreadsheet, you get 11.3 PPB. However, there were 15350 bonus points scored from 1211 bonuses asked during the tournament, which gives 12.7 PPB. In general, I've always thought that 15 PPB was a good goal for a high school tournament, but I don't know which of these two numbers is supposed to be 15 PPB. If it's the former, then the set was significantly too difficult for the field. If it's the latter, then the set was a little too difficult for the field.
David Reinstein
Head Writer and Editor for Scobol Solo, Masonics, and IESA; TD for Scobol Solo and Reinstein Varsity; IHSSBCA Board Member; IHSSBCA Chair (2004-2014); PACE President (2016-2018)
Locked