Sectional Statistics

Dormant threads from the high school sections are preserved here.
Locked
User avatar
Stained Diviner
Auron
Posts: 4725
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Sectional Statistics

Post by Stained Diviner »

I have posted statistics from Round 1 of Sectionals at http://www.ihssbca.org/sectionalround1stats.xls

I hope the column headings make sense. The ones at the very end are the standard deviations of the bonus scores. The data is for ten matches: Timothy Christian vs Lisle, Dwight vs Latin, Belleville West vs Glenwood, Carbondale vs Charleston, Pleasant Plains vs Litchfield, New Berlin vs Marquette, Stevenson vs Fremd, Deerfield vs Buffalo Grove, Altamont vs South Central, and Fairfield vs Paris. The questions were used in 32 matches, but this turns out to be a good cross sample in terms of various measures of diversity, and it's pretty close to all the data I have at this point.

I plan on putting more time into this later in the week. Before I do, let me know if you think of a better way to sort out the data or if you draw any interesting conclusions from it. I also would like to know what people have done in the past with conversion data in terms of figuring out what's good and what's bad. Ideally, all bonuses would be of the same difficulty, but nobody expects conversion averages to be exactly the same for each question, so what do you look for? Similarly, it seems like low bonus conversion standard deviations show that the questions did not really differentiate between teams, while high standard deviations show that too many bonuses either got swept or bageled, so what should be a goal there?

Some of this may tie in with this new thread.
David Reinstein
PACE VP of Outreach, Head Writer and Editor for Scobol Solo and Masonics (Illinois), TD for New Trier Scobol Solo and New Trier Varsity, Writer for NAQT (2011-2017), IHSSBCA Board Member, IHSSBCA Chair (2004-2014), PACE Member, PACE President (2016-2018), New Trier Coach (1994-2011)

User avatar
rjaguar3
Rikku
Posts: 278
Joined: Tue Dec 05, 2006 8:39 am

Re: Sectional Statistics

Post by rjaguar3 »

Why don't you try the bonus plot distribution? Label the x-axis from 0 to 20 and label the y-axis with the number of teams that score each respective amount on the first reading divided by a scaling factor (the area under the curve). This makes it easy to read whether a bonus is good or bad and why it is bad.

Take a look at this key:

Image

You can see some sample bonuses and their corresponding curves.
  • Good bonus: a well-written bonus, where teams are sorted into 0, 10, 20, and 30 groups based on questions with appx. 90-50-10 conversion rate.
  • All-or-nothing bonus: teams either know all the answers or none of them: such as "name the 4 nonreal sixth roots of i" from NT varsity 2006.
  • Too easy: self-explanatory
  • Too hard: also self-explanatory
  • One-two-screw you: two easy parts followed by a third that is near-impossible to get, such as Geronimo-Apache-Loco
  • One-screw you-screw you: one easy part followed by two impossible parts, such as Geronimo-Victorio-Loco
Here are some examples from Fall goldfish 2008. You can see some of the archetypal curves in the bonuses:

Image

And here's the breakdown:
All-or-nothing: none
Too easy: bonus 12 (by far), bonuses 19, 1, 6, and 24
Too hard: bonuses 2, 10, 15, and 22
1-2-screw: bonuses 7, 17 and 18
1-screw-screw: bonuses 13 and 20

Of course, this is systematically biased towards the teams with more knowledge since they are more likely to have control of the bonus, but this is the best we can do.

So, if you have the exact scores on each bonus, please send them my way (gregory dot j dot gauthier at vanderbilt dot edu) and I'll see what I can do with them. Maybe not much, because our sample size is too small, but at least it's worth a shot.
Greg (Vanderbilt 2012, Wheaton North 2008)

User avatar
JackGlerum
Tidus
Posts: 660
Joined: Wed Aug 15, 2007 9:20 pm

Re: Sectional Statistics

Post by JackGlerum »

That's pretty cool.

User avatar
rjaguar3
Rikku
Posts: 278
Joined: Tue Dec 05, 2006 8:39 am

Re: Sectional Statistics

Post by rjaguar3 »

Since Mr. Reinstein asked, here are all the graphs from Round 1 of sectionals. The graphs have text annotations that should be self-explanatory--feel free to ask if you have any questions.

Image

Image

Image

Image

Image

Image

Image

Image
Greg (Vanderbilt 2012, Wheaton North 2008)

User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: Sectional Statistics

Post by Mechanical Beasts »

rjaguar3 wrote: Image
For those of us who can read a graph but are bad at a lot of lines close together, can you provide a Gini-style Gauthier index as to the deviation from the straight line by each category, as a way to quantify the -easy or +hard?
Andrew Watkins

Locked