2012 SCT: Difficulty

[bird bird bird bird bird]

One of NAQT's goals for this year's SCT was to lower the difficulty of the tournament (in both divisions). Did we succeed? Discuss.

[jmannor2]

These questions (DII) seemed extremely easier than last years set. I felt like I was playing a high school NAQT IS set with a slightly harder difficulty. And I mean very slightly.

[marnold]

Did we succeed?

Yep

[RyuAqua]

This year's DI SCT difficulty was a positive success. It showed it's eminently possible to write a sane, difficulty-restricted set which good teams will dominate without alienating the lower half of the field. It kinda makes me wish that more teams had signed up for DI, or that the eligibility requirements hadn't shifted a lot of older players into DII - this set would have been more appropriate than many of the past few Sectionals sets for the kind of player that the rule shift accomodated.

On the whole, I really liked the set, and noticed few flaws. Kudos to those involved!

[Edward Elric]

I would definitely say that DII was much easier than last years set, but it was also very well writen at the same time. Nice Job!

[Bartleby]

I'm sure some of this is a bias from:

a) Having played quiz bowl for a while now
and
b) Playing on a team with no undisputed "number one" sort of player

but, this is the first SCT where I've truly felt able to make a decent contribution to my team's chance at winning, and where I've known a not-trivial number of questions outside of my fields of expertise (sometimes even before the giveaway!). At the same time, the tournament was certainly a challenge, not a cakewalk, and I think that this is a good model for SCT to be based upon in future years.

[GymnogypsCalifornianusWKU]

The DII set was what I expected NAQT collegiate questions to be like when I first started college. They were easier than last year, although still a step up from high school questions. I can't say that there were any questions yesterday that were bad enough to really ruffle my feathers either. I'm very happy to have played this set.

[Madagascar Serpent Eagle]

Maybe I'm wrong because I played on a team with some good people, but I thought that DII Set was not challenging at all. I'm not trying to say that it should be hard to the point where it alienates new schools or anything like that, it's right that it's easy, but there were some tossups (for example, mentioning Edward Everett at the beginning of Gettysburg) where I could think of maybe one or two things that could possibly be better known than a power clue that weren't the giveaway. I can't think of too many examples now, but when I was playing I'd say at least a third of the powers people on my team got made me wonder why that was a power, rather than be impressed at their deep knowledge. Similarly, a decent number of bonuses left me wondering if there was supposed to be any hard part. I'll try to enumerate this better in the specific question thread where there are more answerlines, but I felt like sometimes the DII set was closer to a high school set than a college set that appealed to new teams.

[Cheynem]

I haven't seen the DII set, but it seems like this comes up every year. I dunno, perhaps there were difficulty issues, but:

-I'm not sure how gigantic a difference to be frank a high school set and an easy set intended for new college teams will be.

-If you're a competent player (and Joe certainly is), you will do well at DII SCT. You will probably do very well.

-I don't see anything that wrong with a DII tossup on Gettysburg using Edward Everett as an early clue.

[DumbJaques]

Maybe I'm wrong because I played on a team with some good people, but I thought that DII Set was not challenging at all.

If only there were some kind of more challenging version of the SCT that you could play as a parallel tournament with other people who found the DII set entirely too easy. . . Oh well, I guess we can dream, right?

[Fond du lac operon]

One of the players on Alabama A compared the DII questions to "high-school clues on college-type answer lines." From what little I remember about my high school years, that seemed about right.

Not that that's necessarily a bad thing -- I think the giveaways, and indeed maybe most of the stuff after the power marking, should be clues you might see at a good HS tournament for DII SCT, and I ended up enjoying the set a good deal. But I do think some of the early clues were just way too easy.

If only there were some kind of more challenging version of the SCT that you could play as a parallel tournament with other people who found the DII set entirely too easy. . . Oh well, I guess we can dream, right?

Sarcasm aside, that wasn't an option for me, since Section 4 played a combined field on DII questions. Obviously you shouldn't write primarily for the DI players who are forced to play down, that's not your primary audience, but maybe they're worth keeping in mind?

[Mike Bentley]

One of NAQT's goals for this year's SCT was to lower the difficulty of the tournament (in both divisions). Did we succeed? Discuss.

As a writer, I never received a message that this was a goal of this year's SCT. Did I just miss this, or was it never sent out?

[Marble-faced Bristle Tyrant]

One of NAQT's goals for this year's SCT was to lower the difficulty of the tournament (in both divisions). Did we succeed? Discuss.

As a writer, I never received a message that this was a goal of this year's SCT. Did I just miss this, or was it never sent out?

There was a true needs email on January 22 saying "NAQT is trying to make this year's SCT sets somewhat easier than last year's, so if you were also writing last year, take it down a notch."

[DumbJaques]

If only there were some kind of more challenging version of the SCT that you could play as a parallel tournament with other people who found the DII set entirely too easy. . . Oh well, I guess we can dream, right?

Sarcasm aside, that wasn't an option for me, since Section 4 played a combined field on DII questions. Obviously you shouldn't write primarily for the DI players who are forced to play down, that's not your primary audience, but maybe they're worth keeping in mind?

Sarcasm entirely not aside, uh, that's nice, but this post quoted a comment by an entirely different person who played at another site (where no such issue existed, but that's not really the point). I have absolutely no idea why you seem to have concluded it was directed at you, but the idea that NAQT should write their DII questions with DI players in mind has nothing to do with that stuff anyway. I really have no idea what you're talking about here.

Incidentally, no, they're not really worth keeping in mind. A DI field having to play DII questions is a bad situation nobody wants and everyone works hard to avoid, and really it happens very rarely in competitive regions where it would actually be a big problem. DII serves a very significant purpose in college quizbowl, and it should be entirely crafted with that purpose in mind, not with an eye toward the potential collateral damage it might maybe someday cause to some Alabama grad student's stock clue monocle (I was under the impression James Johnson murdered any grad students who tried to join the Alabama team anyway). You should write DI questions for DI teams, and DII questions for DII teams. If that's not what ends up happening, the solution is not to write harder DII questions just in case; it's to figure out how to get more than three DI teams to show up at a multi-state national qualifier for the largest college tournament of the year.

[samer]

Maybe I'm wrong because I played on a team with some good people, but I thought that DII Set was not challenging at all. I'm not trying to say that it should be hard to the point where it alienates new schools or anything like that, it's right that it's easy, but there were some tossups (for example, mentioning Edward Everett at the beginning of Gettysburg) where I could think of maybe one or two things that could possibly be better known than a power clue that weren't the giveaway. I can't think of too many examples now, but when I was playing I'd say at least a third of the powers people on my team got made me wonder why that was a power, rather than be impressed at their deep knowledge. Similarly, a decent number of bonuses left me wondering if there was supposed to be any hard part. I'll try to enumerate this better in the specific question thread where there are more answerlines, but I felt like sometimes the DII set was closer to a high school set than a college set that appealed to new teams.

FWIW, NAQT's philosophy in general is that power marks indicate the first point that clearly isn't worth power, rather than the last point that is.

Obviously, they are sometimes too early and sometimes too late, but the goal is that about 15% of rooms will power any given tossup.

[Kentari]

Did we succeed?

Yep

/signed

[An Intergalactic Puzzlepalooza]

Quotes not here to prevent over-nesting

Incidentally, no, they're not really worth keeping in mind. A DI field having to play DII questions is a bad situation nobody wants and everyone works hard to avoid, and really it happens very rarely in competitive regions where it would actually be a big problem. DII serves a very significant purpose in college quizbowl, and it should be entirely crafted with that purpose in mind, not with an eye toward the potential collateral damage it might maybe someday cause to some Alabama grad student's stock clue monocle (I was under the impression James Johnson murdered any grad students who tried to join the Alabama team anyway). You should write DI questions for DI teams, and DII questions for DII teams. If that's not what ends up happening, the solution is not to write harder DII questions just in case; it's to figure out how to get more than three DI teams to show up at a multi-state national qualifier for the largest college tournament of the year.

Independent of a DI field being forced to play DII, I also felt that the set was significantly too frequently stock and/or too easy.

I'd like to point out that I probably should be playing DII, which is probably not true of Harrison under perfect circumstances (Those perfect circumstances where your school has a number of DII non-eligible people perfectly divisible by four(Or at least can convince that many people to play DI), and your region doesn't run a combined field). This is my first year playing on pyramidal questions; you'll find more on NAQT's website from searching my younger sister than you'll find from searching me.

Still, there were clues that led in on this that I felt top ASCA high school players, from the universally mocked Alabama state format (And the one I grew up in, which is the least pyramidal stuff on Earth), would have declared easy. Cup of Gold, Achilles killing Penthilsea, and Giles Corey (Even before he was mentioned by name) were the worst specific non-trash offenders of this off the top of my head, but there were a lot of leadins and other early clues (That is, clues early enough to be in power that gave something a large percentage of the room could buzz on, which I'd say clearly indicates the clue shouldn't have been in power) that felt like they would have produced buzzer races among way too many teams with a hugely disparate skill level, which kind of ruins the point of pyramidality in that it's supposed to distinguish between different levels of knowledge (Differing from whatever happened over BARGE in that people just know too much to make buzzer races avoidable).

As someone who thinks he's currently the kind of person for whom DII was designed, though admittedly one with a lot of Quiz Bowl like substance experience for being able to claim newness to this whole thing, that was my take on the difficulty of the set. It certainly was easy, per se, but seemed to be frequently reactionarily too easy.

[Mechanical Beasts]

Everyone who would characterize himself as a "top highschool player," no matter the region, should know some leadins at a sectional tournament on DII questions. The point of leadins is for some people to be buzzing on them and others not to be; you can't have that if zero people think "my word, I KNOW that" on clue one-point-five.

[cornfused]

Cup of Gold

I did laugh to myself when I read that tossup, even though I thought it was a fine leadin for the tourney... Wasn't Steinbeck's first novel being Cup of Gold the ACF writing guide's specific example of a bad leadin?

[Kyle]

I remember thinking the D2 SCT was far too easy when I played it at Yale in 2006. We very nearly lost to Columbia -- or perhaps we did lose, I can't remember -- because we lost four buzzer races for power. But the thing is, neither Harvard nor Columbia had any trouble qualifying for the ICT that year. Once we got to the ICT, the questions were substantially more difficult and these problems sorted themselves out. For the teams that did have trouble qualifying, the SCT questions were exactly the right difficulty. The purpose of the SCT isn't to distinguish the very best teams from the other very best teams, but to distinguish the teams that should make the ICT from the teams that shouldn't. That's why the D2 SCT is explicitly intended to be easier than the HSNCT, but the D2 ICT is explicitly intended to be harder than the HSNCT.

[Fond du lac operon]

Sarcasm entirely not aside, uh, that's nice, but this post quoted a comment by an entirely different person who played at another site (where no such issue existed, but that's not really the point). I have absolutely no idea why you seem to have concluded it was directed at you, but the idea that NAQT should write their DII questions with DI players in mind has nothing to do with that stuff anyway. I really have no idea what you're talking about here.

Incidentally, no, they're not really worth keeping in mind. A DI field having to play DII questions is a bad situation nobody wants and everyone works hard to avoid, and really it happens very rarely in competitive regions where it would actually be a big problem. DII serves a very significant purpose in college quizbowl, and it should be entirely crafted with that purpose in mind, not with an eye toward the potential collateral damage it might maybe someday cause to some Alabama grad student's stock clue monocle (I was under the impression James Johnson murdered any grad students who tried to join the Alabama team anyway). You should write DI questions for DI teams, and DII questions for DII teams. If that's not what ends up happening, the solution is not to write harder DII questions just in case; it's to figure out how to get more than three DI teams to show up at a multi-state national qualifier for the largest college tournament of the year.

Cool your jets, man, I wasn't saying it was directed at me. I was just pointing out that, hey, there are people who have to play DII even though it might be easier than their ideal tournament, whether because of a combined field or because they're very talented frosh/novices on teams with 4n grad students for n integral. (Also, if you are under the impression I am a grad student, I am not. And I don't know who James Johnson is.)

Look, I'm not saying that the entire DII tournament should be written for DI-quality players. But the difficulty should be spread out, and I think the upper tail of that distribution wasn't hard enough to serve the best DII players.

I'd like to point out that I probably should be playing DII, which is probably not true of Harrison under perfect circumstances (Those perfect circumstances where your school has a number of DII non-eligible people perfectly divisible by four(Or at least can convince that many people to play DI), and your region doesn't run a combined field). This is my first year playing on pyramidal questions; you'll find more on NAQT's website from searching my younger sister than you'll find from searching me.

Worth noting: I was DII-eligible for SCT, and I have probably almost as little experience playing on pyramidal questions as Joseph -- there were certainly some good tournaments in Georgia when I played high school, but only 2-3/year that I went to, as well as a bunch of crappy nonpyramidal tournaments. I'm probably an outlier (I've taken quite a few more college courses than the freshmen and sophomores that DII is largely meant for) but I'm still a novice, and other than MOO and BARGE, I hadn't played quizbowl in nearly four years before SCT. I don't think it's too much to ask that people like me should be served by DII as well as players who only know high school clues.

[Frater Taciturnus]

One of NAQT's goals for this year's SCT was to lower the difficulty of the tournament (in both divisions). Did we succeed? Discuss.

Yes, probably a small bit more than I expected despite you saying that this year's set was supposed to be easier.

Will similar adjustments be made for ICT and/or HSNCT, or was this strictly an effort to reduce SCT difficulty?

[An Intergalactic Puzzlepalooza]

Everyone who would characterize himself as a "top highschool player," no matter the region, should know some leadins at a sectional tournament on DII questions. The point of leadins is for some people to be buzzing on them and others not to be; you can't have that if zero people think "my word, I KNOW that" on clue one-point-five.

I'm not completely clear on what the last sentence refers to, so ignore this post if a correct interpretation of that would lead to this post's content not making sense.

However, I agree that lead-ins are worthless if they aren't buzzable. My point in this is that lead-ins are equally worthless if 60% of individuals get them, which it seemed this was rife with.

Don't get me wrong, there were plenty of really appropriate lead-ins which I knew (3 of which happened during our bye round, unfortunately). I thought few enough people in the room (Though I was on a bye round watching two of the worse teams in the tournament when that one and the Drug industry clue, both of which went dead/really late 10ed) knew anything more about the Raven than "Nevermore", and maybe at worst they'd get Lenore. From that same round, the lead-in on Churchill was certainly easy, but oncemore, something that probably only 10% of the rooms in our field would have gotten off that. From a later round, we didn't get to the Shinto tossup, but the Izanami/Izanagi story was literally the first story I learned about Shinto, and it's the lead-in. And that's perfectly appropriate still, because very few DII people know anything about Shinto, and that clue preceded Torii, Worships Kami, and indigenous region of Japan as clues, which is a pretty pyramidal descent. Admittedly, I might have made the worship rituals the lead-in instead, but that one was perfectly fine. Those are the kinds of ones that someone who strongly implied he was a top high school player without actually claiming it should be getting at least a few of (Though I'm not entirely positive that's true, since ASCA is so speed oriented that lead-in type knowledge isn't usually useful.), and those are fine.

I was actually pretty sure that Cup of Gold being Steinbeck's first novel was used as a universal example of an overly stock clue as well, though I can't seem to find anything on google to back that up. In terms of real-world difficulty, that's a very appropriate lead-in, except on that one real world difficulty.

Kyle's take on it, that despite occasional swings into the too easy, it still sufficiently separates the good and bad teams, however, makes a lot of sense to me, and kind of reflects my lack of understanding about this whole process. I don't have a good sense for whether or not we'll qualify, but as far as I could tell, not only should we be right on the borderline of qualifying, but also we will be right on that borderline, and since some of the variables in the D-Values require better statistics than I think I have (It is based off a global PPTH rather than a local one, right?)

(And yeah, who's James Johnson?)

[GymnogypsCalifornianusWKU]

[ I don't think it's too much to ask that people like me should be served by DII as well as players who only know high school clues.

Why should D2 cater to players good enough to play D1? That's what D1 is for. If you have to play D2 questions because of a combined field, that's very unfortunate and is something nobody wants to happen. But the problem here is not that the D2 questions were too easy for a D1 crowd (which they should be) but rather that your region can't find more than 2 teams to play their D1 sectionals. Maybe not hosting SCT in Florida would help solve this problem, but there are a lot more factors in play in the Southeast that causes nobody to play D1.

And I'm generally a person like you in this situation, in that I did very well on this set. Of course I'm also a junior in college that has plenty of quizbowl experience by now. The solution to my predicament isn't to demand that easy sets get harder to cater to my interests, but to play D1 instead.

EDIT: And there wouldn't have been a combined field if Auburn and Alabama B had elected to play D1 instead. You shouldn't generally be required to because players should stick around the circuit for D1 once their D2 eligibility is up, but if this set was too easy for you then you could have played D1 questions

[Fond du lac operon]

Why should D2 cater to players good enough to play D1?

Because the DII field isn't some Platonic ideal of fresh-faced kids straight out of high school who've never played pyramidal tossups before. There are going to be people like Matt Jackson last year, like Joseph and Joey from Auburn, like our grad students who played DII because of the combined field, and like me -- people who could and perhaps should play DI but for whatever reason will play DII instead.

I suppose it comes down to whether one should write pragmatically, considering the distribution of people who are actually going to play a tournament, or ideologically, considering who you think should play that tournament. I'm in favor of the former, obviously, but there are probably good arguments to be made for the latter.

[Vernon Lee Bad Marriage, Jr.]

Why should D2 cater to players good enough to play D1?

Because the DII field isn't some Platonic ideal of fresh-faced kids straight out of high school who've never played pyramidal tossups before. There are going to be people like Matt Jackson last year, like Joseph and Joey from Auburn, like our grad students who played DII because of the combined field, and like me -- people who could and perhaps should play DI but for whatever reason will play DII instead.

I suppose it comes down to whether one should write pragmatically, considering the distribution of people who are actually going to play a tournament, or ideologically, considering who you think should play that tournament. I'm in favor of the former, obviously, but there are probably good arguments to be made for the latter.

No, it comes down to whether one should write pragmatically, considering the distribution of people who are actually going to play the tournament, or stupidly, considering those few posers who will play the tournament and post on the internet about how everything was too easy for them. I'm in favor of the former, and there are no good arguments to be made for the latter. Look dude, you missed one in three bonus parts and failed to power most tossups when power extended through the majority of the question. Statistically, you were not so good that this tournament failed to challenge you. Maybe some individual questions were too easy, but the numbers confirm that the set as a whole was not.

EDIT: So yeah. At your site, 11.8% of tossups were powered, and the average bonus conversion was around 14.7. That's right around NAQT's goal and maybe even a little lower.

[Cheynem]

I think if you were remotely good at high school quizbowl, you're going to do very well at DII SCT and that's perfectly okay with me. We had a DII team with generally little inexperience playing college quizbowl (and in some cases, high school quizbowl I think), and they did pretty well, but still had a sub 15 PPB (four other teams at our site also did). This suggests to me that for actually inexperienced and struggling teams, DII is not too easy. If teams that finished near last were putting up 18 PPB with insane power totals, I would concur that it was too easy, but again, at our site, four teams had 10 or less powers (I'm not saying they're terrible, I'm just saying they weren't dominating the set). At least at our site DII results seemed perfectly normal--sometimes difficult to differentiate between our two best teams (which were of fairly equal strength and have very good players), but that's to be expected.

[Fond du lac operon]

Look dude, you missed one in three bonus parts and failed to power most tossups when power extended through the majority of the question. Statistically, you were not so good that this tournament failed to challenge you. Maybe some individual questions were too easy, but the numbers confirm that the set as a whole was not.

I think "most tossups" is a pretty arbitrary dividing line. I also agree that we shouldn't base any decisions on one asshole claiming the set was too easy for him, particularly when hindsight bias makes people liable to misremember the difficulty. But the top four teams at my site, between them, powered over 30% of the tossups they heard. At least two, and I think three, of those teams were eligible to play DII. At last year's D1 ICT, the top four teams powered just under 20% of the tossups they heard. Statistically, this was considerably easier for its audience.

The other thing is: I'm not actually that good. Outside of philosophy, math/CS, and social sciences, I'm going to miss probably upward of 50% of regular-difficulty questions, even after the giveaway. I know, because I've read packets and tracked my progress. I should not be getting 4.5 powers a game, especially against pretty good competition, at this level.

[Fond du lac operon]

EDIT: So yeah. At your site, 11.8% of tossups were powered, and the average bonus conversion was around 14.7. That's right around NAQT's goal and maybe even a little lower.

Nope. You're double-counting total number of TUH. 23.6% of tossups were powered.

[Cheynem]

I think DII SCT is supposed to be easier for its audience than DI ICT for its audience.

[Vernon Lee Bad Marriage, Jr.]

Look dude, you missed one in three bonus parts and failed to power most tossups when power extended through the majority of the question. Statistically, you were not so good that this tournament failed to challenge you. Maybe some individual questions were too easy, but the numbers confirm that the set as a whole was not.

I think "most tossups" is a pretty arbitrary dividing line. I also agree that we shouldn't base any decisions on one asshole claiming the set was too easy for him, particularly when hindsight bias makes people liable to misremember the difficulty. But the top four teams at my site, between them, powered over 30% of the tossups they heard. At least two, and I think three, of those teams were eligible to play DII. At last year's D1 ICT, the top four teams powered just under 20% of the tossups they heard. Statistically, this was considerably easier for its audience.

The other thing is: I'm not actually that good. Outside of philosophy, math/CS, and social sciences, I'm going to miss probably upward of 50% of regular-difficulty questions, even after the giveaway. I know, because I've read packets and tracked my progress. I should not be getting 4.5 powers a game, especially against pretty good competition, at this level.

OK, this is a somewhat more reasonable post. I think the issue here is that you're deriving an "ought" from an "is." The fact that you don't normally power that many questions doesn't mean that something is wrong if you get more 15's on an intentionally easier set with generous powermarking. Again, statistically, it seems like the set did its job. You also may just be a better player than you thought you were and shouldn't worry about that stuff. It seems like it's an issue that's not worth looking into--you did very well on an easy set, no more, no less.

EDIT: Ah fuck, yeah I noticed that just now. Forgot to factor in the two teams hearing any individual tossup...anyway. 23.6% still isn't really terribly far from the goal of 15%, particularly with a normally ineligible team getting a quarter of the total powers in the field.

[GymnogypsCalifornianusWKU]

But the top four teams at my site, between them, powered over 30% of the tossups they heard. At least two, and I think three, of those teams were eligible to play DII. At last year's D1 ICT, the top four teams powered just under 20% of the tossups they heard. Statistically, this was considerably easier for its audience.

Two of the top four teams at your site were not the target audience for D2, Alabama and Georgia should have been playing D1 if the Southeast region had not had a dismal and embarrassing turnout at that level. You can't just compare a Sectional tournament to a National tournament, ignore the fact that one sample includes D1 teams playing D2 questions, and then declare the two situations to be equal.

The top four D2 teams at you're site powered 16.5% of the tossups they heard, which is actually lower than the example you cited from D1 ICT.

[bird bird bird bird bird]

Will similar adjustments be made for ICT and/or HSNCT, or was this strictly an effort to reduce SCT difficulty?

Similar adjustments, but probably of a smaller scale. I think that both ICT and HSNCT should be easier, but that those changes should be concentrated on the bonuses. (Compare previous SCTs that have had significant numbers of too-difficult tossups as well.)

[Fond du lac operon]

OK, this is a somewhat more reasonable post. I think the issue here is that you're deriving an "ought" from an "is." The fact that you don't normally power that many questions doesn't mean that something is wrong if you get more 15's on an intentionally easier set with generous powermarking. Again, statistically, it seems like the set did its job. You also may just be a better player than you thought you were and shouldn't worry about that stuff. It seems like it's an issue that's not worth looking into--you did very well on an easy set, no more, no less.

Fair enough. I should know better than to pretend to be able to extrapolate from my own experience. Even though that experience wasn't great (which, subjectively speaking, it was not -- it was good, as winning generally is, but not great), it's entirely likely I was in the top few percent of people who actually played DII SCT.

I still think that even the top few percent should be challenged by a decent number of questions on DII sets -- particularly if, like me, they have really shallow general knowledge -- but that should be at best like fifth on the list of concerns when writing SCT, behind stuff like "providing questions that even total novices can get" and "making sure we differentiate between who should and shouldn't go to ICT" and stuff like that. I still think it should be a constraint on the set, but, whatever, it's not worth arguing about. Hopefully ICT will be better, although it may be moot, since I might play DI at ICT (assuming that's allowed).

Edit: Also, Nick, you're double-counting a lot of questions. The top four teams played each other seven times, so they heard fewer questions than the sum of the TUH provided between them. (It's an easy mistake to make... I'm just a combinatorialist, so I'm accustomed to working with sieves, haha.)

[GymnogypsCalifornianusWKU]

Edit: Also, Nick, you're double-counting a lot of questions. The top four teams played each other seven times, so they heard fewer questions than the sum of the TUH provided between them. (It's an easy mistake to make... I'm just a combinatorialist, so I'm accustomed to working with sieves, haha.)

Well I'm not a statistician or anything, so my numbers are likely to be wrong more often than not.

Taking into account everything though, I still doubt the entire DII field's power rate at your site is too far from NAQT's goal. Ask Mercer, Judson, or Stetson if they think this tournament was too easy.

[Mechanical Beasts]

EDIT: So yeah. At your site, 11.8% of tossups were powered, and the average bonus conversion was around 14.7. That's right around NAQT's goal and maybe even a little lower.

Nope. You're double-counting total number of TUH. 23.6% of tossups were powered.

Around 25% has been my goal for as long as I've power-marked--though I didn't powermark any of DII--and so I have less than zero complaints about that. When I saw 11.8%, I was dead scared.

[grapesmoker]

Having lots of tossups powered across the board is not a big deal. The purpose of the SCT is to figure out who qualifies for ICT and create some plausible ranking of those qualified teams. It's not a precision science, it's more of an order-of-magnitude estimation. Considering that the SCT operates with relatively short questions intended to be playable across multiple sites and skill levels, this isn't a problematic outcome at all.

[bird bird bird bird bird]

For the record, NAQT's official target for power percentage is between 15 and 20 percent.

[An Intergalactic Puzzlepalooza]

I'd like to note that although the general arguments have sort of shifted away from where I was, I'm certainly not alleging some sort of set-wide problem. In fact, I agree that statistically speaking overall, I'd think our teams stats lined up with almost exactly what we should have expected to end up with.

The problem it had, rather, was when it would escape into things like the ones I mentioned.

Our DII field had very little margin for error; Games between the top three teams had margins of victory of less than a tossup/bonus in two of them (Alabama B v. Auburn and Auburn v. UCF A), and tossups like those mentioned earlier could have had really bad impacts on those games. That's more of my concern as to what was done with the difficulty, that in making it easier overall, the occasional tossup escaped so far over the line that it did nothing to delineate between two different levels of knowledge and just gave points to whoever won the buzzer race

[grapesmoker]

Our DII field had very little margin for error; Games between the top three teams had margins of victory of less than a tossup/bonus in two of them (Alabama B v. Auburn and Auburn v. UCF A), and tossups like those mentioned earlier could have had really bad impacts on those games. That's more of my concern as to what was done with the difficulty, that in making it easier overall, the occasional tossup escaped so far over the line that it did nothing to delineate between two different levels of knowledge and just gave points to whoever won the buzzer race

This really doesn't matter all that much. I'm guessing that all of those teams are going to qualify for the ICT, where the questions will differentiate better between the teams. For what the SCT is, this result is fine and only says that the three teams in question are generally pretty close.

[setht]

I'm not saying that the entire DII tournament should be written for DI-quality players. But the difficulty should be spread out, and I think the upper tail of that distribution wasn't hard enough to serve the best DII players.

I'm not really sure what this means--are you saying that there should be some tossups that are more challenging than the average, and you think the DII SCT tossup difficulty should have had higher variance? I agree with you on the first part (tossup answers should spread over a range of difficulties) and have no strong opinion on the second, since I don't know what the DII SCT set looked like overall. However, I don't think the DII SCT tossup difficulty should range all the way to a point that would challenge "the best DII players"--presumably that should be reserved for DII ICT.

I don't think it's too much to ask that people like me should be served by DII as well as players who only know high school clues.

Again, to an extent I agree with this, but at some point I think it's entirely reasonable to say "save it for DII ICT." If you play DII ICT and it doesn't "serve" you satisfactorily, then it's possible that the DII ICT set didn't accomplish its goal, or that you really shouldn't be playing DII at all; in the meantime, it sounds like DII SCT is not really supposed to serve people like you, as you put it--it's just supposed to get you your qualification to DII ICT.

...it's entirely likely I was in the top few percent of people who actually played DII SCT.

I still think that even the top few percent should be challenged by a decent number of questions on DII sets -- particularly if, like me, they have really shallow general knowledge -- but that should be at best like fifth on the list of concerns when writing SCT, behind stuff like "providing questions that even total novices can get" and "making sure we differentiate between who should and shouldn't go to ICT" and stuff like that.

If you think about this, I think you'll agree with me that a top team at ICT probably shouldn't be challenged by the SCT set. The SCT set is pitched at one level; the very best teams that played SCT then assemble to play ICT on a set pitched at a different level. I think this is an eminently sensible system, even though it means that some of the best teams in the country maybe don't have a super-exciting time at SCT (I mean, I hope they do, but if they don't I don't think that makes SCT a failure).

Furthermore, I'm guessing the DII SCT set actually did have many questions that "challenged" you, if, as you say, you have really shallow general knowledge. I imagine what you're saying is that you would have liked some questions that would challenge you in your areas of deeper specialized knowledge. I definitely sympathize with that sentiment, but I think it's unreasonable to ask for DII SCT questions that would challenge areas of expertise of a player with some quizbowl experience and several years of college coursework under their belt--it's pretty much directly at odds with the primary goal of DII SCT (distinguishing between teams at the bottom end of the putative DII ICT field). Hopefully you will get the kind of challenge you crave at ICT in whichever division you wind up playing. I strongly encourage you to check out some previous DII ICT sets and see if they feel like a sufficient challenge, and if they don't, play DI if possible.

-Seth

[bradleykirksey]

I'm nowhere near Harrison's level of play. My team had fewer powers than Harrison did. (Though the TD told me that I had more powers than the statistics reflected and said he wasn't allowed to correct it. I couldn't care less as we still finished 10-2, but the sect 6 reported individual statistics should not be the end-all-be-all.)

Maybe it's because I'm a dumb-ass, but I appreciated the level of play. Our program is primarily freshmen. Even though Alabama put up 670+ on both freshmen teams, they found some stuff that made them feel smart, and they're playing even harder. I also think that last year's dII questions would have probably scared off Judson and Stetson for a long time.

I know you also can't write the tournament for people who've never done quiz bowl before either, but it is DII. It is still designed for people who can't play DI yet, and I appreciated that the first pack had both H.M.S. Pinafore and the Sirens. A difficult(ish) question that an inexperienced player in unlikely to know, and another question that brings back the Judsons and Stetsons and UCF Cs.

Then again, this is coming from a guy who buzzed in with "indulgences" for a location. Feel free to overrule me.

edit: Also, couldn't you pretty please take "aliens" for "ancient aliens" up in the power? Pleeeease?
2nd edit: Oops. I counted the wrong UCF Ian's powers. we did have more powers as a team than Harrison.

[Fond du lac operon]

[really long post]

Yeah, on reflection, I agree with pretty much everything Seth says. I'm still a little concerned that a decent number of people thought, subjectively, that the DII SCT was a little too easy for them to fully enjoy it (and I don't think that saying "play DI" really resolves that issue), but I'll concede that I can't make an argument that it was too easy based on any kind of objective criteria.

[setht]

I'm still a little concerned that a decent number of people thought, subjectively, that the DII SCT was a little too easy for them to fully enjoy it (and I don't think that saying "play DI" really resolves that issue), but I'll concede that I can't make an argument that it was too easy based on any kind of objective criteria.

Oops, I actually meant to comment on that too--NAQT has been collecting tournament data for a while now, and I believe the effort to make this year's SCT sets noticeably easier than in the past was prompted by the observation that the data showed noticeable gaps between actual team performance and NAQT's performance targets. It is possible that the DII SCT overshot the mark and wound up too easy, but presumably that's something that will show up in the data once it's collected and analyzed. That will give us some objective criteria for judging the set as a whole. In the meantime, I think people's subjective opinions have worth and it might be useful to share them, so I hope people will continue posting thoughts on the set, including stuff like "I thought the set was too easy/I thought the questions in subject X were generally too easy/I think the following specific questions were too easy." I'm sure the people who did work on the set would like to get as much feedback as possible--it is being used for other tournaments, after all.

-Seth

[The Ununtiable Twine]

Having played a DII set in 2005, 2006, and 2011, and having read all of the sets in between, I think NAQT did a great job of accomplishing its goal of making SCT accessible to most of the teams that you expect to contend for spots at ICT. If my team were challenged by the set, I would have been concerned for the DII fields around the country. DII SCT is supposed to be easy enough for the majority of the field to be challenged and have fun, but tough enough to tell which DII teams are the best, and I thought the previous sets were moving in a direction that made it just a little too challenging for the lower halves of fields, so I'm happy NAQT decided to tone the difficulty down a little bit even if it means that those players that the set is not intended for don't find it very challenging. The set simply does not have upper echelon players in mind as its audience, as it should not. Being too easy is better than being too hard in this case.

[jmannor2]

For the record, I was just stating that the questions were easier than last years. I didn't mean to imply anything like I am far superior to this set, because I definitely am not.

EDIT: And there wouldn't have been a combined field if Auburn and Alabama B had elected to play D1 instead. You shouldn't generally be required to because players should stick around the circuit for D1 once their D2 eligibility is up, but if this set was too easy for you then you could have played D1 questions

Playing D2 was the most logical choice for Auburn considering the experience our team has. I am the only one who has years of experience playing pyramidal, while both Joseph and Neil pretty much started at the beginning of this academic year. And our other player literally started playing 3 weeks ago. I am certain that our choice to play D2 was the best choice.

[GymnogypsCalifornianusWKU]

Playing D2 was the most logical choice for Auburn considering the experience our team has. I am the only one who has years of experience playing pyramidal, while both Joseph and Neil pretty much started at the beginning of this academic year. And our other player literally started playing 3 weeks ago. I am certain that our choice to play D2 was the best choice.

I agree that you should. My team isn't much different than yours and we had the same reason for playing D2. That being said, the region does need to work very hard on getting more D1 teams to play, because that was the problem here. If D2 teams find the set too easy they either qualify for ICT and become D1 eligible next year, or choose to play it anyway. With 2 D1 teams this year and 3 D2 teams who should qualify for ICT, I see no reason why you're region shouldn't have a D1 field next year.

[The Ununtiable Twine]

Playing D2 was the most logical choice for Auburn considering the experience our team has. I am the only one who has years of experience playing pyramidal, while both Joseph and Neil pretty much started at the beginning of this academic year. And our other player literally started playing 3 weeks ago. I am certain that our choice to play D2 was the best choice.

I agree that you should. My team isn't much different than yours and we had the same reason for playing D2. That being said, the region does need to work very hard on getting more D1 teams to play, because that was the problem here. If D2 teams find the set too easy they either qualify for ICT and become D1 eligible next year, or choose to play it anyway. With 2 D1 teams this year and 3 D2 teams who should qualify for ICT, I see no reason why you're region shouldn't have a D1 field next year.

Raise your hand if you tried to avoid a combined field at the Region 6 SCT:

*raises hand

Yo guys, quizbowl is fun and challenging. It's a great opportunity to learn cool new stuff. Don't quit after you lose your DII eligibility, you're doing yourself and your team a disservice. I see so many people that quit after they lose their eligibility. That's sad. Maybe if those people stuck around and played quizbowl, we would have a bigger and better DI field than we did. The fact of the matter is, though, that a lot of the teams around here are full of young players who are going to become DI eligible soon enough.

Also don't forget that Florida was using its manpower on hosting and Tech decided to go to Blacksburg. In addition, don't forget that a lot of the teams that people say are "in the Southeast" (like South Carolina, Clemson, etc) are actually in a different ACUI region, so yes SCT fields in our ACUI region for DI are smaller than the expected size. I also claim that NAQT's changes in eligibility rules convinced a few teams in the region to join the DII field at our sectional and other sectionals around the country. I saw two or three teams in our region change from DI to DII over the course of a week or so. There tends to a problem getting four team DI fields at SCT sites even before the rule changes. While I support the direction NAQT is headed with their rule changes, you have to remember that there are a lot of teams that, given the chance, will elect to play DII instead, which will shrink the DI field size, which it probably did in our case. It's one of the side effects of the rule change.

[49-Mile Scenic Drive]

I have to say I felt the D2 set was rather easy compared to the past ones I've played (Every CCCT) I know alot of the reason I feel that way is because I've been playing quizbowl since I was 11 and NAQT since 8th grade. I realize alot of those questions I found to be "ridiculously easy" were just because I know alot about said subject. However there were a couple that I felt shouldn't have even been power at HSNCT, which I'll post in the questions section after this. The combined field wasn't something I was particularly fond of, but I didn't mind dealing with it. Could we (Alabama B) have played D1? Yes but I don't think any of us, except maybe Harrison, had any desire to. I wasn't even informed he was going to be in D2 with me until a few days before the tournament. Prior to that all signs indicated he'd be a member of D1. Had I not had Harrison as my teammate we probably would've dropped to 2nd or 3rd in D2 and may not have even qualified for ICT.

[Sun Devil Student]

I might point out that part of the issue is that "the best D2 players" (indeed, probably the top 15% or so) are way too good to really enjoy D2 SCT - perhaps if they were allowed to try to qualify for D2 ICT while playing D1 SCT questions they would have a better time. This could be done by letting D2-eligible teams register for D1 at their respective SCTs but choose to not be in contention for a D1 ICT bid, instead seeking to qualify to D2 ICT by statistics (D-value) only. Since they're not playing in D2, they wouldn't be contending for the D2 autobid, and since they're not trying to qualify to D1 ICT, they would not get a D1 autobid even if they win the D1 SCT that they enter themselves in. (Their D-value would certainly be high enough to not need the autobid.) The D1 autobid would then go to the top D1 team (if there are at least 4 actually-D1 teams present) even if that D1 team finished second to a good D2 team.

Actually, I'd also propose eliminating the D2 autobid for winning D2 teams, since this system assumes that the best D2 players will mostly choose to join the D1 field for playing purposes (though not statistical-qualification purposes). The D2 division winners are then no longer the best D2 teams and should not auto-qualify but should have to get D2 bids through D-value. This also removes any remaining incentive for super-good players to stay in the D2 questions because they have to qualify by D-value either way and thus might as well have more fun doing it on D1 questions.

Now, I realize there may be difficulties I'm not thinking of at the moment, but would this idea make everyone happy? The best D2 players can enjoy similarly-leveled competitions (D1 SCT and D2 ICT) without having a boring day of clubbing baby seals, the D1 field can play challenging games against good D2 teams at SCT while waiting to face harder opposition at ICT, and the true novices can stay in D2 until they earn their promotion to D1 by qualifying to D2 ICT statistically. And this way, the D2 field every year will contain only the top 26 D2 teams in the country regardless of which SCT division they chose to play in, no weaker D2 teams getting autobids and leaving stronger teams "stuck" in D2 an extra year. (I've left the 6 community college spots in place in this proposal; that's for another whole discussion.)

Are there any significant parts of our demographic who would be unhappy with this system?