SCT 2009 discussion

[Sen. Estes Kefauver (D-TN)]

Yeah, obviously the whole reason you get an autobid is to have the incentive to not play your best readers.

[OntarioQuizzer]

After taking a look over the stats from the last 3 years of tossups read in d2 sectionals over the last 3 years (since presumably any club will put their best readers in d1), there is a trend that sectionals held in the South (and strangely, Canada) are the 3 or 4 lowest TUHPR, with the exception of Georgia's SCT this year. Also worth noting is that the sectionals the South unfortunately seem to have the lower tossup conversion, so shouldn't they actually get through more tossups because of that? I think the accent thing sounds like a reasonable explanation. Not an excuse or "get my mods of the hook." Just an explanation.

IMO, Canada's low TUH over the past 3 SCT's (since the one that I hosted at Brock, if I recall) is because the increase in quality of moderating in Canada has failed to keep pace with the increase in quality of play, for the most part. When your players aren't as good, your moderators don't have to be as good to get through 20-22 tossups. However, when your players get more tossups, aren't powering as many (due to the fact that NAQT isn't handing power out like Hallowe'en candy like it did at the 2006 SCT), and you as a moderator need to read bonuses in 90% of the cycles, you need to implement "hustle" strategies in order to get through 20-22 cycles a game.

I'll admit -- as the SCT director in 2007 at Brock, I was caught off-guard by the increase in the quality of play between the 2006 and 2007 SCTs. I still don't think that the overall quality of moderators in Canada has caught up.

[naturalistic phallacy]

Hey, I had a random question based on some comments on both this and the private forum:

NAQT editors repeatedly reference that a tossup "played well" in testing when discussing difficulty, clue placement appropriateness, or whether something is a reasonable answer. I'm quite intrigued as to who this group of people ends up being. Most active players participate in SCT, and those that don't usually participate in the ICT. Of the "elder statesmen" who do not participate in sectionals, ICT, or non-open quizbowl, some of them work for NAQT already and a lot of the others I'm in contact with do not, to my knowledge, test NAQT tossups. So, who's in this group? I mean, I would think you'd need at least some active players who have a good grasp of the canon at various levels to effectively do this. Such a burden may be impossible for a tournament that draws so many participants, but if that's the case, I think the fact that something played well in difficulty testing might need to be rethought. In any case, I'd be quite interested to know who does this kind of stuff for NAQT.

I had this question as well. As someone in this potential target group (i.e. a writer for NAQT and a semi-recent college gradudate), I would be interested in being part of this group in the future.

I know that R hosts playtesting of SCT, ICT, and HSNCT at his house in the Twin Cities Metro area, and as far as I know, attendees are members of the quizbowl community who are not playing those tournaments. For SCT/ICT, however, this usually means former college players who are not as intimately connected to the circuit as most players would like.

[DumbJaques]

I know that R hosts playtesting of SCT, ICT, and HSNCT at his house in the Twin Cities Metro area, and as far as I know, attendees are members of the quizbowl community who are not playing those tournaments. For SCT/ICT, however, this usually means former college players who are not as intimately connected to the circuit as most players would like.

Well, this is kind of where I was going with this. This group is inherently limited to both "people who live near R" and "people who don't know incredibly vital things about the circuit/current canon anymore." It's fine to do this, but relying on it as a difficulty check is a tremendously bad idea. It leads to situations where "somebody knew Lin Biao, that proves everyone does!" and nonsense like that, and it's compounded when the people involved aren't even part of the circuit anymore (although the idea that people ever knew Lin Biao on the scale of a tossup answer seems. . . dubious).

Given the capability of the interwebs and noted interweb wizard Mike Bentley's stated interest in participating in this, maybe it's time to upgrade the playtesting procedure to find a group of people who will say "hey, maybe we shouldn't have this question on All my Exes live in Texas?"

[grapesmoker]

"hey, maybe we shouldn't have this question on All my Exes live in Texas?"

I'm fairly certain we could come to this conclusion even without playtesting!

[setht]

I know that R hosts playtesting of SCT, ICT, and HSNCT at his house in the Twin Cities Metro area, and as far as I know, attendees are members of the quizbowl community who are not playing those tournaments. For SCT/ICT, however, this usually means former college players who are not as intimately connected to the circuit as most players would like.

Well, this is kind of where I was going with this. This group is inherently limited to both "people who live near R" and "people who don't know incredibly vital things about the circuit/current canon anymore." It's fine to do this, but relying on it as a difficulty check is a tremendously bad idea. It leads to situations where "somebody knew Lin Biao, that proves everyone does!" and nonsense like that, and it's compounded when the people involved aren't even part of the circuit anymore (although the idea that people ever knew Lin Biao on the scale of a tossup answer seems. . . dubious).

Given the capability of the interwebs and noted interweb wizard Mike Bentley's stated interest in participating in this, maybe it's time to upgrade the playtesting procedure to find a group of people who will say "hey, maybe we shouldn't have this question on All my Exes live in Texas?"

R.'s playtesting session was not the only one; there was also a playtesting session on the West Coast with some Berkeley people (and possibly some non-Berkeley people, I don't know exactly who all was there). Speaking only for myself, I'll say that I found the feedback from the playtesting sessions very useful in late-stage editing, and I wish I'd gotten more of it; if I were doing this again I'd certainly be happy to have Mike Bentley and any other recently-graduated people playtesting the questions (perhaps also some confirmed staffers at host sites?). I don't know how easy it is for NAQT to set up and run these playtesting sessions, but as an editor my preference is to have as much as possible. I'll also say that I didn't swallow all of the feedback whole--I made changes based on almost all of the feedback since I thought almost all of the feedback was on the money, but there were some bits of feedback that I considered and then decided not to act on. I imagine it's been the same for previous set editors, so I don't think anyone should be worried that a small group of playtesters is going to drag a set off in some weird direction with their feedback.

-Seth

[Tower Monarch]

if I were doing this again I'd certainly be happy to have Mike Bentley and any other recently-graduated people playtesting the questions (perhaps also some confirmed staffers at host sites?).

For future years (or even for ICT), consider some high school students who are definitely not playing on the set. For example, I certainly would have been willing and I imagine Ian Eppler would have been as well, and these are just two staffers; Maggie Walker players are decent measures of difficulty for a Div I set, from what I have seen of each.

[cdcarter]

if I were doing this again I'd certainly be happy to have Mike Bentley and any other recently-graduated people playtesting the questions (perhaps also some confirmed staffers at host sites?).

For future years (or even for ICT), consider some high school students who are definitely not playing on the set. For example, I certainly would have been willing and I imagine Ian Eppler would have been as well, and these are just two staffers; Maggie Walker players are decent measures of difficulty for a Div I set, from what I have seen of each.

I know R offered to me and a few other local high school players the opportunity to come to the SCT playtesting, so it is on their radar.

[Wall of Ham]

My two cents in regards to the clock:

As many have figured out, the number of questions read in a timed tournament is related to moderator speed. At good tournaments with good moderators, the clock is no issue, as all moderators get off at least 18 questions per round. However, there will always be certain tournaments where the moderators aren't that fast, and without clocks they will go slower. For a tournament like SCT, which occurs across the country and with a relatively large number of teams, the number and likelyhood of bad moderators increases, making the prescence of clocks at this tournament even more noticable.

The clock allows tournament directors to have more control over the length of the tournament, in a way decreasing the uncertainty in the time of each round while increasing the uncertainty of the number of questions read. The question is, which way do we want to push this uncertainty?

Now in a tournament where all the teams are within a 0-3 hours drive, time is not that big an issue, and most likely people would want more questions read. This might occur in the northeast, or the mid-atlantic if the tournament was in Richmond. However, if teams drove a long time to the tournament or have planes/trains to catch, time is a big issue, and clocks here are useful. This is probably true for sites like Canada/the NW. I know as member of a team 6 hours away from most tournaments I would prefer some rounds where I may hear only 15 questions, so that I don't have to arrive home at 3 AM or stay overnight at some motel.

So the question really is, should clocks should be optional?

With the current (and excellent) rule changes by NAQT, the biggest issue with clocks is that of time vs. # of questions. However, if we make clocks optional, then I am thinking most SCT would not be timed, as the added time of organizing clocks, etc, would be Occam's razor'd by the TD, and then the tournaments that most benefit by having clocks would suffer.


I personally enjoy the usage of clocks, and I see little harm in them. When a tournament uses clocks however, it should focus on giving their moderators some training (pointers, letting them read in practice, etc) for maximum advantage. Doing the either 20 Qs or 20 minutes thing completely erases the need for a clock in the first place (to speed up the game), and monetary compensation for few questions read doesn't seem workable.

[Important Bird Area]

there was also a playtesting session on the West Coast with some Berkeley people (and possibly some non-Berkeley people, I don't know exactly who all was there).

The west coast playtest group was Paul Lujan, Juliana Froggatt, and Larissa Kelly.

This group is inherently limited to both "people who live near R" and "people who don't know incredibly vital things about the circuit/current canon anymore." It's fine to do this, but relying on it as a difficulty check is a tremendously bad idea. It leads to situations where "somebody knew Lin Biao, that proves everyone does!" and nonsense like that, and it's compounded when the people involved aren't even part of the circuit anymore (although the idea that people ever knew Lin Biao on the scale of a tossup answer seems. . . dubious).

Given the capability of the interwebs and noted interweb wizard Mike Bentley's stated interest in participating in this, maybe it's time to upgrade the playtesting procedure to find a group of people who will say "hey, maybe we shouldn't have this question on All my Exes live in Texas?"

It's worth noting that playtesting finished packets and difficulty-checking individual questions are two different systems. NAQT uses an internal email list for the latter, not just random players assembled in Minnesota.

Because the most difficult NAQT questions land in the Division I ICT, NAQT's difficulty-check list is limited to people who have committed to retire from playing any level of NAQT products. Necessarily this means they are somewhat disconnected from the circuit as a whole, because the most active circuit players will still want to play ICT, rather than recuse themselves to tinker with its difficulty level. NAQT is of course open to suggestions, but I think the current system strikes a good balance between question security and circuit activity.

[Susan]

If anyone at NAQT wants to set up IRC (or whatever other remote method) playtesting for the ICT, they can count me in. The difficulty check list is nice, but only a small fraction of the questions in any given tournament come up on it.

[ezubaric]

"hey, maybe we shouldn't have this question on All my Exes live in Texas?"

I'm fairly certain we could come to this conclusion even without playtesting!

Not to defend the question, which I seem to dimly recall being pretty transparent, but this is a song in the country genre that is pretty well known outside of country circles and is not a recent flash in the pan. There have been far, far worse topics of trash questions selected. (Which I won't go into because there's no reason to add more offtopic material to the tread.)

[grapesmoker]

this is a song in the country genre that is pretty well known outside of country circles

Well, there's the problem!

[Blackboard Monitor Vimes]

If anyone at NAQT wants to set up IRC (or whatever other remote method) playtesting for the ICT, they can count me in. The difficulty check list is nice, but only a small fraction of the questions in any given tournament come up on it.

Several of us at MLWGS would most likely be able to help out if this were an option...next year there will be three of us who have played college tournaments intermittently since fall 2007, so we would probably be of some use. Doing this remotely seems like it would allow for a much larger sample size than the current situation and hopefully allow more people with a sense of the current canon to be present...

[at your pleasure]

I would not be adverse to helping to playtest the set.

[Maxwell Sniffingwell]

I could help playtest the ICT.

[Jeremy Gibbs Lemma]

As could I.

[Jeremy Gibbs Paradox]

I think we're forgetting one thing: Southerners just don't speak as quickly as people in other parts of the country, and it adds up on a 20-minute clock. I feel like that's exactly why, even though I screened and trained all of the mods (except one, who had to replace someone with a sudden illness), and told them 10 quesitons per half are expected, 9 is the minimum, half the games ended at 17 or fewer questions read. I never really felt like anything was going too slow though, in fact I thought every moderator was trying to go as fast as they could.

I definitely support allowing matches to go to 20 questions.

You know what I know about Southerners from my relatives in Mississippi? Some of them...can FLY!

And they're made of 90% pixie dust!!!

And every single southerner has a recipe for cornbread: THAT CURES CANCER!

[AKKOLADE]

And every single southerner has a recipe for cornbread: THAT CURES CANCER!

It's pretty tasty cornbread, too!

[Not That Kind of Christian!!]

I could help playtest the ICT.

I as well.

[Important Bird Area]

While NAQT is glad everyone is interested in volunteering to help out with the ICT, we can't accept outside playtesters for reasons of question security. (For instance, if Hannah, Greg, or Kent were to identify a repeat in this year's ICT, we would then be blocked from using the extra question in future ICTs until they retire. Similarly, if one of the Maggie Walker players were to flag a question as "much too easy for ICT level," we wouldn't be able to redirect it to the HSNCT set.)

If, however, you're both interested in testing ICT questions *and* permanently retired from all levels of NAQT play, we'd be glad to have you on board; please contact me or R.