2009 HSNCT question discussion

[Captain Sinico]

...it sound like [CHRIS RAY! and potentially other people who dismiss with Matt Bruce's defense of the "purring" question due to the guessability of its answer from clues] disagree with, say, Andrew Yaphe's description of how the game is played.

You, Seth Teitler, are here failing to draw the crucial distinction between the way questions are/ought be played and the way they ought be written. One ought always write questions that can be answered using real, academic knowledge (and must be answered that way early; are on relevant, important things; etc.) though it is inevitable that players will (with whatever frequency) answer using non-knowledge* of various kinds. It seems to me that the purring question (besides being on something inane and non-academic) is accessible only to guessing, because there were no knowledge-based clues+. That's unacceptable to me and, I suspect, is to most everyone; consequently, saying "Someone might reasonably have guessed the question's answer from its clues" is no defense of the question.

MaS

*I don't think that a precise definition of "knowledge" is necessary here; I trust my readers understand what I mean. If that's not the case, we can bring that up again.
+Perhaps there were some knowledge-based clues that I don't recognize but, at any rate, I see nobody arguing that they had a reasonable expectation of players knowing any of the clues (frankly, even to the end; cats do make other sounds, after all) which is really more relevant. I'm willing to examine this point as well, of course.

[Captain Sinico]

...I disagree tha ... the NHL bonus was difficult...

That bonus was very, very hard.

MaS

PS: Hay guys, don't spend so much time talking about the sports questions from an academic tournament! Wait, what's that you say? 7.1% of the distribution?! Oh, carry on, then.

PPS: In case you didn't pick up on what I was just saying, there's far, far too much trash at NAQT.

[Captain Sinico]

Bhagavad Gita (prompt on "Mahabarata")

Yeah, I felt that that tossup's answer line wasn't appropriate. For example, a fairly early clue stated "This work begins with [how the Bhagavad Gita begins]" which ought make the Mahabarata unpromptably wrong (in something of a screwjob either way.)
Actually, to speak to that more generally, I wish an NAQT representative would delineate its answer policy*, rather than, for example, outright refusing to do so at the moderator meeting before the tournament, having massively wasted the time of more or less every moderator with rules everyone already knew.

MaS

*I should say, either delineate or seriously rethink the answer policy, the latter applying if the "What's on the paper is right" policy issued at the moderator meeting is, in fact, the official policy.

[Terrible Shorts Depot]

...I disagree tha ... the NHL bonus was difficult...

That bonus was very, very hard.

MaS

PS: Hay guys, don't spend so much time talking about the sports questions from an academic tournament! Wait, what's that you say? 7% of the distribution? Oh, carry on, then!

PPS: In case you didn't pick up on what I was just saying, there's far, far too much trash at NAQT.

The main problem with this bonus seemed to be how it was written, not the concept or the answers. Instead of saying "blahblahblah used to be the Nordiques", it should have said something like "blahblahblah used to be the Nordiques blahblah Joe Sakic Peter Forsberg Patrick Roy". Basically, the original bonus required really substantial "real" hockey knowledge, which not too many people 'round here have, thus making it really impossible unless you live in Colorado, New Jersey, or Detroit, though I suppose you could have figured out the Detroit part, due to Michigan geography knowledge. With the addition of some famous player names or other such easier clues, this bonus could have been something other than a guaranteed 0 or 30.

Also, the Lord Kelvin tossup left me wanting. The inclusion of the phrase "some things he wasn't wrong about" tipped me off that it was someone who was famously wrong a lot. Lord Kelvin is most famous for two things: a temperature scale and terrible predictions. That's just a stupid thing to have in a question, so FIFTEEEEEN! for me and anyone else who can guess people who said stupid stuff.

[Important Bird Area]

*I should say, either delineate or seriously rethink the answer policy, the latter applying if the "What's on the paper is right" policy issued at the moderator meeting is, in fact, the official policy.

I don't have access to the precise wording of what was said at the moderator's meeting (it sounds like there was some confusion and that we need to do better next year).

But for the record, our official policy is that moderators should start with what's on the paper; if 1. the moderator believes that what's on the paper is wrong or incomplete and 2. the decision changes the outcome of the game, then the matter should go up to the control room for protest resolution.

Note in particular that "whatever's on the paper is right even if it's wrong" is not (and never has been) NAQT policy, official or unofficial.

It's also true that our questions can and should be more consistent about accepting alternate/variant answers, in particular for common link tossups of the type "one title containing this substance blah blah blah." We'll be talking about this over the summer.

Edit: consistency re: answer lines

[Captain Sinico]

So, NAQT's official policy is that a moderator can not/ought not overrule an answer line in any case? I suggest that be reconsidered, at least for HSNCT.
The issue I raised at the moderator's meeting was this: there were two common link questions that hinged on sifting common words out of series of titles. One had the common word as the answer, but said to accept any of the titles; the other had the common word as the answer and no additional information (which means, given the previously mentioned "paper is always right, let people protest if they think it's wrong" policy, the common word is the only acceptable answer.) I asked which was correct, since in this case "the paper" was clearly at odds with itself. The reply was "I'm not going to answer that question."

MaS

[Important Bird Area]

So, NAQT's official policy is that a moderator can not/ought not overrule an answer line in any case? I suggest that be reconsidered, at least for HSNCT.

The fully correct version is "ought not overrule an answer line without notifying the control room/tournament director." That is: when we make a mistake (and we do), we need to make sure that the change is both propagated to other game rooms (to ensure consistency of results within the tournament) and eventually reported directly to NAQT for future correction (eg, finding a mistake in the first use of an IS set that can be corrected when packets are printed for future uses of the same set).

[Captain Sinico]

The fully correct version is "ought not overrule an answer line without notifying the control room/tournament director."

That is much better. Next year, please have that announced that at the meeting, rather the offensively wrong policy what was announced (at great length.)
So, to restate my question that remains then, what I'd like clarified is: what's NAQT's policy on what the answer line ought to be? For example, what is the answer to my unanswered question?

MaS

[etchdulac]

The fully correct version is "ought not overrule an answer line without notifying the control room/tournament director."

That is much better. Next year, please have that announced that at the meeting, rather the offensively wrong policy what was announced (at great length.)
So, to restate my question that remains then, what I'd like clarified is: what's NAQT's policy on what the answer line ought to be? For example, what is the answer to my unanswered question?

MaS

I heard the question you asked, and understood it was a fairly crucial observation... and here we are five days later, and you're still looking for an answer. I think that sums up the shortcomings of the moderator-meeting experience.

[setht]

...it sound like [CHRIS RAY! and potentially other people who dismiss with Matt Bruce's defense of the "purring" question due to the guessability of its answer from clues] disagree with, say, Andrew Yaphe's description of how the game is played.

You, Seth Teitler, are here failing to draw the crucial distinction between the way questions are/ought be played and the way they ought be written. One ought always write questions that can be answered using real, academic knowledge (and must be answered that way early; are on relevant, important things; etc.) though it is inevitable that players will (with whatever frequency) answer using non-knowledge* of various kinds. It seems to me that the purring question (besides being on something inane and non-academic) is accessible only to guessing, because there were no knowledge-based clues+. That's unacceptable to me and, I suspect, is to most everyone; consequently, saying "Someone might reasonably have guessed the question's answer from its clues" is no defense of the question.

MaS

*I don't think that a precise definition of "knowledge" is necessary here; I trust my readers understand what I mean. If that's not the case, we can bring that up again.
+Perhaps there were some knowledge-based clues that I don't recognize but, at any rate, I see nobody arguing that they had a reasonable expectation of players knowing any of the clues (frankly, even to the end; cats do make other sounds, after all) which is really more relevant. I'm willing to examine this point as well, of course.

I agree with pretty much everything you're saying here, Mike; I guess I still haven't clarified what prompted me to post in the first place. In particular, I agree with your descriptions of how people should write academic questions (presumably trash questions don't always have to start with real, academic knowledge-type clues) and how people play academic questions. I think your criticisms of the purring tossup (few or no clues corresponding to "real academic knowledge," leaving the tossup only accessible to early guessing or buzzing on an inane giveaway) are also spot on. FInally, I agree that the claim "someone might reasonably have guessed the answer from clues" is not a sufficient defense of any question. I did not mean to advance that claim as a defense of the purring tossup, and in fact I don't want to advance any defense of the purring tossup. What I wanted to do was point out that what is wanted is a slightly more nuanced argument against questions like the purring tossup--something along the lines of what you've written should do nicely--that makes it clear that the property "someone might reasonably have guessed the answer from clues" is not automatically an indictment of a question.

It's entirely possible that Chris's post was sufficiently nuanced to start with; he does refer to lateraling/frauding answers, after all. However, my initial reaction was that it sounded like he was sounding off against any tossup where someone could figure out the answer from clues without knowing any single clue cold. I'm pretty sure he doesn't hold that position, but I thought it might be good to clarify. If I'm really the only one that managed to misread Chris's post in that way I apologize.

-Seth

[Captain Sinico]

It would seem that we agree, Seth, as we already knew we did. For the sake of home listeners, we also agree, I think, that saying that a question is highly guessable, especially early, is, in fact, an indictment of that question (and incidentally one that might be read against the "purring" question... indeed, it's perhaps hard to find an indictment we agree on that won't stick to that particular question.)

MaS

[Tanay]

I just had a general question about the packets for the HSNCT. Were the playoff packets designed to be harder than the prelim packets? Additionally, was each playoff packet supposed to be harder than the last (finals questions harder than semifinals, semifinals harder than quarterfinals, etc)?

[Important Bird Area]

I just had a general question about the packets for the HSNCT. Were the playoff packets designed to be harder than the prelim packets? Additionally, was each playoff packet supposed to be harder than the last (finals questions harder than semifinals, semifinals harder than quarterfinals, etc)?

No, all HSNCT packets are intended to meet the same difficulty level.