2011 NSC Question Set Discussion
Posted: Mon Jun 06, 2011 10:59 am
Let's use this thread to discuss the questions at the 2011 NSC. FYI, I edited and wrote a lot of the history and I'll have some more to say about that later.
Sponsored by the Partnership for Academic Competition Excellence (Twitter: @PACENSC)
https://www.hsquizbowl.org/forums/
Part of this may have been because I generally used the feedback from playtesting sessions to remove harder clues from the beginning of tossups and replace bonus parts and occassionally tossup answers that turned out to be too hard. I still think there were still a few things in this category that were too hard that made it into the final set, though.Morraine Man wrote:I playtested a small portion of the history and found it egregiously difficult, but apparently the field disagreed as I have only heard good things about questions.
Wait, this happened? Since all teams are playing in the playoff and superplayoff, why should they be harder than the prelims? I can understand the finals being harder, but it's kinda pointless to make playoffs harder.every time i refresh i have a new name wrote:
- The splitting of difficulty into prelim/playoff/superplayoff/final was a mistake. It's simply infeasible to calibrate difficulty that finely, and it would have put a significantly less strain on the editing crew to make everything uniform in the first place rather than to change the difficulty of individual packets constantly as the format was in flux. Regardless of what the plan was, I noticed several easier questions in the superplayoff rounds and some harder questions in the prelims.
The instructions I received was to group the difficulty into 3 levels: prelims, playoffs and super-playoffs (which were to be at the same difficulty), finals (and I guess All-Star for a fourth level).Wurzel-Flummery wrote:Wait, this happened? Since all teams are playing in the playoff and superplayoff, why should they be harder than the prelims? I can understand the finals being harder, but it's kinda pointless to make playoffs harder.every time i refresh i have a new name wrote:
- The splitting of difficulty into prelim/playoff/superplayoff/final was a mistake. It's simply infeasible to calibrate difficulty that finely, and it would have put a significantly less strain on the editing crew to make everything uniform in the first place rather than to change the difficulty of individual packets constantly as the format was in flux. Regardless of what the plan was, I noticed several easier questions in the superplayoff rounds and some harder questions in the prelims.
I support this generally, but because of the carry-over game I don't think the superplayoffs should have been of a different difficulty than the regular playoffs. I'm all for harder questions to better differentiate similarly-skilled teams, but when you have a carry-over game you have one game in the final standings that's being decided on different difficulty than the other games.every time i refresh i have a new name wrote:- The splitting of difficulty into prelim/playoff/superplayoff/final was a mistake. It's simply infeasible to calibrate difficulty that finely, and it would have put a significantly less strain on the editing crew to make everything uniform in the first place rather than to change the difficulty of individual packets constantly as the format was in flux. Regardless of what the plan was, I noticed several easier questions in the superplayoff rounds and some harder questions in the prelims.
Raynell is absolutely right, and it's unacceptable that Andy Watkins did not think through the obvious implication of having the superplayoffs be harder than the playoffs, which is that it arbitrarily makes some of your superplayoff games on disproportionately easy packets and then tests a somewhat different set of skills for the rest of the games. Incredibly incompetent idea.Bentley Like Beckham wrote:The instructions I received was to group the difficulty into 3 levels: prelims, playoffs and super-playoffs (which were to be at the same difficulty), finals (and I guess All-Star for a fourth level).Wurzel-Flummery wrote:Wait, this happened? Since all teams are playing in the playoff and superplayoff, why should they be harder than the prelims? I can understand the finals being harder, but it's kinda pointless to make playoffs harder.every time i refresh i have a new name wrote:
- The splitting of difficulty into prelim/playoff/superplayoff/final was a mistake. It's simply infeasible to calibrate difficulty that finely, and it would have put a significantly less strain on the editing crew to make everything uniform in the first place rather than to change the difficulty of individual packets constantly as the format was in flux. Regardless of what the plan was, I noticed several easier questions in the superplayoff rounds and some harder questions in the prelims.
There was in fact 5/5 Middle Eastern history in the first 20 questions over the course of 25 rounds (plus 1/1 in the post-20 range), for a total percentage of 20% Middle East and 80% everything else. I laid out the world history distribution back when Auroni was signed on as the history editor. I tried to do so in such a way that the world history would be balanced among all regions. The distribution was (post-20 in parentheses):Papa's in the House wrote:The World History seemed to be 50% middle east and 50% everything else.
As Mike just posted, the superplayoffs were not intended to be more difficult than the playoffs because games from the playoffs were carried over into the superplayoffs.Jeremy Gibbs Freesy Does It wrote:Raynell is absolutely right, and it's unacceptable that Andy Watkins did not think through the obvious implication of having the superplayoffs be harder than the playoffs, which is that it arbitrarily makes some of your superplayoff games on disproportionately easy packets and then tests a somewhat different set of skills for the rest of the games. Incredibly incompetent idea.Bentley Like Beckham wrote:The instructions I received was to group the difficulty into 3 levels: prelims, playoffs and super-playoffs (which were to be at the same difficulty), finals (and I guess All-Star for a fourth level).
I guess it was just sample bias and bad memory that made me think that. Thanks for clarifying.Kyle wrote:There was in fact 5/5 Middle Eastern history in the first 20 questions over the course of 25 rounds (plus 1/1 in the post-20 range), for a total percentage of 20% Middle East and 80% everything else. I laid out the world history distribution back when Auroni was signed on as the history editor. I tried to do so in such a way that the world history would be balanced among all regions. The distribution was (post-20 in parentheses):Papa's in the House wrote:The World History seemed to be 50% middle east and 50% everything else.
5/5 (1/2) Latin America
5/5 (1/1) Middle East
5/5 (1/2) Africa
5/5 (2/1) East Asia
2/2 Southeast Asia
2/2 South Asia
1/1 Australia/Pacific
Also possibly the influence of some ME/North African geography and current events.Papa's in the House wrote:I guess it was just sample bias and bad memory that made me think that. Thanks for clarifying.
It isn't. For something around two thirds of the year, we were operating on a sixty-four team format and editing the set to correspond to that, including the seven round prelims that the sixty-four team format requires. So, for a long time we had seven rounds of prelim-appropriate questions. Once the sixty team format was confirmed I ran through rounds six and seven to do any appropriate switches and tuning necessary to make their difficulty match the rest of the playoffs as well as any pair of rounds' difficulty can match. So, the playoffs were not played on two different difficulties of packet.Jeremy Gibbs Freesy Does It wrote:Sorry, I stupidly wasn't reading that carefully at all. Never mind. Although this 7 rounds thing is a laugh if true.
I'd have to double check but it's also possible the Middle East history questions may have been spread out more into the rounds played by everyone than the rounds not played by everyone (e.g. tiebreakers, finals, emergency packets).Crazy Andy Watkins wrote:Also possibly the influence of some ME/North African geography and current events.Papa's in the House wrote:I guess it was just sample bias and bad memory that made me think that. Thanks for clarifying.
Watkins says it wasn't above. I was told it was by another editor. Contradictory information and a lack of communication to the control rooms was definitely the theme of this past weekend, so it seems only fitting.Jeremy Gibbs Freesy Does It wrote:Sorry, I stupidly wasn't reading that carefully at all. Never mind. Although this 7 rounds thing is a laugh if true.
Argh, sorry about that. That was a sloppy error on my part. I hope that it did not impact the outcome of any games.jonah wrote:and a bonus whose intro quoted some poetry that the first part claimed was by Leon Czolgosz (via asking for Czolgosz) but was actually by Charles Guiteau.
Not as I understand it; in fact, the appearance of three stars is intentionally to signify something later than sundown: in order that we be sure it's been a full day since Shabbat began, we actually make sure it's been slightly more than a full day.Matt Weiner wrote:Isn't "the appearance of three stars" the definition of "sundown" for Jewish religious purposes?
Jonah's right on this one. A sundown-to-sundown holiday or fast starts a bit before the first sundown and ends when three stars are visible after the second sundown (or 18 minutes after sundown if the weather prevents stars from being seen). That said, it is nit-picky and most reform congregations just say sundown.Matt Weiner wrote:Isn't "the appearance of three stars" the definition of "sundown" for Jewish religious purposes?
As someone who worked for the Department of Health and Human Services answering letters addressed to the Secretary, I can tell you that Thompson hasn't been in office since 2005. It was sort of an office joke when we received letters addressed to Thompson, just because it indicated that whoever was writing didn't really have it all together. Your protest definitely deserved a look.Joe N wrote:That being said, I'd just like to point out that the governor of California hasn't been a Republican for over a year and Tommy Thompson hasn't been secretary of Health and Human Services since around January 20, 2009. The former was an annoyance in a bonus along the lines of "Name one of the two states with Republican governors who got most of the money Wisconsin and Ohio turned down for high speed rail." Florida does have a Republican governor, California does not. The Tommy Thompson one did end up costing us our wild-card game: in a history tossup on politicians from Wisconsin, there was a clue along the lines of "The current Secretary of Health and Human Services is from this state..." Buzz "...Tommy" My answer was along the lines of "What? That doesn't make any sense, but the current Secretary of Health and Human Services is from Kansas." My protest was denied because "All the previous clues pointed to Wisconsin and the moderator already said Tommy." I think according to the rules it should have been thrown out and replayed, which would have altered the outcome of the match.
You're right. I'm really sorry about that.Joe N wrote:That being said, I'd just like to point out that the governor of California hasn't been a Republican for over a year
As am I, since that's more or less just a silly thing I should certainly have been able to catch.Kyle wrote:You're right. I'm really sorry about that.Joe N wrote:That being said, I'd just like to point out that the governor of California hasn't been a Republican for over a year
I am similarly sorry about error in the Wisconsin question. I don't know exactly how this error got in. The only thing I can think of is there was a disconnect between me reading "former Secretary of Health and Human Services" and then typing "current Secretary of Health and Human Services" or something. Hopefully this is the last of the errors in the history set that appeared in this tournament--it's already unacceptable that there were so many. Not that I'm currently planning on editing a subject for the tournament next year, but in my future editing efforts I'll make sure to amend my editing process to include a final fact checking pass-through in an attempt to catch things like this or error on the assassination bonus.Kyle wrote:You're right. I'm really sorry about that.Joe N wrote:That being said, I'd just like to point out that the governor of California hasn't been a Republican for over a year
Oh no, you're right. I completely screwed up that entire bonus part. The governor of California thing was just stupid all the way around; the Florida aspect was an instance of me not following the news after I had written the bonus in January. Let this be a lesson to all: write your questions soon before the tournament happens.Westwon wrote:Florida has turned down the high speed rail money.
Lest that recommendation be taken over-broadly.Kyle wrote:Let this be a lesson to all: write your current events questions soon before the tournament happens.
You probably misheard, or perhaps the moderator misspoke. Here is the bonus part I believe you're talking about (as I originally wrote it; it may have been changed, but I don't think so):Westwon wrote:Unless I misheard, I believe a bonus part gave Enoch as Noah's grandfather, though Enoch is generally regarded as Noah's great grandfather.
I felt this way as well, and am curious to see the religion subdistro to see how it was split amongst the various religions.Westwon wrote:Unless I misheard, I believe a bonus part gave Enoch as Noah's grandfather, though Enoch is generally regarded as Noah's great grandfather.
In general, the religion seemed skewed towards Judaism and away from Christianity, though this perception may be due to some of the non-religious questions being related to Judaism.
Yeah that bonus part confused me to no end. Current events questions should be written as close to tournament date as feasible.Kyle wrote:Oh no, you're right. I completely screwed up that entire bonus part. The governor of California thing was just stupid all the way around; the Florida aspect was an instance of me not following the news after I had written the bonus in January. Let this be a lesson to all: write your questions soon before the tournament happens.Westwon wrote:Florida has turned down the high speed rail money.
So that bonus part was doubly wrong, which is why you were rightly confused. Even so, I don't necessarily agree with your second sentence. Obviously it's important to do a better job than I did of making sure that things haven't changed since the question was written (oddly, Andy and I did have a discussion last week about the first part of the bonus because Scott Walker has since made himself more famous than he was a few months ago). But I think it's important that the current events distribution cover more than just the most recent events. For one thing, the NSC is an annual tournament, so a year's worth of current events should be given equal weight. But more important, in my opinion, is that the current events category should focus broadly on things that are important in the world today rather than just on particular events (a single election or a protest or whatever). That's why I wrote the high-speed rail bonus in the first place -- because it's an ongoing topic of continuing importance. I just stopped following it too soon and in the process really, really screwed up that bonus, for which I am sorry. That doesn't mean that current events questions have to be written closer to the tournament date so much as it means that the person writing them has to do a better job of paying attention to the news as it develops.Wurzel-Flummery wrote:Yeah that bonus part confused me to no end. Current events questions should be written as close to tournament date as feasible.Kyle wrote:Oh no, you're right. I completely screwed up that entire bonus part. The governor of California thing was just stupid all the way around; the Florida aspect was an instance of me not following the news after I had written the bonus in January. Let this be a lesson to all: write your questions soon before the tournament happens.Westwon wrote:Florida has turned down the high speed rail money.
Hey look, it's another team eliminated from championship contention on a protest resolution in direct contradiction to the rules!Joe N wrote:The Tommy Thompson one did end up costing us our wild-card game: in a history tossup on politicians from Wisconsin, there was a clue along the lines of "The current Secretary of Health and Human Services is from this state..." Buzz "...Tommy" My answer was along the lines of "What? That doesn't make any sense, but the current Secretary of Health and Human Services is from Kansas." My protest was denied because "All the previous clues pointed to Wisconsin and the moderator already said Tommy." I think according to the rules it should have been thrown out and replayed, which would have altered the outcome of the match.
The text of the tossup read:Joe N wrote:The Tommy Thompson one did end up costing us our wild-card game: in a history tossup on politicians from Wisconsin, there was a clue along the lines of "The current Secretary of Health and Human Services is from this state..." Buzz "...Tommy" My answer was along the lines of "What? That doesn't make any sense, but the current Secretary of Health and Human Services is from Kansas." My protest was denied because "All the previous clues pointed to Wisconsin and the moderator already said Tommy." I think according to the rules it should have been thrown out and replayed, which would have altered the outcome of the match.
The relevant rules states:In this state’s largest city, Victor Berger led the so-called Sewer Socialism movement. In the late 19th century, it was the site of the deadliest fire in American history, the Peshtigo Fire. One politician from this state joined George Norris in filibustering the Armed Ship Bill, much to the chagrin of Woodrow Wilson, and later ran for president with Burton K. Wheeler as his running mate. The current US Secretary of Health and Human Services, (*) Tommy Thompson, was once its governor. Another governor worked with university professors to push for legislation like primary elections and the country’s first state income tax, a policy which became known as this state’s namesake “idea”. For 10 points, name this Midwestern state home to Progressive leader Robert LaFollette, Sr.
ANSWER: Wisconsin
We determined that the clue, though factually incorrect, did not "uniquely describe different answers" and the clue buzzed on could not be said to "definitely...not refer to the same answer as the other clues." This is for two reasons: the player (as we understood it) had heard the word "Tommy," but more importantly, the phrase that led to the buzz did not refer back to the answer. If the question had said "The current secretary of HHS, Kate Sebelius, once lived in this state before moving to Kansas," with the player buzzing after "Kate" with "Kansas," the issue would be essentially the same, and the protest would also be denied (assuming for this example that Sebelius actually did live in Wisconsin at one point). The rule in question confines its application to two very narrow circumstances, neither of which is present here. Both the actual question and the hypothetical above are misleading questions, but there is no recourse in any of the protest rules for this situation.Two or more clues within the question uniquely describe different
answers or one clue definitely does not refer to the same answer as the
other clues—hence, there is no correct answer to the question. This
protest may be lodged by either team at the end of a question
converted by neither team or by the team that did not get the question
after a question is converted by one team. However, if this situation
arises on a tossup that the other team converted before the first clue
creating a contradiction was read, there may be no protest because the
team that missed the question was necessarily not misled.
I want to address the idea that the protest resolutions at this tournament were somehow unfair or ignored the rules. In all instances, we stove to apply the rules as written evenly and fairly to the protested questions. Furthermore, I want to make it clear that we considered only a fair application of the rules to the situation when we were resolving protests. We spent a lot of time reviewing the rules and going over protest situations before the tournament to ensure that we were prepared. When a protest came to us, we consulted the rules directly and applied them as written. When something was confusing or a situation presented a wrinkle we didn't anticipate, we sought out help from outside experts. When we needed to look up facts or consult experts in fields, we did. Any team that brought concerns to us about how we handled their protest received from us a detailed explanation, in some cases requiring us to confirm that our interpretation of the rules was correct. I stand by every protest resolution that we made.jonpin wrote:Hey look, it's another team eliminated from championship contention on a protest resolution in direct contradiction to the rules!
In short, no, this makes absolutely no sense at all. How does it not uniquely describe a different answer? The current Secretary of Health and Human Services, which is what was buzzed on, clearly leads to Kansas. Because the reader got another word out after the buzz should not change this. Even if in some convoluted way the stated rule does not apply (which I think it does anyways), I would think sense would prevail. The clue very clearly did not apply to the answer line and obviously led to the buzz and subsequent miss of the question. I don't see how anyone can try to justify this being the right decision.theMoMA wrote:The text of the tossup read:Joe N wrote:The Tommy Thompson one did end up costing us our wild-card game: in a history tossup on politicians from Wisconsin, there was a clue along the lines of "The current Secretary of Health and Human Services is from this state..." Buzz "...Tommy" My answer was along the lines of "What? That doesn't make any sense, but the current Secretary of Health and Human Services is from Kansas." My protest was denied because "All the previous clues pointed to Wisconsin and the moderator already said Tommy." I think according to the rules it should have been thrown out and replayed, which would have altered the outcome of the match.
The relevant rules states:In this state’s largest city, Victor Berger led the so-called Sewer Socialism movement. In the late 19th century, it was the site of the deadliest fire in American history, the Peshtigo Fire. One politician from this state joined George Norris in filibustering the Armed Ship Bill, much to the chagrin of Woodrow Wilson, and later ran for president with Burton K. Wheeler as his running mate. The current US Secretary of Health and Human Services, (*) Tommy Thompson, was once its governor. Another governor worked with university professors to push for legislation like primary elections and the country’s first state income tax, a policy which became known as this state’s namesake “idea”. For 10 points, name this Midwestern state home to Progressive leader Robert LaFollette, Sr.
ANSWER: Wisconsin
We determined that the clue, though factually incorrect, did not "uniquely describe different answers" and the clue buzzed on could not be said to "definitely...not refer to the same answer as the other clues." This is for two reasons: the player (as we understood it) had heard the word "Tommy," but more importantly, the phrase that led to the buzz did not refer back to the answer. If the question had said "The current secretary of HHS, Kate Sebelius, once lived in this state before moving to Kansas," with the player buzzing after "Kate" with "Kansas," the issue would be essentially the same, and the protest would also be denied (assuming for this example that Sebelius actually did live in Wisconsin at one point). The rule in question confines its application to two very narrow circumstances, neither of which is present here. Both the actual question and the hypothetical above are misleading questions, but there is no recourse in any of the protest rules for this situation.Two or more clues within the question uniquely describe different
answers or one clue definitely does not refer to the same answer as the
other clues—hence, there is no correct answer to the question. This
protest may be lodged by either team at the end of a question
converted by neither team or by the team that did not get the question
after a question is converted by one team. However, if this situation
arises on a tossup that the other team converted before the first clue
creating a contradiction was read, there may be no protest because the
team that missed the question was necessarily not misled.
I hope this all makes sense. If the full logic of the protest resolution did not get back to the teams in question, I apologize.
I don't agree with this reasoning at all. The clue that Joe buzzed on was deep enough into the question that it was obvious the answer was a state. Thus when one hears "the current secretary of HHS," one can assume it is asking for that person's home state (and, in fact, it was asking for the home state of the person the question writer mistakenly thought was current HHS secretary). An equivalent situation would be if it were a geography question on Oregon and it contained a clue saying "The site of Yosemite National Park (*) is this state's neighbor, California." Who wouldn't buzz after Yosemite and say California if they knew where Yosemite was? The fact that he heard "Tommy" is irrelevant because he had also already heard the clue describing a different answer. By any reasonable interpretation of the rules, the question should've been thrown out and replaced.theMoMA wrote:The text of the tossup read:Joe N wrote:The Tommy Thompson one did end up costing us our wild-card game: in a history tossup on politicians from Wisconsin, there was a clue along the lines of "The current Secretary of Health and Human Services is from this state..." Buzz "...Tommy" My answer was along the lines of "What? That doesn't make any sense, but the current Secretary of Health and Human Services is from Kansas." My protest was denied because "All the previous clues pointed to Wisconsin and the moderator already said Tommy." I think according to the rules it should have been thrown out and replayed, which would have altered the outcome of the match.
The relevant rules states:In this state’s largest city, Victor Berger led the so-called Sewer Socialism movement. In the late 19th century, it was the site of the deadliest fire in American history, the Peshtigo Fire. One politician from this state joined George Norris in filibustering the Armed Ship Bill, much to the chagrin of Woodrow Wilson, and later ran for president with Burton K. Wheeler as his running mate. The current US Secretary of Health and Human Services, (*) Tommy Thompson, was once its governor. Another governor worked with university professors to push for legislation like primary elections and the country’s first state income tax, a policy which became known as this state’s namesake “idea”. For 10 points, name this Midwestern state home to Progressive leader Robert LaFollette, Sr.
ANSWER: Wisconsin
We determined that the clue, though factually incorrect, did not "uniquely describe different answers" and the clue buzzed on could not be said to "definitely...not refer to the same answer as the other clues." This is for two reasons: the player (as we understood it) had heard the word "Tommy," but more importantly, the phrase that led to the buzz did not refer back to the answer. If the question had said "The current secretary of HHS, Kate Sebelius, once lived in this state before moving to Kansas," with the player buzzing after "Kate" with "Kansas," the issue would be essentially the same, and the protest would also be denied (assuming for this example that Sebelius actually did live in Wisconsin at one point). The rule in question confines its application to two very narrow circumstances, neither of which is present here. Both the actual question and the hypothetical above are misleading questions, but there is no recourse in any of the protest rules for this situation.Two or more clues within the question uniquely describe different
answers or one clue definitely does not refer to the same answer as the
other clues—hence, there is no correct answer to the question. This
protest may be lodged by either team at the end of a question
converted by neither team or by the team that did not get the question
after a question is converted by one team. However, if this situation
arises on a tossup that the other team converted before the first clue
creating a contradiction was read, there may be no protest because the
team that missed the question was necessarily not misled.
I hope this all makes sense. If the full logic of the protest resolution did not get back to the teams in question, I apologize.