The Quizbowl Resource Center

A fairly common trend in quizbowl is to tag some of the harder questions for the finals of a tournament. If I recall, this got some discussion at one point and there was some pushback against doing it, but I don't remember if that went anywhere. This isn't related to any specific tournament, but I'll throw out my thoughts and see what people think:

-I think as a whole, we should try to avoid tagging harder questions for the finals. At an extreme level, you end up in a very hard finals packet that is borderline unplayable. I realize that arguably the two best teams in the tournament should be playing the finals, but I think it's fair that the packet they confront should be similar to what they've encountered so far in the tournament.

-i should note that I generally dislike the concept of questions getting harder in the playoffs. I realize that this might be a good idea in things like MSNCT (but that's more to do about length), but as a player, I've never been a fan.

-At its worst and most extreme, it becomes a crutch. Rather than try to figure out if the question can be made easier, it gets shifted to the finals.

-I do think it's okay to note 1-2 of the harder questions in the tournament and put that in the finals.

Cheynem wrote: ↑Mon Mar 18, 2019 11:52 am A fairly common trend in quizbowl is to tag some of the harder questions for the finals of a tournament. If I recall, this got some discussion at one point and there was some pushback against doing it, but I don't remember if that went anywhere. This isn't related to any specific tournament, but I'll throw out my thoughts and see what people think:

-I think as a whole, we should try to avoid tagging harder questions for the finals. At an extreme level, you end up in a very hard finals packet that is borderline unplayable. I realize that arguably the two best teams in the tournament should be playing the finals, but I think it's fair that the packet they confront should be similar to what they've encountered so far in the tournament.

-i should note that I generally dislike the concept of questions getting harder in the playoffs. I realize that this might be a good idea in things like MSNCT (but that's more to do about length), but as a player, I've never been a fan.

-At its worst and most extreme, it becomes a crutch. Rather than try to figure out if the question can be made easier, it gets shifted to the finals.

-I do think it's okay to note 1-2 of the harder questions in the tournament and put that in the finals.

Heartily co-sign. When aggregated, this effect can raise a set by half a difficulty level or more. That's a bit unfair to players who signed up to play a tournament of a certain difficulty, especially if they're not informed of the rise ahead of time. The existence of finals packets shouldn't be an excuse to derelict your difficulty-control duties, editors.

I'm against making the questions harder in the finals, but I'm perfectly fine with shifting some of the harder answerlines in the set to later packets, and we should make sure we're distinguishing between the two when we talk about it.

heterodyne wrote: ↑Mon Mar 18, 2019 3:25 pm I'm against making the questions harder in the finals, but I'm perfectly fine with shifting some of the harder answerlines in the set to later packets, and we should make sure we're distinguishing between the two when we talk about it.

Could you clarify this distinction? I think I understand what you mean but don't want to assume.

wcheng wrote: ↑Mon Mar 18, 2019 4:12 pm

heterodyne wrote: ↑Mon Mar 18, 2019 3:25 pm I'm against making the questions harder in the finals, but I'm perfectly fine with shifting some of the harder answerlines in the set to later packets, and we should make sure we're distinguishing between the two when we talk about it.

Could you clarify this distinction? I think I understand what you mean but don't want to assume.

To piggyback on Weijia's request, could you also provide a justification for shifting harder answerlines in the set to later packets? It seems like that would also lead to an inconsistent difficulty experience for teams.

I believe that the usual justification for this practice is that the better teams play each other in the finals (and playoffs) and thus harder answerlines are less likely to go dead in matches that will decide final placement. (With the perverse side "benefit" that more tossups go dead in lower bracket games, resulting in shorter matches.)

I actually buy into this, with the caveat that, if the answerlines are too hard, then you've created dead tossups in the most consequential matches of your tournament! That's obviously bad.

So, I think the idea is: if you have some easier answerlines and some harder ones, it's OK or even possibly desirable to put many of the harder ones in later rounds, but do not select harder answerlines explicitly for those rounds, since you're likely to overshoot and create dead tossups in important games.

My point is something like this: it's clearly possible to have equally hard tossups on answerlines that, in themselves, have rather different "familiarity rates." If, as Jacob says, the questions are written with an eye to keeping difficulty consistent and then, after the fact, you decide to move some of the "harder answerlines" into the finals, that should not create a difficulty spike, since you have written the tossups at a difficulty consistent with the other tossups in the set.

In theory, it's OK to increase difficulty in the finals because the teams are better. In practice, I think it's almost always a bad idea and encourages set editors to write sadistically hard questions.

I've seen finals between good teams that turned into brutal endurance tests, and they didn't seem engaging or fun.

I think that having more difficult questions in later packets (i.e. "the playoffs") is a bad idea because many sites will have different schedules, resulting in different packets being in the playoffs. It makes absolutely no sense to have a more difficult packet in the sixth round of prelims if you have a six-round prelim format, let alone having different difficulty in the first and second halves of a 12 team round robin. This is justifiable for nationals where there is only one site, but nothing else.

As someone who recently helped co-edit a set, it's really easy to fall into the mindset of "this is too hard, oh I suppose we can just put it in the finals packets." At first, I was okay with this, but on some reflection and advice from John Lawrence, we elected to packetize all the questions (selectively) random -- and I think this is the right choice. It forces you to control difficulty a lot closer than to just throw everything "too hard" for your tournament into the final two packets. At any tournament, I think the finals should be about the same difficulty -- after all, it's the difficulty that got you into the finals. The argument that the "teams are better" is suspicious to me. You really have no way of knowing how good any team is at scaling up to more difficult answer lines.

I also think I agree with Alston that if you do have "harder answer lines" (which is what I'm presuming since harder questions on easier answer lines should be easy to edit down -- though this distinction could also be discussed), they shouldn't be in the first or second round. I'd argue this because people typically aren't at their highest level of play at the start of the day and while harder answer lines are still fine here or there, they'll definitely throw people off if placed in the first round. Its the same reason you typically don't match the best teams in the pool against each other in round 1. Again, there's inevitably going to be some harder questions in any tournament, so at least give players a chance to ease into the set first.

You also owe it to your writers and players of your set that this is the plan ahead of time. Otherwise you end up with *some* harder questions and bonuses (!!!!!) in your final while others are still just regular difficulty. The bonus thing strikes me as particularly important. If one team gets a recognizably harder bonus, that's definitely not okay.

Milhouse wrote: ↑Mon Mar 18, 2019 7:14 pm I think that having more difficult questions in later packets (i.e. "the playoffs") is a bad idea because many sites will have different schedules, resulting in different packets being in the playoffs. It makes absolutely no sense to have a more difficult packet in the sixth round of prelims if you have a six-round prelim format, let alone having different difficulty in the first and second halves of a 12 team round robin. This is justifiable for nationals where there is only one site, but nothing else.

The other problem is that, while the playoff see strong teams play each other, in modern formats they also see weaker teams play each other. A battle between two lower-bracket teams is tough enough (by definition), and if you put it on harder questions, it becomes a dreadful slog.

vinteuil wrote: ↑Mon Mar 18, 2019 4:42 pm I actually buy into this, with the caveat that, if the answerlines are too hard, then you've created dead tossups in the most consequential matches of your tournament! That's obviously bad.

I think this is an "underrated bad". Writers don't live in sufficient fear of creating this outcome, and players don't complain about it as loudly as they should. There absolutely have been quizbowl tournaments over the years, even important ones, that were decided because questions went dead even in the presence of some of the game's top specialists in that subject.