Protest Resolution: Best Practices?

[matt979]

Hi Everyone,

Prequel question (inserted on edit, inspired by the longer ICT thread):
What elements of protest resolution would you say absolutely must be covered in the pre-tournament moderator meeting? Here we're trying to achieve the right balance between good preparation and information overload; we want to be thorough but we don't want teams waiting around half an hour before Round 1 because staff isn't out of their meeting yet.

There's an increasing sense among people I've talked to that protests are just denied as a matter of keeping things running on time rather than on merit.

Jeff Hoppes relayed this statement by e-mail (I didn't immediately catch whether it was a post or a chat) to ICT planners, and a couple of us immediately (to our consternation) realized that, although this statement is false, one could easily get that impression from outside.

To what extent you all agree with the statements below? Assume we're talking about ICT, though most of this also applies to HSNCT. None of these have been vetted by other NAQTers, and please note that a couple of these are infeasible (or just very bad ideas IMO), but all are natural brainstorms. I'm interested in which of these get resounding agreement (or even resounding condemnation of doing things any other way) as opposed to benign apathy (EDIT) or resounding disagreement.

1. Before any national championship, NAQT should form (from some subset of the members/editors at the tournament) a "protest committee"

1a. ...and tell teams in advance who those people are

2. Any time a protest would determine who wins a game (after any conditional or hypothetical play is carried out, of course), someone in the control room should hear from the protesting team a concise explanation of its exact point(s) of contention.

2a. ...and hear from the other team a concise rebuttal, if the other team so wishes.

3. Every protest resolution should involve a reasonably senior NAQT member concisely explaining the ruling, and in brief the procedure of coming to that conclusion, to both teams.

4. There should be a dedicated resource to carrying out all necessary factual research; that person would have no other duties (i.e. not a reader, scorekeeper, stat person, or control room head)

4a. That person should be on-site.

5. Sometimes it's just not possible to make a quick determination. Wherever the schedule doesn't depend on the outcome of that game (ICT-specific example: as of round 9 the opponents are set through round 14, but a round 7 game would likely affect the necessity of round 8 tiebreaks and the round 9 onward pairings), NAQT should tell both teams that no immediate resolution is available and give them an ETA for the resolution.

5a. Sometimes (i.e. more frequently than current practice), if it's just not possible to make a quick determination, a tossup or bonus should simply be replaced (even if it's reasonable to believe that some amount of research would lead to a resolution in whose correctness we're fairly confident).

6. Correct protest resolution is so important that at an ICT, somebody of the caliber of R. or Andrew should be in a control room instead of reading.

What other statements should be here?

Thanks,
Matt

P.S. If you find it relevant: This year's ICT room layout and scoresheet/packet procedures left me with a de facto control room, the implications of which I didn't ponder in depth beforehand. (Every protest that moderators brought to my attention, I called upstairs.) Otherwise I'm heavily involved in tournament planning but not nearly as well qualified to address factual issues as R., Andrew, Jeff, Matt K., et al. For HSNCT I've typically run a Saturday control room (but relied heavily on R. for protest handling) and handled the Sunday playoff control room. In past years, HSNCT playoff protests have basically paused the tournament while we do everything in our power to get the right resolution quickly.

[Matt Weiner]

Hi Everyone,

There's an increasing sense among people I've talked to that protests are just denied as a matter of keeping things running on time rather than on merit.

Jeff Hoppes relayed this statement by e-mail (I didn't immediately catch whether it was a post or a chat) to ICT planners, and a couple of us immediately (to our consternation) realized that, although this statement is false, one could easily get that impression from outside.

It sure does seem that somewhere around 100% of people who have been involved in protests in the last four ICTs got that impression. At the end of this year's ICT we were told to immediately vacate the rooms because NAQT's reservations had run out. It was also my definite impression in 2006 that no one had any time to research protests (all 3 of the people who talked to me about the protest at various points were also reading games and were in a hurry to get back to doing so) and that NAQT values keeping to its schedule more than nearly anything else, fairness of game resolutions included, at the ICT.

To what extent you all agree with the statements below?

...

1. Before any national championship, NAQT should form (from some subset of the members/editors at the tournament) a "protest committee"

1a. ...and tell teams in advance who those people are

2. Any time a protest would determine who wins a game (after any conditional or hypothetical play is carried out, of course), someone in the control room should hear from the protesting team a concise explanation of its exact point(s) of contention.

2a. ...and hear from the other team a concise rebuttal, if the other team so wishes.

This all seems sensible.

3. Every protest resolution should involve a reasonably senior NAQT member concisely explaining the ruling, and in brief the procedure of coming to that conclusion, to both teams.

I don't care what rank someone holds in NAQT; if I have a protest on a history question, I'd want to hear from someone who understands how quizbowl works and knows something about the historical topic, who NAQT has designated to handle history-related protests for those reasons and not because of his position within the organization.

4. There should be a dedicated resource to carrying out all necessary factual research; that person would have no other duties (i.e. not a reader, scorekeeper, stat person, or control room head)

4a. That person should be on-site.

Probably not needed, as there will rarely be more than 2 or 3 protests at a tournament and oftentimes even a large national championship will have none. If you do have the luxury of such redundant staff, that's fine, but in large events I've been involved in, there is a three-person protest committee which is convened as needed, and a stat person or scorekeeper will just take that round off to cover for their moderating duties if necessary.

5. Sometimes it's just not possible to make a quick determination. Wherever the schedule doesn't depend on the outcome of that game (ICT-specific example: as of round 9 the opponents are set through round 14, but a round 7 game would likely affect the necessity of round 8 tiebreaks and the round 9 onward pairings), NAQT should tell both teams that no immediate resolution is available and give them an ETA for the resolution.

This is also standard at many other tournaments and seems perfectly fine.

5a. Sometimes (i.e. more frequently than current practice), if it's just not possible to make a quick determination, a tossup or bonus should simply be replaced (even if it's reasonable to believe that some amount of research would lead to a resolution in whose correctness we're fairly confident).

No, this is Chip Beallian in its insidiousness. All it does is penalize people for protesting. If a team protests that their -5 should have been a 15 and loses the game by 10, making them play a replacement question just means that they now have a 50% chance of losing the game even though they already have, in actuality, won. The use of replacement questions in that scenario should absolutely never be considered.

6. Correct protest resolution is so important that at an ICT, somebody of the caliber of R. or Andrew should be in a control room instead of reading.

Again, I think it is probably a better overall use of resources to hold people from reading just 1 round if they are needed to research a protest, and no tournament is really bursting at the seams with excellent moderators, so that's probably the way to go.

What other statements should be here?

Teams should have the right to use an Internet-enabled computer in a control room to bring relevant materials to the attention of the protest committee within a reasonable timeframe.

[fleurdelivre]

I don't care what rank someone holds in NAQT; if I have a protest on a history question, I'd want to hear from someone who understands how quizbowl works and knows something about the historical topic, who NAQT has designated to handle history-related protests for those reasons and not because of his position within the organization.

With regards to this, wouldn't it make sense to have, instead of a 'protest committee' per se, a listing of "category protest handlers" akin to the category editors (and, seeing how many of the NAQT writing/editing staff were on hand, probably made up of the same people?) If they're pre-determined, finding the appropriate subject expert doesn't take very long (see note: having everyone's cell phone #s), and teams can know beforehand that their protests will be heard by someone trusted (or can complain in advance, instead of as these things are being judged).

[Frater Taciturnus]

also, it works well for if the protest committee doesn't know which players are involved so no accusations of bias are possible. I definitely agree with a protest committee, and would rather have protests handled by actual specialists rather than someone appointed by rank (ie I want Jeff Hoppes deciding my geography protest, not Dwight Kidder.).

[Susan]

What elements of protest resolution would you say absolutely must be covered in the pre-tournament moderator meeting? Here we're trying to achieve the right balance between good preparation and information overload; we want to be thorough but we don't want teams waiting around half an hour before Round 1 because staff isn't out of their meeting yet.

I think it would be a good idea to ensure that the tournament staff know what they're doing before they even get to the tournament. (I mean, you're flying these people out and housing them on your dime; I think you're entitled to ask them to do some preparation, and my impression has been, from some of the questions people ask at the HSNCT staff meeting, that some people don't bother looking over the rules before coming.) Would it be possible to have people, say, pass a quiz on the rules before being accepted as staff? If there are other ways to assess whether people actually have read the rules, that would be fine, too.

Obviously I'm not suggesting that you replace the moderator meeting with what I suggest above, but this should help make that meeting more of a refresher course.

[Important Bird Area]

3. Every protest resolution should involve a reasonably senior NAQT member concisely explaining the ruling, and in brief the procedure of coming to that conclusion, to both teams.

I don't care what rank someone holds in NAQT; if I have a protest on a history question, I'd want to hear from someone who understands how quizbowl works and knows something about the historical topic, who NAQT has designated to handle history-related protests for those reasons and not because of his position within the organization.

4. There should be a dedicated resource to carrying out all necessary factual research; that person would have no other duties (i.e. not a reader, scorekeeper, stat person, or control room head)

4a. That person should be on-site.

Probably not needed, as there will rarely be more than 2 or 3 protests at a tournament and oftentimes even a large national championship will have none. If you do have the luxury of such redundant staff, that's fine, but in large events I've been involved in, there is a three-person protest committee which is convened as needed, and a stat person or scorekeeper will just take that round off to cover for their moderating duties if necessary.

5a. Sometimes (i.e. more frequently than current practice), if it's just not possible to make a quick determination, a tossup or bonus should simply be replaced (even if it's reasonable to believe that some amount of research would lead to a resolution in whose correctness we're fairly confident).

No, this is Chip Beallian in its insidiousness. All it does is penalize people for protesting. If a team protests that their -5 should have been a 15 and loses the game by 10, making them play a replacement question just means that they now have a 50% chance of losing the game even though they already have, in actuality, won. The use of replacement questions in that scenario should absolutely never be considered.

On 3, the person explaining the protest to the teams is assumed to be the head of the protest committee, not necessarily whoever did the factual research. I think it's reasonable that the former hold some rank in NAQT for an NAQT tournament; and of course it's obviously good that any protest committee would contain subject-matter experts in (at least) the big three subject areas.

On 4. I think there should be a dedicated protest staffer, at least for HSNCT, which is unusually large.

On 5, I agree with Matt W. that if the protest can be resolved with reasonable amounts of research, then replacement questions are a bad idea. They should be reserved for the (in my experience extremely rare) situations in which the amount of research required would exceed the available time before the completion of the tournament. (and other similar fair things such as instant-resolution of "protest-to-tie" scenarios)

[AlphaQuizBowler]

2. Any time a protest would determine who wins a game (after any conditional or hypothetical play is carried out, of course), someone in the control room should hear from the protesting team a concise explanation of its exact point(s) of contention.

2a. ...and hear from the other team a concise rebuttal, if the other team so wishes.

I do like allowing teams to state their protest rather than having moderators relay the information.

5a. Sometimes (i.e. more frequently than current practice), if it's just not possible to make a quick determination, a tossup or bonus should simply be replaced (even if it's reasonable to believe that some amount of research would lead to a resolution in whose correctness we're fairly confident).

This is my least favorite form of protest resolution, as it basically says, "We can't or don't want to resolve your protest correctly, so we'll just pretend like that didn't happen."

[matt979]

With regards to this, wouldn't it make sense to have, instead of a 'protest committee' per se, a listing of "category protest handlers" akin to the category editors

Makes sense to me. On factual disputes this is actually how it worked (for example, the decisive factor in resolving Minnesota's biology protest was Matt Keller); I'd be interested in learning to what extent teams would value hearing this formally stated.

also, it works well for if the protest committee doesn't know which players are involved

This is true, though to some extent it competes with the notion that the protesting team should get to make its case directly. Using Pompadour as an example, Chris Borglum saw Maryland lodge the protest; I had no idea what schools were involved (but on the flip side did not hear the protest argued directly from the source). Someone like me should probably know the team and hear the direct argument; someone like Andrew/Jeff/Matt should probably hear the second-hand argument but not know the team. Not sure where R. slots in.

I think it would be a good idea to ensure that the tournament staff know what they're doing before they even get to the tournament.

We did e-mail a Moderator Guide before the tournament, and handed out hard copy of that same guide upon check-in. Would it be excessive of me to copy and paste that here? (Assuming I find the version that actually went out as opposed to previous drafts batted around internally.)

[SnookerUSF]

Would it be possible to have people, say, pass a quiz on the rules before being accepted as staff? If there are other ways to assess whether people actually have read the rules, that would be fine, too.

Other ways...you say? Clearly you mean a deadly serious and simultaneously hilarious meta-round on the NAQT rules held during the moderator's meeting.

The fifth part of this section suggests that a previously stated rule does not apply, when a *certain feature is not visible to both teams. In section 4.4, the word "begun" has recently been added, and a further stipulation regarding "Preliminary(*)Statements" such as "Here's the next tossup" states that such statements do not constitute part of the tossup-bonus cycle near the end of a half. For 10 points -- name this section of the NAQT rules that defines the role and function of the clock during a match.
answer: Time [accept Section F]

*notice and appreciate NA-cutie appropriate "coy giveaway" in the second line

[Auks Ran Ova]

Other ways...you say? Clearly you mean a deadly serious and simultaneously hilarious meta-round on the NAQT rules held during the moderator's meeting.

The fifth part of this section suggests that a previously stated rule does not apply, when a *certain feature is not visible to both teams. In section 4.4, the word "begun" has recently been added, and a further stipulation regarding "Preliminary(*)Statements" such as "Here's the next tossup" states that such statements do not constitute part of the tossup-bonus cycle near the end of a half. For 10 points -- name this section of the NAQT rules that defines the role and function of the clock during a match.
answer: Time [accept Section F]

*notice and appreciate NA-cutie appropriate "coy giveaway" in the second line

<3

[cvdwightw]

1. Before any national championship, NAQT should form (from some subset of the members/editors at the tournament) a "protest committee"

1a. ...and tell teams in advance who those people are

2. Any time a protest would determine who wins a game (after any conditional or hypothetical play is carried out, of course), someone in the control room should hear from the protesting team a concise explanation of its exact point(s) of contention.

2a. ...and hear from the other team a concise rebuttal, if the other team so wishes.

3. Every protest resolution should involve a reasonably senior NAQT member concisely explaining the ruling, and in brief the procedure of coming to that conclusion, to both teams.

As people have said, having a designated protest committee is something that needs to happen. Ad hoc protest committees are bad. My recommendation is that a three-man protest committee be formed of:
1. R., or if for whatever reason he's unavailable, some other higher-up. This person is responsible for explaining the final protest decision to each team and the reason for that decision.
2. Someone in the control room. This person serves as the "point man" for the protests; he or she listens to the moderator's explanation of the protest, the protesting team's argument, and the opposing team's argument (if there is one). This person then conveys the relevant information about the protest to the other two members of the protest committee, without mentioning names (e.g., "Team A is protesting that 'Eumenides' should be acceptable for 'Furies'; Team B does not know enough to either agree or disagree with Team A").
3. A "category expert." This person should be an NAQT member/editor/writer, and someone whose knowledge of both the NAQT rules and the category involved in the protest is respected by both teams. For HSNCT, SCT, and ICT, it would make sense to have this person either on site or on call. This person is responsible for researching and providing factual information regarding the protest topic, arriving at an initial ruling on the protest, fielding questions about the initial ruling from the other two members of the committee, and arriving at a final ruling once the other two members of the committee are out of pertinent questions.

In general, these positions should belong to three distinct people. Also, the list of "category experts" should be listed publicly (or otherwise conveyed to teams) for each tournament so we do not have the Beallian "we just called a so-called expert who has absolutely no relevance to the protested topic, and we ruled this way" issue.

4. There should be a dedicated resource to carrying out all necessary factual research; that person would have no other duties (i.e. not a reader, scorekeeper, stat person, or control room head)

4a. That person should be on-site.

In general, there are nowhere near enough protests to justify such a position. Having a group of "category experts" will take many of the time-related problems out of so-called "Internet research," as presumably these people will know exactly what they're looking for or who would be a more appropriate person to call, and don't need to take 5 minutes to interpret all the scientific jargon on the website. I would like to think that there are enough competent moderators at any NAQT tournament that two of them can get by without a scorekeeper for one round until the protest is resolved, or that the control room can function with three fewer people for a round.

5. Sometimes it's just not possible to make a quick determination. Wherever the schedule doesn't depend on the outcome of that game (ICT-specific example: as of round 9 the opponents are set through round 14, but a round 7 game would likely affect the necessity of round 8 tiebreaks and the round 9 onward pairings), NAQT should tell both teams that no immediate resolution is available and give them an ETA for the resolution.

This seems fair. However, this opens up the possibility of a team finding its own facts in between the arguments and the resolution, and then either counter-protesting or being really upset with the resolution. You need to make clear to the teams up to what point you are willing to accept factual information (e.g. citation from a textbook, reliable internet website, etc.) from those teams in support of their protest.

5a. Sometimes (i.e. more frequently than current practice), if it's just not possible to make a quick determination, a tossup or bonus should simply be replaced (even if it's reasonable to believe that some amount of research would lead to a resolution in whose correctness we're fairly confident).

A tossup or bonus should only be replaced due to moderator error: e.g., if a team protests that it should have been prompted, and the moderator finds that he should have prompted, but didn't, then the question needs to be tossed; or if a moderator accepts a wildly wrong answer for no apparent reason, and the opposing team protests, then that question needs to be tossed. In any case, tossing a question should not penalize teams by making the maximum number of tossups heard 25.

Tossing a question for the reason that "we don't want to bother resolving this protest" is even worse than making a quick, haphazard ruling for the reason that "we don't want to bother resolving this protest."

6. Correct protest resolution is so important that at an ICT, somebody of the caliber of R. or Andrew should be in a control room instead of reading.

As I've indicated, someone like R. or Andrew (or both) should be part of the protest committee, but wasting a quality moderator for the sake of protest resolution disadvantages the 26 teams that would be in their room at ICT (because they end up with a poorer moderator) for the sake of less than 10 teams involved in meaningful protests. Taking a moderator out for a round to deal with the protest disadvantages the 2 teams in the room, but advantages the 2 teams involved in the protest, so there's no net disadvantage.

[jonpin]

Would it be possible to have people, say, pass a quiz on the rules before being accepted as staff? If there are other ways to assess whether people actually have read the rules, that would be fine, too.

Other ways...you say? Clearly you mean a deadly serious and simultaneously hilarious meta-round on the NAQT rules held during the moderator's meeting.

The fifth part of this section suggests that a previously stated rule does not apply, when a *certain feature is not visible to both teams. In section 4.4, the word "begun" has recently been added, and a further stipulation regarding "Preliminary(*)Statements" such as "Here's the next tossup" states that such statements do not constitute part of the tossup-bonus cycle near the end of a half. For 10 points -- name this section of the NAQT rules that defines the role and function of the clock during a match.
answer: Time [accept Section F]

*notice and appreciate NA-cutie appropriate "coy giveaway" in the second line

On that note, is it just me or is section 5 (if the clock is not visible to both teams) entirely redundant now that any cycle in progress when time expires is finished no matter what?
The lead-in to section 6, on the other hand, is vital: "The team with more points at the end of the game wins."

[theMoMA]

One of the more infuriating things about our protest was that we were told that the protest committee was "unanimous" in its decision. First, we did not know who the protest committee was. Second, we had no idea on what merits our protest was decided. I would rather have heard that Matt Keller singlehandedly quashed our protest, since I know he has the technical knowledge necessary to rule on the issue, than have R. tell me that a nebulous group that may or may not include any chemists was "unanimous."

Also, I think that the issue of our protest opens up major issues about what constitutes a promptable answer. It's my opinion that reasonable categories one step removed from the answer given should be generally promptable (i.e. assassinating the monarch of Britain for assassinating Charles I, Songs of Innocence and Experience for Songs of Experience, or fullerenes for nanotubes). Question writers should stop trying to exclude broader answers with technicalities early in the question (if you say "the first attempt to do this..." on an "attempting to assassinate Charles I" tossup, and you deny the protest based on that not being the "first" time someone tried to assassinate a British monarch, you're screwing people who are legitimately knowledgeable but confused about the specificity of the answer).

Instead, writers should attempt to be technically precise, but also be mindful of potential right, but more general answers that should be promptable. In my mind, what constitutes promptability is whether the answer is both a) only one step removed from the answer in question (as in, nanotubes are a direct subclass of fullerenes), and b) it's reasonable to assume there could be a tossup on the more general answer (buzzing in on an "alkanes" tossup with "hydrocarbons" is not analogous to the situations I've outlined, because it's not reasonable to expect a tossup on hydrocarbons).

None of my argument is based on how something necessarily is; I'm simply arguing how it ought to be. But it seems necessary to have more generous prompts for right answers when questions cannot easily delineate the degree of specificity that they are looking for. For example, there was really no way to discern whether the tossup on "nanotubes" was going to have clues on nanotubes exclusively, or if it would have C-60 clues later, based on the early clues.

There are obviously always sticky cases, like the Dartmouth protest about "immune cells" for "macrophages." For my part, I would have included a prompt for "immune cells" in an answer line for macrophages, had I written that tossup, because the degree of specificity required was not readily apparent, and I tend to err on the side of knowledge rather than gamesmanship. But once the protest was lodged, I can understand being wary of granting a new tossup to be read when it is seemingly dubious that the player could have come up with the right answer if he were prompted (since there are so many subclasses of immune cells, and really just two subclasses of fullerenes, I hardly think this applies to our protest, but that's immaterial).

[vandyhawk]

The issue on the Minnesota protest of not being prompted on "fullerenes" answered for the "nanotubes" tossup is an interesting one I think. I don't think anyone has disputed that the two clues in the first sentence are unique to nanotubes and don't apply to other fullerenes. Since the question said "these molecules" in that first sentence and gave two clues unique to nanotubes, I (and Samer, R., and Emily) felt like there wasn't a need to prompt on more general answers at this point. If the question were going for fullerenes, the wording of the question would have been quite poor, and should have said "one type of these molecules..." or something similar since the clues don't apply to buckyballs and other random fullerenes. Andrew H's post (and my IM conversation with him that evening after prelims) raises a lot of questions about what should generally be promptable, since there really is no clear rule on the matter.

As for the handling of the nanotubes protest, I think this one was done much better than the Pampadour one. Emily emailed both Samer and me to get our inputs (w/out telling us which teams were involved), and also did research herself. When I didn't respond quickly, I probably should have been called though, since it wound up with me only getting involved right at the end. R. also did his own research I think after the prelims before making his ruling, with a late word of support from me.

In general, a more formal procedure should be adopted though I think. An important part should be making sure that the teams involved get a chance to make their cases to experienced people, rather than relying on the moderator to explain the protest (though the mod should also be consulted to make sure the teams don't misrepresent anything). Presumably R. should have final word, and provided he's there and it isn't science, Andrew Y should be involved as well. The subject editor of the question should also be consulted.

I guess I'll finish up with a word on the macrophages protest. I feel stronger about this one not deserving a prompt on "immune cells." There had already been several clues unique to macrophages, with the most non-prompt-deserving part being the clue about vitamin D-binding protein being a namesake activator; it's called macrophage activating factor, so I think that pretty well rules out any other answer. As I think someone noted, "immune cells" is a very broad class too, and I think it just doesn't make sense as an answer to give at that point.

[matt979]

As people have said, having a designated protest committee is something that needs to happen. Ad hoc protest committees are bad. My recommendation is that a three-man protest committee be formed

This seems like a reasonable basic framework. If I'm reading it correctly, #1 would almost always be R. (sometimes Chad Kubicek for ICT Division II if R. wasn't immediately available). #2 would be whoever ran the control room into which that game room fed (so most frequently Joel, Emily, or myself), and the obvious #3 would be someone like Andrew/Jeff/Matt K., basically an NAQT subject editor.

How would this be tweaked (if at all) for protests that turn on correct rules application as opposed to factual disputes?

enough competent moderators at any NAQT tournament that two of them can get by without a scorekeeper for one round until the protest is resolved, or that the control room can function with three fewer people for a round.

This year the (upstairs) control room had exactly three people. I suppose we're at 4.5 if you count Leon from Alabama (based in my room; did some of the scoresheet intake and most of the basement-to-11 shuttling: did I mention just how far beyond the call of duty he went?) and split me in half.

So while I agree that sometimes we should pull people off to handle protests optimally, it's not as straightforward as it looks. For example, say Andrew was about to read Harvard-Chicago but a protest resolution required his time that round. We're not going to stick just anyone into that game. Off the top of my head I'd probably bump up Matt W. that round, and bump someone else up into Matt's room, and so on. This is the sort of thing that NAQT ought to pull off without a hitch, but obviously would need to be anticipated in advance to avoid a fustercluck.

This seems fair. However, this opens up the possibility of a team finding its own facts in between the arguments and the resolution, and then either counter-protesting or being really upset with the resolution. You need to make clear to the teams up to what point you are willing to accept factual information (e.g. citation from a textbook, reliable internet website, etc.) from those teams in support of their protest.

That makes sense. While we're here, earlier in the thread someone recommended that NAQT always make the control room internet connection available to the protesting team. I have practical misgivings about being able to promise this, but in a world where so many phones are Internet-capable, I generally approve of the idea that teams lodging a factual protest marshal their evidence.

[theMoMA]

I think a lot of Matt's logic is based on people being able to hear and clearly comprehend clues as if they were looking at the paper. That's simply not the case, especially in timed games with short tossups. While it may not be the correct protest resolution to allow a tossup to be replayed in many cases, there is absolutely nothing wrong with putting a generous prompt in the answer line of a question. That way, all possible confusion is avoided during the game situation and not in the TD room.

[jonpin]

Just for reference, how does one fix a "should've prompted, but rejected" protest? Let's say I answer "Roosevelt" and the moderator rejects it. The other team eventually gets it right with "Franklin Roosevelt" and I protest. Once the protest that I should've been prompted is accepted, what's the resolution? I don't think you could automatically give me the points, because it's possible that upon prompting I would've said Teddy. Does the whole tossup get thrown out and replaced?

[AKKOLADE]

I don't see why one wouldn't take as much time as is necessary to determine the correct decision in the case of a protest. If the teams have to go back and play a few more questions later, that's what it takes.

[cvdwightw]

Just for reference, how does one fix a "should've prompted, but rejected" protest? Let's say I answer "Roosevelt" and the moderator rejects it. The other team eventually gets it right with "Franklin Roosevelt" and I protest. Once the protest that I should've been prompted is accepted, what's the resolution? I don't think you could automatically give me the points, because it's possible that upon prompting I would've said Teddy. Does the whole tossup get thrown out and replaced?

It seems to me that the proper course is to remove any neg occurred as a result of the question and the points gained by the opponent, and play a replacement tossup answerable by both teams, analogous to the situation where the moderator inadvertently ruled the answer. Remember, it is also possible that the opposing team is thinking, "This is Franklin Roosevelt, but if it were actually Franklin Roosevelt, then the moderator would have prompted, so I'm going to guess it's Harry Truman or something," so in a somewhat bizarre situation, a team that finds themselves down by <45 at the end of the game could counter-protest that the lack of a prompt on the other team's buzz meant that the moderator gave conflicting information leading to a lack of a response (by not prompting, the moderator implies that that the answer is not a Roosevelt, which is clearly wrong; I'm about 90% sure that this protest would be denied and 100% sure that it would be denied on most situations not involving people's names, but I can see where it might happen). However, I can't find any documentation about this in the NAQT Rules, and HSAPQ's version of the ACF Rules (which I'm assuming is more-or-less equivalent to the ACF Rules) does not even mention insufficient prompting as being protestable.

[jonpin]

However, I can't find any documentation about this in the NAQT Rules, and HSAPQ's version of the ACF Rules (which I'm assuming is more-or-less equivalent to the ACF Rules) does not even mention insufficient prompting as being protestable.

Which is why I was asking. To be sure, HSAPQ/ACF rules do state:

5. Nothing else is protestable, including but not limited to: [...] the failure of the question to require a prompt on an opposing team’s answer.

While intended for cases where an answer was outright accepted instead of prompted, this has the effect of eliminating any possibility of protests of the type you mentioned.

[Important Bird Area]

3. A "category expert." This person should be an NAQT member/editor/writer, and someone whose knowledge of both the NAQT rules and the category involved in the protest is respected by both teams.

Dwight, I agree with essentially all of your post (protest committees announced in advance, and straightforward communication with teams), but I don't see how the last bit is remotely enforceable. How would anyone know in advance whether someone's knowledge of a particular category is, in fact, respected by every team in the tournament? What happens when a team produces a metaprotest that they do not, in fact, respect the knowledge of the category expert?

(In short, I think this is clearly a good idea but impossible to codify as policy.)

[cvdwightw]

(In short, I think this is clearly a good idea but impossible to codify as policy.)

I think it would be sufficient enough to have the category editor of the question be the third member of the protest committee. Presumably, if complaints about the quality of questions/protest resolutions keep popping up in a single subject area, then NAQT would replace that category's editor.

[First Chairman]

Late to discussion: protest resolution committee rosters should be declared ahead of time, either at the first team meeting or on paper at the start. A process for declaring a formal protest and situations should obviously be clearly articulated. Ideally, I would want the committee to consist of subject experts and one tournament director (not voting, but executes); at NSC, I prefer the chief editor to be involved in the committee. At other competitions I've been involved with, we usually have a designated person "on call" for all calculation questions, and at least one designated person who does the internet confirmation.

Nowadays, with broadband internet and 3G access, it may be easier to circumvent a formal protest, so it may be more important for moderators to know what options to resolve protests informally should be.