Assigning Moderators

[Aaron Goldfein]

I was wondering if anyone had any opinions to share on how to assign staff to matches when you're the tournament director. I'll work under the assumption that you have a range of staff of various abilities, and that you probably have slightly more than the number of rooms you have so you can have some rooms with two staff but not all. In general, I think most would agree you want to assign the less experienced staffers to be scorekeepers, and then you want to give your less experienced moderators the scorekeepers and have your more experienced moderators work solo.

The general consensus on this topic seems to be that you assign your more experienced moderators to the more competitive brackets and your less experienced moderators to the less competitive brackets. However, there is often a conflict between this and the fact that many staffers often have a connection to one of the teams. How much do you worry about the whole "avoiding bias" thing? For example, if you had a strong moderator who was the coach from a school at your tournament, and that school had teams in the top brackets, would you then assign him to the weakest bracket in order to avoid the possibility of bias? Or would this be a lesser concern than keeping the stronger moderators in the top bracket?

[Whiter Hydra]

I don't think I've ever seen a case in a tournament where a team was concerned about conflict of interest. I have played in many matches where the moderator either was attending, went to, or was a coach at the opposing team's school, and I have never seen any hint of bias.

[njsbling]

I was wondering if anyone had any opinions to share on how to assign staff to matches when you're the tournament director. I'll work under the assumption that you have a range of staff of various abilities, and that you probably have slightly more than the number of rooms you have so you can have some rooms with two staff but not all. In general, I think most would agree you want to assign the less experienced staffers to be scorekeepers, and then you want to give your less experienced moderators the scorekeepers and have your more experienced moderators work solo.

The general consensus on this topic seems to be that you assign your more experienced moderators to the more competitive brackets and your less experienced moderators to the less competitive brackets. However, there is often a conflict between this and the fact that many staffers often have a connection to one of the teams. How much do you worry about the whole "avoiding bias" thing? For example, if you had a strong moderator who was the coach from a school at your tournament, and that school had teams in the top brackets, would you then assign him to the weakest bracket in order to avoid the possibility of bias? Or would this be a lesser concern than keeping the stronger moderators in the top bracket?

Thankfully I've never had any problems with the team/coaching bias when assigning moderators to certain brackets and rooms. Most of the time our best moderators can also scorekeep so I generally prefer to have the newer moderators work with a scorekeeper. I suppose it also depends on how many rounds are being run in the brackets (need the better moderators to be in the bracket with more rounds).

[Aaron Goldfein]

I've never been so concerned with moderators being actually biased (it's fairly difficult for a moderator to have much influence on a match anyway) as much as I've been concerned with a team accusing a moderator of being biased when they lose a close match. Teams can look for excuses when they lose close games, and knowing that the moderator is, for example, the coach of the other team could be all that team needs to justify to itself that something unfair happened.

[Skepticism and Animal Feed]

I've been in quizbowl for well over a decade. I have seen all sorts of quizbowl scandals and alleged scandals, some of them bizarre. I can't recall a single scandal involving a moderator who was allegedly biased against one team during a game because of team affiliation. The closest I can think of is the allegation that It's Academic was often rigged so that certain schools would win, but this is the "David Stern rigging the NBA so big market teams will win to increase ratings" type of scandal, not the hypothetical "guy who went to team A rigs a game for Team A" you describe. I suppose another analogue may be the scandal where one guy who was editing a tournament (or at least had access to the questions) leaked answers (ineptly) to his team, but that did not involve a moderator making biased decisions during game play.

So, yeah, I would not really consider bias a factor. Put the best moderators on the games that matter most. Sub out slow moderators/give them scorekeepers if you have the human resources to do so. Hold basic training before the tournament so moderators know how to avoid easy mistakes that slow down games.

[Ndg]

So, yeah, I would not really consider bias a factor. Put the best moderators on the games that matter most. Sub out slow moderators/give them scorekeepers if you have the human resources to do so. Hold basic training before the tournament so moderators know how to avoid easy mistakes that slow down games.

This is exactly right. Teams are much more likely to complain about bad moderators than about any kind of conflict of interest. Don't sacrifice moderator quality for this reason.

[jonah]

For the prelims, I seed moderators in more or less the same way one seeds teams. Just as one might make adjustments to seeding to keep teams from the same school or region apart, I make minor adjustments to keep moderators away from teams with which they have been associated — not because I am concerned about bias, but, as Aaron suggested, because I don't want to deal with claims about possible bias, a discussion of which is guaranteed to leave no one happy — and occasionally to ensure staff who have made requests get to see teams they want to see, especially if they haven't before.

For the playoffs, I generally put the best moderators in the top pool, but below that, it's some combination of "worse moderators go in lower pools" and "make sure each pool has at least one good moderator". I avoid putting even excellent moderators in the top pool if they have been associated with a team I expect to make the top pool, for the same reason, but below that I don't really worry about it (you think you got screwed from fourteenth place down to fifteenth? deal with it.).

Within pools, I assign matches between closer seeds to better moderators. (That doesn't always mean, say, the game between the two top seeds. For instance, in a 24 team tournament with 4 pools of 6 in the prelims, the 8-9 game is "more important" than the 1-8 game.)

[jonpin]

What all of the people above me have said. There is also a consideration to keep in mind if your tournament is spread out over a large area or on multiple floors, where everyone comes back to one tab room after each round, and that is to make sure the moderators who have to travel the furthest are some combination of fastest readers and fastest walkers. A lot of this seems minor, but saving 5 minutes a round over the course of a tournament can save nearly an hour altogether.

[Kevin]

One of the things I learned was that if I have a tournament with multiple preliminary pools, I should put my best moderators together. Yes, this means some teams get stuck with worse moderators, but it also means that if the best moderators can finish more quickly, they can go relieve the weaker moderators. A pool is only as fast as its slowest moderator.

For playoffs, I want my best moderators with the best teams, generally speaking.

As far as bias goes, if I have a situation where I have a high school house team, I try to make sure my student-staffers aren't reading for them. I've been lucky enough to have enough adult volunteers or college students that it hasn't been an issue. Strictly speaking I don't think this is necessary, but why not avoid the issue if it's possible?

[Ndg]

One of the things I learned was that if I have a tournament with multiple preliminary pools, I should put my best moderators together. Yes, this means some teams get stuck with worse moderators, but it also means that if the best moderators can finish more quickly, they can go relieve the weaker moderators. A pool is only as fast as its slowest moderator.

It's worth emphasizing that this is good only for non-timed tournaments, though. I once made the mistake once of putting my best moderators in the same few preliminary brackets (because those rooms would end up having the top playoff bracket), with the result that the teams in the bracket with the worst moderators had much fewer tossups heard. So make sure to distribute the best readers evenly if you're using timed rounds.

[Steeve Ho You Fat]

One of the things I learned was that if I have a tournament with multiple preliminary pools, I should put my best moderators together. Yes, this means some teams get stuck with worse moderators, but it also means that if the best moderators can finish more quickly, they can go relieve the weaker moderators. A pool is only as fast as its slowest moderator.

It's worth emphasizing that this is good only for non-timed tournaments, though. I once made the mistake once of putting my best moderators in the same few preliminary brackets (because those rooms would end up having the top playoff bracket), with the result that the teams in the bracket with the worst moderators had much fewer tossups heard. So make sure to distribute the best readers evenly if you're using timed rounds.

This is a really bad idea whether or not you're using timed rounds. If you have all your top bracket (best) moderators in the same prelim bracket and all your bottom bracket (worst) in another, the first will get miles ahead of the second. The relieving the bad moderators works only if the one bracket is more than a full round ahead of the other. You should never have that situation, because it means some teams will have extremely long/short breaks, and because then you can have a hallway full of people who have heard a round's with of questions they can spoil accidentally or otherwise for teams that haven't yet started that round.

You have to distribute your best moderators among the prelim brackets and then just make them move to the top when you rebracket.

[Cody]

One of the things I learned was that if I have a tournament with multiple preliminary pools, I should put my best moderators together. Yes, this means some teams get stuck with worse moderators, but it also means that if the best moderators can finish more quickly, they can go relieve the weaker moderators. A pool is only as fast as its slowest moderator.

It's worth emphasizing that this is good only for non-timed tournaments, though. I once made the mistake once of putting my best moderators in the same few preliminary brackets (because those rooms would end up having the top playoff bracket), with the result that the teams in the bracket with the worst moderators had much fewer tossups heard. So make sure to distribute the best readers evenly if you're using timed rounds.

This is a really bad idea whether or not you're using timed rounds. If you have all your top bracket (best) moderators in the same prelim bracket and all your bottom bracket (worst) in another, the first will get miles ahead of the second. The relieving the bad moderators works only if the one bracket is more than a full round ahead of the other. You should never have that situation, because it means some teams will have extremely long/short breaks, and because then you can have a hallway full of people who have heard a round's with of questions they can spoil accidentally or otherwise for teams that haven't yet started that round.

You have to distribute your best moderators among the prelim brackets and then just make them move to the top when you rebracket.

Joe is 100% correct. Always try to distribute good moderators evenly between prelim brackets and switch people for playoffs.

[Kevin]

Sorry, I disagree (although I'll concede the point as far as timed tournaments go). I had a tournament with 21 teams split into 3 groups of 7 teams. So I needed 9 moderators. I had 3 really good moderators, a couple of okay college kids, a few of my high schoolers who were okay, and one or two very raw and inexperienced high schoolers. It was very likely that if I'd have had the three good moderators together, they would have been able to finish at least a round ahead. That experience has made me very hesitant to divide up the best moderators in the future.

In the tournaments I've run since then, I've had enough good moderators to make things go smoothly. That's the best way to fix the problem.

[Steeve Ho You Fat]

Yes, but even if they do get a whole round ahead, then the teams in that bracket have heard round 7 and are standing around in the hallway with teams from the slow bracket that just finished round 6. You don't have to be paranoid to see the ways, accidentally or intentionally, that this could cause problems.

Also as a secondary issue you're being kind of mean to the teams in the slow bracket by making them have 6/7 rounds read to them by the worst moderators at the tournament. As a tertiary issue, you now have the teams in the bracket with the best readers sitting around for an entire round while the slow bracket catches up.