setht wrote: ↑Thu Feb 21, 2019 12:21 amDI teams play 5 prelim games, then 5 playoff games, then 3 super-playoff games. The 6 teams that wind up in the top super-playoff brackets will not have played against the same opponents in the playoffs. What games should/should not be used in the final rankings? In particular, do people think it is best to throw out all non-common-opponent matches and rank the top 6 super-playoff teams on the basis of their 5 games against each other?
Yes. I will develop this train of thought further below.
setht wrote: ↑Thu Feb 21, 2019 12:21 amOr does the sorting of teams into playoff brackets reduce the “variance in skill between . . . two sets of teams” (as Cody puts it) sufficiently that we stand to gain by determining final rankings based on 9 games (1 prelim game + 5 playoffs + 3 super-playoffs), all played against top-playoff-bracket opponents? (E.g. do we think that ranking based on 5 games will boost the probability of circles of death that might need to be broken on half-packets?)
I do not think it does (re: variance). (The probability of circles of death is guaranteed to increase, but I do not think it renders the placement policy worse than the alternative.)
I want to note that the variance / skill (team strength) argument is, in my opinion, a precondition for accepting a ranking based on overall record. If you cannot make that argument, then there is (again, in my opinion) no point in further discussion because it is clear that using overall record is fundamentally unfair compared to the alternative. I use it because the implications are quite plain and it’s easy to see that the argument is not true; the disparity is often huge, even with great seeding. (And even if it were true, this variance is easily eliminated, which is a positive.)
However, variance / skill (team strength) is not the only argument against counting games between teams not in the field of common opponents. See below.
setht wrote: ↑Thu Feb 21, 2019 12:21 amAlso note that it is possible that two teams who play each other in the prelims can wind up playing each other again in the super-playoffs. It seems a little weird to me to say “we need to throw out all games against non-common opponents; as a result, the result of your prelim match will not count towards the final rankings, but the result of your super-playoff match will.”
I don’t see it as weird / see it as sensible due to the following chain. The alternative would need to be that the super-playoff match against that opponent be a bye. If you played the super-playoff match and counted the prelim match then you would be counting extra games for some teams but not others and/or you are counting extra games against different opponents for some teams but not others. I think such an alternative scheme (not replaying the match in the super-playoffs) is actually perfectly fine, but it would be suboptimal for a few reasons.
First, it’s a bit of a logistical nightmare for no gain (you don’t save a round or eliminate a room) and puts a lot of pressure on the tournament staff to make sure they have identified all these matches since it would propagate through all 6 super-playoff brackets. Second, tabula rasa, it’s accepted that certain inefficiencies of these types may occur that require you to replay matches and that this is a perfectly fair compromise that can be worthwhile (for example, because teams will be more in tune with the set in the afternoon than in 5 rounds in the morning). I say tabula rasa because once the results of the game are known, I imagine some teams would strenuously disagree with that statement :)
____________________________________________________________
There is a more fundamental argument to be had, which I mentioned in my postscript previously and which I have referenced in the previous two quotes: what is the purpose of bracketed, round-robin preliminary rounds? I would like to develop this from a low-level as follows. (To be clear, I understand this is a regurgitation of known or intuited knowledge for many people and especially you, Seth.)
We accept that the simplest and fairest possible format is a single round robin of all the teams in a tournament because every team plays every other team. But, when you have more than 14 teams, this format imposes significant logistical hurdles and starts to affect the fairness of other aspects of quizbowl (consistent packet difficulty and quality being an obvious example).
The community standard of bracketed round-robin preliminary & playoff rounds (or any other extensions, including super-playoffs) are a compromise that significantly improve on other formats (single/double/x-elimination, whatever NHBB’s monstrosity is) while being much easier to execute than the other fair compromise format we have accepted (power matching, which excels at large field sizes). A properly executed bracketed format meets many of the fundamental standards of fairness embodied by a whole-field round robin: a single loss will not eliminate a team from winning the tournament (hence also the “advantaged final”), teams play as much of the field as feasible, teams play as many other teams of similar strength as feasible, etc.
A key to the bracketed format is seeding / re-seeding. You cannot take all the teams from one bracket to a given tier in the playoffs, else we would not need to use a bracketed format. So, we snake seed teams for preliminary rounds based on a value judgement of their strength to spread them out and try to ensure that the appropriate teams make each tier of the playoffs. (Imperfections in seeding are the critical weakness of a bracketed format because they block teams from making the tier they may have “deserved” to make based on their strength. It’s a small but significant reason that power matching excels for large field sizes, circa 150+ teams. In the presence of perfect seeding, bracketed formats would be almost as perfect as a whole-field round robin.) At the conclusion of preliminary rounds, teams are reseeded based on win-loss and tiebreakers and put into playoff brackets where they play other teams of similar strength.
The critical take-away from the bracketed format scheme is that the preliminary rounds are designed specifically to seed teams for the playoffs in the fairest format we can devise, accepting that some imperfections will come into play (mitigated by proper design of the playoff format). The playoffs, against teams of similar strength, are how we determine the final standings. By this understanding, there is no tension in dropping games from the preliminary rounds in determining placement because they have served their purpose in slotting teams into playoff brackets.
At some point, someone recognized that throwing out all the games from the preliminary rounds might be suboptimal. A game played between two teams in the same playoff bracket has no less legitimacy because it occurred in the preliminary rounds. (Else there would be bigger problems!) Hence: the crossover. If you carry those games over and count them in the playoffs, you do not need to replay them. As a consequence, you can take more teams into a given playoff bracket and keep more teams in contention for a given tier of placement. (This results because the number of games for a crossover bracket with an even number of teams is {#teams / 2} instead of {#teams – 1} as in a round robin.) (In the example from my previous post for 16 teams, you can keep teams in contention for places #1 – #8 and #9 – #16. If you did not carry over games and instead had everyone play a strict round robin in the playoffs you would split into places #1 – #4, #5 – #8, #9 – #12, #13 – #16.)
Using a crossover becomes more complicated for larger tournaments, which is why NAQT is using a prelims -> playoffs -> super-playoffs format for 36 teams. (I am disregarding the prelims -> playoffs tiebreaker because it is the same in both formats.) A prelims -> playoffs crossover would take 5 + 10 rounds because you would need to take the top 2 teams from every bracket in order to not eliminate teams for a single loss. (The current ICT format only takes 13 rounds due to the use of statistical tiebreakers from the playoffs -> super-playoffs stage.) And so, the preliminary rounds are designed to slot teams into a playoff bracket within a playoff flight. (Playoff flights contain two or more brackets in “parallel contention”, e.g. at ICT there are two brackets in the top playoff flight for places #1 – #12.) And the playoff rounds are designed to slot teams into a super-playoff bracket half the size (in ICT’s case) of the playoff flights. Similarly to the argument for dropping games as you transition from preliminary rounds to playoff rounds, dropping games from the playoffs is
fine because the playoffs served their purpose in slotting teams into the super-playoffs as fairly as can be devised. (Except that this is not the case at ICT currently because overall record and overall statistics are used at the playoff -> super-playoff stage.)