Is this a joke?Tees-Exe Line wrote:Any tiebreaker to enter the final will first regard teams’ records in their head-to-head matches, and if that is insufficient (or the tournament is bracketed), points per bonus.
[Split from this thread--mgmt]
Is this a joke?Tees-Exe Line wrote:Any tiebreaker to enter the final will first regard teams’ records in their head-to-head matches, and if that is insufficient (or the tournament is bracketed), points per bonus.
I'm not wedded to the format I posted here, but I must say I haven't heard a convincing argument against head-to-head tiebreakers. In particular, the proposition that it double-values a single game is not correct. Normalizing the weight on every other game to 1, the weight on the head-to-head game is in the range (1,2). If that game were in fact double-counted, there would be no tie. So the question is whether the head-to-head match should have a weight epsilon greater than the weight on every other match, for epsilon < 1, and I think there's a strong argument that it should. The tiebreaker is supposed to answer the question "which is the better of these two teams?" and the data from the match in which those teams met is more informative than data from other matches; hence a higher weight is justified.grapesmoker wrote:Matt is right: h2h is a terrible tiebreaker (double-values a single game)
Read this thread from 2008 and then get back to us.Tees-Exe Line wrote: I'm not wedded to the format I posted here, but I must say I haven't heard a convincing argument against head-to-head tiebreakers.
Not to mention the team that won the head-to-head matchup is also likely the team that took the worse loss. (In a 10-team round robin, consider two teams tied at 8-1. If the #1B team beat the #1A team, then the #1B team necessarily lost to a team at #3 or below, which the #1A team necessarily beat. Yet the #1B team gets the tiebreaker. This seems perverse to me.)Horned Screamer wrote:It's incredibly simple. The tie wouldn't exist in the first place if not for one team beating another, so it makes no sense to break the tie with a result that caused the tie in the first place.
Thanks for re-stating the very point I already addressed in my previous post. But when it comes to totally unnecessary, pointlessly hostile forum posts, we know whom to turn to. At least this time you didn't out and out lie.Horned Screamer wrote:It's incredibly simple. The tie wouldn't exist in the first place if not for one team beating another, so it makes no sense to break the tie with a result that caused the tie in the first place.
I'm not exactly sure what you're saying, but I guess relative to other tiebreakers head-to-head DOES reward the team that did NOT do well in a less important match (which is downweighted by definition)--and that's a plus!Cheynem wrote:But if you use head to head, does that not reward the team who did not do well in "unimportant" matches (i.e., losing to a team below them)?
as a premise, nothing you have argued leads to this:IF you agree with me that the point of a tiebreaker is to choose which of two teams has done better at a given tournament
In fact, people are taking great pains to point out that overall performance over the course of the tournament is a better indicator of which of two teams has done better at a given tournament, rather than which of two teams has done better at in a given game. You're engaged in basic question-begging here.you should not be satisfied with the status quo.
No it doesn't. I can't believe you just said this. Look at the stats to find out how well they actually did against the field.Tees-Exe Line wrote:The very fact that there's a tie says that both teams did equally well against the field!
Your conclusion again fails to follow from your premise. You are, again, simply begging the question by advancing the data which will be used in the head-to-head tiebreaker to the status of "most relevant" data. The relevance of the data is in fact what's in dispute here.Since it has to be broken somehow and we're operating from the agenda "produce a binary ranking," then the most relevant data (and hence the data deserving a higher weight) is from the head-to-head match(es).
Here's a pro-tip: sometimes the majority consensus is wrong, to be sure. But sometimes the majority consensus is not only right, but right for such a simple and straightforward reason that arguing against it devolves into circular argumentation and tautology. Which horn is represented by this conversation is left as an exercise to the reader.I'm not sure what more there is to be said on this subject since we're all just repeating ourselves.
Actually, by your logic, the team that lost the head-to-head matchup did better against the field, as they had one fewer loss against common opponents.Tees-Exe Line wrote:The very fact that there's a tie says that both teams did equally well against the field!
This.Communi-Bear Silo State wrote:Actually, by your logic, the team that lost the head-to-head matchup did better against the field, as they had one fewer loss against common opponents.Tees-Exe Line wrote:The very fact that there's a tie says that both teams did equally well against the field!
I was thinking about this earlier, and I think the question is, what is a tiebreaker supposed to do? Is it supposed to pick out the better of two teams in "a hypothetical two team tournament," as you suggest, or is it supposed to pick out the team that's done better against the full field of the tournament, or the team that's more likely to do well in a top bracket or finals match (or whatever the tiebreaker is being used to determine)? Given that finals matches should always be played, it seems to me that picking out the better of two teams in a hypothetical two team tournament pretty much never matches the actual in-tournament situation in which a tiebreaker arises: a tie in record at the end of prelims or playoffs that affects which teams have a shot at the title. In other words, at the end of a prelim round-robin, if teams A and B are tied in record, and you're trying to decide which one should advance to the top playoff bracket, I'm not sure that the question "which of these teams do I think would win a hypothetical two team tournament?" is the right one to ask. It seems like the more relevant question is something like "which of these teams do I think has the better shot at the title?" Or, if you're looking at a tiebreaker for a non-top bracket, the question could be rephrased as "which of these teams do I think will place higher in this bracket if they get in?" Once the question is phrased that way, I think it makes perfect sense to look at data beyond which team beat which other team. For instance, suppose team A beat team B by 5 points, lost to team C by 300 points, and beat all other teams by 5 points. Meanwhile, team B beat all teams except for team A by 150 points. Which of these teams would you guess has a better shot at success in the top bracket? In a finals match against (say) team C?prodski wrote:Put me down in the minority that doesn't understand why h2h isn't the first tiebreaker. If beating a team head to head doesn't matter, why even keep w/l records? Just rank everyone according to PPG?
At a standard quizbowl tournament the final placement is already determined by your win-loss record within the playoff pool, and any leftover ties that need to be played off will be broken by a packet anyway.prodski wrote:I agree with you Seth, if you are looking to move a team up into a bracket, that makes perfect sense. I obvioulsy am looking at it for crowning a champion, or final placement at a tournament when it is over and there are no other tiebreakers or matches to be played. Your point is well taken. Thanks.
This is largely correct, except that statistics were not calculated across the tournament, but rather only in games against common opponents and each other. Thus, a team did not benefit by playing a game against a poor team its opponent did not play.Tees-Exe Line wrote:As for people whose arguments actually deserve discussion, if the thread Jeff linked above is what established the consensus in favor of a PPG tiebreaker and against head-to-head, that consensus ought to be revisited. As I understand the analysis, the sample was taken from tournaments featuring repeated match-ups. Each of the candidate tiebreakers was calculated before the second (or third...) match between heretofore-tied teams, and the "prediction" about the outcome of that match was tested against the actual outcome, yielding statements like "PPG predicted the outcome of the repeat match 70% of the time." If I've misunderstood the procedure, please correct me.
You will note that this proposition has been a commonly accepted proposition well before this thread (in fact, it was an axiom of the thread under review). I'm not going to respond to Andrew Hart's critique since I think that criticism is somewhat right (for instance, there may have been factors such as question quality and relative team strength that biased the results one way or another; for another, H2H was transformed to "H2H differential" if the teams had played multiple times such that there would be an actual H2H-based tiebreaker if H2H was split evenly between the teams). I don't feel that it's right "enough" to totally invalidate the collected data. If I understand your twist on the critique, however, you are arguing that a "regular" repeat match in a tournament between two teams of the exact same composition as the original match is subjectively different than a "tiebreaker" repeat match, which, furthermore, is subjectively different from a "finals" repeat match. I'm not sure what the subjective difference here is (we used all three types, IIRC). Question quality/difficulty? Should be the same. Distribution? I'd hope that "whether or not that third arts tossup is opera or architecture" would be unbiased as to regular vs. tiebreaker matches. Pressure? I haven't seen any objective or subjective evidence that teams play differently in tiebreaker matches compared to other matches.That procedure has a number of flaws, but probably the most important one to address and resolve is one of interpretation about the nature of a tiebreaker. First, let me propose that the best way to break a tie is to play a complete packet between the two tied teams and award the tiebreaker to the winner of that match.
Marshall, I've already copped to using the wrong statistical test in this thread and re-analyzed the data using what I believe to be the correct test. The second test used a 2x2 matrix of outcomes (e.g. H2H better won/PPG better won, H2H better lost/PPG better won, H2H better won/PPG better lost, H2H better lost/PPG better lost). I did not make any corrections for the number of tests that were done but a one-tailed McNemar's test between PPB and H2H is still significant at the 5% level even using Bonferroni corrections.Second, the outcome in this case is a binary prediction: was the tiebreaker in question was "right" or "wrong" about the outcome of the repeat match? If, as I suspect, a linear probability model was used, than in some sense that is required to be incorrect by construction. At the very least we'd need corrected standard errors around the percentage-correct estimates. Since the inputs have a lot of variance empirically, it would be far better would to use probit or logit. (Were the PPG and PPB tiebreakers specified as binary or continuous variables in that procedure, ie "Team A has higher PPG" or "Team A's PPG - Team B's PPG" or some variation thereof?)
As the person who largely combined the suggestions of Andrew Hart and others into the D-Value, I sought to accurately rank teams who had played wildly different sets of opponents on potentially different packet sets. The D-Value roughly translates into "how many points would we expect you to score against a hypothetical nationally-average team on the appropriate set of questions," which to me (and apparently NAQT) seems to be as good as any a way of ranking every college team that played SCT without making the calculations too arcane for the average quizbowler (there are some adjustments to the raw rankings to ensure that order of finish within a given tournament takes precedence over statistical measures). Of course a D-Value or equivalent future statistic can be calculated for any individual tournament. However, not knowing your experience, I am unsure whether the second sentence here is due to your inexperience TDing anything larger than ~10-15 teams. Reconstructing the results of an entire bracket, by hand, in a small time window, using scoresheets with borderline unintelligible writing is something that happens all the time in moderately-sized high school tournaments. The best statistical tiebreakers are things that can be computed relatively easily by hand if one of the thousand things that could go wrong with stats-entering does go wrong. I'm not saying it can't be done, but that it would be wholly impractical to do it.Finally, I should say that it's quite possible to use a function of several tiebreakers. That seems to be the philosophy behind the NAQT "D-value," and I don't see why it can't be done here. I imagine it wouldn't be hard to program such a function into the tournament statistics software, and we can discuss what the function would be.
No one is arguing that we should be satisfied with the status quo, or that a 4.5-year-old thread is somehow the final word on tiebreakers. However, the objective analysis done in 2008 (and corrected in 2011 due to being wrong) was broadly consistent with subjective quizbowl experience that:Let me conclude by saying that IF you agree with me that the point of a tiebreaker is to choose which of two teams has done better at a given tournament, you should not be satisfied with the status quo. To my mind, by rewarding teams for doing well in unimportant matches, it overvalues a lot of uninformative data.