Fred wrote:Saying that "most" housewrites have a roughly equivalent difficulty is, well... wrong. Sorry, but HFT had definite differences from BHSAT and LIST.
But those housewrites are more similar to HSAPQ than they are to NAQT in my opinion.
Fred wrote:Saying that "most" housewrites have a roughly equivalent difficulty is, well... wrong. Sorry, but HFT had definite differences from BHSAT and LIST.
theflyingdeutschman wrote:Fred wrote:Saying that "most" housewrites have a roughly equivalent difficulty is, well... wrong. Sorry, but HFT had definite differences from BHSAT and LIST.
But those housewrites are more similar to HSAPQ than they are to NAQT in my opinion.
Edward Powers wrote:Ashvin,
I noticed one error, an understandable one, on your October 5th update.
You accidentally added the PHSAT results for Blair Academy of New Jersey to the Maryland Fall Tournament results of Montgomery Blair High School of Maryland. The error probably occurred because Montgomery Blair is often referred to simply as "Blair", and was so designated at Maryland's Fall Tournament. Still, these records should be disentangled, restoring Montgomery Blair to its earlier 8-4 record while giving Blair Academy of NJ its own separate record.
whitesoxfan wrote:Why haven't teams that played at UIUC Earlybird been added to the rankings yet?
whitesoxfan wrote:Tristan Wiley = Macomb's data doesn't appear to be entered.
Charbroil wrote:Are you not going to incorporate ACF Novice into these rankings?
Excelsior (smack) wrote:Charbroil wrote:Are you not going to incorporate ACF Novice into these rankings?
No. I have literally no way of correlating teams at Collegiate Novice with the teams that will be playing regular college tournaments throughout the year. While I'm usually a fan of having more data, Collegiate Novice data would be so actively unhelpful that I think they would reduce the quality of the rankings.
First, high school teams appear to suffer a ranking penalty for playing college tournaments, if games are weighted by the difficulty of the set they were played on. It's been noted already that college tournaments are overweighted by the current scheme, because they are less similar to nationals than nationals would be but they get higher weight than nationals would. With college tournaments included, however, set difficulty is positively correlated with opponent strength -- college teams are, in principle, the strongest opponents, and they are only faced at the toughest tournaments. Since weight is monotonically increasing in set difficulty, and probability of losing in opponent strength, it follows that weight is positively correlated with probability of losing for high school teams who play college tournaments. A few Ls against college opposition much stronger than you, on questions that don't look much like nationals, lower your ranking many times more than your ranking is raised by a few Ws, on questions that don't look much like nationals, against high school opposition much weaker than you -- even though the latter are theoretically potential nationals opponents and the former are not. In fact, this is probably true to some extent for high school tournaments as well -- better teams on average seek out harder tournaments -- but the effect size should be small; with the inclusion of college tournaments, however, the effect is stark.
The second big issue is one of security. Eligibility rules in some states frown on high school teams playing college tournaments, and those teams who do so anyway often play under fictitious names to avoid repercussions with state bureaucracies. The rankings do not appear to respect this anonymity, but instead go ahead and translate those fictitious names into real ones, eliminating teams' anonymity and leaving them open to eligibility recriminations.
Ultimately, I recommend replacing the KRACH probability model at the heart of these rankings with DP-FRIAR in order to both: (a) incorporate all the information contained in scores as opposed to just W-L records;
(b) estimate difficulty from the data, at the packet level, instead of arbitrarily assuming difficulty levels for each set.
Update with IHSSBCA kickoffs?
Excelsior (smack) wrote:Do note that I was going to start determining set-level difficulty from PPB data eventually; the arbitrary weight's I've picked so far are just stopgaps for the meantime.
jonpin wrote:Excelsior (smack) wrote:Do note that I was going to start determining set-level difficulty from PPB data eventually; the arbitrary weight's I've picked so far are just stopgaps for the meantime.
I might have gotten an answer for this previously, but I don't recall it... I still don't understand what the motivation is for "harder tournament = more important".
large, visible red letters on the Excelsior web page wrote:The rankings may not be updated until after New Year's, because I am currently in the midst of rewriting the entire site to make it more maintainable and extensible.
I don't speak officially for NAQT, but I probably know at least as much about database things as Jeff does, so I'll speak unofficially. We don't anticipate changing that column in the foreseeable future, and if we did, it's not like those are a trade secret—we could probably arrange to get you the necessary data to switch things over.Excelsior (smack) wrote:One potential issue (which I would have to ask Jeff or someone about) is that if NAQT decides to change their numbering system altogether in any way at all (including shuffling schools from one number to another and so forth), this would be a bit of a hassle for me. If NAQT has no plans to change their numbering system in the foreseeable future (or if I can at least get a heads up in advance if the system is going to change), I will probably implement a system based on NAQT's org_id's.
jonah wrote:I don't speak officially for NAQT, but I probably know at least as much about database things as Jeff does, so I'll speak unofficially. We don't anticipate changing that column in the foreseeable future, and if we did, it's not like those are a trade secret—we could probably arrange to get you the necessary data to switch things over.
Do schools know their own NAQT ID numbers? Could this be a line added to registration forms? Is it too much to ask tournament directors to look up ID numbers and include them in their statistical output?
mithokie wrote:Out of curiosity, does it look like these will be ready in May or is it looking more like a 2012-2013 restart?
Thanks!
Return to National and Regional Comparisons and Discussions
Users browsing this forum: No registered users and 0 guests