There's a few different areas I want to address here, so I suppose I will go with this one first.
Fred Morlan, at the beginning of this thread wrote:This isn't endorsed by any organization or anyone else, and is barely endorsed by me. I do not mean to slight anyone by my rankings and, if they cause any offense, I apologize in advance for that. That's the last thing I want to do with this process. At some point, though, I do need to stick someone next to some number.
Having these rankings was a good thing. It was a good source of discussion about all sorts of things. The fact that people are apparently taking these things way too seriously and complaining about being mis-ranked is a bad thing. Without putting words in Fred's mouth, I think he did this because he really cares about the game and the people who play it and wanted to reflect all the hard work that people across the nation have put in, even if no one else saw it. If you want to encourage Fred not to do this next year, keep posting.
---END TERRIBLE META(-META?)-DISCUSSION HERE---
So, the whole reason I initiated the ranking, at first, when I very first thought of it back in June or July of last summer, was simply because I thought that the poll I ran on here was wrong, in that teams with "name value" benefited from something of a bias, as compared with teams that didn't have traditionally strong programs. I was hoping, if not to be "more correct," to at least point out teams in regions that don't post a million times on the board every day and say "hey, this team is pretty good, you should think about them some" or "I know region X has teams A & B, but it also has team C which is pretty good" or whatever dumb analogy you want to come up with. And that was the entire thought process behind it. I thought I'd give it a shot, see how asinine it was, and then quit in November or something if it turned sour or someone mailed me a toe with pink nail polish or whatever.
As I started getting ready to do it, though, I realized that there were a couple of benefits that could be gained from the rankings beyond what one would normally gain from rankings - that is, the ability to tell your administration, "hey, let us go to nationals this year, we just got voted the 20th best team in the nation!", or tell that to anyone who would donate to your team, or etc.
First of all, no one has really looked at the difficulty of sets in the statistical sense, or at least not in any big sense of it. On here, people would declare sets as easy or hard, often (it seemed) on the basis of a couple of questions. GSAC, for example, was typically treated as a hard set. I believe that the Maggie Walker editing team made some decisions to change how they handled their set this year, but it turned out to not be a particularly hard set, this year at least.
I don't claim to be a master statistician. I'm probably a bit too much like Bill James in how I look at statistics - there was a good number of gut decisions in the rankings, and with my adjustments for sets and so forth. But I am able to put data into an Excel sheet and make it spit out means and quartiles and standard deviations, or just differences in PPB between teams I'm watching, and using my vast knowledge accumulated from watching the occasional episode of Card Sharks, determining that numbers that are higher indicate that that set is easier than a set with lower numbers. That's what I did this year.
This is beneficial, I believe, because quiz bowl really needs to watch its difficulty to be successful. I come from West Virginia, an area that, since I've been active in national quiz bowl (primarily via crappy posts), has had exactly two or three teams of any real talent as far as having a national impact (the Parkersburg squads of the last 3 years). The rest of the teams have ppbs that, at best, go around 15, but probably have a median of about 8. If you're going to bring a new set in and try to appeal to these teams, you aren't going to do it if you're swinging around with easy parts on Mishima and Jasper Johns in your bonuses. You're going to turn those teams off.
This is a whole different thread about this subject, probably a ton of threads. But one of my goals with this thing was to track sets and kind of let people know just what to expect from a set, because an announcement thread with "regular season difficulty" in it is often meaningless, and sets often get reputations about their difficulty from the slightest thing. You'd think Tommy Casalaspi had drove a school bus through a group of nuns because he helped write CaTo/TaCo a few years ago, in a set that was intended to be really hard.
Also, no one had really ever tried to track teams' performances in all sets and keep them in one area publicly. I wasn't good about uploading these regularly, and hope to improve in that regard next year if I do this again, but I think this was a first. NAQT has its public database, of course, but that doesn't include HSAPQ sets or Prison Bowl or whatever, for obvious reasons. And I sent this spreadsheet to Andrew Hart, Gautam Kandlikar, Jeff Hoppes, and R. Hentzel in May, before their nationals. I honestly forget if I sent it to Matt Weiner or anyone in HSAPQ; I'd hope I did, but I might have just forgotten. So I believe HSNCT and NSC, and maybe NASAT, benefited in this regard.
But, basically, I used the rankings to draw eyes to these things. To the fact that teams like Parkersburg, Champlain Valley, and University School of Nashville, while not real title contenders, were certainly solid squads that just happened to be stuck in areas that lacked "real" quiz bowl in terms of what you would say in Illinois or around D.C. or even in Texas with the deep LASA & Seven Lakes teams and the other good squads that are there. To the fact that GSAC and Prison Bowl certainly weren't as hard as their reputations might make some think. That the field of national title contenders wasn't really limited to just a couple of teams this year, but was a little more open than any of us would have thought a year ago.
I don't know that I ever really cared so much about ranking the top teams as much as highlighting the middle of the pack teams. I think, speaking in terms of the growth of the game nationally and ensuring more people get involved in this great activity, we probably spend too much time talking about the best teams and trying to point some things towards them that don't necessarily have to be pointed to them. The nationals should be about them, of course, because the point of a national championship is to fairly determine what team is actually the best in the country, or get as close to that as possible. I don't think it's good to frame discussion of sets that are intended to appeal to a wide spread of teams as just in terms of "how will this set be received/played on by TJ, Maggie Walker, State College, LASA, ..." That should be part of it, but there's only so many teams in that range.
I think pointing out things like "hey, there's a team in this place called Columbia, Kentucky, that has 800 students total 9-12, public school, and they put together a damn good team" and then asking what we can do to get more situations like that going is key to the growth of quiz bowl. Or, "this Frenchtown team is really committed to quiz bowl, despite being stuck in what is essentially the middle of nowhere for the activity, how can we get more teams like them?"
tl;dr: There was a lot more behind why I did what I did than just having some obsessive need to order quiz bowl teams like cards in the 1990 Donruss baseball set; coincidentally, I did this to draw attention to those things.
Anyway, enjoy this hot mess of a post. I'll try and actually address some concerns raised with the rankings, starting from a top-down prospective.