Player Poll

Old college threads.
Locked
User avatar
Cheynem
Sin
Posts: 7220
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Player Poll

Post by Cheynem »

Good evening. After Gautam J. Kandlikar declined to run this year's player poll, I took up the burden.

Please submit a ranking of your top 25 players. Please try to submit rational ballots and don't make crazy regional things. You can vote for high schoolers if you like. For the purposes of the poll, please consider only those players still in college and thus eligible to, say, play ICT and ACF Nationals (so don't vote for, like, Jonathan Magin or Matt Weiner). You can vote for players who did not play either nationals (Eric Mukherjee) or players who played very sparingly (Jeff Hoppes).

My e-mail address is in the profile. I will close the balloting at an appropriate time (probably sometime in May).
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Cheynem
Sin
Posts: 7220
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Player Poll

Post by Cheynem »

I received a partial ballot, which I would rather not encourage, but I guess I will allow if you have particularly strong opinions about the top of your rankings but not the bottom or if you don't feel comfortable ranking different regions or something.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Cheynem
Sin
Posts: 7220
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Player Poll

Post by Cheynem »

Just a bump here--I'd like to see more ballots (I have 4). I'll probably leave this open until mid May at least--I realize people are swamped with finals and national writing right now, but some more would be great.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Cheynem
Sin
Posts: 7220
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Player Poll

Post by Cheynem »

I will probably close this by the end of next week.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Cheynem
Sin
Posts: 7220
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Player Poll

Post by Cheynem »

I received a ballot today--I will take them probably until Tuesday or so and then publish the aggregate findings.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Cheynem
Sin
Posts: 7220
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Player Poll

Post by Cheynem »

Here are the results from the 2011 player poll:

1. Mike Sorice, Illinois
2. Eric Mukherjee, Penn
3. Rob Carson, Minnesota
4. Matt Bollinger, Virginia
5. Chris Ray, Maryland
6. Andrew Hart, Minnesota
7. Evan Adams, VCU
8. Matt Jackson, Yale
9. Auroni Gupta, UCSD
10. Selene Koo, Chicago, and Guy Tabachnick, Brown
12. John Lawrence, Yale
13. Kurtis Droge, Michigan
14. Ted Gioia, Harvard
15. Andy Watkins, Harvard
16. Ike Jose, Illinois
17. Jeff Hoppes, Berkeley
18. Dallas Simons, Harvard
19. Aaron Rosenberg, Brown
20. Will Butler, Virginia
21. Gautam Kandlikar, Minnesota
22. Kevin Koai, Yale
23. Trevor Davis, Carnegie Mellon
24. Dwight Wynne, Irvine
25. Henry Gorman, Rice

Also Receiving Votes (in order of number received): Sam Bailey, Robert Harden, Tommy Casalapsi, Neil Gurram, Ray Luo, Marshall Steinbaum, Aaron Cohen, Stephen Liu, Graham Moyer, MIke Cheyne, Ben Cohen, Will Nediger

Voters: Matt Bollinger, Jerry Vinokurov, Andrew Hart, Joey Montoya, Matt Jackson, Gautam Kandlikar, Zach Foster, Libo Zeng

When I have more time, perhaps published ballots and statistical blatherings.

I apparently did not receive a ballot, so I edited the rankings to take that ballot into account.
Last edited by Cheynem on Mon May 16, 2011 11:37 pm, edited 1 time in total.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: Player Poll

Post by Mechanical Beasts »

I'll note that barely under 75% of the top 25 players per this poll are undergraduates, indicating that the anti-grad students lunacy is still just that.
Andrew Watkins
geekjohnson
Wakka
Posts: 180
Joined: Fri Dec 23, 2005 7:10 am
Location: Flomaton, AL

Re: Player Poll

Post by geekjohnson »

And I will note that my advocacy on such was merely a suggestion. One which I posited irrespective of acheivements of the demographic(s). Though why would one expect people to actually take what is actually said. Alas..Deesian logic.
James A. Johnson
University of West Florida
formerly of Alabama and Faulkner State

I am an alum of southern quizbowl, the worst form imaginable, and hope and pray that the bearded quizgods shine down their rotund grace upon us all.
User avatar
Cheynem
Sin
Posts: 7220
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Player Poll

Post by Cheynem »

I'm having some problem with sharing Google Docs, so here's an attachment of the ballots in Excel spreadsheet form. Please let me know if my math is flawed in some way.
Attachments
Individual Player Poll 2011.xls
(9.5 KiB) Downloaded 526 times
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: Player Poll

Post by Mechanical Beasts »

Why was Voter 5's ballot counted that way? Seeing as it only ranked two players, and Rob would have moved up a spot if Voter 5 thought Rob was better than like 20th. (And if that ballot held Matt to be third, Matt wouldn't have been able to move into second place like Rob would, but he would have tied for second at least.)

Granted, one ballot left Eric entirely unranked, which is an exercise in absurdity in and of itself.
Andrew Watkins
User avatar
TheKingInYellow
Rikku
Posts: 310
Joined: Sun Sep 14, 2008 5:13 pm

Re: Player Poll

Post by TheKingInYellow »

Voter 3 knows whats up
Graham Moyer
State College 2011
Harvard 2015
User avatar
Cheynem
Sin
Posts: 7220
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Player Poll

Post by Cheynem »

Voter 5 was someone who offered a very clear, passionate opinion about who he felt were the best and second best players in the game. I was reluctant to count his ballot, but since there were very few ballots, I did so. Frankly, with Eric not ranked on the one ballot, I felt it evened things out.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
marnold
Tidus
Posts: 706
Joined: Wed Jan 17, 2007 12:32 pm
Location: NY

Re: Player Poll

Post by marnold »

I'm pretty sure hsquizbowl polls end up with bizarre results any time there isn't a preceding thread where general discussion sorts players into tiers and the obvious names are thrown out so people know to vote for them. There's at least one comically glaring omission from this poll I'm sure wouldn't have happened with such a thread and it also might prime the pump a bit so more ballots come in.
Michael Arnold
Chicago 2010
Columbia Law 2013

2009 ACF Nats Champion
2010 ICT Champion
2010 CULT Champion
Member of Mike Cheyne's Quizbowl All-Heel Team

Fundamental Theorem of Quizbowl (Revised): Almost no one is actually good at quizbowl.
User avatar
Skepticism and Animal Feed
Auron
Posts: 3238
Joined: Sat Oct 30, 2004 11:47 pm
Location: Arlington, VA

Re: Player Poll

Post by Skepticism and Animal Feed »

To be fair, it's mostly our collective fault for not voting and for letting an extremely small sample determine the results.

I tried to put together a ballot many times, but was put off by the new trend of tournaments releasing stats for different rounds separately. Too much work to look at both the playoff and regular rounds. I have some strong opinions about certain players that I would like to have expressed (I'd have Matt Jackson higher and Matt Bollinger lower, for starters) but it's just a lot of work to responsibly determine how to correctly rank the players you know less about.
Bruce
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source
User avatar
Adventure Temple Trail
Auron
Posts: 2754
Joined: Tue Jul 15, 2008 9:52 pm

Re: Player Poll

Post by Adventure Temple Trail »

marnold wrote:There's at least one comically glaring omission from this poll I'm sure wouldn't have happened with such a thread
Out of curiosity, who or what do you have in mind here? Is this referring to the lack of 3-25 on one ballot or do you think there's someone missing?
Matt Jackson
University of Chicago '24
Yale '14, Georgetown Day School '10
member emeritus, ACF
User avatar
marnold
Tidus
Posts: 706
Joined: Wed Jan 17, 2007 12:32 pm
Location: NY

Re: Player Poll

Post by marnold »

RyuAqua wrote:
marnold wrote:There's at least one comically glaring omission from this poll I'm sure wouldn't have happened with such a thread
Out of curiosity, who or what do you have in mind here? Is this referring to the lack of 3-25 on one ballot or do you think there's someone missing?
The latter. I'll give a hint: his name rhymes with Flahntanoo Flah. Obviously people didn't know he was eligible to be in this poll, which proves my point.
Michael Arnold
Chicago 2010
Columbia Law 2013

2009 ACF Nats Champion
2010 ICT Champion
2010 CULT Champion
Member of Mike Cheyne's Quizbowl All-Heel Team

Fundamental Theorem of Quizbowl (Revised): Almost no one is actually good at quizbowl.
User avatar
Cheynem
Sin
Posts: 7220
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Player Poll

Post by Cheynem »

Is it really that egregious to not vote for Flahntanoo considering he barely played anything? (I think he in fact told people not to vote for him last year)

Also: Certainly this poll had a small sample size, but I really don't think it's overly ludicrous. I didn't vote, but it's roughly where I would place most people.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
DumbJaques
Forums Staff: Administrator
Posts: 3109
Joined: Wed Apr 21, 2004 6:21 pm
Location: Columbus, OH

Re: Player Poll

Post by DumbJaques »

Shantanu really didn't play anything this year (the last tournament I'm aware of him playing was when we played Sun n Fun, which is still the tail end of last year). If Shantanu had been playing stuff this year, he almost certainly would have run away with any such polls, because he's ridiculous. The intersection of ACF Nationals this year being way too real for normal humans and Shantanu being Shantanu would have caused everyone great suffering, though it probably still wouldn't have earned him a spot on the inestimable ballot 8.
Chris Ray
OSU
University of Chicago, 2016
University of Maryland, 2014
ACF, PACE
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: Player Poll

Post by grapesmoker »

Rankings are pretty useless because once you come down from the few players who can reliably corner a category plus get stuff across multiple other categories, you've got a bunch of people amongst whom there isn't that much difference. They're all pretty good but not dominant and distinguishing between #10 and #14 or #5 and #8 isn't terribly meaningful.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
User avatar
Sima Guang Hater
Auron
Posts: 1958
Joined: Mon Feb 05, 2007 1:43 pm
Location: Nashville, TN

Re: Player Poll

Post by Sima Guang Hater »

I'm glad someone felt so strongly that I should be ranked above Mike to just rank the two of us. Though I wouldn't agree with this person.

Also, I'll enjoy my 2-month Lady-Jane-Grey-like reign as Matt Bollinger surpasses me.
Eric Mukherjee, MD PhD
Brown 2009, Penn Med 2018
Instructor/Attending Physician/Postdoctoral Fellow, Vanderbilt University Medical Center
Coach, University School of Nashville

“The next generation will always surpass the previous one. It’s one of the never-ending cycles in life.”
Support the Stevens-Johnson Syndrome Foundation
User avatar
Cheynem
Sin
Posts: 7220
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Player Poll

Post by Cheynem »

Voters, voters, voters!!!

Matt Bollinger 4th? Andrew Hart 6th?

Jeff Hoppes behind Ike Jose?

These are only a few of what appear to be many strange final rankings. Are you trying to provoke many pollsters to correct you in an alternate player poll? It seems you are, for how else explain such errors at the Top of your rankings when the Nationals that have just passed seeemed to suggest other conclusions, and if you have erred here, through haste or deliberate provocation, I think, then surely your rankings from 10 to 25 were also seriously rushed.

Some brief illustrations at or near the top of your rankings to help make my point, with some necessary supporting evidence:

Matt Bollinger played and excelled at both the ICT and ACF Nationals. Eric Mukherjee did not play either. Matt Bollinger's SCT D-value score was much higher. So the placing of Bollinger behind Mukherjee is very perplexing. Eric Mukherjee is a great player, but how in the world could you place him ahead of Matt Bollinger? Matt Bollinger played much more in the regular season, winning almost every tournament he entered with extraordinary stats, and more important, actually played both nationals.

Meanwhile, Andrew Hart's teams finished 2nd in BOTH Nationals, ahead of an admittedly terrific Chris Ray both times, and he is BEHIND Chris Ray? With what rationale? i bet Andrew Hart and his family would love to hear it. Again, Chris Ray had a great team and great season, but I think Chris would wonder how he can place NATIONALLY ahead of a player whose team had much stronger performances at Nationals?

Additionally, Kurtis Droge played stronger than Auroni Gupta all year, finishes ahead of him in the one National both attend, and Auroni Gupta, who had a great run at Nationals, but who still finished behind Kurtis here, finishes ahead in your rankings? Again, this is NOT a knock on Auroni Gupta, but an argument for comparing apples with apples and oranges with oranges.

And Jeff Hoppes has a legit beef as well---he has won more nationals than any other player ranked on the poll--and he cannot crack the top ten? Again, perplexing. Is he being punished for not playing ICT?

It seems you rushed your list; my guess is future polls l will correct some of these seeming errors---unless of course it is I who has erred, but I am willing to let other pollsters decide this. And given the errors at the very Top of the rankings, the easiest place NOT to err, I think, you will have awakened many other pollsters to help you see the light.

Fortunately, this is all for fun and even with the best of intentions you were bound to upset some people, but I fear however good you, the voters' intentions were, your apparent haste here will provoke more pollsters to show you the error of your ways---this at least, is my guess.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Edward Elric
Tidus
Posts: 506
Joined: Sun Jan 04, 2009 11:31 pm
Location: Wheaton, IL

Re: Player Poll

Post by Edward Elric »

ITT Mike "Powers" Cheyne
Mike Perovanovic
University of Ill.-Chicago '13
Wheaton North '09
User avatar
jonpin
Auron
Posts: 2266
Joined: Wed Feb 04, 2004 6:45 pm
Location: BCA NJ / WUSTL MO / Hackensack NJ

Re: Player Poll

Post by jonpin »

Not enough hyphens.
Jon Pinyan
Coach, Bergen County Academies (NJ); former player for BCA (2000-03) and WUSTL (2003-07)
HSQB forum mod, PACE member
Stat director for: NSC '13-'15, '17; ACF '14, '17, '19; NHBB '13-'15; NASAT '11

"A [...] wizard who controls the weather" - Jerry Vinokurov
User avatar
Auks Ran Ova
Forums Staff: Chief Administrator
Posts: 4295
Joined: Sun Apr 30, 2006 10:28 pm
Location: Minneapolis
Contact:

Re: Player Poll

Post by Auks Ran Ova »

DUDE!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Rob Carson
University of Minnesota '11, MCTC '??, BHSU forever
Member, ACF
Member emeritus, PACE
Writer and Editor, NAQT
Edward Powers
Auron
Posts: 1107
Joined: Thu Sep 25, 2008 2:52 pm

Re: Player Poll

Post by Edward Powers »

Definitely not enough----what are they called again-------------------?????????????-----oh yes, HYPHENS!

EDIT: In re-reading this, I reflected deeply---for hours, in fact--- and took Jon Pinyan's advice to heart, so I edited the above post and added 3 hyphens, just so everyone would know that this is really MY post and NOT Mikle Cheyne's, about whose post above see my commentary in the post below.
Last edited by Edward Powers on Thu Jun 30, 2011 3:17 pm, edited 1 time in total.
Ed Powers
Coach
SJHS Academic Team
Metuchen, NJ
Edward Powers
Auron
Posts: 1107
Joined: Thu Sep 25, 2008 2:52 pm

Re: Player Poll

Post by Edward Powers »

Mike Cheyne---or is it Mike (Powers) Cheyne???,

Is it parody or imitation that is the kind of whimsy known to heal the soul of its foolish temptations to surrender to easy meta-insults and other sundry offenses?------Oh, I remember now---a screech or something sounding like Jon Pinyan's voice has whispered in my ear to help me----it is imitation that is the highest form of flattery, and it is parody that allows us to whimsically laugh at ourselves and thus purge our souls of their infinite capacities to make fools of ourselves--or something like that. So, thanks for the whimsy---or is it parody---or is it imitation?

BTW---Jeff Hoppes DOES have a legitimate beef! ( Wouldn't you agree, Andy Watkins???????????)!!!!!!.....

A Pre-editing EDIT( an unnecessary novelty, I know) : Do you think I should add or subtract some hyphens or other grammatical paraphanalia (sic), just to make sure that you're not confused with (or is it bemused by) me ?
Ed Powers
Coach
SJHS Academic Team
Metuchen, NJ
Magister Ludi
Tidus
Posts: 677
Joined: Thu Dec 01, 2005 1:57 am
Location: Washington DC

Re: Player Poll

Post by Magister Ludi »

Not to derail this "hilarious" thread with substantive comments, but reading all the discussion of the other rankings reminded me I wanted to post in this thread earlier. Basically, I think that whoever runs this poll should revamp the way it's done. As the poll stands now with only 7-8 ballots from people who self-select if they want to submit a ballot, it strikes me as basically meaningless. I think it would yield more valid results if the person running it would select a committee of 15-20 people who know college quizbowl well to cast ballots. Obviously, one would pick voters from geographically diverse regions who represent different methodologies/approaches to quizbowl (in the way Bruce would differ from Jerry). Also, if some sort of committee were chosen I think the people selected would feel inclined to actually cast a ballot so there would be an acceptable number of ballots.

I doubt some of the voters in this poll have the experience or frame of reference to really have any informed opinion about whether Kevin Koai is better than Aaron Rosenburg. For example, I think the only time I voted in this player poll was as a freshman when I cared more about these kinds of things, and I foolishly didn't include Selene in my top 25 because for aesthetic reasons I didn't want to have three people from the same Chicago team in my top 25. At that time, most of my poll was really guesswork. And while I don't doubt younger players like Matt Jackson or Matt Bollinger (or Zach Foster, I guess) know the circuit much better than I did as a freshman, I think it would generally be more meaningful to have people who have played/seen Selene at least a few times judging where she should be placed on a poll.

It's by no means critical to fix this poll, and I don't want to get into an argument about this issue. I simply wanted to make a suggestion.
Ted Gioia - Harvard '12
Editor ACF, PACE
User avatar
AKKOLADE
Sin
Posts: 15782
Joined: Thu Apr 24, 2003 8:08 am

Re: Player Poll

Post by AKKOLADE »

Magister Ludi wrote:Not to derail this "hilarious" thread with substantive comments, but reading all the discussion of the other rankings reminded me I wanted to post in this thread earlier. Basically, I think that whoever runs this poll should revamp the way it's done. As the poll stands now with only 7-8 ballots from people who self-select if they want to submit a ballot, it strikes me as basically meaningless. I think it would yield more valid results if the person running it would select a committee of 15-20 people who know college quizbowl well to cast ballots. Obviously, one would pick voters from geographically diverse regions who represent different methodologies/approaches to quizbowl (in the way Bruce would differ from Jerry). Also, if some sort of committee were chosen I think the people selected would feel inclined to actually cast a ballot so there would be an acceptable number of ballots.

...

It's by no means critical to fix this poll, and I don't want to get into an argument about this issue. I simply wanted to make a suggestion.
This is actually how the top 25 high school team poll was run by Matt, pre-dating the forums (I think extending into the forums a little bit? I'm not sure). I think this would lead to as many, if not more, sources of complaints as the current format.

I think the biggest problem with a "please send me ballots" approach is actually getting people who are knowledgeable to participate. There's a variety of reasons - not wanting to be yelled at by people because of 'undervaluing their team/players' seems to be a big one, with 'not wanting to look stupid because you forgot someone fairly obvious on your ballot' as a notable one too.
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
Magister Ludi
Tidus
Posts: 677
Joined: Thu Dec 01, 2005 1:57 am
Location: Washington DC

Re: Player Poll

Post by Magister Ludi »

I'm not suggesting the high school rankings be run this way, but the collegiate player poll which would have more semi-retired players who aren't affiliated with any teams. As much as anything, the benefit of picking a committee would be preventing unqualified people from voting or preventing too many voters from one region from voting.
Ted Gioia - Harvard '12
Editor ACF, PACE
User avatar
theMoMA
Forums Staff: Administrator
Posts: 5993
Joined: Mon Oct 23, 2006 2:00 am

Re: Player Poll

Post by theMoMA »

I remain unconvinced that even a prolific cast of ballot-casters knows what makes a player "good." My personal strategy has usually been to think about the players I'd least want to face , putting a little more emphasis on nationals-level questions than regular season (and of course, inflating my own ranking as much as reasonably possible).

Things that I think that people do in rankings that they shouldn't:
1. Ignore the regular season. Quizbowl happens in months that aren't May.
2. Run down the top scorers at nationals and ICT instead of thinking pragmatically about who's good.
3. Ignore individual playoff stats at ICT and emphasize them at Nationals. People do this because it's not easy to sort playoff stats in NAQT's system but playoff stats are the ones that show up when you look at the final rankings of teams at Nationals.
4. Place ICT at the same level as Nationals. I'm not saying that ICT is less legitimate, because nothing could be further from the truth. But there are three ACF-style tournaments for every NAQT tournament, so ability at the more prolific format should count for more.
5. Emphasize stats at all. Teammate effects cause drastic differences in playing style and buzzing opportunities; simply comparing stats across tournaments doesn't always give you the best idea of who's better.
Andrew Hart
Minnesota alum
User avatar
Cheynem
Sin
Posts: 7220
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Player Poll

Post by Cheynem »

Man, apparently posting a Ed Powers whimsical parody is the best way to generate discussion here.

One thing that I've always wondered on is what criteria people use when they vote. For instance, the simplest brute-force way of evaluating players is to ponder what would happen if they all played solo against each other and seeing who would do better. Another way might be to evaluate the quality of a player's game within context more. There are a few players who I would rank fairly well of which I'm not sure how they would fare as a solo player, but which work within their team so skillfully that I must give them credit. How much should we weigh performances in the "big games" as opposed to the regular season? What about skill on bonus questions which is not something which shows up in the stats? Which players possess the special skills at just playing the game independent of pure knowledge?

I am not sure if a blue-ribbon committee would produce dramatically different results, but I would be interested in seeing if such an institution could be formed.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
theMoMA
Forums Staff: Administrator
Posts: 5993
Joined: Mon Oct 23, 2006 2:00 am

Re: Player Poll

Post by theMoMA »

Cheynem wrote:One thing that I've always wondered on is what criteria people use when they vote. For instance, the simplest brute-force way of evaluating players is to ponder what would happen if they all played solo against each other and seeing who would do better. Another way might be to evaluate the quality of a player's game within context more. There are a few players who I would rank fairly well of which I'm not sure how they would fare as a solo player, but which work within their team so skillfully that I must give them credit. How much should we weigh performances in the "big games" as opposed to the regular season? What about skill on bonus questions which is not something which shows up in the stats? Which players possess the special skills at just playing the game independent of pure knowledge
I wrestle with the idea of whether to do rankings every year, but in the end, I feel a founder's obligation to carry on the tradition even as the better part of me says that they're dumb (which is why I stopped organizing them). For all the things that I've claimed are wrong with rankings, I will admit that it takes me about five minutes to make mine, and they're not scientific in any way. I tend to spit out a list of names, corroborate with nationals and ICT stats to make sure I'm not missing anyone, and then rank on a subjective scale of "goodness" (perhaps with downgrades for the chronically absent). I would say that my scale is about equal parts raw generalist ability on regular-difficulty questions, ability to contribute to meaningful team wins, and individual/team performance at the national tournaments. All of those things add up to a sort of fear factor: How much does the karma change when this player walks in the room, Nate Mattison style?
Andrew Hart
Minnesota alum
User avatar
Maximus
Lulu
Posts: 24
Joined: Tue Jul 07, 2009 6:53 am
Location: Pineville, KY

Re: Player Poll

Post by Maximus »

It seems to me that in almost every sport any type of player ranking is based almost entirely on the player's skill in isolation. If it is a team game, then rankings tend to try and remove as many effects from other players as possible in order to isolate how good the player would be if he played by himself or "one-on-one." Sometimes, in some sports more than others, there is perhaps a "winner" factor given to players who seem to inordinately boost the performance of any team they are a part of. When we are talking about quizbowl, skill is determined by how many questions a player in isolation can answer, and how early and quickly he can answer them. Some skill points are also awarded for how that player can do on bonuses, although our conceptions of skill in quizbowl seem to be much more heavily weighted towards tossup performance. So I believe when we want to do a ranking of players in quizbowl, we should look mostly at how they would do in a solo match against another player, and compare them. This is to say, I suppose, that a generalist type player would win most of the time against a specialist type player, and so we should rank generalists higher than specialists, in general. Now specialists are always extremely valuable to have on any team, and will always help you get points and help the team win. Perhaps like an elite point guard in basketball, who always makes the right passes and always motivates his team. Any team in quizbowl will have a hard time winning without some specialist type players. Buy when we are talking about skill at the GAME of quizbowl, I think almost any outside observer would say that generalists have more skill. Of course we should look at the depth of any generalists' knowledge, and some specialists who have deep knowledge will undoubtedly have more skill than some generalists with entirely superficial knowledge. I've just always thought that if one wants to be good at quizbowl, they should have to learn about all the subjects that are a part of it. To me that is the spirit of the game, and we should rank players who do so accordingly.

That turned out a bit longer than I had anticipated, haha.
Richard Mason, Yale
Magister Ludi
Tidus
Posts: 677
Joined: Thu Dec 01, 2005 1:57 am
Location: Washington DC

Re: Player Poll

Post by Magister Ludi »

theMoMA wrote:I remain unconvinced that even a prolific cast of ballot-casters knows what makes a player "good." My personal strategy has usually been to think about the players I'd least want to face , putting a little more emphasis on nationals-level questions than regular season (and of course, inflating my own ranking as much as reasonably possible).

Things that I think that people do in rankings that they shouldn't:
2. Run down the top scorers at nationals and ICT instead of thinking pragmatically about who's good.
3. Ignore individual playoff stats at ICT and emphasize them at Nationals. People do this because it's not easy to sort playoff stats in NAQT's system but playoff stats are the ones that show up when you look at the final rankings of teams at Nationals.
5. Emphasize stats at all. Teammate effects cause drastic differences in playing style and buzzing opportunities; simply comparing stats across tournaments doesn't always give you the best idea of who's better.
I was suggesting saying that an "experts' poll" or any voting form would yield perfect or even reliable rankings, but I think a poll would create better rankings. Just for the two reasons it would guarantee a certain minimum number of votes and it would prevent uninformed people from casting ballots overly based on ppgs and reputations--a trend Andrew rightfully disparages. I actually disagree with Mike's claim that the rankings would basically be the same because while the top couple players would certainly stay the same I think the 6-25 would be different

I agree with Andrew that some of the most negative trends in voting derive from emphasizing individual stats without thinking about stats critically. The highest I've ever been rated on this poll was after my freshman year when I put up a high ppg in the second bracket of ACF Nats with no one else on my team who knew anything about lit or arts. Generally overvaluing individual players who put on high ppgs in the second or lower brackets of ACF playoffs and not thinking about shadow effects are a consistent mistake. Just judging from my team--and I think Andy would agree--that Dallas is objectively the best player and he jumped from being 8th on this poll last year to 18th even though he put up the third highest number of powers at the ICT and had great playoff stats just because he had a bad run of seven games at the worst ACF Nationals in recent memory. Basically, his position swung radically based on one, completely non-representative set of 7 games.

While I'm talking about this issue, I also want to say that I think people look at power numbers the wrong way. I hear people putting blind faith in looking at someone's total number of powers without thinking about those numbers in context. It's significantly more impressive that Andrew and Rob powered 19 and 25 questions respectively at TIT playing on the same team than a generalist picking up 30 powers playing on a team that didn't have other people competing with him on humanities questions.

But as an extension of that line of reasoning, I think it's also much more impressive for someone who only buzzes on science questions (and can only power 4/20 questions) a packet to put up 14 powers in a tournament than a generalist to get 25 powers playing without great teammates in which that guy can feasibly power all the non-science questions in a packet. I've kept a stat I call "power percentage" for the Harvard team the last few tournaments we played, which records what percentage of tossups in a category someone powers. So someone powered X out of 10 myth tossups over the tournament; which gives a better sense for how good people really are in certain categories.
Ted Gioia - Harvard '12
Editor ACF, PACE
User avatar
Skepticism and Animal Feed
Auron
Posts: 3238
Joined: Sat Oct 30, 2004 11:47 pm
Location: Arlington, VA

Re: Player Poll

Post by Skepticism and Animal Feed »

The problem with the way that these rankings are done now is that you get a very small pool of self-selected voters. This is not because quizbowl people are lazy: quizbowl players may be lazy about writing questions, but they have an incredible work ethic when it comes to voicing opinions about quizbowl. Rather, it is because it is very costly to become informed about how good other quizbowl players are, especially those that you don't play regularly.

E.g., the reason that ACF Nats/ICT stats are weighed too much is that they're the one tournament where you can click on the stats and expect to see every major player in the country. To find ACF Regional stats for every major player in the country, it takes up to a half-dozen times as many clicks.

If any committee of people wants to create a better college player ranking, what they should do is create a central database of easy to find stats, perhaps sorted in some way. This still leaves open the entire debate about how stats should be interpreted, but it eliminates a major problem.
Bruce
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source
User avatar
Cheynem
Sin
Posts: 7220
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Player Poll

Post by Cheynem »

I actually wasn't saying the blue ribbon poll would produce the same results, I just wasn't sure.

I agree with Andrew and Ted that voters tend to be impressed with potentially empty stats. I'm certainly not saying I should have been ranked, but I felt like I played some of the best quizbowl of my life at the ICT playoffs, which--doesn't show up on the individual scoring page which only looks at prelims and was obviously overshadowed by my superior teammates. I bet there are way better players than me with similar "beefs."
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Ondes Martenot
Tidus
Posts: 688
Joined: Sat Sep 29, 2007 1:06 pm
Location: Troy, N.Y.

Re: Player Poll

Post by Ondes Martenot »

Maybe I'm looking at this the wrong way, but it seems like what I'm hearing is that people are essentially punishing players who don't have the luxury of strong teammates when they vote. In my opinion from personal experience, if you don't have strong teammates, it is a much better investment of your time to try to become a decent/competent player in a wide range of categories rather than being excellent in one category (you can of course be excellent in all categories, but that is a rather daunting task in the college game). This is of course results in a smaller number of powers but a lot more tens, which people tend to automatically assume is the mark of a weak player who is faking it, when I feel this may not actually be the case.
Aaron Cohen, Bergen County Academies '08, RPI '12, NYU-???, NAQT writer, HSAPQ writer, PACE writer
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: Player Poll

Post by grapesmoker »

The purpose of all ranking is to generate rankings threads in which discussions of those rankings occur. By that metric, these rankings are an unqualified success.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
User avatar
No Rules Westbrook
Auron
Posts: 1238
Joined: Mon Nov 22, 2004 1:04 pm

Re: Player Poll

Post by No Rules Westbrook »

Some combination of Bruce and Hart is where the truth lies with these rankings. Statistics are terrible approximators, but people use them because they don't have the information to pragmatically evaluate a lot of players that they don't see very often.

"Thinking pragmatically about what skill set a player has" would be what I would recommend as the best standard to evaluate.

I've always said that qb players are way too results-focused, and way too team-focused - and don't do a good job of evaluating individual players as atoms. Unless you believe there's such a thing as a "clutch player" or "Great team player" in quizbowl, you really should be looking at everyone in isolation, in a vacuum.
Ryan Westbrook, no affiliation whatsoever.

I am pure energy...and as ancient as the cosmos. Feeble creatures, GO!

Left here since birth...forgotten in the river of time...I've had an eternity to...ponder the meaning of things...and now I have an answer!
User avatar
Skepticism and Animal Feed
Auron
Posts: 3238
Joined: Sat Oct 30, 2004 11:47 pm
Location: Arlington, VA

Re: Player Poll

Post by Skepticism and Animal Feed »

No Rules Westbrook wrote: Unless you believe there's such a thing as a "clutch player" or "Great team player" in quizbowl, you really should be looking at everyone in isolation, in a vacuum.
This is an interesting question and something I think we should discuss. I've long noted that Jerry Vinokurov, in particular, seems to be able to make the players around him better. Maybe it's all a coincidence and he was blessed with multiple recruits who were willing to work hard and get better, but a lot of freshmen went into Brown unheralded and left as elite players.
Bruce
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: Player Poll

Post by grapesmoker »

The War (boxing) wrote:I've long noted that Jerry Vinokurov, in particular, seems to be able to make the players around him better.
I'm the Steve Nash of quizbowl.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
User avatar
Cheynem
Sin
Posts: 7220
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Player Poll

Post by Cheynem »

What has always impressed me is what I call the "sheer fighting power" of a player. Anyone can destroy a question on a topic near and dear to them, but what about being able to fight your way through a packet full of questions you don't know very well? The confidence and care to play in those circumstances can be pretty amazing, like playing with Andrew Hart when he knuckled down and plowed his way through some ICT questions or Rob Carson nailing a Nats question to send us to the finals. I've seen it in other players too--in the Nats final, Matt Jackson answered a number of questions against us in categories I know are not his main forte.

This also can perhaps be explained with the sports cliche of "hating to lose." I know nobody enjoys losing, but there are certain players I've played with, however briefly, or seen, that just hate losing so much that they will not give in even against all odds. Mike Sorice, Jerry Vinokurov, and Brendan Byrne are the players I've played with or seen who exemplify this in my mind, who took their games to almost another level when the odds were against them.

There are also players I just enjoy playing with independent of their quizbowl skills in that they make you feel like a better player through their encouragement and trust. Outside of my regular teammates, Seth Teitler (by far), Jonathan Magin, and Matt Lafer fit that pattern.

I'm certainly not saying anything earth-shattering by saying that Seth, Sorice, Jerry, Andrew, etc. are good players, but I think it's also key to think in my mind just why they are good independent of "they know a lot."
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Sima Guang Hater
Auron
Posts: 1958
Joined: Mon Feb 05, 2007 1:43 pm
Location: Nashville, TN

Re: Player Poll

Post by Sima Guang Hater »

The War (boxing) wrote:A lot of freshmen went into Brown unheralded and left as elite players.
we all wanted to beat him.
Eric Mukherjee, MD PhD
Brown 2009, Penn Med 2018
Instructor/Attending Physician/Postdoctoral Fellow, Vanderbilt University Medical Center
Coach, University School of Nashville

“The next generation will always surpass the previous one. It’s one of the never-ending cycles in life.”
Support the Stevens-Johnson Syndrome Foundation
User avatar
Cheynem
Sin
Posts: 7220
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Player Poll

Post by Cheynem »

I will hastily add that Matt Weiner may be the quintessential "fighting spirit" type player, having won a number of tournaments in dramatic fashion against the odds. I didn't put him on the list originally because I haven't seen him play as much directly as the other people.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
Locked