Crowdsourced Head to Head Rankings
- sephirothrr
- Quizbowl Detective Extraordinaire
- Posts: 210
- Joined: Sun Mar 23, 2008 11:00 pm
- Location: Louisville, KY
Crowdsourced Head to Head Rankings
While trying to rank my top 25 for Mike's midseason poll, I came upon a small problem - I'm really bad at ranking things in a list. To that end, I cobbled this little thing together.
Presenting the creatively named PriyaRank!
While I had designed it to help me find my top 25, I realized that one person alone would take a long time to simulate the necessary match results.
Since Quizbowl lives by the idea that the only way to rank teams is to throw them all in a giant round robin, that's just what I did!
The current rankings are on the home page, and you can contribute in the arena by clicking "Rank" and choosing a winner for a hypothetical match.
Over time, assuming good faith choices, the system will gradually approach what people believe to be the "true" rank, so I'm curious to see how this little experiment goes.
Presenting the creatively named PriyaRank!
While I had designed it to help me find my top 25, I realized that one person alone would take a long time to simulate the necessary match results.
Since Quizbowl lives by the idea that the only way to rank teams is to throw them all in a giant round robin, that's just what I did!
The current rankings are on the home page, and you can contribute in the arena by clicking "Rank" and choosing a winner for a hypothetical match.
Over time, assuming good faith choices, the system will gradually approach what people believe to be the "true" rank, so I'm curious to see how this little experiment goes.
Ramapriya
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Birdofredum Sawin wrote: ↑Fri Apr 04, 2014 1:52 pmIf you don't want to be regarded as a "raving lunatic," it might be advisable to rave less, or at least to do so in a less loony manner.
Re: Crowdsourced Head to Head Rankings
If nothing else, I suspect that playing around with this will help prevent people forgetting about teams/having regional availability bias when picking lower-ranked teams on their ballots.
Jacob R., ex-Chicago
- sephirothrr
- Quizbowl Detective Extraordinaire
- Posts: 210
- Joined: Sun Mar 23, 2008 11:00 pm
- Location: Louisville, KY
Re: Crowdsourced Head to Head Rankings
Interestingly enough, there's already a strong showing of people not being aware of the strengths of teams in various regions, especially in the middle of the pack.
Ramapriya
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Birdofredum Sawin wrote: ↑Fri Apr 04, 2014 1:52 pmIf you don't want to be regarded as a "raving lunatic," it might be advisable to rave less, or at least to do so in a less loony manner.
Re: Crowdsourced Head to Head Rankings
This is lots of fun.
Libo
Washington '14, Michigan '13, Troy High School '09
Washington '14, Michigan '13, Troy High School '09
Re: Crowdsourced Head to Head Rankings
I think you're probably neutering the true wisdom of the crowd a little bit by having the current record display below each team. Relative independence of opinions is important to getting better data in this kind of ranking.
Andrew Hart
Minnesota alum
Minnesota alum
- sephirothrr
- Quizbowl Detective Extraordinaire
- Posts: 210
- Joined: Sun Mar 23, 2008 11:00 pm
- Location: Louisville, KY
Re: Crowdsourced Head to Head Rankings
Yeah, I definitely noticed that, so that's gone now.theMoMA wrote:I think you're probably neutering the true wisdom of the crowd a little bit by having the current record display below each team. Relative independence of opinions is important to getting better data in this kind of ranking.
Ramapriya
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Birdofredum Sawin wrote: ↑Fri Apr 04, 2014 1:52 pmIf you don't want to be regarded as a "raving lunatic," it might be advisable to rave less, or at least to do so in a less loony manner.
- Excelsior (smack)
- Rikku
- Posts: 386
- Joined: Sun Jan 25, 2009 12:20 am
- Location: Madison, WI
Re: Crowdsourced Head to Head Rankings
This is awesome! (In retrospect, I can't believe nobody had thought of this before, seeing as it makes _so_ much sense!)
Ashvin Srivatsa
Corporate drone '?? | Yale University '14 | Sycamore High School (OH) '10
Corporate drone '?? | Yale University '14 | Sycamore High School (OH) '10
Re: Crowdsourced Head to Head Rankings
One possible drawback (and, of course, drawback to every ranking system) is imperfect information about lineups. If all rankers were considering "Minnesota" to be a team with Mike Cheyne on it, as it may be for ICT, I think that Minnesota would do a lot better in the ranks. But I agree that this is a very, very good idea that, in my opinion, has already produced very credible results.
Andrew Hart
Minnesota alum
Minnesota alum
-
- Auron
- Posts: 2248
- Joined: Sat Jan 10, 2009 1:08 am
- Location: Kentucky
Re: Crowdsourced Head to Head Rankings
Considering I've voted against Minnesota most times they've come up, this is probably a pertinent point.theMoMA wrote: If all rankers were considering "Minnesota" to be a team with Mike Cheyne on it, as it may be for ICT, I think that Minnesota would do a lot better in the ranks.
Nicholas C
KQBA member
KQBA member
Re: Crowdsourced Head to Head Rankings
This is cool.
Are you goofy enough to do this for high schoolers?
Are you goofy enough to do this for high schoolers?
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
- sephirothrr
- Quizbowl Detective Extraordinaire
- Posts: 210
- Joined: Sun Mar 23, 2008 11:00 pm
- Location: Louisville, KY
Re: Crowdsourced Head to Head Rankings
This deployment was really just a Phase I proof of concept. By the end of the day I should be ready to deploy Phase II, which is more in line with what I believe are the goals of the original rating system - namely, records and ratings will be hidden until Mike Cheyne publishes his results, at which point I shall publish mine. Hopefully, that will allow people to choose winners based on their own opinion of relative skill and not possibly incorrect trends in the data. (read: trolls)
It wouldn't be any extra work on the software side, if, say, someone wanted to provide me with a text file of relevant teams, one team per line...Grams's Go-Go Boots wrote:This is cool.
Are you goofy enough to do this for high schoolers?
Ramapriya
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Birdofredum Sawin wrote: ↑Fri Apr 04, 2014 1:52 pmIf you don't want to be regarded as a "raving lunatic," it might be advisable to rave less, or at least to do so in a less loony manner.
Re: Crowdsourced Head to Head Rankings
I just sent you a list of all the schools that have entered an NAQT tournament this year for which we have results.
Jonah Greenthal
National Academic Quiz Tournaments
National Academic Quiz Tournaments
-
- Auron
- Posts: 2248
- Joined: Sat Jan 10, 2009 1:08 am
- Location: Kentucky
Re: Crowdsourced Head to Head Rankings
A pool of hundreds of high school teams is going to lead to a lot of people making selections on scant information. There may be some interesting results however.
Nicholas C
KQBA member
KQBA member
Re: Crowdsourced Head to Head Rankings
In that case, it might be better just to include teams that have signed up for HSNCT or NSC.Sulawesi Myzomela wrote:A pool of hundreds of high school teams is going to lead to a lot of people making selections on scant information. There may be some interesting results however.
Tanay
ex-Berkeley, ex-Bellarmine
ex-Berkeley, ex-Bellarmine
Re: Crowdsourced Head to Head Rankings
I could just post my stat file.
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
- sephirothrr
- Quizbowl Detective Extraordinaire
- Posts: 210
- Joined: Sun Mar 23, 2008 11:00 pm
- Location: Louisville, KY
Re: Crowdsourced Head to Head Rankings
Sweet! However, while setting that up, I made a realization - with that many teams, my purely random approach to generating matches would fail due to the staggeringly high number of simulation matches needed. Unfortunately, the HSNCT field will have a similar effect. While that's a problem I intend to fix, in the meantime, I'm looking to cap the field size at 100 teams. However, I know very little about the high school quiz bowl landscape, so I've decided to take the top 100 teams from Fred's HSQBRank, unless anyone more knowledgeable than me can offer a suggestion.jonah wrote:I just sent you a list of all the schools that have entered an NAQT tournament this year for which we have results.
Ramapriya
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Birdofredum Sawin wrote: ↑Fri Apr 04, 2014 1:52 pmIf you don't want to be regarded as a "raving lunatic," it might be advisable to rave less, or at least to do so in a less loony manner.
Re: Crowdsourced Head to Head Rankings
You could also consider doing something like this: http://www.basketball-reference.com/about/elo.html in terms of match-up control.
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
- The Ununtiable Twine
- Auron
- Posts: 1058
- Joined: Fri Feb 02, 2007 11:09 pm
- Location: Lafayette, LA
Re: Crowdsourced Head to Head Rankings
There are so many interesting things that could come of this, say if you make everyone make an account and give some additional info to sign into this. With enough data on each individual voter you can surely weed out trolls. You can also develop a national ranking as well as regional ones, and...get this...we may finally have a way to measure regional bias and prove that it exists and to what degree! I still think you should start with a blank slate and not show the results to anyone, then compare to the Cheyne poll. You can share the results of certain individuals that participated in both PriyaRank and the Cheyne poll to see if their ballots were indeed consistent with how they participated in PriyaRank to get even more interesting results. This is a rather fantabulous thing and I love it very very much.
EDIT: You could even ask people to tell you how much emphasis they put on ACF/mACF with respect to NAQT or you could even ask who would win on NAQT or ACF/mACF. There are so many things that you can do to expand this and get super-interesting data.
EDIT: You could even ask people to tell you how much emphasis they put on ACF/mACF with respect to NAQT or you could even ask who would win on NAQT or ACF/mACF. There are so many things that you can do to expand this and get super-interesting data.
Jake Sundberg
Louisiana, Alabama
retired
Louisiana, Alabama
retired
- The Ununtiable Twine
- Auron
- Posts: 1058
- Joined: Fri Feb 02, 2007 11:09 pm
- Location: Lafayette, LA
Re: Crowdsourced Head to Head Rankings
Additionally, it might be noted that Yogesh plays for New Mexico State so I doubt they're the second-worst team here. A complete list of rosters somewhere may also help with this - perhaps you could make that available on your site?
Jake Sundberg
Louisiana, Alabama
retired
Louisiana, Alabama
retired
Re: Crowdsourced Head to Head Rankings
Rosters would definitely be a good thing to show. Even better would be to have some sort of statistical summary like on the Basketball-Reference and Baseball-Reference Elo Raters, although I don't know how much effort it would take to maintain something like that.The Ununtiable Twine wrote:Additionally, it might be noted that Yogesh plays for New Mexico State so I doubt they're the second-worst team here. A complete list of rosters somewhere may also help with this - perhaps you could make that available on your site?
As is, this is still a really neat thing to have.
Andrew Nadig
Mannhiem Mannheim Manheim Township, 2005-11
Carnegie Mellon University, 2011-15
Carnegie Mellon University, 2011-15
Re: Crowdsourced Head to Head Rankings
I agree with both points on this post.Ndg wrote:Rosters would definitely be a good thing to show. Even better would be to have some sort of statistical summary like on the Basketball-Reference and Baseball-Reference Elo Raters, although I don't know how much effort it would take to maintain something like that.The Ununtiable Twine wrote:Additionally, it might be noted that Yogesh plays for New Mexico State so I doubt they're the second-worst team here. A complete list of rosters somewhere may also help with this - perhaps you could make that available on your site?
As is, this is still a really neat thing to have.
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
- The Ununtiable Twine
- Auron
- Posts: 1058
- Joined: Fri Feb 02, 2007 11:09 pm
- Location: Lafayette, LA
Re: Crowdsourced Head to Head Rankings
Well, it looks like you'll have to find a different twitter handle for PriyaRank.
https://twitter.com/priyarank
What the hell does any of this mean? I wish Jerry were around to translate - I see something about Angry Birds...Angry Russian Birds.
https://twitter.com/priyarank
What the hell does any of this mean? I wish Jerry were around to translate - I see something about Angry Birds...Angry Russian Birds.
Jake Sundberg
Louisiana, Alabama
retired
Louisiana, Alabama
retired
- The Ununtiable Twine
- Auron
- Posts: 1058
- Joined: Fri Feb 02, 2007 11:09 pm
- Location: Lafayette, LA
Re: Crowdsourced Head to Head Rankings
Dartmouth (39), Louisville (47), UVA (10), Penn (9) be gettin' trolled! Meanwhile, someone also hates Rob and Bernadette (32)! Could someone at Vassar (24) or Claremont (15) possibly be behind this???
Last edited by The Ununtiable Twine on Tue Feb 25, 2014 1:04 am, edited 1 time in total.
Jake Sundberg
Louisiana, Alabama
retired
Louisiana, Alabama
retired
-
- Auron
- Posts: 2248
- Joined: Sat Jan 10, 2009 1:08 am
- Location: Kentucky
Re: Crowdsourced Head to Head Rankings
This sounds like it could be a...The Ununtiable Twine wrote:someone also hates Rob and Bernadette (32)
Conspiracy theory.
Nicholas C
KQBA member
KQBA member
- Good Goblin Housekeeping
- Auron
- Posts: 1100
- Joined: Sun May 23, 2010 10:03 am
Re: Crowdsourced Head to Head Rankings
This is really neat! I'm a bit confused on a few things though, namely how stuff like this happens
Andrew Wang
Illinois 2016
Illinois 2016
-
- Wakka
- Posts: 187
- Joined: Sat Nov 12, 2011 5:09 pm
Re: Crowdsourced Head to Head Rankings
The Ununtiable Twine wrote:There are so many interesting things that could come of this, say if you make everyone make an account and give some additional info to sign into this. With enough data on each individual voter you can surely weed out trolls.
EDIT: You could even ask people to tell you how much emphasis they put on ACF/mACF with respect to NAQT or you could even ask who would win on NAQT or ACF/mACF. There are so many things that you can do to expand this and get super-interesting data.
I agree with both of these things. I was tempted to help UCF out by saying we'd beat Yale, but I didn't. I might just be an exceptionally bad person, but it was tempting. And having done surveys for undergrad research, I can promise you that there are people sad and lonely enough to lie for the fun of it. Maryland B being ranked 3 above Maryland A might be an example of not enough people voting, but it might also be that.
I also think that some teams are a lot better at ACF than NAQT. We played Alabama very close a few years back on NAQT tossups (losing on the last tossup of course). Then when they removed a lot of American current events and Jesus tossups and geography, we got crushed at ACF. It might be fun to try to measure A) how much better teams actually are at one format than the other and B) how people perceive how much better some teams are than others on one format than the other.
I really do like the system, and I'll be interested to see how this turns out at the end when it evens out.
And then there's also the fact that you have people like me rating, and really, why would I know if Minneanapolis CTC is better than Alabama? I mean, I feel like they are (and honestly that maybe we shouldn't be ahead of either but maybe should be ahead of Vasser or WUSTL, but honestly I've never seen Rob Carson or any of the Vasser players in my life and I wouldn't know.) There's only a handful of players who have been at enough national tournaments to really say this for sure.
But like I said, I do like the system and I'll be very interested in where it goes.
Edit: Since I started that reply, Yale took a nose dive and Virginia isn't moving up. UCF is in the top 20. I think that's trolls at work?
Bradley Kirksey
Mayor of quiz bowl at the University of Central Florida (2010-2015)
The club at Reformed Theology Seminary Orlando (2017 - 2021)
Mayor of quiz bowl at the University of Central Florida (2010-2015)
The club at Reformed Theology Seminary Orlando (2017 - 2021)
- The Ununtiable Twine
- Auron
- Posts: 1058
- Joined: Fri Feb 02, 2007 11:09 pm
- Location: Lafayette, LA
Re: Crowdsourced Head to Head Rankings
That was at a combined field on D2 questions, but it was a very fun match indeed!bradleykirksey wrote: I also think that some teams are a lot better at ACF than NAQT. We played Alabama very close a few years back on NAQT tossups (losing on the last tossup of course).
EDIT: Perhaps I could attribute this to your team having an unusual power spike and much higher PPB than in previous rounds, but I am not Joseph Nation and hence I will not overanalyze the results of a combined-field SCT or attribute that close game to anything other than that particular game being close, however many Jesus questions may or may not have been in the packet.
Last edited by The Ununtiable Twine on Tue Feb 25, 2014 12:18 am, edited 2 times in total.
Jake Sundberg
Louisiana, Alabama
retired
Louisiana, Alabama
retired
-
- Auron
- Posts: 2248
- Joined: Sat Jan 10, 2009 1:08 am
- Location: Kentucky
Re: Crowdsourced Head to Head Rankings
Maybe the Claremont administration found out about this and decided to make their team look better? After all, they do cheat at college rankings.
Nicholas C
KQBA member
KQBA member
Re: Crowdsourced Head to Head Rankings
Yeah, this might need to be implemented for this to retain any usefulness, in light of the fact that someone has invested enough time in trolling this to give Claremont a sizable lead at #1. (And to put Maryland B at #4, four spots ahead of Maryland A.)The Ununtiable Twine wrote:There are so many interesting things that could come of this, say if you make everyone make an account and give some additional info to sign into this.
Andrew Nadig
Mannhiem Mannheim Manheim Township, 2005-11
Carnegie Mellon University, 2011-15
Carnegie Mellon University, 2011-15
- Frater Taciturnus
- Auron
- Posts: 2463
- Joined: Mon Dec 12, 2005 1:26 pm
- Location: Richmond, VA
Re: Crowdsourced Head to Head Rankings
*sigh*
This... this is why we can't have nice things.
This... this is why we can't have nice things.
Janet Berry
[email protected]
she/they
--------------
J. Sargeant Reynolds CC 2008, 2009, 2014
Virginia Commonwealth 2010, 2011, 2012, 2013,
Douglas Freeman 2005, 2006, 2007
[email protected]
she/they
--------------
J. Sargeant Reynolds CC 2008, 2009, 2014
Virginia Commonwealth 2010, 2011, 2012, 2013,
Douglas Freeman 2005, 2006, 2007
- The Ununtiable Twine
- Auron
- Posts: 1058
- Joined: Fri Feb 02, 2007 11:09 pm
- Location: Lafayette, LA
Re: Crowdsourced Head to Head Rankings
*cough 340-100 *cough - it is I who am driving Rob and Bernadette's ranking into the ground! There is simply no other explanation as to why the indefeatable Minneanapolis CTC's team could be doing so badly in the Rangaraju rankings!bradleykirksey wrote:
And then there's also the fact that you have people like me rating, and really, why would I know if Minneanapolis CTC is better than Alabama?
In all seriousness let's be a little more serious about seeing what kind of results we get when the PriyaRank gets reset. I think we're all curious to see how this correlates to what comes out of our normal polling methods.Frater Taciturnus wrote:*sigh*
This... this is why we can't have nice things.
Jake Sundberg
Louisiana, Alabama
retired
Louisiana, Alabama
retired
-
- Wakka
- Posts: 187
- Joined: Sat Nov 12, 2011 5:09 pm
Re: Crowdsourced Head to Head Rankings
It stayed about a tossup apart the entire time. The only way it could have been more fun was if we got the last one of course, lol.The Ununtiable Twine wrote:That was at a combined field on D2 questions, but it was a very fun match indeed!bradleykirksey wrote: I also think that some teams are a lot better at ACF than NAQT. We played Alabama very close a few years back on NAQT tossups (losing on the last tossup of course).
As for Minnesota CTC, I didn't realize that actually. I just remember them finishing one above you at ICT and not losing any starters. But since I've never seen them play, I'm really only going off of one tournament ever. Which isn't too fair. I basically have no clue what I'm talking about, and you've got people like me voting. (Though I feel like the actual Mike Cheyne polls have the same problem. We murdered WUSTL last year something like 305-100 and finished ahead of them at ICT. We rightfully got no votes in the poll but they got ranked 25th. No offense to Charles Hang, but I still don't understand why people voted for them over the only teams that finished ahead of us both.)
And Andrew and George, if UCF is ever ahead of Chicago, there's something seriously wrong. I don't think this can be a small sample size problem.
Edited because I am stupid
Edit part 2: We had a mild power spike (tied for 3rd most on the day) and relaly barely a PPB spike (less than 2.0 higher than on the day average) but we didn't neg ourselves into oblivion like usual. We only negged twice against you guys, compared to 8 times against our C team
Last edited by bradleykirksey on Tue Feb 25, 2014 7:06 am, edited 1 time in total.
Bradley Kirksey
Mayor of quiz bowl at the University of Central Florida (2010-2015)
The club at Reformed Theology Seminary Orlando (2017 - 2021)
Mayor of quiz bowl at the University of Central Florida (2010-2015)
The club at Reformed Theology Seminary Orlando (2017 - 2021)
Re: Crowdsourced Head to Head Rankings
I really hope you guys aren't surprised by this.
Edit: although if someone set up a script to do it and took the site down with it, that'd be pretty annoying.
Edit: although if someone set up a script to do it and took the site down with it, that'd be pretty annoying.
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
- The Ununtiable Twine
- Auron
- Posts: 1058
- Joined: Fri Feb 02, 2007 11:09 pm
- Location: Lafayette, LA
Re: Crowdsourced Head to Head Rankings
I think we would have been surprised if it DIDN'T happen.Grams's Go-Go Boots wrote:I really hope you guys aren't surprised by this.
Edit: although if someone set up a script to do it and took the site down with it, that'd be pretty annoying.
Jake Sundberg
Louisiana, Alabama
retired
Louisiana, Alabama
retired
-
- Wakka
- Posts: 187
- Joined: Sat Nov 12, 2011 5:09 pm
Re: Crowdsourced Head to Head Rankings
Surprised isn't the word. I think disappointed is.Grams's Go-Go Boots wrote:I really hope you guys aren't surprised by this.
Edit: although if someone set up a script to do it and took the site down with it, that'd be pretty annoying.
Bradley Kirksey
Mayor of quiz bowl at the University of Central Florida (2010-2015)
The club at Reformed Theology Seminary Orlando (2017 - 2021)
Mayor of quiz bowl at the University of Central Florida (2010-2015)
The club at Reformed Theology Seminary Orlando (2017 - 2021)
-
- Auron
- Posts: 2248
- Joined: Sat Jan 10, 2009 1:08 am
- Location: Kentucky
Re: Crowdsourced Head to Head Rankings
FWIW, Ram and I basically talked about trolls hitting the site as soon as he set it up, and I think this was basically just a demonstration version anyway.bradleykirksey wrote:Surprised isn't the word. I think disappointed is.Grams's Go-Go Boots wrote:I really hope you guys aren't surprised by this.
Edit: although if someone set up a script to do it and took the site down with it, that'd be pretty annoying.
Nicholas C
KQBA member
KQBA member
-
- Auron
- Posts: 2248
- Joined: Sat Jan 10, 2009 1:08 am
- Location: Kentucky
Re: Crowdsourced Head to Head Rankings
I think those fuckers did it. Good job guys, you crashed a quizbowl alliance's website so you can have fun.Grams's Go-Go Boots wrote:I really hope you guys aren't surprised by this.
Edit: although if someone set up a script to do it and took the site down with it, that'd be pretty annoying.
Nicholas C
KQBA member
KQBA member
-
- Wakka
- Posts: 187
- Joined: Sat Nov 12, 2011 5:09 pm
Re: Crowdsourced Head to Head Rankings
I guess that makes me feel a little better, but I'm definitely all about Jake's suggestion of having to log in so we can keep out trolls.
When we do that, I'll be very interested by the data we get from it.
When we do that, I'll be very interested by the data we get from it.
Bradley Kirksey
Mayor of quiz bowl at the University of Central Florida (2010-2015)
The club at Reformed Theology Seminary Orlando (2017 - 2021)
Mayor of quiz bowl at the University of Central Florida (2010-2015)
The club at Reformed Theology Seminary Orlando (2017 - 2021)
- sephirothrr
- Quizbowl Detective Extraordinaire
- Posts: 210
- Joined: Sun Mar 23, 2008 11:00 pm
- Location: Louisville, KY
Re: Crowdsourced Head to Head Rankings
I knew as soon as I posted this that it was only a matter of time until it went to shit.
Well, now that everyone's had their fun, looks like it's time to reset the rankings and add some security. And also boot Claremont off the list.
Well, now that everyone's had their fun, looks like it's time to reset the rankings and add some security. And also boot Claremont off the list.
Ramapriya
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Birdofredum Sawin wrote: ↑Fri Apr 04, 2014 1:52 pmIf you don't want to be regarded as a "raving lunatic," it might be advisable to rave less, or at least to do so in a less loony manner.
- sephirothrr
- Quizbowl Detective Extraordinaire
- Posts: 210
- Joined: Sun Mar 23, 2008 11:00 pm
- Location: Louisville, KY
Re: Crowdsourced Head to Head Rankings
Okay, either this is a strange coincidence, or they took down my web hosting provider as well.
Edit: No wait, I gave them too much credit. While they did destroy the rankings, it wasn't them who took down the site. My provider had a scheduled downtime from 11:00 - 12:00 EST tonight.
Edit: No wait, I gave them too much credit. While they did destroy the rankings, it wasn't them who took down the site. My provider had a scheduled downtime from 11:00 - 12:00 EST tonight.
Last edited by sephirothrr on Tue Feb 25, 2014 12:53 am, edited 1 time in total.
Ramapriya
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Birdofredum Sawin wrote: ↑Fri Apr 04, 2014 1:52 pmIf you don't want to be regarded as a "raving lunatic," it might be advisable to rave less, or at least to do so in a less loony manner.
-
- Wakka
- Posts: 187
- Joined: Sat Nov 12, 2011 5:09 pm
Re: Crowdsourced Head to Head Rankings
I'm assuming that that part's a joke, right? They're definitely not the worst team on the list.sephirothrr wrote: And also boot Claremont off the list.
Sorry though. It's frustrating when trolls break nice things.
Edit: Took down the whole provider? Forgive my stupid social-science-major question, but is that possible?
Edit x 2: Well good. I couldn't imagine the logistics of that, but I would think that would be pretty terrible.
Last edited by bradleykirksey on Tue Feb 25, 2014 12:34 am, edited 1 time in total.
Bradley Kirksey
Mayor of quiz bowl at the University of Central Florida (2010-2015)
The club at Reformed Theology Seminary Orlando (2017 - 2021)
Mayor of quiz bowl at the University of Central Florida (2010-2015)
The club at Reformed Theology Seminary Orlando (2017 - 2021)
- sephirothrr
- Quizbowl Detective Extraordinaire
- Posts: 210
- Joined: Sun Mar 23, 2008 11:00 pm
- Location: Louisville, KY
Re: Crowdsourced Head to Head Rankings
I'm not actually going to remove them. Probably.bradleykirksey wrote:I'm assuming that that part's a joke, right? They're definitely not the worst team on the list.sephirothrr wrote: And also boot Claremont off the list.
Sorry though. It's frustrating when trolls break nice things.
Edit: Took down the whole provider? Forgive my stupid social-science-major question, but is that possible?
Ramapriya
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Birdofredum Sawin wrote: ↑Fri Apr 04, 2014 1:52 pmIf you don't want to be regarded as a "raving lunatic," it might be advisable to rave less, or at least to do so in a less loony manner.
- sephirothrr
- Quizbowl Detective Extraordinaire
- Posts: 210
- Joined: Sun Mar 23, 2008 11:00 pm
- Location: Louisville, KY
Re: Crowdsourced Head to Head Rankings
Well, since I have to wait for my database to come back online, what else would you like to see added? I've fixed the exploit that was used to screw up the ranking, and I've added Google/Facebook authentication to hopefully keep people honest. I'm a little torn on hiding rankings, because while it might help the results a little, I'm afraid that people will be less likely to participate if they can't see the effect they're having.
Ramapriya
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Birdofredum Sawin wrote: ↑Fri Apr 04, 2014 1:52 pmIf you don't want to be regarded as a "raving lunatic," it might be advisable to rave less, or at least to do so in a less loony manner.
-
- Wakka
- Posts: 187
- Joined: Sat Nov 12, 2011 5:09 pm
Re: Crowdsourced Head to Head Rankings
My guess would be that keeping it private would be better for getting peoples' actual opinions on rankings. Keeping it public would make it a much more fun little toy. The call is yours I guess, but I think that that's what the decision boils down to.sephirothrr wrote:Well, since I have to wait for my database to come back online, what else would you like to see added? I've fixed the exploit that was used to screw up the ranking, and I've added Google/Facebook authentication to hopefully keep people honest. I'm a little torn on hiding rankings, because while it might help the results a little, I'm afraid that people will be less likely to participate if they can't see the effect they're having.
Good news guys! Cleremont is about to get passed again! By Truman State!
Bradley Kirksey
Mayor of quiz bowl at the University of Central Florida (2010-2015)
The club at Reformed Theology Seminary Orlando (2017 - 2021)
Mayor of quiz bowl at the University of Central Florida (2010-2015)
The club at Reformed Theology Seminary Orlando (2017 - 2021)
Re: Crowdsourced Head to Head Rankings
The best apart this is it outs people who waste their time voting for themselves on these things!
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
Re: Crowdsourced Head to Head Rankings
Mike instructed Top-25 voters to assume that each school had its full A-team. Richard Yu played D2 solo at the ICT, so the D1 WUSTL team was significantly weaker.bradleykirksey wrote: We murdered WUSTL last year something like 305-100 and finished ahead of them at ICT. We rightfully got no votes in the poll but they got ranked 25th. No offense to Charles Hang, but I still don't understand why people voted for them over the only teams that finished ahead of us both.)
Matt Hayes
Unaffiliated
Unaffiliated
- Muriel Axon
- Tidus
- Posts: 729
- Joined: Wed Mar 21, 2012 12:19 am
Re: Crowdsourced Head to Head Rankings
What determines the rating and the rank? Why is Texas A&M (126/423/25 currently) above Maryland B (494/281/19)?
Shan Kothari
Plymouth High School '10
Michigan State University '14
University of Minnesota '20
Plymouth High School '10
Michigan State University '14
University of Minnesota '20
- sephirothrr
- Quizbowl Detective Extraordinaire
- Posts: 210
- Joined: Sun Mar 23, 2008 11:00 pm
- Location: Louisville, KY
Re: Crowdsourced Head to Head Rankings
It uses the Elo system, so who you play matters as well. In this case, people have figured out how to exploit the system and choose specific matches, causing the rankings to be out of order.
Ramapriya
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Birdofredum Sawin wrote: ↑Fri Apr 04, 2014 1:52 pmIf you don't want to be regarded as a "raving lunatic," it might be advisable to rave less, or at least to do so in a less loony manner.
- Whiter Hydra
- Auron
- Posts: 1418
- Joined: Tue Dec 04, 2007 8:46 pm
- Location: Fairfax, VA
- Contact:
Re: Crowdsourced Head to Head Rankings
In addition, the Elo system only cares about your current rating, not how many games you've played. So if a team with a record of 100-400 were to suddenly beat a team with a record of 400-100 10 times in a row, the former's Elo will most likely shoot up past the latter's.
Harry White
TJHSST '09, Virginia Tech '13
Owner of Tournament Database Search and Quizbowl Schedule Generator
Will run stats for food
TJHSST '09, Virginia Tech '13
Owner of Tournament Database Search and Quizbowl Schedule Generator
Will run stats for food
- sephirothrr
- Quizbowl Detective Extraordinaire
- Posts: 210
- Joined: Sun Mar 23, 2008 11:00 pm
- Location: Louisville, KY
Re: Crowdsourced Head to Head Rankings
Also true, but in theory if users were acting in good faith it wouldn't matter.
Ramapriya
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Birdofredum Sawin wrote: ↑Fri Apr 04, 2014 1:52 pmIf you don't want to be regarded as a "raving lunatic," it might be advisable to rave less, or at least to do so in a less loony manner.