ANNOUNCEMENT: Gaddis Experiment II

[yoda4554]

I'm probably in for this too.

[Maxwell Sniffingwell]

Yep.

Looks like I won't be at HSNCT, so I'm out.

[ulls66]

I'd like to play in this.

[powerplant]

I'd like to sign up.

[Strongside]

I would like to sign up for this. It looks like fun.

Also, has a decision been made on

1. Whether the tournament will be tossups only or tossups and bonuses.

2. How points will be scored, and how many buzzes/negs will be allowed on each question, etc.

3. What time the tournament will start.

4. How teams will be formed. I am assuming we can pick our teams.

Those four things won't affect whether I play or not, but I figured other people were also curious.

[Deckard Cain]

I'd like to play this as well.

[Sir Thopas]

Yeah, I'm not making this. Drat.

[swwFCqb]

I just realized I didn't post in this thread, but I'm definately up for playing this.

[magin]

I would like to sign up for this. It looks like fun.

Also, has a decision been made on

1. Whether the tournament will be tossups only or tossups and bonuses.

2. How points will be scored, and how many buzzes/negs will be allowed on each question, etc.

3. What time the tournament will start.

4. How teams will be formed. I am assuming we can pick our teams.

Those four things won't affect whether I play or not, but I figured other people were also curious.

Sorry about being so late with my responses; I've been writing literature tossups on the Diels-Alder reaction. Here are some answers to your questions, Brendan:

The tournament will be tossup only, to speed things along. There will be superpowers and powers (worth 30 and 20 points, respectively), and negs worth -5. I'm not sure if there should be a limit to buzzes per question, because if six people neg, then everyone just got -30 for that question (which should be an incentive not to neg). Then again, would people like some restriction on the amount of buzzes per tossup? I'm amenable to suggestions.

I would like the tournament to start by 7:30 PM Saturday night; does that seem reasonable?

There won't be teams for this tournament in the usual sense; instead, everyone will pick a number out of a hat (or some random process of assigning numbers), and you'll play individually with random teammates that change each round. For instance, for 64 people, if you draw number 1, you could possibly play with players 2, 3, 4, 5, 6, 7, and 8, or players 13, 35, 60, 3, 44, 11, and 49. For every round, your score equals your individual score plus the total points the whole room scored.

I've set up a public spreadsheet to sign up for this tournament, also. Please only sign up if you're going to play (since if you sign up and don't play, it screws with the schedule).
http://spreadsheets.google.com/ccc?key= ... hG0gdJb5OA

[Mike Bentley]

This format seems even worse than before. By the luck of the draw the best player in this tournament can lose because he is playing crappy players all day (thus, less overall points). Additionally, someone can effectively clinch the tournament well before the final match, even if they haven't played the other top players yet. Finally, scorekeeping this is going to be a pain in the ass if there are more than 1 negs per team. You won't be able to do it with any computer programs I know of, and you might even need to make custom scoresheets.

Why, again, isn't this just a regular tournament?

[magin]

Mike, I originally proposed allowing people to form teams of their choice, but people pooh-poohed that idea. Would that be better?

[at your pleasure]

Or you could just keep the teams the same throughout the tournament.

[magin]

Or you could just keep the teams the same throughout the tournament.

Yes, that was the original proposal I was referring to.

[cvdwightw]

This format seems even worse than before. By the luck of the draw the best player in this tournament can lose because he is playing crappy players all day (thus, less overall points). Additionally, someone can effectively clinch the tournament well before the final match, even if they haven't played the other top players yet.

Ideally, each person plays at least one round in a room with each other person, so the "playing crappy players all day" argument won't hold. There will be presumably some kind of randomness issue in the "extra" people each person plays with, but I don't think it's as bad as you're making it out to be.

[Important Bird Area]

Agree with Dwight that the problem here is the "purely random" selection method, not the concept of rotating teams and collective points. I'm reasonably sure there's a fair way to work this, although I don't particularly care to devise it myself.

Actually, it might look quite a bit like the card system used by the HSNCT itself.

[Mike Bentley]

I guess I misunderstood the format a bit, but I still think everyone would be better served with a regular format.

For instance, negative performance can have an impact beyond one's own team, which goes against the idea of having the most knowledgable team win. Say I'm playing this format in a game with Jonathan Magin. I obviously am not going to win this tournament, but it's possible Jonathan might. However, I have a grudge with Jonathan for making me play this format. Thus, I be a bad sport and neg 20 times in this game, artificially lowering Jonathan's score by at least 100 points (I would automatically be losing him 100 points, and would also lose him the value of any questions I would have otherwise gotten before anyone else in the room would have gotten them). I could also be less of an obvious jerk and just not buzz on things I know, which would still serve to hurt Jonathan's prospects of winning. In a regular match, the only people I'd be hurting my doing this type of stuff would be my own team.

[magin]

I guess I misunderstood the format a bit, but I still think everyone would be better served with a regular format.

For instance, negative performance can have an impact beyond one's own team, which goes against the idea of having the most knowledgable team win. Say I'm playing this format in a game with Jonathan Magin. I obviously am not going to win this tournament, but it's possible Jonathan might. However, I have a grudge with Jonathan for making me play this format. Thus, I be a bad sport and neg 20 times in this game, artificially lowering Jonathan's score by at least 100 points (I would automatically be losing him 100 points, and would also lose him the value of any questions I would have otherwise gotten before anyone else in the room would have gotten them). I could also be less of an obvious jerk and just not buzz on things I know, which would still serve to hurt Jonathan's prospects of winning. In a regular match, the only people I'd be hurting my doing this type of stuff would be my own team.

Well, there are two or three reasons I don't think that will happen. First of all, people like answering tossups correctly and buzzing in on stuff they know, which negging 20 times will prevent them from doing. Secondly, out of pure self-interest, you're losing 100 points by doing that, not to mention the points that someone might get for answering the tossup correctly. Finally, deliberately cutting off your points to spite one person will incur a huge amount of disfavor and pretty much everyone will hate it and know that you did it, completely lowering your standing in the quizbowl community.

For those three reasons, I think that players at this tournament have the incentive to behave in good faith and play normally.

[DumbJaques]

For those three reasons, I think that players at this tournament have the incentive to behave in good faith and play normally.

But in regular tournaments, you don't need to write paragraphs arguing why people have the incentive to behave normally, nor do you have to hope that you are in fact correct.

[magin]

For those three reasons, I think that players at this tournament have the incentive to behave in good faith and play normally.

But in regular tournaments, you don't need to write paragraphs arguing why people have the incentive to behave normally, nor do you have to hope that you are in fact correct.

Well, I mean, if someone decided to neg 20 times in a game in a regular tournament out of spite towards another player, that would be frowned upon for very similar reasons. Just as no one actually does that in regular tournaments, I don't think that people will actually act that way at Gaddis. I mean, surely quizbowl players wouldn't be that foolish.

I understand critiques of the tournament like "it creates a bit more randomness than a regular tournament," but I think you and Mike are misplaced here, Chris, in arguing that aberrant behavior like deliberately negging 10+ times is encouraged by this tournament.

[No Rules Westbrook]

Yeah, I too am wary of a completely random method of assigning teammates and opposition per round. It seems to me that one of the most important factors in your placing at this tournament is going to be how often you happen to get stuck in a room with a bunch of good players.

But, actually, what I dislike more is this 30 points for superpowers business. I know Jonathan loves people with deep knowledge who make super-awesome real buzzes, but 30 points is an insane amount to get for one buzz. Even the 20-15-10 power structure gets a bit unfair sometimes, but powers are cool so it's fine...this seems crazy and adds even more randomness to the outcome. As for the negging issue, well, that's just another complication - usually I dont even like negs at tourneys like these, but I can see some value in encouraging people not to just go on crazy lateral buzzing odysseys (especially if the reward is a whopping 30 points!)

And yeah, I know we all promised we wouldn't grouse about this format anymore, so sorry for doing so.

[No Rules Westbrook]

I think the only fair way of doing this without having set teams would be to develop some kind of system which ensures that you don't play in a room with the same person (either on your team or on the other team) more than 1 or 2 times.

[Cheynem]

I am a little kerfluffle over the format, but since I am confident Jonathan will bring the awesome questions-wise, I have no worries.

[Skepticism and Animal Feed]

If it doesn't work out, we can just relentlessly mock him on the internet for it. Magin has long been a proponent of shaming as a tool for social regulation.

[magin]

Hey,

If you all would like to play, please email me (jsmagin at yahoo dot com) or add yourself to the spreadsheet linked earlier in this thread. Right now fewer than ten people have signed up, and I think many more people are actually interested, so if you are, please sign up. Remember that the cost is zero dollars and that I promise better-quality questions than last year's Gaddis.

Oh yeah, it will not be completely random. Ideally, every person will play with every other person once. Also, Bruce, I'm not really a proponent of public shaming, and I don't know where you got it from (I guess it's because I think monetary penalties are foolish and not enforceable, while shaming people involves no actual resources).

[Auks Ran Ova]

Also, Bruce, I'm not really a proponent of public shaming, and I don't know where you got it from (I guess it's because I think monetary penalties are foolish and not enforceable, while shaming people involves no actual resources).

He was probably being ironic, given that your history of opposition to, e.g., "yelling at people".

[magin]

The following people have posted that they would like to play, but have not added themselves to the spreadsheet or emailed me:

Dave Letzler
Evan Nagler
Matt Chadbourne
Matt Weiner
Steven Wellstead
Arnav Moudgil
Anurag Kashyap
Aaron Cohen
Gordon Arsenoff
George Stevens
Bruce Arthur
Charlie Dees
Mike Sorice
Seth Teitler
Micah Manary
Ray Luo

If you are one of those people, please add your name to the spreadsheet below or email me (at jsmagin at yahoo dot com) to tell me that you're not playing. If you're not one of those people, I'm assuming you're not playing unless you add yourself to the spreadsheet or email me.

I've set up a public spreadsheet to sign up for this tournament, also. Please only sign up if you're going to play (since if you sign up and don't play, it screws with the schedule).
http://spreadsheets.google.com/ccc?key= ... hG0gdJb5OA

[Maxwell Sniffingwell]

There is a 15-20% chance that I'll be playing this.

[cvdwightw]

I like this idea a lot, Dwight. Instead of complete randomization, would assigning one-eighth of the players the number 1, one-eighth the number 2, and so on up to 8 work, and then doing permutations of that work?

Jonathan, I sent you an e-mail regarding this, but I thought I'd post here about my proposed schedule for Gaddis and look for feedback from people other than Jonathan.

Round 1 begins with randomly selected teams of 7 and 8 playing Gaddis-style against the packet.

At the end of Round 1, the team score is tabulated, and each player's individual contribution is calculated. Each individual's total score for that round is TEAM POINTS + INDIVIDUAL POINTS. The player in each room with the highest score for that round (corresponding to the most individual points) receives the 1 seed in that room, the next highest score the 2 seed, etc. Breaking ties is something I haven't figured out yet.

Players are then rotated into new rooms based on a single schedule. Because players get a new seed every round, but the schedule by seed stays the same, only one round's worth of schedule need be copied down. Each room will contain each of the seeds 1-8 from different rooms in the previous round, unless seed 8 is nonexistent.

It is unlikely that more than 1-2 elite players will ever be in a room together if we judiciously select initial teams (this can be construed as either an advantage or disadvantage depending on how you look at it), and each room is balanced by seed (again, an advantage or disadvantage depending on how you look at it). There will never be fewer than 7 people in a room (advantage), and because it will be relatively easy to make a schedule on the fly regardless of how many people show up, we can get started fairly quickly even if hordes of people that didn't sign up decide to play.

I don't see Donald Taylor anywhere in this thread, so I'll volunteer to run logistics and stats. Real-time stats will probably have to be kept in Excel and then tossup stats entered into SQBS or something after the tournament.

[Mike Bentley]

This system still has problems. It gives people an incentive to intentionally try for a lower seed in several matches. A player who always get just below the first seeded player (but is good enough to be the first seeded player) is going to win this tournament. Again, this is another tournamnet format that can be won by withholding knowledge.

[magin]

This system still has problems. It gives people an incentive to intentionally try for a lower seed in several matches. A player who always get just below the first seeded player (but is good enough to be the first seeded player) is going to win this tournament. Again, this is another tournamnet format that can be won by withholding knowledge.

Wait, I don't understand. If someone scores fewer points than the top player in every round, how will they win? The odds of anything like that happening seem astronomical.

[Mike Bentley]

Because they do this over 8 rounds.

Say I'm a player capable of being a #1 seed. I keep track of my points during the round and the points of the 2nd best player in the room. I notice near tossup 18 that we both have 60 individual points. Rather than buzzing on clues I know on the last two tossups, I sit on questions and let the other player buzz in hopes that he will beat me by the minimum number of points. This will lose me maybe 10-20 individual points for this round, but will net me more team points in the next round since I'll be played with a better player (essentially two number 1 seeds will be playing). I can do this for the entire tournament and win based on the strength of my teammates.

Also, what are we doing about ties in this format?

[magin]

Well, if you do that over eight rounds, then you're 100-200 points in the hole, and therefore have a much, much smaller chance to win the tournament. It's simply not rational to act that way instead of buzzing in and answering the tossup if you want to maximize your points.

[cvdwightw]

This system still has problems. It gives people an incentive to intentionally try for a lower seed in several matches. A player who always get just below the first seeded player (but is good enough to be the first seeded player) is going to win this tournament. Again, this is another tournamnet format that can be won by withholding knowledge.

Wait, I don't understand. If someone scores fewer points than the top player in every round, how will they win? The odds of anything like that happening seem astronomical.

I will concede that it does provide a perverse incentive for, say, top seed Chris Ray to hold off buzzing on the last few tossups in the second-to-last round so that he is paired with top seed Matt Weiner for the last round rather than a second seed from a different room. As long as Chris is ahead of Matt at the end of that second-to-last round, there is nothing Matt can do to pass Chris, and in fact there would be the distinct possibility of both players passing the top overall player at the end of the second-to-last round.

I'll throw another wacky format out there and see what people think of this. Ultimately it's up to Jonathan to decide which of these formats we end up playing.

As with the original Gaddis plan, each player is on a team with 2 or 3 other people, and competes in a room with another team of 3 or 4 people. The goal, as with the original plan, is to score as many points as possible. The scoring system, however, does not reward teams for how many points they score. Instead, it rewards teams for how many other rooms and how many other teams they outscore (ties count as 0.5). So, you get 4r+t points, where r is the number of rooms your room collectively outscores and t is the number of teams your team outscores individually. This system overvalues the team "cooperation" tactic because you get 3 times the number of rooms you and your opponent collectively outscore, but also keeps a slight "competition" tactic by adding a (one-half weight) score for individual team performance within that.

I believe this removes all perverse incentives not to buzz, since your goal is now to score as many points as possible in all scenarios. Being a bad sport will still hurt both teams, but it will hurt your team more.

[magin]

This system still has problems. It gives people an incentive to intentionally try for a lower seed in several matches. A player who always get just below the first seeded player (but is good enough to be the first seeded player) is going to win this tournament. Again, this is another tournamnet format that can be won by withholding knowledge.

Wait, I don't understand. If someone scores fewer points than the top player in every round, how will they win? The odds of anything like that happening seem astronomical.

I will concede that it does provide a perverse incentive for, say, top seed Chris Ray to hold off buzzing on the last few tossups in the second-to-last round so that he is paired with top seed Matt Weiner for the last round rather than a second seed from a different room. As long as Chris is ahead of Matt at the end of that second-to-last round, there is nothing Matt can do to pass Chris, and in fact there would be the distinct possibility of both players passing the top overall player at the end of the second-to-last round.

I'll throw another wacky format out there and see what people think of this. Ultimately it's up to Jonathan to decide which of these formats we end up playing.

As with the original Gaddis plan, each player is on a team with 2 or 3 other people, and competes in a room with another team of 3 or 4 people. The goal, as with the original plan, is to score as many points as possible. The scoring system, however, does not reward teams for how many points they score. Instead, it rewards teams for how many other rooms and how many other teams they outscore (ties count as 0.5). So, you get 4r+t points, where r is the number of rooms your room collectively outscores and t is the number of teams your team outscores individually. This system overvalues the team "cooperation" tactic because you get 3 times the number of rooms you and your opponent collectively outscore, but also keeps a slight "competition" tactic by adding a (one-half weight) score for individual team performance within that.

I believe this removes all perverse incentives not to buzz, since your goal is now to score as many points as possible in all scenarios. Being a bad sport will still hurt both teams, but it will hurt your team more.

I like this a lot, and like allowing teams to form whatever teams they want. What do you all think?

[cvdwightw]

I will present an example of this:

There are 4 teams: Team Magin, Team Wynne, Team Bentley, and Team Ray.

Team Magin and Team Wynne play in the first round and Team Magin wins 150-105.
Team Bentley and Team Ray play in the first round and Team Ray wins 150-75.

The Magin/Wynne combo scored 255 total points and the Bentley/Ray combo scored 225 points.

Thus Team Magin for round one gets 4*1+2.5 = 6.5 points (as they outscored the other room and 2 other teams, and tied with 1 other team). Team Wynne gets 4*1+1 = 5 points (as they outscored the other room and 1 other team). Team Ray gets 4*0+2.5 = 2.5 points (as they outscored 2 other teams and tied with 1 other team). Team Bentley gets 0 points.

Round 2: Magin/Bentley: Magin wins 200-45. Ray/Wynne: Ray wins 140-105. Each room scored 245 points. Team Magin gets 4*0.5+3 = 5 points; Team Ray gets 4*0.5+2 = 4 points;
Team Wynne gets 4*0.5+1 = 3 points; Team Bentley gets 4*0.5+0 = 2 points.

After Round 2:
Magin 11.5 points
Wynne 8 points
Ray 6.5 points
Bentley 2 points

Round 3: Magin/Ray: Ray wins 165-150. Wynne/Bentley: Wynne wins 160-85. Team Magin gets 4*1+1 = 5 points; Team Ray gets 4*1+3 = 7 points; Team Wynne gets 4*0+2 = 2 poitns; Team Bentley gets 4*0+0 = 0 points.

Thus after a full round robin:
Magin 16.5 points
Ray 13.5 points
Wynne 10 points
Bentley 2 points

With many more teams, it is theoretically possible for the second-place team to overtake the first-place team, if they are within enough points, by outscoring them by enough points that they make up those points with t. If the top two teams are tied and play in the last round, then obviously the team that wins wins the tournament.

EDIT: Note that in this scenario, the team that won all 3 games did not win the tournament, because the overall goal is to score points, not win games. However, a team that wins an individual game will always get more points than the team they beat in that game.

[theMoMA]

That format doesn't have the pitfall of rewarding people for playing with good teammates on easier or harder packets, but doesn't it seem like individual performance should factor into this somehow?

[cvdwightw]

That format doesn't have the pitfall of rewarding people for playing with good teammates on easier or harder packets, but doesn't it seem like individual performance should factor into this somehow?

That depends. The initial move to a more singles-style tournament was made because of issues with pitfalls with the whole "I played with Seth Teitler and Matt Weiner and we scored 600 points; someone else played with that team and only scored 400" issue - in this format we use a normalized scoring system (the maximum possible points is 19 per round for a 32-person tournament) that eliminates round-by-round variability.

I suppose we could also use a normalized individual scoring system (e.g. points are now 40r+10t+i, where i is the number of individuals you outscored that round), but since people are now choosing their own teams instead of playing with randomly selected or seed-selected teammates, I'm not sure that it's worth it. Also, we have the issue of lots of people scoring 0 points in rounds, and just a raw score of "number of points you scored" eliminates the corrections that a fully normalized scale brings.

Thoughts on changing the scoring system to make it incorporate individual performance are welcome, though.

[Mike Bentley]

So do we have schedules and scorekeeping guidelines laid out for this tournament?

Have teams been formed already? Is it worth it to just have a draft in order to get some sort of even teams this late in the game (especially with people probably showing up at the tournament who haven't signed up)?

[Blackboard Monitor Vimes]

(especially with people probably showing up at the tournament who haven't signed up)?

3-4 Gov kids would like to do this, if it is ok with Magin/whoever else may have an organizational role.

[ihavenoidea]

My voice is a little hoarse, but I'll read for this and my teammates are willing to help as well.

[No Rules Westbrook]

Well, whatever wacky format we choose - if teams are going to be constant throughout the tournament, it seems like we should just draft. I know Magin said he didn't want to, partly cause it would slow things down, and it probably will...but if there are enough readers/staff, I assume anyone who wants to play will play. So, it might be better to just have a draft instead of expecting people to pair up themselves, which will likely lead to very imbalanced teams.

[magin]

This tournament will start an hour and a half after the Saturday rounds of HSNCT finish. If you would like to play, please meet at that time by the escalator/stairs in the basement of the hotel (by the suites). There will be a brief draft before the tournament; we will take measures to ensure that the draft goes briskly.

Anyone who wants to play (or staff) is welcome, and the fee is zero dollars. It's going to be pretty awesome.

[Skepticism and Animal Feed]

Hey guys, guess what happened.

Teams started playing Gaddis and began trying to answer tossups before the other team could. After 10 and 15 tossups, they asked for scorechecks. If they negged and the other team picked it up, they cursed. If they scored more points than the other team, they high-fived each other. When they passed each other in the hallway, players would ask each other what their PPG was, or what their teams win-loss record was. The team that went undefeated boasted of having potentially cleared the field.

In other words, teams ignored the format and treated this like a regular tournament. As far as I could tell, most people were having fun too.

[Mike Bentley]

I believe Weiner's team and Westbrook's team both were like 7-2 and the prelims if normal scoring was done (Weiner's team won by like 6 points using the whacky format). In the "finals", Weiner's team ended up winning from what I heard.

[No Rules Westbrook]

Heh, I'm guessing Bruce had that post ready to go pre-Gaddis, and it was a simple copy and paste operation. Bentley is right about the results, btw.

[Cheynem]

I can't believe that these questions aren't cleared for discussion, but just in case, I'll be vague. I enjoyed this set and particularly enjoyed the American lit, which ran close to my own interests, including a few works that I had tossed up before in the sample packets of my vanity lit tournament (always good to recognize clues you've used).

Regarding Magin's thesis: Since I'm not sure what exactly it was, I won't bother trying to explain if it worked or not. I'll just say: I think side events typically create a more relaxed, communal atmosphere anyway, since you're playing a lot of times with unfamiliar people on different sorts of questions. I greatly enjoyed playing both this and Ryan's earlier Experiment.

[Blackboard Monitor Vimes]

I believe Weiner's team and Westbrook's team both were like 7-2 and the prelims if normal scoring was done (Weiner's team won by like 6 points using the whacky format). In the "finals", Weiner's team ended up winning from what I heard.

Confirming that Matt's team won. Thanks for doing this and letting people play for free, Magin, it definitely improved this weekend as someone who was greatly disappointed in the HSNCT set. I don't have much of a frame of reference since I hadn't played a tournament near this difficulty level before, but I enjoyed it thoroughly.

[cvdwightw]

Full stats will be up sometime tonight, with an explanation of what the weird PPG is doing. No guarantee that individual stats are accurate - if somehow your awesome superpower got attributed to someone else, I apologize in advance.

From what I remember off the top of my head, Weiner's team cleared the field by about 5 points over the 4-5 Rob Carson team.

I don't presume to speak for Jonathan, but at least for me, there were two major hypotheses:
1. Players enjoy playing on (mostly) awesome questions on (mostly) awesome things, regardless of the way a winner is determined.
2. The primary point of quizbowl may be to win games, but an equally important point is to score points. Therefore, a scoring system can be devised that rewards players for scoring points and is normalized for difficulty of packet/strength of opponent.

I think (1) went a long way toward being proven at this tournament. Regarding (2), I think this tournament showed that this hypothesis has promise - it is a good idea to reward teams for losing to good teams (rather than just the plain 0 for losing), but this scoring system may not have been the ideal one, and it may not be appropriate (or efficient) to run an entire tournament using this format.

I do not know whether there are round-by-round data for all 2009 NAQT Sectionals, but I may try to better elucidate what exactly is going on in this scoring system and see if I (or an interested person at NAQT, if they have more time than I do in the next few weeks) can use this data and optimize parameters for room and team scores (in this case PPTH) to develop a "strength of schedule" score for the S-Value.

[Skepticism and Animal Feed]

At most, you proved that people will prefer oddly scored quizbowl to no quizbowl at all.