Some Thoughts on Bonuses

Elaborate on the merits of specific tournaments or have general theoretical discussion here.
Post Reply
User avatar
Gene Harrogate
Lulu
Posts: 82
Joined: Fri Aug 04, 2017 11:05 pm

Some Thoughts on Bonuses

Post by Gene Harrogate »

After tournaments, more often than not, I end up talking with people about bonuses: what functions they serve, what knowledge they test, how they should be written. In this post I'll share what I see as bonus best practices and gesture a bit towards their future. All of my examples will be hypothetical (I'm not a fan of the j'accuse! school of forum posting).

I'll begin by stating my core postulate: quizbowl is first and foremost a game. The rest of my post will assume that the primary function of bonuses is to differentiate between teams competively, at three different levels of skill. There are other nice things about bonuses--leadins are usually where writers stash interesting facts or context that they can't fit elsewhere. But these considerations are secondary to making gradated, answerable questions.

Bonus intricacy, moderating, and cognitive load

In an effort to write more creative or educational questions, writers try structure their questions around unique themes and include as many interesting clues on the subject as possible. The problem with this focus is that these questions often take up much more space. They tend to balloon or employ a succession of intricate clauses to thematically link bonus parts. Sometimes, in an effort to make an easy part "interesting," you end up with questions like:
This modern-day country traces its founding to the ancient king Dudley Princesymbol, who used flaming weasels to repel the Numenorians from his ancient capital of Labradoria, therebye fulfilling Jim Bob Beaverton's prophecy that "fire will be stronger than water." For 10 points each:
[10e] Name this country, where, in the 1950s, Dudley Princesymbol's legacy divided the Borgesians and the Quixotians at French-backed universities overseen by Leopold Churchill. Dirk Lastrade's Pulizter-winning history Restless City recounts how Dudleyite paramilitaries used pitchforks and torches to clash with the Borgesian-aligned LPLA in this country's capital of Coolesville.
This sort of long exercise in capital-matching is common enough, and usually elicits a groan from someone in the room. It's fine to keep players interested by mixing in some extra clues, or to give them something to look up later. But long, intricate bonuses create competitive problems:

1) Moderators have finite amounts of energy. Tournaments with systematically long and complex bonuses drain readers, who begin to make more mistakes. They lose their voices, mispronounce words, and mis-emphasize things. That's not fun for players either! Not coincidentally, NAQT sets are the most fun to read.

2) Intricate bonuses increase cognitive load, thereby making questions unecessarily hard to answer. We've all seen the good team that sits through 3 lines of "After...that...which..., while..." only to go "what are they asking for?". Quizbowl is a difficult game that requires a lot of sustained thinking over a long period of time. Part of being an empathetic writer is imaging oneself as a tired player, listening and trying to keep track of all the clues being thrown at them (this is where reading your own questions out loud helps). Note relatively short questions can also unduly increase cognitive load by trying to fit too many clauses or clues into a short amount of space.

3) Intricate bonuses disproportionately punish newer players. And here, I mean the players who haven't yet become skilled at cutting through the quizbowlese, sifting through extraneous information to realize the writer is just asking for the guy who wrote "The Raven." Most easy tournaments do a pretty good job at this, but there is room for improvement.

I am of the opinion that there is rarely reason to go over two lines for a bonus part, and that it is usually possible to fit interesting, gettable clues within 1.5 lines---with exceptions, like scientific formulas and score clues. Someday I'd like to see a tournament try a hard cap of 2 lines with at least one part of each bonus required to stay on one line. My guess is that sort of limitation would improve quizbowl writing by forcing tighter clues.

What are easy parts for?

It is a truth universally acknowledged that easy parts should legitimately test knowledge. Easy parts are there to differentiate between the bottom of the field. Compare two possible easy parts for a bonus making a connection between the Latvian novel The Electioneer's Daughter and Anna Karenina:
[10e] In The Electioneer's Daughter, Sylvia throws herself in front of a train, much like this author's character Anna Karenina.
ANSWER: Leo Tolstoy
[10e] Both Sylvia in The Electioneer's Daughter and Tolstoy's Anna Karenina throw themselves on the tracks of this sort of vehicle, which in Russia run on a "Trans-Siberian" structure.
ANSWER: a train
The first style of bonus is "boring," but it accurately sorts bottom bracket teams based on whether they know who Leo Tolstoy is. The second style of bonus asks you to know what vehicles run on tracks, and then throws in a non-literature clue. You probably haven't and won't see this type of bonus for Anna Karenina specifically, but you've definitely seen it for other topics and subjects.

Most teams probably know Anna Karenina threw herself in front of a train. A related type of bonus asks you to make the obvious guess without concretely knowing any of the clues:
[10e] The scarcity of the plant used to make this substance is a recurring motif throughout The Electioneer's Daughter. In Lando Daugavins' novel Fire on the Neva, Otto drinks this substance to wake up in the morning.
ANSWER: coffee
Coffee of course is the obvious guess here, but I don't actually know anything about Fire on the Neva. For all I know, Otto is a Satanist who drinks a fresh cup of goat's blood every morning. What this bonus asks the player to do is successfully recognize that it's the easy part, and not out-think their way past the obvious guess. Instead of differentiating literature knowledge, this bonus differentiates between teams who can guess what someone might drink in the morning and teams who go wrong somewhere along the way ("uh, tea has caffeine I guess").

This example might seem absurd, but I swear I've heard a similar question in just about every recent academic tournament. Usually it happens when an author has a niche theme they don't want to break (Latvian literature, prehistoric Japanese artifacts, Romanian mythology, etc.). In these circumstances, I would suggest writers choose the lesser of two evils and find a way to make the easy part on Tolstoy. The answer can even still be an object, event, substance, etc.---vary your answerline types, of course. But make sure you're 1) testing knowledge of the actual subject 2) giving a concrete clue you expect most of the field to know.

I'd like to emphasize that I don't think these sorts of easy parts make the game more interesting for the bottom of the field---they're not the ones burnt out on Tolstoy questions. I think these questions are written out of the fear that very good quizbowlers who have heard it all before will criticize the bonus for being boring. Newer players are usually just happy to get points for knowing something.

Most of quizbowl has recognized that it is better to write a tossup on a simple answerline with creative clues than a tossup with a creative but confusing answerline. I'm not sure we've paid similar attention to bonuses. We should be clear that there is nothing wrong with a simple easy part. In some ways, it takes a lot of confidence to write "Anna Karenina is by this author of War and Peace." Don't let the need to do something new get in the way of writing a bonus that distinguishes between bottom bracket teams.

Bonuses and the End of Quizstory

I've talked a bit about how bonuses are currently written, and I've probably said a lot of things you already know about writing simple, direct questions. But any bonus discussion eventually runs into the big problem: why does a mostly redundant knowledge test take up more than half of game time? By some accounts, bonuses decide less than 10 percent of games, and are about 1/12 as predictive of ICT finishes as tossups (source: some threads I read once but can't find).

Arguments in favor of the current three-part bonus usually focus on the teamwork aspect, providing opportunities for weaker players to contribute, testing knowledge that doesn't fit well into pyramidal tossups, and quizbowl's uniquely educational ethos (i.e., bonuses give more space for writers to provide important context and fun facts). These are all good reasons for keeping bonuses as they are, and I don't think the state of the game today is exactly dire. But given how much time we put into bonuses versus their importance for deciding games, I think it's worth considering ways to keep their positive aspects while improving game efficiency.

We seem to have reached the End of Quizstory, where we've become focused on producing the best possible version of a fixed, immutable game. There's no real reason why this needs to be the case. I would personally enjoy playing academic housewrites that take big risks with the format (successful distributional experiments like FLOPEN have already taken some steps in this direction). Here are some barely thought-out mechanical adjustments:
  1. Try out one-part and two-part bonuses. Even combine this with an increase in tossups, and more distributional space for "Other" questions.
  2. Allow conferring after the other team negs, while cutting back on bonuses. For many players--like Reach for the Top vets--this is actually very intuitive. You get more of the pace of tossups, while keeping the inclusive aspects of bonuses.
  3. Play around with point values. Maybe tossups should be worth zero points: they'd be plenty valuable anyway. Introduce two-part bonuses worth 20 points each.
It's obviously fine for the vast majority of tournaments to continue with the three part bonus. And every easy part does not need be less than a line long. There's plenty of room to include interesting clues in concise evocative bonuses, and the best writers do this all the time. But there is no need to consistently push the limits of bonus length, nor to try to include every interesting clue on the subject. It is much more forgiving to moderators and players to err on the side of simplicity and brevity.

Many very good writers and editors have made these most of points elsewhere, and I don't think I've put down any unique insights here. I would love to see further discussion on the broader purpose of quizbowl bonuses, and how we can be writing them better.
Last edited by Gene Harrogate on Tue Jan 18, 2022 12:04 am, edited 4 times in total.
Henry Atkins
McGill
User avatar
Stained Diviner
Auron
Posts: 5011
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: Some Thoughts on Bonuses

Post by Stained Diviner »

This might be the thread you were looking for.

While I am in this thread, I am casting votes for short bonuses and easy parts that test knowledge.
David Reinstein
Head Writer and Editor for Scobol Solo and Masonics, TD for Scobol Solo and Reinstein Varsity, Writer for NAQT (2011-2017), IHSSBCA Board Member, IHSSBCA Chair (2004-2014), PACE Member, PACE President (2016-2018), New Trier Coach (1994-2011)
User avatar
Sam
Rikku
Posts: 328
Joined: Sat Nov 07, 2009 2:35 am

Re: Some Thoughts on Bonuses

Post by Sam »

I agree with most of this post but the section on easy parts is especially astute. When writing bonuses I (and I suspect many others) am tempted to include interesting but superfluous clues in the easy part since teams will "get it anyway," but Henry's right that it can make them more confusing. It also often does less injury to the bonus to cut a fun fact from the easy part than to remove a clue from hard or medium where it may actually be helpful, if length is becoming an issue.

I think this post also gets to a general point that is good to remember when writing and editing: rather than striving to make each individual question (or even each bonus part) a unique and fascinating gem that can stand alone, it's better to focus on an interesting variety of question content and question difficulty across the packets. Both a packet with every easy part a half-line author question and a packet where every easy part is three lines are probably going to be less fun than one with a mix.
Sam Bailey
Minnesota '21
Chicago '13
User avatar
Mike Bentley
Sin
Posts: 6274
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: Some Thoughts on Bonuses

Post by Mike Bentley »

I'm an advocate for trying out different things with bonuses. I haven't thought a ton about different ways to do this but I encourage other people to do so. I am a little skeptical that 1 or 2 part bonuses are the answer because they produce more variance and/or end up being a slog for the bottom teams because you've dropped the easy parts or whatever.

It's generally not my experience that people write obtuse easy parts because they're trying to be creative. I think the main culprit is a lack of strict length limits on bonus parts. When you have this, you will by nature start removing the fat from those parts so you can better support the middle and hard parts.

I'm not opposed to the occasional bonus part that involves some degree of figuring it out. As you mention, quizbowl is a game. Even well-written bonus parts are going to have players doing some degree of figuring it out as they determine "is this the hard part" when deciding between 2 answers to give. And tossups have all sorts of gamesmanship going on as you determine what's askable in the answer space and how much of the question you've heard. That being said, don't go overboard with these things. And as much as possible, keep the content within one category and break theme if needed.
Mike Bentley
VP of Editing, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
User avatar
ErikC
Rikku
Posts: 251
Joined: Sat Sep 24, 2016 12:44 pm

Re: Some Thoughts on Bonuses

Post by ErikC »

I'd also extend Henry's point about how to write easy parts to the ends of tossups too. I've noticed tossups trying to massage a harder answerline into a tournament by adding clues from outside the category, or ending the question with a similar clue to the "coffee to wake up in the morning" trope Henry points out. Obviously dead tossups are far more consequential to a game's outcome and player experience, but I always found it very inconsistent when tournaments vary from basic, core giveaways (name this author of the Golden Notebook) versus a coffee in the morning kind of giveaway.

I think a bit of consensus needs to be made about whether the standard bonus format is aiming for 90% of teams to get the easy part or 100%, because way too often I'm seeing parts that I wouldn't expect any team to miss right next to literature bonuses that work just fine for most teams are frequently missed by newer players. I don't think this is a coincidence - history questions often end with a geo clue or a capital clue even at regionals level which relies on information I really expect most newer teams (which I spent a lot of time playing with, against, or reading for before Canada matured as a regionals) to still know. This isn't true for many other subjects that often don't or can't employ unusually easy parts.
Erik Christensen
University of Waterloo - School of Planning Class of '18
I write trash
Defending VETO top scorer
User avatar
Skepticism and Animal Feed
Auron
Posts: 3220
Joined: Sat Oct 30, 2004 11:47 pm
Location: Arlington, VA

Re: Some Thoughts on Bonuses

Post by Skepticism and Animal Feed »

At almost every tournament that isn't a championship or a qualifier for a championship, convincing marginal teams and players that quizbowl is something they want to do again is a far more important goal than fine-grade differentiating between teams. One theory says that extremely obvious bonus parts and extremely obvious clues in the last sentence of a tossup serve this goal by giving even the least experienced teams the satisfaction of having answered questions correctly and scored some points.

Further, I was raised to believe that superfluous clues in the introductions or easy parts of bonuses are a place where you can either educate your audience further on the matter at hand, or insert clues that would not be appropriate to ask about.
Before becoming President, this man owned a newspaper called the Buffalo Commercial Advertiser.
[10] Name this Whig who became President upon the death of Zachary Taylor
ANSWER: Millard Fillmore
At most tournaments, "Buffalo Commercial Advertiser" is not going to be acceptable even as a third bonus part, but now I've introduced an entire field of people to this fact. People can start using this knowledge in their lives. Perhaps, one day, a kid playing this tournament will answer a Chicago Open tossup on Millard Fillmore because they remembered that from my lead-in.

I'm not saying any of this is correct, or justified, or has stood the test of time, etc. I'm just trying to provide insight into why some writers do or did this kind of stuff.
Bruce
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source
User avatar
Cheynem
Sin
Posts: 6985
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Some Thoughts on Bonuses

Post by Cheynem »

I think that clue about Fillmore is fine added information because it's short and comes very early. It's when we get bonus parts like the ones that Henry mentioned where the added information is sprinkled throughout the part and sometimes drowns out the easy clue or information.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Gene Harrogate
Lulu
Posts: 82
Joined: Fri Aug 04, 2017 11:05 pm

Re: Some Thoughts on Bonuses

Post by Gene Harrogate »

Cheynem wrote: Mon Jan 17, 2022 11:28 pm I think that clue about Fillmore is fine added information because it's short and comes very early. It's when we get bonus parts like the ones that Henry mentioned where the added information is sprinkled throughout the part and sometimes drowns out the easy clue or information.
Mike's 100 percent right here. Saying easy parts should never include any unnecessary information at all would be tantamount to saying bonuses should never begin with the easy part, which would be silly to argue.
Skepticism and Animal Feed wrote: Mon Jan 17, 2022 11:20 pm At almost every tournament that isn't a championship or a qualifier for a championship, convincing marginal teams and players that quizbowl is something they want to do again is a far more important goal than fine-grade differentiating between teams. One theory says that extremely obvious bonus parts and extremely obvious clues in the last sentence of a tossup serve this goal by giving even the least experienced teams the satisfaction of having answered questions correctly and scored some points.

Further, I was raised to believe that superfluous clues in the introductions or easy parts of bonuses are a place where you can either educate your audience further on the matter at hand, or insert clues that would not be appropriate to ask about.
Before becoming President, this man owned a newspaper called the Buffalo Commercial Advertiser.
[10] Name this Whig who became President upon the death of Zachary Taylor
ANSWER: Millard Fillmore
At most tournaments, "Buffalo Commercial Advertiser" is not going to be acceptable even as a third bonus part, but now I've introduced an entire field of people to this fact. People can start using this knowledge in their lives. Perhaps, one day, a kid playing this tournament will answer a Chicago Open tossup on Millard Fillmore because they remembered that from my lead-in.

I'm not saying any of this is correct, or justified, or has stood the test of time, etc. I'm just trying to provide insight into why some writers do or did this kind of stuff.


I agree that retaining players is an important part of question writing. To me, it's self-evident that long and confusing bonuses won't make new players want to stick around. It's admittedly hard to guess why people leave. After all, these conversations are between the ones who stayed. But my guess is that marginal players don't stick around because they get "free points," but because they feel quizbowl rewards their intellectual curiosity. Bonuses shouldn't make these players feel dumb by throwing a bunch of "easy quizbowl core" at them that they aren't likely to have known beforehand--looking at you E.A. Robinson!--but I also don't think they should condescend by asking intelligent adults what kind of vehicles use sails.*

*This applies to bonuses. I'm in favor of stopping tossups from going dead at easy difficulties whenever possible.
Henry Atkins
McGill
User avatar
everdiso
Wakka
Posts: 100
Joined: Sun Nov 08, 2015 10:36 pm

Re: Some Thoughts on Bonuses

Post by everdiso »

I was very glad to see this post. Quizbowl is really good at writing interesting and competitive tossups, but I think bonuses are the weaker part of basically every tournament. I think most of what Henry wrote makes a lot of sense.
ErikC wrote: Mon Jan 17, 2022 11:17 pm I think a bit of consensus needs to be made about whether the standard bonus format is aiming for 90% of teams to get the easy part or 100%
Erik highlights a key question here. Imagine a bonus in which the hard part was absolutely ungettable – it asked for the 19th-century mayor of some Canadian town or for a character who speaks one sentence in a minor scene of Henry James’s 12th-most famous novel. No teams would get this hard part, meaning that everyone who got the medium part would wind up with 20 points. Would this be an acceptable bonus?

Clearly it would not be, because it completely fails to differentiate between good teams and great teams. Anybody good enough to get a 20 will wind up doing equally well, and there will be no advantage to having knowledge beyond that point. This hard part would fail at its task of differentiating the best teams from merely pretty-good ones, so it would not be allowed to be used in a set.

We have the right attitude on this when it comes to top teams, but not when it comes to weak teams. The equivalent of this for them would be a bonus in which the easy part was so easy that every team of functioning human beings would get it: guaranteeing everyone 10 makes the easy part useless, just like making 30 impossible makes the hard part useless. And just like that fails to differentiate between teams that are good at this subject and teams that are great at it, this would fail to distinguish between teams below average at the subject and teams very bad at it. Next time you might be tempted to write an easy part like “this planet is named for the Roman god of war and has a red surface”, realise that that’s the functional equivalent of a hard part like “Identify this word, the 14th word on page 198 of the Penguin edition of Austen’s book Emma. ANSWER: Brittle”. (Note: I did not actually check page 198 of Emma, this is probably wrong.)

Another important thing about easy parts is that, very simply, they ought to test knowledge of the bonus's subject. They fail to do this surprisingly often. We’ve all seen the easy parts – most often in literature – that don’t even try to clue from that subject, like “Gordimer was from this country, where she wrote about its black people’s struggles under Apartheid”. This is not a literature question in any way. It may not matter to the top teams, who will be getting easy parts regardless of what they’re about, but it certainly matters to the weaker, usually newer teams that these questions are for. Again, imagine that, instead of the easy part, it was the hard part of a Nadine Gordimer bonus that suddenly swerved into history: that after two literature questions, the hard part of the literature bonus asked a difficult question about a South African politician of the Apartheid era. Clearly, nobody would even write this bonus, let alone consider it fine. We should have the same attitude towards those all-too-common easy parts that abandon the bonus’s supposed subject. Many teams 10 most bonuses, so for those teams, the easy part is often the only important one. That means that writing an easy part that fails to ask about the bonus’s intended subject is tantamount to not having a bonus in that subject at all for weaker teams.

This will make choosing easy parts more difficult for writers, no question. But that’s a good thing. As Henry wrote, we should give as much attention to the easy questions, meant for weaker teams, as to the hard questions that only the best teams will be competing on. Easy parts may be an afterthought for great teams as they play them, but they shouldn’t be an afterthought for writers and editors.

I think this also applies to novice/into tournaments. Of course, easy parts there will have to be easier, but I think they still shouldn't be at the free-ten-points level of easy. As Henry wrote in his latest comment, it might feel worse to be patronised with 10 points for touching your butt after missing two actual trivia questions than to go 0/3 on a subject you're not interested in.

---

On the topic of experimental changes to bonus design, I had one such idea recently: always ordering bonuses easy-medium-hard, and ending them after a team misses one part. The idea would be to save teams that don’t know much about a subject from sitting through two questions way above their level before finally hearing one gettable question at the end (that they may still miss).

This would be a relatively undisruptive change, as it would keep the current three-part, difficulty-gradated structure of bonuses and wouldn’t change scoring at all. One drawback would be the loss of the occasional times when a team gets a medium or hard part while missing an easier one, but I suppose the idea would now be that teams have to “qualify” for the harder questions by getting the earlier ones, much like they qualify for the bonus by getting a tossup. Another potential drawback would be that some weak teams might find it more frustrating to rarely get to hear the harder parts of their bonuses in games against better teams who hear most medium and hard parts of theirs than to sit through long questions without being able to answer them.

I'd be interested in hearing people's thoughts on this.
Paul Kasiński
University of Toronto, 2020
Reigning VETO champion
User avatar
Cheynem
Sin
Posts: 6985
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Some Thoughts on Bonuses

Post by Cheynem »

Paul, I think a number of your thoughts are pretty good, but I guess I will say this in regards to two of them:

-In a lot of tournaments, "Name this planet named for the Roman god of war that has a red surface" would be too easy for an easy part. However, I do think rare is the part where there is 100% or even extremely close to 100% conversion. People are always going to trip up on parts or blank on knowledge. This does not mean we should force feed super easy parts ("This color, the color of catsup"), but that even what appears to be trivial knowledge ("Roman god of war," "author of War and Peace") can still be a valid easy part at most levels.

-In regards to the apartheid clue for the Gordimer bonus...I don't know. Apartheid is actually a major theme in Gordimer's works. Yes, it adds in history knowledge, but it is thematically quite relevant while still being a part that most teams can answer. I think this works a lot better than some other potential parts, like "Mahfouz is an author from this country that has the pyramids," etc. In short, I think having some slightly cross-category knowledge for easy parts can work if we are still being thematically relevant.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
everdiso
Wakka
Posts: 100
Joined: Sun Nov 08, 2015 10:36 pm

Re: Some Thoughts on Bonuses

Post by everdiso »

Cheynem wrote: Tue Jan 18, 2022 3:45 pm ...even what appears to be trivial knowledge ("Roman god of war," "author of War and Peace") can still be a valid easy part at most levels.
Yeah, I definitely agree that "author of War and Peace" and even "Roman god of War" (at lower levels) are non-trivial. I guess it's more the "red surface" part that I was focusing on here, since I doubt anybody who's ever played university quizbowl has not known that. Of course, the specifics of this hypothetical example are not too important.
Cheynem wrote: Tue Jan 18, 2022 3:45 pm Apartheid is actually a major theme in Gordimer's works. Yes, it adds in history knowledge, but it is thematically quite relevant while still being a part that most teams can answer.
Yeah, this is true. But I don't think it makes the easy part play as a literature question for weaker teams. If you're struggling to 10 this bonus, you probably don't know anything about Nadine Gordimer's writing (or maybe even who she was), so this part wouldn't play as "oh, of course, Gordimer wrote about Apartheid", but as "well, I don't know who she is, but Apartheid was in South Africa". For stronger teams this is relevant, and it's true that introduces a key fact about Gordimer (it's a good example of the educational aspect of easy parts). But nobody would 10 this bonus from literature knowledge.

I certainly agree that your Mahfouz example is more obviously indefensible, and I think there are lots of those sorts of easy parts bouncing around.
Paul Kasiński
University of Toronto, 2020
Reigning VETO champion
User avatar
Mike Bentley
Sin
Posts: 6274
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: Some Thoughts on Bonuses

Post by Mike Bentley »

I think there's a spectrum to the cross-category nature of bonus parts. Quizbowl has traditionally had a more narrow definition of subjects than you encounter in a class / casual learning. If you're reading about Gordimer there's no way you're not discussing that she's a South African author and the context of her writing her books. And quizbowl is inherently a game, so sometimes you need to do things that make it still work fairly. There are obviously ways you can do this with more or less elegance.

In general, placing more attention on the easy parts of bonuses is a good idea. Especially for harder tournaments. I've played many harder events where some editors think you need to work for the easy part and others still have the mindset of basically 100% of teams should get those parts. This can make a difference in low-scoring matches.

The E/M/H that ends when you miss it is an interesting idea. I'd play a tournament that experiments with doing it.

I do think you'll probably run into cases where people keep asking what the later parts in the question were. It also adds more constraints on writers since there are some bonuses where it's a lot easier to have the hard part come first.

I think another thing that could work to make bonuses more interesting for some experimental online tournament is letting teams select their category. This would need to be integrated into the buzzing software to make this not take forever. Basically, once a bonus starts the team gets maybe 3 seconds to choose a remaining category (the software knowing which ones are left). If they don't choose in time it automatically loads the next / random bonus. This would add some strategy around knowing what your team and the other team are good at. It has the obvious downside that it's less costly to be weak at some categories. But, again, I'd like to see some experimental tournament try something like this.
Mike Bentley
VP of Editing, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
User avatar
everdiso
Wakka
Posts: 100
Joined: Sun Nov 08, 2015 10:36 pm

Re: Some Thoughts on Bonuses

Post by everdiso »

Mike Bentley wrote: Tue Jan 18, 2022 4:46 pm I do think you'll probably run into cases where people keep asking what the later parts in the question were. It also adds more constraints on writers since there are some bonuses where it's a lot easier to have the hard part come first.
Yeah, both of these are probably true. When a hard part is something like the most famous book by an author, it can only really be a hard part if teams don't yet know it's by that author. I wonder if this would just force a rethinking of how to write about some topics or if it would really make some things undoable in bonuses, which would be a bad thing
I think another thing that could work to make bonuses more interesting for some experimental online tournament is letting teams select their category. This would need to be integrated into the buzzing software to make this not take forever. Basically, once a bonus starts the team gets maybe 3 seconds to choose a remaining category (the software knowing which ones are left). If they don't choose in time it automatically loads the next / random bonus. This would add some strategy around knowing what your team and the other team are good at. It has the obvious downside that it's less costly to be weak at some categories. But, again, I'd like to see some experimental tournament try something like this.
This sounds like a very fun angle to add to a tournament, and a cool kind of teamwork (the deciding). A weird side effect of this is that it would make the early tossups more valuable than the late ones, because by the end the bonuses that the teams are good at will be gone. I'm not sure if that's a problem or just a quirk.
Paul Kasiński
University of Toronto, 2020
Reigning VETO champion
User avatar
Gene Harrogate
Lulu
Posts: 82
Joined: Fri Aug 04, 2017 11:05 pm

Re: Some Thoughts on Bonuses

Post by Gene Harrogate »

Sam wrote: Mon Jan 17, 2022 9:12 pm I think this post also gets to a general point that is good to remember when writing and editing: rather than striving to make each individual question (or even each bonus part) a unique and fascinating gem that can stand alone, it's better to focus on an interesting variety of question content and question difficulty across the packets. Both a packet with every easy part a half-line author question and a packet where every easy part is three lines are probably going to be less fun than one with a mix.
This is a really great point that I don't want to get lost. It's sometimes hard to remember while chipping away at a set one question at a time that you're trying to produce a good forest, not a bunch of cool trees. I'm all in favor of mixing up answerlines, varying clue style and type of content, and varying bonus part length (with the caveat that if one's easy part is pushing two lines one should probably be trimming fat in the other parts).
everdiso wrote: Tue Jan 18, 2022 3:37 pm On the topic of experimental changes to bonus design, I had one such idea recently: always ordering bonuses easy-medium-hard, and ending them after a team misses one part. The idea would be to save teams that don’t know much about a subject from sitting through two questions way above their level before finally hearing one gettable question at the end (that they may still miss).

This would be a relatively undisruptive change, as it would keep the current three-part, difficulty-gradated structure of bonuses and wouldn’t change scoring at all. One drawback would be the loss of the occasional times when a team gets a medium or hard part while missing an easier one, but I suppose the idea would now be that teams have to “qualify” for the harder questions by getting the earlier ones, much like they qualify for the bonus by getting a tossup. Another potential drawback would be that some weak teams might find it more frustrating to rarely get to hear the harder parts of their bonuses in games against better teams who hear most medium and hard parts of theirs than to sit through long questions without being able to answer them.
I would also be down to try this.
Henry Atkins
McGill
User avatar
ThisIsMyUsername
Yuna
Posts: 948
Joined: Wed Jul 15, 2009 11:36 am
Location: New York, NY

Re: Some Thoughts on Bonuses

Post by ThisIsMyUsername »

everdiso wrote: Tue Jan 18, 2022 3:37 pm On the topic of experimental changes to bonus design, I had one such idea recently: always ordering bonuses easy-medium-hard, and ending them after a team misses one part. The idea would be to save teams that don’t know much about a subject from sitting through two questions way above their level before finally hearing one gettable question at the end (that they may still miss).

This would be a relatively undisruptive change, as it would keep the current three-part, difficulty-gradated structure of bonuses and wouldn’t change scoring at all. One drawback would be the loss of the occasional times when a team gets a medium or hard part while missing an easier one, but I suppose the idea would now be that teams have to “qualify” for the harder questions by getting the earlier ones, much like they qualify for the bonus by getting a tossup. Another potential drawback would be that some weak teams might find it more frustrating to rarely get to hear the harder parts of their bonuses in games against better teams who hear most medium and hard parts of theirs than to sit through long questions without being able to answer them.

I'd be interested in hearing people's thoughts on this.
This would be a massive change to how bonuses are written. A good proportion of bonuses that are currently written would not work if they were reordered as E-M-H. Making that order automatic severely limits what you can do as a writer.

Failing to convert an easy part but then converting one or more of the other two parts is more common than you suggest. And when it does happen, it can be really satisfying. I also think it's far more demoralizing (and patronizing) to be told that you don't deserve to hear more questions than to have to sit through more questions that you don't convert.

The policy you're suggesting would also rob everyone in the room of hearing the interesting content in the remainder of the bonus. From a writer's perspective: many are already frequently disappointed when one of their packets isn't played; now imagine how disappointed they will be if several bonus parts per packet never see the light of day in most rooms! All that effort goes for naught. From a player's perspective: no one in the room gets to learn anything interesting about the parts they don't know.

As a general response to both this thread and the other thread: I don't like the idea that quizbowl questions should cater to people who are put off by hearing lots of information that they don't know, even if the set as a whole is accessible. That's far from the only argument in these threads being made for shortening questions, and some of the other arguments are good. (To be clear, I'm pro-shortening questions in general.) But that idea nonetheless runs through some of these posts and is pretty antithetical to the spirit of quizbowl.
John Lawrence
Yale University '12
King's College London '13
University of Chicago '20

“I am not absentminded. It is the presence of mind that makes me unaware of everything else.” - G.K. Chesterton
User avatar
Gene Harrogate
Lulu
Posts: 82
Joined: Fri Aug 04, 2017 11:05 pm

Re: Some Thoughts on Bonuses

Post by Gene Harrogate »

ThisIsMyUsername wrote: Wed Jan 19, 2022 11:40 am As a general response to both this thread and the other thread: I don't like the idea that quizbowl questions should cater to people who are put off by hearing lots of information that they don't know, even if the set as a whole is accessible. That's far from the only argument in these threads being made for shortening questions, and some of the other arguments are good. (To be clear, I'm pro-shortening questions in general.) But that idea nonetheless runs through some of these posts and is pretty antithetical to the spirit of quizbowl.
I actually haven't noticed anyone saying that they're put off by information that they don't know, and I wouldn't want anyone to come away from this post with that idea. The point of the first part of my post is that introducing extraneous information must be done responsibly so that bonuses are 1) readable over 10 hour days and 2) understandable by tired players. I have no problem with the sort of example Bruce posted, or even somewhat more ornate easy parts occasionally--I myself have gone out and read a book because I found a leadin anecdote interesting more than once. But right now, I'm concerned quizbowl is emphasizing didacticism at the expense of playability. We should also recognize that the vast majority of players will only ever hear a given question once, and that bonus parts with more clues than one can reasonably take in at a single time are really only being written for a small group of packet studiers.

EDIT: I'll note here that I think some excessive complexity comes from writers underestimating the differences between reading questions to themselves and out loud, especially when questions are only playtested as text in discord. I'll again plug reading every question out loud as a responsible writing habit.
Henry Atkins
McGill
User avatar
ValenciaQBowl
Auron
Posts: 2522
Joined: Thu Feb 05, 2004 2:25 pm
Location: Orlando, Florida

Re: Some Thoughts on Bonuses

Post by ValenciaQBowl »

This is an interesting thread, and I'm enjoying the discourse!

I wanted to comment on the hypothetical Gordimer bonus and Paul's assertion that throwing the word "apartheid" into the last part when asking for her nationality dilutes the "literature-ness" of a literature bonus. This is a fair point, as one obviously needs no knowledge of Gordimer's work to get it. Unfortunately, I can say with 100% certitude that plenty of weaker teams won't know Gordimer at all. Sure, this may be less likely among four year teams, even novice teams, but as someone immersed in the CC circuit, I can assure you that most of our teams won't have players (or coaches!) who have heard of Gordimer. Thus, I commonly throw in such off-topic, general knowledge clues for easy parts when writing Delta Burke. The top teams all know Gordimer, but they make up a quarter of the field.

The E/M/H thing, while intriguing, can lead to some difficulties when writing bonuses since, as John noted, there are often composition/flow reasons to not structure that way. Also, though this is probably not too common, among serious college teams there could always be the person who's read the book or randomly knows the historical detail clued in the hard part even if no one on that team knew the middle part.
Chris Borglum
Valencia College Grand Poobah
User avatar
Gene Harrogate
Lulu
Posts: 82
Joined: Fri Aug 04, 2017 11:05 pm

Re: Some Thoughts on Bonuses

Post by Gene Harrogate »

ValenciaQBowl wrote: Wed Jan 19, 2022 12:51 pm This is an interesting thread, and I'm enjoying the discourse!

I wanted to comment on the hypothetical Gordimer bonus and Paul's assertion that throwing the word "apartheid" into the last part when asking for her nationality dilutes the "literature-ness" of a literature bonus. This is a fair point, as one obviously needs no knowledge of Gordimer's work to get it. Unfortunately, I can say with 100% certitude that plenty of weaker teams won't know Gordimer at all. Sure, this may be less likely among four year teams, even novice teams, but as someone immersed in the CC circuit, I can assure you that most of our teams won't have players (or coaches!) who have heard of Gordimer. Thus, I commonly throw in such off-topic, general knowledge clues for easy parts when writing Delta Burke. The top teams all know Gordimer, but they make up a quarter of the field.
Glad you're enjoying it Chris! I think Paul and I would both agree that Nadine Gordimer isn't a proper easy part for an introductory quizbowl set. Our (I imagine) shared contention--and what I'm arguing for in the second part of my post--is that it's better to try to connect Gordimer to an easy part testing literature knowledge. So you could have a bonus on Gordimer where the easy part is the Nobel Prize in Literature, or some well-known author who influenced her.

I also agree that knowing harder parts but missing easier parts happens fairly often, and your and John's point that unused bonuses would be a bummer for writers who already put much work into a poorly compensated activity is well taken. But I would still like to see some experiments at the fringes of the college game.
Henry Atkins
McGill
User avatar
ThisIsMyUsername
Yuna
Posts: 948
Joined: Wed Jul 15, 2009 11:36 am
Location: New York, NY

Re: Some Thoughts on Bonuses

Post by ThisIsMyUsername »

Gene Harrogate wrote: Wed Jan 19, 2022 12:12 pm I actually haven't noticed anyone saying that they're put off by information that they don't know, and I wouldn't want anyone to come away from this post with that idea.
I agree that your initial post didn't say that, and I actually agree with much of what you said in that post. But both threads bring up the point that I drew attention to.

In this thread, Paul wrote:
The idea would be to save teams that don’t know much about a subject from sitting through two questions way above their level before finally hearing one gettable question at the end (that they may still miss).
And in the other thread, Eric Yin wrote:
Even if an answer line is easy, a longer tossup will have a greater number of harder clues that a given team wouldn't know, which makes for a less exciting game and more things to feel disenchanted about not knowing (which can sway people on the fence of participating). In contrast, shorter questions are over sooner, meaning that even if a question overshoots in difficulty it'll have less of an impact on players' overall experience listening to the packet.
I am strongly objecting to the idea that we should be "saving" people from this.
John Lawrence
Yale University '12
King's College London '13
University of Chicago '20

“I am not absentminded. It is the presence of mind that makes me unaware of everything else.” - G.K. Chesterton
User avatar
theMoMA
Forums Staff: Administrator
Posts: 5852
Joined: Mon Oct 23, 2006 2:00 am

Re: Some Thoughts on Bonuses

Post by theMoMA »

The original post raises several interesting points (and the ensuing discussion is also interesting). I have a couple "good writing practices" thoughts that I'll include later, but for now, I want to raise an objection to the framing of the part about the energy placed on bonuses in light of their supposed lack of importance to the competitive aspect of the game.

The primary objection I have to that is that bonuses are fun and serve extremely important noncompetitive purposes regardless of the number of games decided on bonus conversion. They allow for deep dives into subject matters in a way that tossups do not. They support themes and answers that are not suitable for tossups. They introduce players to interesting clues and topics, and they introduce writers to new ways to organize information, new connections between known subjects, and new ways of asking about old subjects. They provide an opportunity for players to contribute meaningfully to their team's success even if they're not getting a lot of tossups (particularly important for gunshy newer players). They allow moments for social interaction among teammates that otherwise do not exist in the atomized gameplay on tossups--and this is particularly meaningful because the wide variety of possible subjects that could give your team access to bonus points means that the player who got the tossup often has to rely on their teammates to do well on the bonus, and the good vibes from working together in this way are a huge part of what makes playing on a team worthwhile.

Bonuses also serve an important function in quizbowl's overall knowledge-testing scheme. Being an excellent bonus team/player requires solid, firmed-up knowledge across a variety of subjects. Naively modeled, clues exist in the mind as a knowledge binary (title/author, leader/country, battle/war, etc.). (Binary isn't quite right, because, at least for me, topics exist in my mind at the core of a cloud of associated topics, with varying strengths of connection between them--but this can be broken down into many, many single binaries existing in close proximity.) As Matt Bollinger teaches us with his well-known statement "that's a thing, Mike Sorice," knowledge monads (your knowledge of the mere existence of a thing with no associated knowledge to connect it to) can be nothing more than a starting point; without at least a weak association with some other piece of information, knowing that a thing exists has no possible cash-out value in points.

By contrast, weak knowledge binaries are extremely useful for answering tossups. There are many clues that, if they came up in a tossup, I'd be able to buzz correctly on instinct, feel, and the cumulative thrust of the previous clues, even though my knowledge of the actual information contained in the clues is at best tenuous. It goes without saying that, for many of these weak connections, I would not be able to get points if the clue were asked about in its own right.

Exploiting weak binaries to suss out a plausible tossup answer may be a valuable quizbowl skill (given the importance of tossup conversion, likely one of the most valuable skills), but it doesn't demonstrate mastery of the material. True mastery exists when a player can demonstrate a "both-way connection" on a knowledge binary--being able to identify not just a plausible answer from the clue, but to identify the clue when it's asked about on its own merits. Although tossup answers should vary in difficulty to keep players honest, they have to stick within a particular range and distribution to achieve desirable conversion rates. This is all good and proper, of course, and I wouldn't have it another way. But it means that tossups are not a particularly good way to test a team's both-way connections. Bonuses, by contrast, are the ideal format for asking about both-way connections and, in so doing, probing the depths of a team's knowledge about worthwhile topics in a way that tossups (even when buzzed early) by their nature cannot.

Finally, I take issue with the idea that bonus play is not competitively relevant to quizbowl. It's true that, within a certain band of competence, bonus conversion is much less likely to impact a game result than tossup conversion; we've seen time and again that teams with lower bonus conversions often win games and tournaments when they have roughly similar tossup conversion skill as teams that look a little better on PPB. (And this makes perfect sense, because marginal bonus conversion advantage of 3 PPB cashes out to a 30-point advantage over ten tossups converted, while the marginal value of each additional tossup converted is, for a good team at a hard tournament, 10 tossup points, plus ~20 bonus points, plus stopping the other team from getting 10-40 points, and ~60 marginal points is much more than 30.) I think it's safe to conclude that a team with a "true" bonus conversion of 19 is only slightly disfavored against a team with a "true" bonus conversion of 22 if both teams have a roughly similar "true" tossup conversion; the real test will almost certainly be which team can get that crucial 11th tossup (or 10th if one goes dead, etc.).

It's also true that, because tossup and bonus conversion both correlate to a team's overall "knowledge base," teams with good bonus conversions tend to have good tossup conversions. But, and I think this is very important to stress, this is not necessarily so! Bonus conversion is a separate skill, as I've just outlined, perhaps overly extensively, above. Although there's likely a pretty big "zone of indifference" in which bonus conversion doesn't matter much compared to tossup conversion, it's still very hard to get your team into this zone, and it requires a different focus to develop a separate skillset as answering tossups. It also requires the development of strong team-interaction skills that are not relevant to tossup play, without which a team will almost certainly lose crucial points in some close game when bickering or doubt leads to a flubbed bonus part.

It's very likely that there are some (perhaps many) teams with a good enough tossup conversion to be just about equally likely to win any given game, but cumulatively, their BBP disadvantage across several key matches means they would need to get really lucky to win the tournament. This is especially likely at a big tournament with a really competitive field such as Nationals or ICT, where if your lower PPB means even one additional expected loss, you're really unlikely to win the tournament. (Even if your team gets into the lower end of the zone of indifference, you're still starting at a 20-30 point disadvantage against an equally tossup-competent team with a 2-3 point PPB advantage, which means you're much more likely to need an outright tossup advantage to win, and in a big field with a lot of competitive teams, a 20-30 point starting disadvantage in a couple key matches is likely the difference between a really good chance to win the tournament and needing to get really, really lucky to win the tournament.)

In short, even if tossup conversion is 90%+ of the competitive aspect of the game, which it probably is, a 5-10% disadvantage is gigantic enough to spell almost certain doom for a title run across the four or five key games that a team needs to win at a national tournament. And if a team wants to close that gap or hold the PPB advantage themselves, they'll need to develop a distinct set of skills: individually, firming up knowledge binaries to demonstrate both-way knowledge, and collectively, developing team skills to make sure that they identify and give the correct answer.
Andrew Hart
Minnesota alum
User avatar
cchiego
Yuna
Posts: 874
Joined: Fri Jun 06, 2008 7:14 pm
Contact:

Re: Some Thoughts on Bonuses

Post by cchiego »

From a reader's perspective, I really hate seeing teams getting tripped up on the "easy" parts of a bonus because there are 5 lines of text between the lead-in and answer to the first part or there are 3 lines of difficult-to-parse quizbowlese on a middle part that includes one "obvious" clue alongside 2-3 "time to learn" clues. It's also frustrating when, by the time round 9 comes along, everyone's tired (especially if reading online) and you hit a 11-12-line bonus that gets painfully bageled. Not sure if the E-M-H-until-missing-a-part setup would be a good solution to that, but I appreciate the creative ideas like that generated in this thread.

I agree with Henry's point about NAQT questions being the easiest to read, in part because it feels less like lecturing and more engaging when the reader isn't talking so much. Making bonus parts in general less verbose (yes, even if it means team hear 'fewer clues') by doing hard limits on 2 lines a part or encouraging more 1 or 2 clue parts would be a great way to help teams and readers retain focus without appreciably hurting the overall didactic aspect of quizbowl. It simply isn't necessary to include so many overstuffed bonus parts, especially since they can confuse teams and wear readers out even more.

There also seems to be an unstated law to make sure that basically *all* teams get 10 PPB too, which is what can lead to "this red-colored planet"-like clues or more obvious examples of the "drink you have when you wake up in the morning" clues [loved that example, seen several bonuses play like that recently]. I would be interested to see what people think about this law of 10 PPB for all teams, since even though it's well-intended and ideally teams would be playing on sets in which the easy parts are sufficiently easy to mostly 10 (I suspect this set difficulty-experience mismatch is one of the big issues more broadly), it does seem to lead to a weird constriction of bonuses and to more of the painfully obvious gimme "easy 10s." And I do wonder as well if certain categories end up with more of these easy "gimme 10s" than others (checking the advanced stats for a comparison of the subjects that get more 0s vs. 10s on bonuses among lower PPB teams would be interesting). Perhaps this is where experimenting with two part M-H bonuses would help.
Chris C.
Past: UGA/UCSD/Penn
Present: Solano County, CA
User avatar
Mike Bentley
Sin
Posts: 6274
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: Some Thoughts on Bonuses

Post by Mike Bentley »

I think there is probably some psychological benefit to giving a correct answer even if everyone in the tournament is giving it. Hard to say how much this is worth in the grand scheme of things though.

I like Andrew's point of bonuses at national tournaments being something you need to be good at in order to compete at the top. And I definitely agree that bonuses promote teamwork in ways that tossups don't. However, I do think there's the opportunity cost of being able to play fewer tossups / games because you're spending such a long amount of time on bonuses.

I continue to think that some summer events should experiment with different formats that, when refined, could make their way back into regular quizbowl. For instance, I think that the next tossup-only event that I run will include some degree of conferring. Maybe this will be a disaster and we'll never do it again. But I also think it could be a way to make these events more fun to play as a team.
Mike Bentley
VP of Editing, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
User avatar
Cody
2008-09 Male Athlete of the Year
Posts: 2855
Joined: Sun Nov 15, 2009 12:57 am
Location: Richmond

Re: Some Thoughts on Bonuses

Post by Cody »

theMoMA wrote: Wed Jan 19, 2022 3:13 pmFinally, I take issue with the idea that bonus play is not competitively relevant to quizbowl. It's true that, within a certain band of competence, bonus conversion is much less likely to impact a game result than tossup conversion; we've seen time and again that teams with lower bonus conversions often win games and tournaments when they have roughly similar tossup conversion skill as teams that look a little better on PPB.
I’m not sure where the idea that bonus play is not competitively relevant comes from – though as Andrew notes, it wouldn’t particularly matter if it were true – but there is a plethora of data to test this on because almost everyone hosts their statistics on the hsquizbowl.org Tournament Database. The fact that tournaments, by design, do not feature exclusively competitive games means that many games are not “competitively relevant” and cannot be decided by bonuses, but that has no bearing on the “competitive relevance” of bonuses. (And, indeed, those games are important to the overall function of a tournament and quizbowl as a whole...just like bonuses.)

Below (and linked here) is data for the 2020–2021 ACF tournaments that kept statistics using YellowFruit. Bonus conversion is incredibly relevant at all levels of the college game, and is frequently the margin in any game that where the teams get within 2-tossups (which are themselves 20-35% of all games at a tournament). (Only one data point in this set featured a team overcoming a 3-tossup deficit.)

Note: this does not include data points where bonus points were the deciding factor for the winning team because they negged, though that is another way that bonuses can decide a game.

Image

Code: Select all

2021 ACF Nationals from August 07-08, 2021

Chicago A 285 (9 bheard, 200 bpts) def. WUSTL 245 (11 bheard, 140 bpts)
Harvard A 270 (10 bheard, 180 bpts) def. Stanford 240 (10 bheard, 150 bpts)
Stanford 250 (9 bheard, 180 bpts) def. North Carolina 225 (10 bheard, 130 bpts)
Brown 205 (9 bheard, 130 bpts) def. Chicago C 175 (9 bheard, 90 bpts)
Brown 195 (8 bheard, 130 bpts) def. Harvard B 185 (9 bheard, 100 bpts)
Princeton 195 (8 bheard, 120 bpts) def. Chicago C 145 (8 bheard, 80 bpts)
Columbia 220 (9 bheard, 140 bpts) def. North Carolina 200 (9 bheard, 120 bpts)
Texas 250 (9 bheard, 160 bpts) def. WUSTL 205 (9 bheard, 130 bpts)
Vanderbilt 145 (8 bheard, 70 bpts) def. Princeton 120 (8 bheard, 50 bpts)
Stanford 260 (9 bheard, 180 bpts) def. Columbia 245 (9 bheard, 170 bpts)
Maryland 260 (9 bheard, 180 bpts) def. Illinois 240 (9 bheard, 170 bpts)
Minnesota 220 (8 bheard, 150 bpts) def. Yale 200 (9 bheard, 120 bpts)
Texas 210 (9 bheard, 140 bpts) def. North Carolina 205 (9 bheard, 120 bpts)
Columbia 245 (8 bheard, 170 bpts) def. Stanford 235 (10 bheard, 160 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	14 of 173 total games – 8.1%
	14 of 83 close games (<= 3 bonuses heard margin) – 16.9%
	14 of 59 close games (<= 2 bonuses heard margin) – 23.7%
	of games –
		lowest winning PPB: 8.75
		median winning PPB: 17.89
		highest winning PPB: 22.22
		
note: 8 of 66 top-bracket games were decided this way.

================================================================================

2021 ACF Regionals High School Mirror on January 31, 2021

Kinkaid 255 (10 bheard, 170 bpts) def. "bees" 230 (10 bheard, 150 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	1 of 31 total games – 3.2%
	1 of 16 close games (<= 3 bonuses heard margin) – 6.2%
	1 of 13 close games (<= 2 bonuses heard margin) – 7.7%
	of games –
		lowest winning PPB: 17.00
		median winning PPB: 17.00
		highest winning PPB: 17.00

================================================================================

2021 ACF Regionals at University of Toronto on January 30, 2021

Toronto C 225 (9 bheard, 140 bpts) def. Queen's 205 (10 bheard, 120 bpts)
Waterloo A 260 (9 bheard, 170 bpts) def. McMaster 240 (10 bheard, 150 bpts)
McGill 310 (10 bheard, 220 bpts) def. Toronto A 290 (10 bheard, 190 bpts)
Kiel 230 (9 bheard, 150 bpts) def. Waterloo A 205 (9 bheard, 120 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	4 of 37 total games – 10.8%
	4 of 16 close games (<= 3 bonuses heard margin) – 25.0%
	4 of 10 close games (<= 2 bonuses heard margin) – 40.0%
	of games –
		lowest winning PPB: 15.56
		median winning PPB: 17.78
		highest winning PPB: 22.00

================================================================================

2021 ACF Regionals at Georgia Tech on January 30, 2021

Liberty C 190 (8 bheard, 110 bpts) def. UNF 165 (10 bheard, 90 bpts)
Alabama A 160 (9 bheard, 100 bpts) def. Liberty A 150 (9 bheard, 70 bpts)
South Carolina A 235 (9 bheard, 170 bpts) def. Virginia B 175 (9 bheard, 100 bpts)
Florida B 180 (8 bheard, 100 bpts) def. South Carolina B 150 (8 bheard, 80 bpts)
UNF 185 (8 bheard, 110 bpts) def. Liberty B 165 (8 bheard, 90 bpts)
Virginia B 190 (9 bheard, 110 bpts) def. UNF 180 (9 bheard, 100 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	6 of 82 total games – 7.3%
	6 of 37 close games (<= 3 bonuses heard margin) – 16.2%
	6 of 24 close games (<= 2 bonuses heard margin) – 25.0%
	of games –
		lowest winning PPB: 11.11
		median winning PPB: 13.12
		highest winning PPB: 18.89

================================================================================

2021 ACF Regionals at Nebraska on January 30, 2021

NC State 210 (8 bheard, 130 bpts) def. Northwestern C 180 (9 bheard, 110 bpts)
Northwestern B 235 (9 bheard, 150 bpts) def. WUSTL B 205 (10 bheard, 120 bpts)
Rice A 245 (9 bheard, 170 bpts) def. WUSTL B 175 (9 bheard, 110 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	3 of 63 total games – 4.8%
	3 of 20 close games (<= 3 bonuses heard margin) – 15.0%
	3 of 12 close games (<= 2 bonuses heard margin) – 25.0%
	of games –
		lowest winning PPB: 16.25
		median winning PPB: 16.67
		highest winning PPB: 18.89

================================================================================

2021 ACF Regionals at Minnesota on January 30, 2021

Minnesota C 185 (8 bheard, 140 bpts) def. Case Western B 145 (8 bheard, 70 bpts)
Michigan State A 155 (7 bheard, 90 bpts) def. Michigan State B 135 (7 bheard, 70 bpts)
Case Western A 230 (9 bheard, 140 bpts) def. Indiana 215 (9 bheard, 130 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	3 of 60 total games – 5.0%
	3 of 27 close games (<= 3 bonuses heard margin) – 11.1%
	3 of 16 close games (<= 2 bonuses heard margin) – 18.8%
	of games –
		lowest winning PPB: 12.86
		median winning PPB: 15.56
		highest winning PPB: 17.50

================================================================================

2021 ACF Regionals Invitational on January 30, 2021

Illinois A 315 (10 bheard, 230 bpts) def. MIT A 285 (10 bheard, 190 bpts)
Texas A 350 (10 bheard, 250 bpts) def. Stanford A 300 (10 bheard, 210 bpts)
Chicago A 325 (10 bheard, 230 bpts) def. Ohio State A 270 (10 bheard, 180 bpts)
Stanford A 310 (9 bheard, 230 bpts) def. Maryland A 300 (10 bheard, 210 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	4 of 67 total games – 6.0%
	4 of 26 close games (<= 3 bonuses heard margin) – 15.4%
	4 of 23 close games (<= 2 bonuses heard margin) – 17.4%
	of games –
		lowest winning PPB: 23.00
		median winning PPB: 24.00
		highest winning PPB: 25.56

================================================================================

2021 ACF Regionals at New York University on January 30, 2021

UNC A 270 (9 bheard, 190 bpts) def. Duke A 260 (9 bheard, 180 bpts)
Johns Hopkins A 265 (9 bheard, 190 bpts) def. Columbia C 245 (11 bheard, 140 bpts)
Penn A 200 (9 bheard, 130 bpts) def. Johns Hopkins B 195 (9 bheard, 110 bpts)
Johns Hopkins A 300 (10 bheard, 200 bpts) def. Duke B 260 (10 bheard, 170 bpts)
Princeton A 230 (9 bheard, 150 bpts) def. Penn State A 215 (9 bheard, 140 bpts)
Johns Hopkins B 185 (9 bheard, 100 bpts) def. GWU A 175 (9 bheard, 90 bpts)
Princeton A 310 (10 bheard, 230 bpts) def. Columbia B 265 (10 bheard, 180 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	7 of 83 total games – 8.4%
	7 of 34 close games (<= 3 bonuses heard margin) – 20.6%
	7 of 27 close games (<= 2 bonuses heard margin) – 25.9%
	of games –
		lowest winning PPB: 11.11
		median winning PPB: 20.00
		highest winning PPB: 23.00

================================================================================

2021 ACF Regionals at UW/USC on January 30, 2021

UBC A 245 (9 bheard, 170 bpts) def. Oregon 235 (10 bheard, 140 bpts)
UBC A 275 (10 bheard, 180 bpts) def. Caltech 250 (10 bheard, 150 bpts)
UBC A 300 (10 bheard, 200 bpts) def. UC  Davis 265 (10 bheard, 170 bpts)
UBC B 230 (9 bheard, 150 bpts) def. Caltech 200 (10 bheard, 110 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	4 of 40 total games – 10.0%
	4 of 21 close games (<= 3 bonuses heard margin) – 19.0%
	4 of 13 close games (<= 2 bonuses heard margin) – 30.8%
	of games –
		lowest winning PPB: 16.67
		median winning PPB: 18.44
		highest winning PPB: 20.00

================================================================================

2021 ACF Regionals at Imperial on January 30, 2021

Glasgow 265 (10 bheard, 170 bpts) def. Southampton B 250 (10 bheard, 150 bpts)
Edinburgh 225 (9 bheard, 140 bpts) def. Southampton B 170 (9 bheard, 90 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	2 of 78 total games – 2.6%
	2 of 27 close games (<= 3 bonuses heard margin) – 7.4%
	2 of 17 close games (<= 2 bonuses heard margin) – 11.8%
	of games –
		lowest winning PPB: 15.56
		median winning PPB: 16.28
		highest winning PPB: 17.00

================================================================================

2020 ACF Winter at Iowa on November 07, 2020

Iowa A 255 (9 bheard, 170 bpts) def. Grinnell A 245 (10 bheard, 160 bpts)
Grinnell A 265 (9 bheard, 180 bpts) def. Minnesota C 235 (9 bheard, 150 bpts)
Nebraska B 170 (7 bheard, 110 bpts) def. Iowa C 155 (7 bheard, 100 bpts)
Minnesota C 190 (9 bheard, 110 bpts) def. Nebraska A 175 (10 bheard, 90 bpts)
Iowa C 175 (8 bheard, 100 bpts) def. Nebraska B 155 (8 bheard, 80 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	5 of 45 total games – 11.1%
	5 of 11 close games (<= 3 bonuses heard margin) – 45.5%
	5 of 9 close games (<= 2 bonuses heard margin) – 55.6%
	of games –
		lowest winning PPB: 12.22
		median winning PPB: 15.71
		highest winning PPB: 20.00

================================================================================

2020 ACF Winter at Ohio State University on November 07, 2020

Carnegie Mellon B 245 (9 bheard, 160 bpts) def. Michigan State A 215 (10 bheard, 120 bpts)
Michigan B 315 (10 bheard, 220 bpts) def. Case Western A 280 (10 bheard, 200 bpts)
West Virginia 300 (10 bheard, 210 bpts) def. Carnegie Mellon A 270 (10 bheard, 180 bpts)
Carnegie Mellon B 205 (9 bheard, 130 bpts) def. Wayne State 165 (11 bheard, 60 bpts)
Michigan A 325 (10 bheard, 230 bpts) def. Carnegie Mellon A 235 (10 bheard, 150 bpts)
Michigan C 235 (9 bheard, 160 bpts) def. Carnegie Mellon B 230 (9 bheard, 150 bpts)
Case Western B 200 (9 bheard, 110 bpts) def. Wayne State 165 (9 bheard, 90 bpts)
Michigan State B 210 (10 bheard, 120 bpts) def. Wayne State 180 (10 bheard, 80 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	8 of 52 total games – 15.4%
	8 of 15 close games (<= 3 bonuses heard margin) – 53.3%
	8 of 14 close games (<= 2 bonuses heard margin) – 57.1%
	of games –
		lowest winning PPB: 12.00
		median winning PPB: 17.78
		highest winning PPB: 23.00

================================================================================

2020 ACF Winter at Vanderbilt University on November 07, 2020

Auburn A 205 (8 bheard, 130 bpts) def. South Carolina B 200 (10 bheard, 110 bpts)
South Carolina B 290 (9 bheard, 200 bpts) def. Florida B 275 (11 bheard, 180 bpts)
Florida C 235 (9 bheard, 150 bpts) def. South Carolina A 225 (10 bheard, 140 bpts)
Florida C 240 (10 bheard, 150 bpts) def. Georgia A 215 (10 bheard, 120 bpts)
South Carolina B 205 (8 bheard, 130 bpts) def. Georgia B 185 (9 bheard, 100 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	5 of 65 total games – 7.7%
	5 of 17 close games (<= 3 bonuses heard margin) – 29.4%
	5 of 15 close games (<= 2 bonuses heard margin) – 33.3%
	of games –
		lowest winning PPB: 15.00
		median winning PPB: 16.25
		highest winning PPB: 22.22

================================================================================

2020 ACF Winter South Central on November 07, 2020

Texas A&M A 240 (9 bheard, 150 bpts) def. Rice B 220 (10 bheard, 120 bpts)
Texas A 300 (10 bheard, 210 bpts) def. Oklahoma A 270 (10 bheard, 180 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	2 of 17 total games – 11.8%
	2 of 7 close games (<= 3 bonuses heard margin) – 28.6%
	2 of 5 close games (<= 2 bonuses heard margin) – 40.0%
	of games –
		lowest winning PPB: 16.67
		median winning PPB: 18.83
		highest winning PPB: 21.00

================================================================================

2020 ACF Winter at Rutgers on November 07, 2020

Princeton B 240 (8 bheard, 170 bpts) def. Lehigh 230 (11 bheard, 130 bpts)
Columbia B 350 (10 bheard, 250 bpts) def. Penn State 280 (10 bheard, 190 bpts)
Gettysburg 230 (10 bheard, 130 bpts) def. Yale 185 (10 bheard, 100 bpts)
Lehigh 190 (8 bheard, 120 bpts) def. Columbia C 155 (8 bheard, 90 bpts)
NYU A 300 (10 bheard, 200 bpts) def. Princeton B 290 (10 bheard, 190 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	5 of 81 total games – 6.2%
	5 of 21 close games (<= 3 bonuses heard margin) – 23.8%
	4 of 15 close games (<= 2 bonuses heard margin) – 26.7%
	of games –
		lowest winning PPB: 13.00
		median winning PPB: 20.00
		highest winning PPB: 25.00

================================================================================

2020 ACF Winter Toronto on November 07, 2020

British Columbia B 275 (10 bheard, 180 bpts) def. McMaster 210 (10 bheard, 120 bpts)
Toronto C 285 (9 bheard, 200 bpts) def. Waterloo A 245 (10 bheard, 150 bpts)
Toronto C 245 (9 bheard, 170 bpts) def. Toronto B 205 (9 bheard, 120 bpts)
Waterloo A 225 (9 bheard, 140 bpts) def. Toronto B 205 (9 bheard, 120 bpts)
Waterloo A 210 (8 bheard, 130 bpts) def. Western 170 (9 bheard, 100 bpts)
McGill C 275 (10 bheard, 180 bpts) def. Waterloo A 255 (10 bheard, 160 bpts)
British Columbia B 285 (10 bheard, 200 bpts) def. Toronto C 270 (10 bheard, 180 bpts)
McGill A 340 (10 bheard, 250 bpts) def. Toronto A 315 (10 bheard, 220 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	8 of 51 total games – 15.7%
	8 of 20 close games (<= 3 bonuses heard margin) – 40.0%
	8 of 15 close games (<= 2 bonuses heard margin) – 53.3%
	of games –
		lowest winning PPB: 15.56
		median winning PPB: 18.44
		highest winning PPB: 25.00

================================================================================

2020 ACF Winter at UW on November 07, 2020

Colorado B 195 (8 bheard, 120 bpts) def. Boise State 165 (9 bheard, 110 bpts)
Colorado A 245 (10 bheard, 160 bpts) def. Oregon 235 (10 bheard, 150 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	2 of 30 total games – 6.7%
	2 of 4 close games (<= 3 bonuses heard margin) – 50.0%
	2 of 3 close games (<= 2 bonuses heard margin) – 66.7%
	of games –
		lowest winning PPB: 15.00
		median winning PPB: 15.50
		highest winning PPB: 16.00

================================================================================

2020 ACF Winter @ USC on November 07, 2020

Caltech A 290 (10 bheard, 200 bpts) def. Stanford B 280 (10 bheard, 190 bpts)
Caltech B 220 (8 bheard, 140 bpts) def. USC B 175 (9 bheard, 90 bpts)
Berkeley B 245 (9 bheard, 170 bpts) def. Stanford B 235 (10 bheard, 150 bpts)
Caltech A 260 (9 bheard, 190 bpts) def. USC B 220 (9 bheard, 130 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	4 of 48 total games – 8.3%
	4 of 17 close games (<= 3 bonuses heard margin) – 23.5%
	4 of 15 close games (<= 2 bonuses heard margin) – 26.7%
	of games –
		lowest winning PPB: 17.50
		median winning PPB: 19.44
		highest winning PPB: 21.11

================================================================================

2020 ACF Winter Midwest on November 07, 2020

Chicago C 290 (10 bheard, 200 bpts) def. Chicago D 240 (10 bheard, 140 bpts)
Chicago B 290 (10 bheard, 210 bpts) def. Purdue A 285 (10 bheard, 200 bpts)
Chicago A 230 (9 bheard, 160 bpts) def. WUSTL A 215 (9 bheard, 140 bpts)
Chicago A 295 (10 bheard, 200 bpts) def. Illinois B 255 (10 bheard, 170 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	4 of 64 total games – 6.2%
	4 of 15 close games (<= 3 bonuses heard margin) – 26.7%
	4 of 12 close games (<= 2 bonuses heard margin) – 33.3%
	of games –
		lowest winning PPB: 17.78
		median winning PPB: 20.00
		highest winning PPB: 21.00

================================================================================

2020 ACF Winter High School Mirror on November 07, 2020

Lambert A 290 (10 bheard, 200 bpts) def. Sandburg A 245 (10 bheard, 150 bpts)
Lambert A 260 (9 bheard, 180 bpts) def. Detroit Catholic Central A 255 (9 bheard, 170 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	2 of 20 total games – 10.0%
	2 of 8 close games (<= 3 bonuses heard margin) – 25.0%
	2 of 6 close games (<= 2 bonuses heard margin) – 33.3%
	of games –
		lowest winning PPB: 20.00
		median winning PPB: 20.00
		highest winning PPB: 20.00

================================================================================

2020 ACF Winter at Warwick on November 07, 2020

Oxford D 260 (9 bheard, 170 bpts) def. Edinburgh A 215 (9 bheard, 160 bpts)
Oxford C 275 (10 bheard, 190 bpts) def. Oxford Brookes A 185 (10 bheard, 90 bpts)
Imperial B 270 (9 bheard, 190 bpts) def. Cambridge C 250 (10 bheard, 160 bpts)
Warwick A 240 (10 bheard, 150 bpts) def. Warwick B 210 (10 bheard, 120 bpts)
Oxford A 350 (10 bheard, 260 bpts) def. Oxford B 310 (10 bheard, 210 bpts)
Cambridge A 315 (10 bheard, 220 bpts) def. Southampton A 300 (10 bheard, 210 bpts)
Southampton A 315 (10 bheard, 220 bpts) def. Oxford A 300 (10 bheard, 200 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	7 of 121 total games – 5.8%
	7 of 38 close games (<= 3 bonuses heard margin) – 18.4%
	7 of 28 close games (<= 2 bonuses heard margin) – 25.0%
	of games –
		lowest winning PPB: 15.00
		median winning PPB: 21.11
		highest winning PPB: 26.00

================================================================================

2020 ACF Fall at McGill University on October 17, 2020

Toronto A 315 (10 bheard, 230 bpts) def. British Columbia B 310 (10 bheard, 220 bpts)
Toronto B 300 (10 bheard, 210 bpts) def. Toronto D 260 (10 bheard, 170 bpts)
Toronto A 290 (10 bheard, 200 bpts) def. Waterloo A 265 (10 bheard, 170 bpts)
McGill B 320 (10 bheard, 220 bpts) def. Waterloo A 250 (10 bheard, 150 bpts)
McMaster A 290 (10 bheard, 190 bpts) def. Waterloo B 235 (10 bheard, 140 bpts)
British Columbia B 285 (9 bheard, 190 bpts) def. McGill B 275 (11 bheard, 170 bpts)
McGill A 330 (9 bheard, 240 bpts) def. Western 315 (11 bheard, 210 bpts)
McMaster A 250 (9 bheard, 180 bpts) def. Toronto C 240 (9 bheard, 170 bpts)
McGill B 315 (10 bheard, 230 bpts) def. McMaster A 295 (10 bheard, 210 bpts)
Toronto A 275 (9 bheard, 200 bpts) def. Western 270 (11 bheard, 170 bpts)
Queen's B 275 (9 bheard, 190 bpts) def. McGill E 260 (11 bheard, 160 bpts)
Toronto D 270 (9 bheard, 190 bpts) def. Waterloo A 245 (10 bheard, 160 bpts)
McMaster A 295 (9 bheard, 210 bpts) def. Queen's A 260 (11 bheard, 180 bpts)
British Columbia A 340 (10 bheard, 260 bpts) def. McGill A 335 (10 bheard, 240 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	14 of 82 total games – 17.1%
	14 of 40 close games (<= 3 bonuses heard margin) – 35.0%
	14 of 29 close games (<= 2 bonuses heard margin) – 48.3%
	of games –
		lowest winning PPB: 19.00
		median winning PPB: 21.56
		highest winning PPB: 26.67

================================================================================

2020 ACF Fall at New York University on October 17, 2020

Swarthmore A 270 (9 bheard, 180 bpts) def. Johns Hopkins B 215 (10 bheard, 120 bpts)
Vassar C 190 (8 bheard, 130 bpts) def. Vassar B 145 (8 bheard, 70 bpts)
Rutgers B 290 (10 bheard, 200 bpts) def. Gettysburg B 275 (10 bheard, 180 bpts)
Princeton B 290 (10 bheard, 190 bpts) def. Swarthmore A 280 (10 bheard, 180 bpts)
Johns Hopkins A 305 (9 bheard, 220 bpts) def. Penn A 280 (11 bheard, 180 bpts)
Swarthmore A 275 (10 bheard, 180 bpts) def. Gettysburg B 225 (10 bheard, 130 bpts)
Swarthmore A 270 (9 bheard, 180 bpts) def. Rutgers B 265 (11 bheard, 170 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	7 of 82 total games – 8.5%
	7 of 24 close games (<= 3 bonuses heard margin) – 29.2%
	7 of 21 close games (<= 2 bonuses heard margin) – 33.3%
	of games –
		lowest winning PPB: 16.25
		median winning PPB: 20.00
		highest winning PPB: 24.44

================================================================================

2020 ACF Fall at Stony Brook University on October 17, 2020

Colgate A 240 (9 bheard, 160 bpts) def. Brandeis B 235 (9 bheard, 150 bpts)
RIT A 245 (10 bheard, 170 bpts) def. Colgate A 240 (10 bheard, 140 bpts)
Georgetown C 240 (9 bheard, 150 bpts) def. Colgate B 200 (10 bheard, 110 bpts)
RIT B 195 (8 bheard, 140 bpts) def. Brandeis B 160 (8 bheard, 90 bpts)
Williams A 315 (10 bheard, 220 bpts) def. Tufts 275 (10 bheard, 190 bpts)

Games from combined statistics report where the winning team scored more bonus
points despite hearing equal or fewer bonuses:
	5 of 82 total games – 6.1%
	5 of 33 close games (<= 3 bonuses heard margin) – 15.2%
	5 of 28 close games (<= 2 bonuses heard margin) – 17.9%
	of games –
		lowest winning PPB: 16.67
		median winning PPB: 17.50
		highest winning PPB: 22.00

================================================================================

Summary statistics for total games:
	min: 2.6%
	first quartile: 6.0%
	median: 7.9%
	third quartile: 10.6%
	max: 17.1%

Summary statistics for close games (<= 3 bonuses heard margin):
	min: 6.2%
	first quartile: 15.6%
	median: 23.7%
	third quartile: 29.4%
	max: 53.3%

Summary statistics for close games (<= 2 bonuses heard margin):
	min: 7.7%
	first quartile: 24.0%
	median: 28.7%
	third quartile: 40.0%
	max: 66.7%
[Made Cody's gigantic graph much smaller --staff]
Cody Voight (he/him), VCU ’14. I wrote lots of science and am an electrical engineer.
VCU Tournament Director ’13-’17. HSAPQ President ’15-’16. ACF Treasurer ’19-’20. ACF Nats ’21 TD.
Hero of Socialist Quizbowl Labor (NSC ’14). “esteemed colleague” of Snap Wexley, ca. 2016. Stats Hero (Nats ’16). “stats god in the flesh, the ominous Indominus Rex” (Winter ’20).
User avatar
Gene Harrogate
Lulu
Posts: 82
Joined: Fri Aug 04, 2017 11:05 pm

Re: Some Thoughts on Bonuses

Post by Gene Harrogate »

Cody wrote: Thu Jan 20, 2022 4:17 pm I’m not sure where the idea that bonus play is not competitively relevant comes from – though as Andrew notes, it wouldn’t particularly matter if it were true – but there is a plethora of data to test this on because almost everyone hosts their statistics on the hsquizbowl.org Tournament Database. The fact that tournaments, by design, do not feature exclusively competitive games means that many games are not “competitively relevant” and cannot be decided by bonuses, but that has no bearing on the “competitive relevance” of bonuses. (And, indeed, those games are important to the overall function of a tournament and quizbowl as a whole...just like bonuses.)
I appreciate you bringing some hard data into this thread, Cody. I agree that Andrew's post makes several good points about the intangible value of bonuses, and it is not my contention that bonuses have no value at all.

You quote the phrase “competitively relevant”; you'll note I never use that phrase in my post, just as I never claim bonuses are useless. What I do claim is that bonuses affect the outcome of something like less than 10 percent of games, and mostly function as a tiebreaker--which as far as I can tell is exactly what your data says.

The point in the third part of my post is that bonuses take up a disproportionate amount of time compared to their competitive utility. If that doesn't seem like a problem to you or others, that's fine. As I write in that section, there are many good things about bonuses. One of my proposed experiments is to actually increase the value of bonuses relative to tossups--bringing their competitive decisiveness more in line with the time spent writing and playing them. I do recognize that bonuses will always be secondary to tossups to some degree as long as bonuses heard depends on tossups gotten.
Henry Atkins
McGill
User avatar
naan/steak-holding toll
Auron
Posts: 2447
Joined: Mon Feb 28, 2011 11:53 pm
Location: Los Angeles, CA

Re: Some Thoughts on Bonuses

Post by naan/steak-holding toll »

Seconding the thanks for Cody's insightful post and analysis.
Will Alston
Bethesda Chevy Chase HS '12, Dartmouth '16, Columbia Business School '21
Mickey Mouse Ring-Bearer (ICT 2021)
User avatar
Gene Harrogate
Lulu
Posts: 82
Joined: Fri Aug 04, 2017 11:05 pm

Re: Some Thoughts on Bonuses

Post by Gene Harrogate »

One thing I would be interested in seeing: what happens with the ACF data when you remove tossup points? How dramatically does the importance of bonuses in deciding games increase? It would be interesting to treat tossups as fights over scoring opportunities, and bonuses as the actual scoring--more strongly rewarding the "both-way connections" that Andrew discussed. Intuitively, this changes the 1 tossup:3 ppb advantage in Andrew's example from 60:30 to 40:30.

EDIT: Just noticed there's a data link, awesome! I'll try to answer my own question if I can find the time soon.
Henry Atkins
McGill
User avatar
Cody
2008-09 Male Athlete of the Year
Posts: 2855
Joined: Sun Nov 15, 2009 12:57 am
Location: Richmond

Re: Some Thoughts on Bonuses

Post by Cody »

Gene Harrogate wrote: Thu Jan 20, 2022 10:12 pm One thing I would be interested in seeing: what happens with the ACF data when you remove tossup points? How dramatically does the importance of bonuses in deciding games increase? It would be interesting to treat tossups as fights over scoring opportunities, and bonuses as the actual scoring--more strongly rewarding the "both-way connections" that Andrew discussed. Intuitively, this changes the 1 tossup:3 ppb advantage in Andrew's example from 60:30 to 40:30.

EDIT: Just noticed there's a data link, awesome! I'll try to answer my own question if I can find the time soon.
Edit: a couple sites are messed up here; will exclude or troubleshoot tomorrow. fixed.

A different variation on the above – may get to this query tomorrow.

Image
full-size data (strictly fewer) data (equal or fewer)
Last edited by Cody on Fri Jan 21, 2022 5:26 pm, edited 6 times in total.
Cody Voight (he/him), VCU ’14. I wrote lots of science and am an electrical engineer.
VCU Tournament Director ’13-’17. HSAPQ President ’15-’16. ACF Treasurer ’19-’20. ACF Nats ’21 TD.
Hero of Socialist Quizbowl Labor (NSC ’14). “esteemed colleague” of Snap Wexley, ca. 2016. Stats Hero (Nats ’16). “stats god in the flesh, the ominous Indominus Rex” (Winter ’20).
User avatar
Cheynem
Sin
Posts: 6985
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Some Thoughts on Bonuses

Post by Cheynem »

I think there's four reasons I can think of in favor of bonuses beyond simply "do they affect who wins the game" and "do they take up more time than the net score might indicate."

1. We learn from them. While this can be taken to unhelpful and ludicrous extremes, I do think we can learn a lot from bonuses. Unlike tossups, in which a particularly good team or knowledgeable player can prevent a lot of clues from being heard, in most formats and play, we are more likely to hear more clues in bonus parts. In particular, lead-ins or unique hard (or even middle) parts can be very direct ways to learn information that could be hard to sprinkle into a lot of tossups.

2. They ask about things in different ways than tossups. This is especially true of bonus parts that are not simply on a person or work or "named" thing. There are just some bonus parts, even easy parts, that cannot be satisfactorily asked about in a tossup form. To use a very easy example, a bonus part asking for the phrase that begins the book of Genesis ("In the beginning"). I'm sure you can think of more creative and interesting and more difficult examples, which requires a different sort of knowledge--maybe more applied or deeper or less rote memorization than tossups.

3. They provide ways for less aggressive teammates to answer questions. This was helpful to me when I was starting college quizbowl. It's very intimidating sometimes to play on very good teams and knowing you don't want to neg. But I remember my teammates basically deferring to me on some categories like Bible. Even nowadays when I'm older and far more experienced, having a clunker of a game tossup-wise, but being able to ferret out answers or figure out bonus parts is very encouraging. Even simply being able to offer course correction is very empowering--to be able to say "No, they wanted the author, not the work," or "I don't know his name, but I think they're talking about the composer of Rigoletto." This is stuff you can't do in most tossup formats.

4. They provide a break from continually listening to tossups and being ready to buzz in. Perhaps even a too long break, of course, if you're on a team not getting a lot of bonuses. But assuming a reasonably fair distribution of buzzes going to both teams, it is nice to be able to relax for a few seconds on another team's bonus or even your own bonus, the mental equivalent of stepping out of the batter's box or slowly dribbling the basketball for a few seconds. (Playing solo, of course, on a tossup-bonus format is ridiculously mentally taxing.) Even on your own bonus--it's cool that if you recognize a clue, you can still wait for more information and you don't have to worry about another team buzzing against you.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Cody
2008-09 Male Athlete of the Year
Posts: 2855
Joined: Sun Nov 15, 2009 12:57 am
Location: Richmond

Re: Some Thoughts on Bonuses

Post by Cody »

Gene Harrogate wrote: Thu Jan 20, 2022 10:12 pm One thing I would be interested in seeing: what happens with the ACF data when you remove tossup points? How dramatically does the importance of bonuses in deciding games increase? It would be interesting to treat tossups as fights over scoring opportunities, and bonuses as the actual scoring--more strongly rewarding the "both-way connections" that Andrew discussed. Intuitively, this changes the 1 tossup:3 ppb advantage in Andrew's example from 60:30 to 40:30.

EDIT: Just noticed there's a data link, awesome! I'll try to answer my own question if I can find the time soon.
Image
full-size data (incl. ties) data (excl. ties)


a bunch in one:
Image
full-size
Last edited by Cody on Fri Jan 21, 2022 5:28 pm, edited 2 times in total.
Cody Voight (he/him), VCU ’14. I wrote lots of science and am an electrical engineer.
VCU Tournament Director ’13-’17. HSAPQ President ’15-’16. ACF Treasurer ’19-’20. ACF Nats ’21 TD.
Hero of Socialist Quizbowl Labor (NSC ’14). “esteemed colleague” of Snap Wexley, ca. 2016. Stats Hero (Nats ’16). “stats god in the flesh, the ominous Indominus Rex” (Winter ’20).
User avatar
Cody
2008-09 Male Athlete of the Year
Posts: 2855
Joined: Sun Nov 15, 2009 12:57 am
Location: Richmond

Re: Some Thoughts on Bonuses

Post by Cody »

Gene Harrogate wrote: Thu Jan 20, 2022 5:02 pm
Cody wrote: Thu Jan 20, 2022 4:17 pm I’m not sure where the idea that bonus play is not competitively relevant comes from – though as Andrew notes, it wouldn’t particularly matter if it were true – but there is a plethora of data to test this on because almost everyone hosts their statistics on the hsquizbowl.org Tournament Database. The fact that tournaments, by design, do not feature exclusively competitive games means that many games are not “competitively relevant” and cannot be decided by bonuses, but that has no bearing on the “competitive relevance” of bonuses. (And, indeed, those games are important to the overall function of a tournament and quizbowl as a whole...just like bonuses.)
I appreciate you bringing some hard data into this thread, Cody. I agree that Andrew's post makes several good points about the intangible value of bonuses, and it is not my contention that bonuses have no value at all.

You quote the phrase “competitively relevant”; you'll note I never use that phrase in my post, just as I never claim bonuses are useless. What I do claim is that bonuses affect the outcome of something like less than 10 percent of games, and mostly function as a tiebreaker--which as far as I can tell is exactly what your data says.

The point in the third part of my post is that bonuses take up a disproportionate amount of time compared to their competitive utility. If that doesn't seem like a problem to you or others, that's fine. As I write in that section, there are many good things about bonuses. One of my proposed experiments is to actually increase the value of bonuses relative to tossups--bringing their competitive decisiveness more in line with the time spent writing and playing them. I do recognize that bonuses will always be secondary to tossups to some degree as long as bonuses heard depends on tossups gotten.
To be sure, apologies for giving the wrong impression.

I do not agree with the 10% number, though, because I do not think it is appropriate to consider all games in a vacuum. Tossup and bonus conversion data is irrelevant to a significant percentage of games because a significant number of games are played between teams of disparate skill. You could reformat a quizbowl game any number of fair ways and those result would not change, including eliminating tossups and scoring the result of directing 10 bonuses to each team. Though this structure is extremely important to all competitions, it means that an analysis of competitive utility would be mistaken to draw conclusions based on the data from all games.

Given that everyone has to play the same format, the best data for which to consider the competitive utility of bonuses would be the playoffs of a bracketed round-robin, but this drastically reduces relevant data from local tournaments and eliminates many outright. Even with the best data, though, attempting to make an argument about how bonuses affect the outcome of games is extremely difficult because they are inextricably linked to the format. The data about subtracting TUpts demonstrates this because bonus opportunities are directly correlated with tossup points, so subtracting tossup points has little effect on results and you cannot really draw any conclusion from the data.

Even “removing” the “tiebreaker” aspect (requiring a winner with strictly fewer tossup points and equal or fewer bonuses heard) yields a lot of close games won on bonuses. While competitive games* are distinct but overlapping with close games**, fairly determining the winner of competitive and close games is the most important aspect of a quizbowl question format and so the results of close games are an important thing to key in on.

* games between teams of broadly similar skill – e.g. you would expect the same winner no more than, say, 3 out of 4 times
** empirical results within a certain score range, or proxy thereof – e.g. bonuses heard

Image
full-size

This is, I think, the kernel of my disagreement: inasmuch as an ex post facto analysis of box scores can proportion competitive utility (quizbowl win probability when?), I do not believe that an analysis shows that bonuses soak up more game time than their competitive utility would dictate, and I strongly disagree that this is a (big) problem for bonuses (or discussion thereof). While I don’t have any objection to tinkering with the tossup-bonus format (plenty of side events have done so), I find this Andrew Hart post to be right on the money.

csv of all 1,554 games for qualifying 2020–21 ACF tournaments

Btw, I do love the first two parts of the post – they are crucial considerations for all writers. To some extent, then, I have to apologize for the distraction from your main point (even this late in the thread), but I think this is an important point of disagreement.
Cody Voight (he/him), VCU ’14. I wrote lots of science and am an electrical engineer.
VCU Tournament Director ’13-’17. HSAPQ President ’15-’16. ACF Treasurer ’19-’20. ACF Nats ’21 TD.
Hero of Socialist Quizbowl Labor (NSC ’14). “esteemed colleague” of Snap Wexley, ca. 2016. Stats Hero (Nats ’16). “stats god in the flesh, the ominous Indominus Rex” (Winter ’20).
User avatar
theMoMA
Forums Staff: Administrator
Posts: 5852
Joined: Mon Oct 23, 2006 2:00 am

Re: Some Thoughts on Bonuses

Post by theMoMA »

Gene Harrogate wrote: Thu Jan 20, 2022 5:02 pmThe point in the third part of my post is that bonuses take up a disproportionate amount of time compared to their competitive utility. If that doesn't seem like a problem to you or others, that's fine. As I write in that section, there are many good things about bonuses. One of my proposed experiments is to actually increase the value of bonuses relative to tossups--bringing their competitive decisiveness more in line with the time spent writing and playing them. I do recognize that bonuses will always be secondary to tossups to some degree as long as bonuses heard depends on tossups gotten.
I'm not sure if this came through in my original post, but I take issue with the idea that the marginal role of bonus conversion in actual games means that the bonuses are not "competitively useful" in gameplay.

Although bonus skills do not often decide games, they are pretty much necessary to win games. A team that could only eke out the easy part on most bonuses would be at a tremendous disadvantage in pretty much every game, even if that team could keep pace with the best of them on the tossups. It so happens that everyone intuits this, and so they develop their individual skills and compose their teams in such a way that they're competitive on bonuses.

And it is not fait accompli that a team with good tossup skills are also good at bonuses. Answering bonuses well is very hard! And it requires a different mix of skills (remembering more minute details accurately) from the skill of answering tossups, even though there is overlap. Put another way, if bonuses didn't exist, there would be little reason for players to develop the specific skills that being good on bonuses requires (although presumably some players would still be naturally good at remembering more minute or precise details rather than just the general thrust of a particular clue). If you think the skill of remembering precise details about topics worth knowing is worthwhile, then bonuses serve an extremely important competitive purpose by making it virtually impossible to win games without at least baseline competence at this skill.

Bonuses also make games more complex, satisfyingly lengthy, and competitively fair and interesting. It's a lot harder to hide your lack of knowledge when a tossup in your wheelhouse can lead to a bonus in an embarrassing gap.
Andrew Hart
Minnesota alum
User avatar
Ben Dillon
Rikku
Posts: 319
Joined: Thu Apr 28, 2005 10:47 pm
Location: South Bend, IN
Contact:

Re: Some Thoughts on Bonuses

Post by Ben Dillon »

I find myself surprised to see some of the ideas in this thread. Some of them sound like a return to the Chip/NAC/Questions Unlimited format of bonuses. For those who aren't acquainted, they are four parts, are worth 5-10-15-20, are ordered by increasing difficulty to match the point value, end when the controlling team misses a part, and allow the other team to rebound only that missed part.
everdiso wrote: Tue Jan 18, 2022 3:37 pm On the topic of experimental changes to bonus design, I had one such idea recently: always ordering bonuses easy-medium-hard, and ending them after a team misses one part.
Years ago, the quiz bowl community reached a consensus about bonuses that ran almost diametrically opposed to the QU approach:
  • There should be three bonus parts;
  • Each part should be worth the same, regardless of difficulty; and
  • The parts should be of varying difficulty, but not necessarily ordered
An argument could be – and was at the time – based on the fact that this structure would create uniformity for writers, players, and scorekeepers. And a common argument in this forum was that proper quiz bowl should not be played with rules that shifted from question to question.

When my school played QU almost exclusively, I would notice lots of problems with QU's ability to accurately order the parts by difficulty. For example, their philosophy on math bonuses would be that the five-pointer would be 9th grade math, the ten-pointer would be 10th grade math, etc. So the first part might be "solve this difficult system of equations" whereas the last part might be a very easy "find the simple derivative".

And, unlike what the ACF data is showing, I found that the QU matches we played were often decided by the round in which bonuses applied. Even an inferior team with a substantial lead because of the bonus round almost always won.

It should be noted that QU was not the only holdout: College Bowl at the time also took a bit of a Wild West approach to this. There were "30-20-10" bonuses that decreased in difficulty, bonuses that were entirely related to the previous tossup, and bonuses with overly creative rubrics, e.g. "For five points apiece, name the six wives of Henry VIII."

I guess the point I'm really trying to make is that, although the experimentation can be fun, any broader decision about bonus structure should still keep writers, players, and scorekeepers in mind.
Ben Dillon, Saint Joseph HS

"Why, sometimes I've believed as many as
six impossible things before breakfast!"
User avatar
Gene Harrogate
Lulu
Posts: 82
Joined: Fri Aug 04, 2017 11:05 pm

Re: Some Thoughts on Bonuses

Post by Gene Harrogate »

theMoMA wrote: Sat Jan 22, 2022 8:59 pm
Gene Harrogate wrote: Thu Jan 20, 2022 5:02 pmThe point in the third part of my post is that bonuses take up a disproportionate amount of time compared to their competitive utility. If that doesn't seem like a problem to you or others, that's fine. As I write in that section, there are many good things about bonuses. One of my proposed experiments is to actually increase the value of bonuses relative to tossups--bringing their competitive decisiveness more in line with the time spent writing and playing them. I do recognize that bonuses will always be secondary to tossups to some degree as long as bonuses heard depends on tossups gotten.
I'm not sure if this came through in my original post, but I take issue with the idea that the marginal role of bonus conversion in actual games means that the bonuses are not "competitively useful" in gameplay.

Although bonus skills do not often decide games, they are pretty much necessary to win games. A team that could only eke out the easy part on most bonuses would be at a tremendous disadvantage in pretty much every game, even if that team could keep pace with the best of them on the tossups. It so happens that everyone intuits this, and so they develop their individual skills and compose their teams in such a way that they're competitive on bonuses.

And it is not fait accompli that a team with good tossup skills are also good at bonuses. Answering bonuses well is very hard! And it requires a different mix of skills (remembering more minute details accurately) from the skill of answering tossups, even though there is overlap. Put another way, if bonuses didn't exist, there would be little reason for players to develop the specific skills that being good on bonuses requires (although presumably some players would still be naturally good at remembering more minute or precise details rather than just the general thrust of a particular clue). If you think the skill of remembering precise details about topics worth knowing is worthwhile, then bonuses serve an extremely important competitive purpose by making it virtually impossible to win games without at least baseline competence at this skill.

Bonuses also make games more complex, satisfyingly lengthy, and competitively fair and interesting. It's a lot harder to hide your lack of knowledge when a tossup in your wheelhouse can lead to a bonus in an embarrassing gap.
I think we're talking past each other a bit here. I absolutely agree that bonuses are competitively useful as they exist currently, and was only arguing that they were relatively unimportant in comparison to tossups (despite taking up about half of game time). Cody has convinced me that the 90% number I was throwing around wasn't really the right way to think about things. I think it's still fair to say that bonuses are a secondary mechanic to tossups.

I enjoyed reading your points on the differences between knowledge required on tossups and bonuses. I agree that bonuses get at things tossups can't test and are harder to game in the moment (whether bonuses demonstrate more complete "mastery" of the subject is, I think, a more abstract question that can't get a complete discussion here--certainly bonuses get those "both-way" connections you mention, but it's my experience that tossups often better tap into into the vague recollections I have of actually reading something, whereas bonuses often reward me for memorizing a breadth of context-free proper nouns).

You, Mike, and Cody have all offered convincing reasons why bonuses are important, and I would oppose moving to a tossup-only format for regular tournaments. To me, the question is: can we keep the aspects of bonuses we all like while making the game more dynamic? I don't see any harm in brainstorming ideas, trying them out, and seeing how they play. You bring up that the current three part bonus is the equilibrium reached after a long period of development, and that's fair, but I'm not sure anything else has seriously been tried in the modern era of good writing practices. Cody is also right that I would stress the writing points of my post more than format tweaks--as long as we're writing three part bonuses, I'd like to see more attention paid to moderators and bottom-bracket teams.
Ben Dillon wrote: Sun Jan 23, 2022 10:32 pm I find myself surprised to see some of the ideas in this thread. Some of them sound like a return to the Chip/NAC/Questions Unlimited format of bonuses. For those who aren't acquainted, they are four parts, are worth 5-10-15-20, are ordered by increasing difficulty to match the point value, end when the controlling team misses a part, and allow the other team to rebound only that missed part.
everdiso wrote: Tue Jan 18, 2022 3:37 pm On the topic of experimental changes to bonus design, I had one such idea recently: always ordering bonuses easy-medium-hard, and ending them after a team misses one part.
Years ago, the quiz bowl community reached a consensus about bonuses that ran almost diametrically opposed to the QU approach:
  • There should be three bonus parts;
  • Each part should be worth the same, regardless of difficulty; and
  • The parts should be of varying difficulty, but not necessarily ordered
An argument could be – and was at the time – based on the fact that this structure would create uniformity for writers, players, and scorekeepers. And a common argument in this forum was that proper quiz bowl should not be played with rules that shifted from question to question.

When my school played QU almost exclusively, I would notice lots of problems with QU's ability to accurately order the parts by difficulty. For example, their philosophy on math bonuses would be that the five-pointer would be 9th grade math, the ten-pointer would be 10th grade math, etc. So the first part might be "solve this difficult system of equations" whereas the last part might be a very easy "find the simple derivative".

And, unlike what the ACF data is showing, I found that the QU matches we played were often decided by the round in which bonuses applied. Even an inferior team with a substantial lead because of the bonus round almost always won.

It should be noted that QU was not the only holdout: College Bowl at the time also took a bit of a Wild West approach to this. There were "30-20-10" bonuses that decreased in difficulty, bonuses that were entirely related to the previous tossup, and bonuses with overly creative rubrics, e.g. "For five points apiece, name the six wives of Henry VIII."

I guess the point I'm really trying to make is that, although the experimentation can be fun, any broader decision about bonus structure should still keep writers, players, and scorekeepers in mind.
I think it's unfair to compare anyone in this thread to Questions Unlimited. As far as I know Paul Kasinski isn't an apyramidal plagiarizer or an anti-competive crook--though he is a Leafs fan, which is probably worse. More to the point, I think the idea that good quizbowl necessarily needs to have three part bonuses of 10 points each is a bit dogmatic. As long as the format is consistent and well-written with difficulty targets in mind, I don't see any reason why alternatives would compromise the fairness of the game (I don't think anyone is advocating variable-point bonuses within a single packet, for instance). There are reasons to think that changing bonus formats might decrease statistical sensitivity and such, but these are tradeoffs with game speed and efficiency that already exist (we're not running four part bonuses right now).
Henry Atkins
McGill
User avatar
Ben Dillon
Rikku
Posts: 319
Joined: Thu Apr 28, 2005 10:47 pm
Location: South Bend, IN
Contact:

Re: Some Thoughts on Bonuses

Post by Ben Dillon »

Gene Harrogate wrote: Sun Jan 23, 2022 11:41 pm I think it's unfair to compare anyone in this thread to Questions Unlimited.
Well, I'm certainly not trying to do that! I just found it weird that we might be opening the door to bonuses that look more like the QU format. I love the discussion about bonus length/difficulty, but I guess I've thought of a three-part bonus as more of "settled law" at this point.
Ben Dillon, Saint Joseph HS

"Why, sometimes I've believed as many as
six impossible things before breakfast!"
User avatar
Sima Guang Hater
Auron
Posts: 1899
Joined: Mon Feb 05, 2007 1:43 pm
Location: Nashville, TN

Re: Some Thoughts on Bonuses

Post by Sima Guang Hater »

I was going to start a thread about overly tortuous bonuses and cognitive load, and I see that Henry had gone and done it already. I want to flesh out that part of the post a little bit more after my experience reading ACF regionals this weekend.

First, with easy parts, I want to in particular call out the idea of making easy parts overly didactic or burying the actual clue under a torrent of extra information - particularly when that extra information is difficult to pronounce. This isn't the best example, but this bonus part from this weekend partly illustrates the point
ACF Regionals 2022 wrote: 2. This title plant “thrives along disturbed edges” according to a book that recounts the myth of Skywoman Falling
and uses the life force “Puhpowee” to illustrate the “grammar of animacy.” For 10 points each:
[10h] Name this title plant of a book that outlines a reciprocal accord with nature called the Honorable Harvest; that
book titled for “braiding” this plant is by the Citizen Potawatomi author of Gathering Moss, Robin Wall Kimmerer.
ANSWER: sweetgrass [or Mary’s grass or manna grass or vanilla grass or holy grass or bison grass or
Hierochloe odorata or Anthoxanthum nitens; accept Braiding Sweetgrass; prompt on grass or Poaceae or
Gramineae]
[10m] Braiding Sweetgrass draws on Keewaydinoquay Peschel’s expertise in “botany” given this prefix. This prefix
modifies “medicine” and “psychopharmacology” to describe traditional ecological knowledge of Indigenous people.
ANSWER: ethno- [accept ethnobotany or ethnomedicine or ethnopsychopharmacology; prompt on folk medicine]
[10e] Richard E. Schultes studied the ethnobotany of these ritual experiences caused by entheogens. Terence
McKenna, the curandera María Sabina, and Michael Pollan have promoted these sensory events caused by
psychedelics.
This easy part is the third part of the bonus, which already features a fair amount of difficult-to-pronounce words that all but the best moderators would likely struggle with. The nucleus of this bonus part is that hallucinations and trips are caused by psychedelics. I understand why the writer specifically wanted to use the word "entheogens", because it was created to get rid of the negative connotation of the prefix "psych" in psychedelic or "hallucin-" in hallucinogens. At least this bonus puts the easiest clue at the end - it's also annoying to hear a line of extra clues after the easiest clue, which is something I (and other writers) am guilty of doing. However, you could test the same information by taking out a few of the names:
Richard E. Schultes studied the ethnobotany of these ritual experiences caused by entheogens. The curandera María Sabina has also promoted these sensory events caused by psychedelics.
Or if you wanted to be even more generous and not have people figure out how to pronounce "curandera", you could do this:
Richard E. Schultes studied the ethnobotany of these ritual experiences caused by entheogens. Terence McKenna promoted the use of psychedelics to trigger these events.
Cutting out the extra clues doesn't make the bonus any harder - I doubt there's going to be anyone in the field who doesn't know what psychedelics do that would get to the answer by hearing Michael Pollan or Maria Sabina's name. You could even cut out all three names if you wanted and do this:
Richard E. Schultes studied the ethnobotany of these ritual experiences caused by entheogens. These sensory events are caused by psychedelics like LSD.
The point is that you're not really adding much to the experience by adding a bunch of names to this easy part, in my opinion. Richard Schultes has to be in there to tie this bonus part to the rest of the bonus, but the rest of the names are either didactic in function or are supposed to help you get to the answer - and since they're clearly not doing the former, they're doing the latter, and I'd be wary of doing that excessively.

The other thing I want to call out is making bonus parts of all difficulties overly dense and difficult to digest (both from a reader and player perspective). A good example is this one:
ACF Regionals wrote:12. In 2014, the FDA recommended halving the dosage of this drug for women, since 10 to 15 percent of women
who take it may risk impaired driving eight hours later, compared to 3% of men. For 10 points each:
[10h] Name this imidazopyridine (“imid-azo-pyridine”) GABAA (“GAB-uh-A”) receptor agonist. This psycholeptic
Z-drug is prescribed far more often than other nonbenzodiazepines (“non-ben-zoh-dai-AA-zuh-peens”), like
eszopiclone (“ess-ZAHP-ih-clone”) or zaleplon (“zuh-LEP-lahn”).
ANSWER: zolpidem [accept Ambien CR or Intermezzo or Edluar or Zolpimist or Stilnox or Stilnoct or Sanval
or Ivadal; prompt on sleeping pills or sleep aids by asking “what specific drug?”; prompt on unfamiliar drug names
(many of which start with Zol) by requesting “please give a common American name”]
First of all, just step back and look at this bonus part - it looks like it's over half pronunciation guides, which is a problem in itself. And the PGs aren't even excessive, these are words that need them (I've apparently been pronouncing zaleplon wrong this whole time). There's a few core facts in this bonus - it's a sleep aid, it's a Z-drug/not a benzodiazepine, and it's not Lunesta (eszopiclone) or Sonata (zaleplon). The leadin about dosing in women isn't particularly well-known unless you're prescribing it regularly, but it's interesting and fine as a leadin. You can test the same information by doing a few cuts:
In 2014, the FDA recommended halving the dosage of this sedative-hypnotic for women, because it is more likely to cause impaired driving 8 hours later. For 10 points each:
[10h] Name this GABA-A (“GAB-uh-A”) receptor agonist. This non-benzo is prescribed more often than other Z-drugs like eszopiclone (“ess-ZAHP-ih-clone”) or zaleplon (“zuh-LEP-lahn”).
I took out the clue about it being an imidazopyridine because (while uniquely identifying for zolpidem over eszopiclone and zaleplon) it's not particularly helpful and chemical names are hard to pronounce for most moderators. I cut the word "psycholeptic" in favor of "sedative-hypnotic" since that's it's actual class, but you could use the word "psycholeptic" where I have the word "sedative-hypnotic" if you were worried that being too easy. I also cut down the leadin because the actual percentages and numbers aren't helpful, but the fact itself is important and interesting. Finally, I used "benzo" instead of "benzodiazepine", which is a little colloquial but I don't think anyone would mind you doing it. If you wanted to make it even easier to process you could use the brand names Lunesta and Sonata (I strongly oppose using brand names personally because clinicians using generic names helps lower drug costs, but I don't think this applies to quizbowl tournaments), though you'd run into the problem that brand names change depending on what country you're in. You could also just say it's the most prescribed Z-drug, which would be a little harder but still uniquely identifying enough that people will get it if they know the subject. Here's a minimal version of the bonus part that might be overdoing it:
In 2014, the FDA recommended halving the dosage of this sedative-hypnotic for women, because it is more likely to cause impaired driving 8 hours later. For 10 points each:
[10h] Name this GABA-A (“GAB-uh-A”) receptor agonist. This non-benzo is the most prescribed Z-drug in the US.
I would personally prefer the previous iteration (with the other Z-drug names, at least) since this one is a little threadbare, but that's an aesthetic choice. Either way, this bonus is now much easier to read, with all the important information, just by cutting a few words here and there and trying to prune the need for PGs.

In short - sometimes fewer facts are ok, and try to minimize the tortuous language and difficult-to-pronounce words as much as possible, please.
Eric Mukherjee, MD PhD
Brown 2009, Penn Medicine 2018
Vanderbilt Dermatology PGY-4
Writer, NAQT, NHBB, IQBT, ACF, PACE

“The next generation will always surpass the previous one. It’s one of the never-ending cycles in life.”
Support the International Pemphigus and Pemphigoid Foundation
User avatar
Mike Bentley
Sin
Posts: 6274
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: Some Thoughts on Bonuses

Post by Mike Bentley »

theMoMA wrote: Sat Jan 22, 2022 8:59 pm
Gene Harrogate wrote: Thu Jan 20, 2022 5:02 pmThe point in the third part of my post is that bonuses take up a disproportionate amount of time compared to their competitive utility. If that doesn't seem like a problem to you or others, that's fine. As I write in that section, there are many good things about bonuses. One of my proposed experiments is to actually increase the value of bonuses relative to tossups--bringing their competitive decisiveness more in line with the time spent writing and playing them. I do recognize that bonuses will always be secondary to tossups to some degree as long as bonuses heard depends on tossups gotten.
I'm not sure if this came through in my original post, but I take issue with the idea that the marginal role of bonus conversion in actual games means that the bonuses are not "competitively useful" in gameplay.

Although bonus skills do not often decide games, they are pretty much necessary to win games. A team that could only eke out the easy part on most bonuses would be at a tremendous disadvantage in pretty much every game, even if that team could keep pace with the best of them on the tossups. It so happens that everyone intuits this, and so they develop their individual skills and compose their teams in such a way that they're competitive on bonuses.

And it is not fait accompli that a team with good tossup skills are also good at bonuses. Answering bonuses well is very hard! And it requires a different mix of skills (remembering more minute details accurately) from the skill of answering tossups, even though there is overlap. Put another way, if bonuses didn't exist, there would be little reason for players to develop the specific skills that being good on bonuses requires (although presumably some players would still be naturally good at remembering more minute or precise details rather than just the general thrust of a particular clue). If you think the skill of remembering precise details about topics worth knowing is worthwhile, then bonuses serve an extremely important competitive purpose by making it virtually impossible to win games without at least baseline competence at this skill.

Bonuses also make games more complex, satisfyingly lengthy, and competitively fair and interesting. It's a lot harder to hide your lack of knowledge when a tossup in your wheelhouse can lead to a bonus in an embarrassing gap.
I'm quite skeptical that there are players / teams out there that have the ability to beat better teams consistently on tossups but are terrible on bonuses. Yes, getting bonus hard parts can sometimes require different skills than getting an early clue in a tossup. But I think these in practice are highly correlated. You can certainly find two teams that often get anywhere between 7 and 13 tossups against each other but differ maybe by 2 PPB. In these games, bonuses can be an important factor in deciding the game. But I continue to contend that they do take up an awfully long amount of gameplay for the relatively limited cases where they're deciders. I would really like to see quizbowl explore ways to keep some of the positive aspects while improving on the status quo (which I do not agree is some magical ideal because it's just been that way for a long time). Some of the obvious ones, mentioned elsewhere in this thread, is to trim down bonus parts lengths. But I think there are more radical changes that some lower-stakes tournaments should explore.
Mike Bentley
VP of Editing, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
Post Reply