Marking bonus parts as hard, medium, and easy

Elaborate on the merits of specific tournaments or have general theoretical discussion here.
Post Reply
User avatar
TaylorH
Lulu
Posts: 96
Joined: Wed Jun 18, 2014 7:31 pm

Marking bonus parts as hard, medium, and easy

Post by TaylorH »

While playing SCT last weekend and puzzling at the intended difficulty of parts of a bonus we heard, a thought occurred to me: why is it not standard practice to mark the intended hard, medium and easy parts of bonuses in the question text? Doing this would only be a tax of a few seconds on the part of the question writer/editor, and it could be an aid to players when packet studying or offering critique of question difficulty. I know I've seen writers mark their bonus parts as [e], [m] or [h] in various writing projects, only to have these tags edited out in the question text. Obviously, adding these tags to bonuses will not affect gameplay in any way and would only be useful post hoc. Standardizing such tags could be a valuable learning tool for new writers looking at older questions ("okay, so this answerline was a medium part at this past tournament") or could act as a learning guide for players studying old questions when all parts of a bonus are unfamiliar ("okay, I don't know anything in this bonus, so I should learn about [e] before looking into [m] and [h]"). In most cases, adding these tags is unnecessary as it is quite clear what the intended difficulty of each part is, but I've certainly lost count of the number of times I've heard players in practice or tournaments ask "what was the easy/hard part there?"

My recommendation would just be to slip the "e","m", or "h" into the [10] indicator that appears in the vast majority of modern bonuses. Something like:

Answer these questions about bonus difficulty, for 10 points each:
[10e] Bonuses should have certainly feature one of these parts that 90 percent of teams could answer.
ANSWER: _easy_ part
[10m] A three part bonus should also include these tougher parts that only 50 percent of teams could answer.
ANSWER: _medium_ part
[10h] Finally, a bonus should include a one of these challenging third parts that only teams with strong, specialized knowledge could answer.
ANSWER: _hard_ part

Let me know if this has ever been standard practice in the past, or if there are any drawbacks to this I have not considered.
Taylor Harvey
University of Florida Quiz Bowl
UF B.S. Nuclear Engineering '17
UF PhD Nuclear Engineering '21

User avatar
AKKOLADE
Sin
Posts: 15325
Joined: Thu Apr 24, 2003 8:08 am

Re: Marking bonus parts as hard, medium, and easy

Post by AKKOLADE »

Context matters a lot as to the difficulty of a bonus part. Some of this is based off of difficulty level, and some of it is based off of clue selection. You could make someone like George Washington a hard part at any level. I feel like this undermines the value of these hypothetical markings.
Fred Morlan
PACE President, 2018-19
International Quiz Bowl Tournaments, co-owner
University of Kentucky CoP, 2017
hsqbrank manager, NAQT writer (former subject editor), former hsqb Administrator/Chief Administrator, 2012 NASAT TD

User avatar
vinteuil
Auron
Posts: 1421
Joined: Sun Oct 23, 2011 12:31 pm

Re: Marking bonus parts as hard, medium, and easy

Post by vinteuil »

AKKOLADE wrote:
Sat Feb 15, 2020 11:37 pm
Context matters a lot as to the difficulty of a bonus part. Some of this is based off of difficulty level, and some of it is based off of clue selection. You could make someone like George Washington a hard part at any level. I feel like this undermines the value of these hypothetical markings.
How? I believe Taylor's point is precisely that the markings indicate the actual difficulty of the bonus part, as opposed to, e.g., what you might assume based on the answerline.
Jacob Reed (he/him/his)
Chicago ~'25 | Yale '19, '17 | East Chapel Hill '13
"...distant bayings from...the musicological mafia"―Denis Stevens

User avatar
Ike
Yuna
Posts: 948
Joined: Sat Jul 26, 2008 5:01 pm
Contact:

Re: Marking bonus parts as hard, medium, and easy

Post by Ike »

I don't think there is too much value or harm done in adding these tags. By and large, I think that one part of being good is intuitively learning what is a hard part and a medium part, and certainly these tags would help a beginner on some sense.

Also, I occasionally write bonuses that have two disparate parts that I would classify as being of the same difficulty - medium and hard. One example might be a bonus on George Washington where one of the two parts is on some aspect of his life and the other is on some other aspect. Some fraction of teams would convert part A, some other fraction convert part B. But having a player who has sufficient depth in both areas gets both parts, and thus 30s the bonus. In this case, (as opposed to the more popular case where you write three parts on one topic or aspect of increasing depth, it doesn't lend itself well to the model you describe.) I know I've seen a number of other writers do this in their questions, so I don't think I'm a particularly unique case in using this format.
Ike
UIUC 13

User avatar
AKKOLADE
Sin
Posts: 15325
Joined: Thu Apr 24, 2003 8:08 am

Re: Marking bonus parts as hard, medium, and easy

Post by AKKOLADE »

vinteuil wrote:
Sat Feb 15, 2020 11:43 pm
AKKOLADE wrote:
Sat Feb 15, 2020 11:37 pm
Context matters a lot as to the difficulty of a bonus part. Some of this is based off of difficulty level, and some of it is based off of clue selection. You could make someone like George Washington a hard part at any level. I feel like this undermines the value of these hypothetical markings.
How? I believe Taylor's point is precisely that the markings indicate the actual difficulty of the bonus part, as opposed to, e.g., what you might assume based on the answerline.
I could see it leading to someone who's new to writing questions and lacks the context to understand what it means for an answer to be labeled in something in one circumstance. I worry that that will lead to issues with assumptions and decisions being made with insufficient information, and that resulting in bad questions.

That may not be a good argument to not label bonus parts. I don't know that it is enough on its own. I just think it's a potential hazard that could come from this.
Fred Morlan
PACE President, 2018-19
International Quiz Bowl Tournaments, co-owner
University of Kentucky CoP, 2017
hsqbrank manager, NAQT writer (former subject editor), former hsqb Administrator/Chief Administrator, 2012 NASAT TD

User avatar
Carlos Be
Wakka
Posts: 153
Joined: Sun Jun 25, 2017 11:34 pm

Re: Marking bonus parts as hard, medium, and easy

Post by Carlos Be »

I think this would have a noticeable benefit for the version of the set read at a tournament or posted on the archives. Furthermore, I think it would have an enormous benefit for set production. On every set I've worked on, the most ubiquitous type of comment is stuff along the lines of "is this emh or meh?" It'd save a lot of time to have those questions answered by default.

I'd suggest that the labels be [E], [M], and [H]. Bonus parts that are not worth 10 points don't need to exist, anyway.
Ike wrote:
Sun Feb 16, 2020 1:02 am
Also, I occasionally write bonuses that have two disparate parts that I would classify as being of the same difficulty - medium and hard.
I think this actually furthers the case for labels. It would be very hard to figure out that a bonus has intended difficulty (mh)/e/(mh) (for example) without some sort of label indicating that. You could use something like [MH] for the ambiguous parts.
Justin/e French
UCLA

User avatar
ValenciaQBowl
Auron
Posts: 2391
Joined: Thu Feb 05, 2004 2:25 pm
Location: Orlando, Florida

Re: Marking bonus parts as hard, medium, and easy

Post by ValenciaQBowl »

I think this is a swell idea and hope that some set takes it up soon. Heck, I'll start by saying I'll use it for Delta Burke next year (actually a good tournament for such indicators, as when the set is mirrored for university teams I often get feedback that too many bonuses have two easy parts, an evaluation that doesn't fully take into account DB's core audience of first-time CC players in Orlando).

As a side note, I agree with Justin that prefacing bonus parts with [10] isn't something that seems necessary anymore (if it really ever was). I used A,B,C for bonus parts for a long time until it seemed wise to mirror ACF/mACF formatting. If you have a bonus part that's 5,5 or something you can just say so in the bonus text.
Chris Borglum
Valencia College Grand Poobah

User avatar
Sima Guang Hater
Auron
Posts: 1877
Joined: Mon Feb 05, 2007 1:43 pm
Location: Nashville, TN

Re: Marking bonus parts as hard, medium, and easy

Post by Sima Guang Hater »

I enjoy writing Schrodinger bonuses where the e/m/h change based on whether you're coming at it from having studied old packets vs real knowledge of the subject. Other than the difficulty of marking that I support this idea.
Eric Mukherjee, MD PhD
Brown University, 2009
Perelman School of Medicine at the University of Pennsylvania, 2018
Medicine Intern, Yale-Waterbury, 2018-9
Dermatology Resident, Vanderbilt University Medical Center, 2019-
Lord, Fiefdom of Impressionable Children, Unknown-

Member Emeritus, ACF
Member, PACE
Writer, NAQT, NHBB, IQBT

"The next generation will always surpass the previous one. It's one of the never-ending cycles in life."

User avatar
Cheynem
Sin
Posts: 6698
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Marking bonus parts as hard, medium, and easy

Post by Cheynem »

I think this would be an interesting thing, but I would urge people to treat the markings like the ordering of clues in tossups--they represent what the writers/editors think are the proper difficulty. They probably are "correct" for the most part, but not always. In other words, they do not represent the "objectively correct" ranking of difficulty and should not (in isolation) be used alone when considering difficulty for questions.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger

User avatar
Zealots of Stockholm
Rikku
Posts: 349
Joined: Fri Jul 10, 2015 3:28 am
Location: Auburn, AL

Re: Marking bonus parts as hard, medium, and easy

Post by Zealots of Stockholm »

Sima Guang Hater wrote:
Tue Feb 18, 2020 12:08 pm
I enjoy writing Schrodinger bonuses where the e/m/h change based on whether you're coming at it from having studied old packets vs real knowledge of the subject. Other than the difficulty of marking that I support this idea.
could you give an example of this?
Chandler West
Auburn University '20
Good Hope High School (Cullman, AL) '16
Member, PACE

User avatar
Mike Bentley
Auron
Posts: 5937
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: Marking bonus parts as hard, medium, and easy

Post by Mike Bentley »

100% Clean Comedian Dan Nainan wrote:
Tue Feb 18, 2020 2:04 pm
Sima Guang Hater wrote:
Tue Feb 18, 2020 12:08 pm
I enjoy writing Schrodinger bonuses where the e/m/h change based on whether you're coming at it from having studied old packets vs real knowledge of the subject. Other than the difficulty of marking that I support this idea.
could you give an example of this?
I see this in contemporary literature. There will often be a clue that rewards pretty surface-level knowledge of a recently-written book that's a leadin or a hard clue for a bonus part. Someone reasonably clued in to what's trendy in this scene will be able to buzz pretty easily on these clues with the assumption that because they haven't come up before, they're harder than the more canonical clues later in the question.
Mike Bentley
VP of Editing, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008

User avatar
Auks Ran Ova
Forums Staff: Chief Administrator
Posts: 4094
Joined: Sun Apr 30, 2006 10:28 pm
Location: Minneapolis
Contact:

Re: Marking bonus parts as hard, medium, and easy

Post by Auks Ran Ova »

Cheynem wrote:
Tue Feb 18, 2020 12:13 pm
I think this would be an interesting thing, but I would urge people to treat the markings like the ordering of clues in tossups--they represent what the writers/editors think are the proper difficulty. They probably are "correct" for the most part, but not always. In other words, they do not represent the "objectively correct" ranking of difficulty and should not (in isolation) be used alone when considering difficulty for questions.
This is one reason I'm not a big fan of this proposal--it lends an "officialness" to what's ultimately the best guess of a writer/editor that can be misleading, especially given that people already cite things like "number of database hits (in a vacuum)" to justify or criticize difficulty. The overall negative impact would probably be pretty low, of course, but I do think it'd be more beneficial to encourage less experienced writers to, like, ask a teammate, or another member of the community, about things like relative bonus part difficulty in some old packet.

[10] labels certainly aren't explicitly necessary--NAQT uses A./B./C., for example--but I don't really see a problem with continuing to use them for continuity's sake. (also, I wouldn't be me if I didn't point out that "Bonus parts that are not worth 10 points don't need to exist" is dogmatic and silly; perfectly reasonable 5/5/10/10 or (5,5)/10/10-type bonuses can and have been written, for example, though for the most part sticking to 10/10/10 for ease and simplicity is perfectly reasonable.)
Rob Carson
University of Minnesota '11, MCTC '??
Member, ACF
Member, PACE
Writer and Editor, NAQT

User avatar
vinteuil
Auron
Posts: 1421
Joined: Sun Oct 23, 2011 12:31 pm

Re: Marking bonus parts as hard, medium, and easy

Post by vinteuil »

justinfrench1728 wrote:
Sun Feb 16, 2020 10:51 pm
I think this would have a noticeable benefit for the version of the set read at a tournament or posted on the archives. Furthermore, I think it would have an enormous benefit for set production. On every set I've worked on, the most ubiquitous type of comment is stuff along the lines of "is this emh or meh?" It'd save a lot of time to have those questions answered by default.
This is just as easily solved by having people mark all bonuses as "h/m/e" "e/h/m" etc. in the answer spreadsheet (a practice I encourage).
Jacob Reed (he/him/his)
Chicago ~'25 | Yale '19, '17 | East Chapel Hill '13
"...distant bayings from...the musicological mafia"―Denis Stevens

User avatar
Auroni
Auron
Posts: 3059
Joined: Thu Nov 15, 2007 6:23 pm
Location: Urbana

Re: Marking bonus parts as hard, medium, and easy

Post by Auroni »

I like this idea — it has the potential to be a friendly, easily usable guide for newer players or people who are just unfamiliar with a given bonus’s subject, which most players are likely to be with respect to very specialized categories, such as different areas of science. The concerns raised in this thread can be addressed by just tuning the language in implementation. The phrase “intended e/h/m,” for instance, could be used to avoid making a categorical claim as to the absolute difficulty of a given answer. Also, a bonus with two parts that might be medium or easy based on complementary sets of knowledge might be marked “intended e or m/h/e or m.”
Auks Ran Ova wrote:
Wed Feb 19, 2020 2:34 am
I do think it'd be more beneficial to encourage less experienced writers to, like, ask a teammate, or another member of the community, about things like relative bonus part difficulty in some old packet.


While we’re certainly not lacking for people in this community who would be happy to field questions like these, it’s important to recognize that this may not be an option for everyone — it’s not guaranteed that newer players will have either easy access to people with the requisite knowledge or the level of comfort needed to ask a stranger for help, and it may be prohibitively annoying to repeatedly ask questions about several bonus parts. It seems better to have an easy to implement system that provides the answer at a glance to cover every situation from people wondering what the easy part was of a particularly rough bonus mid-tournament, to teams needing to know which is the medium or hard part was to a deep-looking bonus in a category they’re trying to study by reading packets.
Last edited by Auroni on Wed Feb 19, 2020 9:49 am, edited 1 time in total.
Auroni Gupta
UIUC
ACF

"Slaves aren't people lmao." - Ike Jose

User avatar
CPiGuy
Tidus
Posts: 739
Joined: Wed Nov 16, 2016 8:19 pm
Location: Ann Arbor, MI

Re: Marking bonus parts as hard, medium, and easy

Post by CPiGuy »

Auks Ran Ova wrote:
Wed Feb 19, 2020 2:34 am
"Bonus parts that are not worth 10 points don't need to exist" is dogmatic and silly; perfectly reasonable 5/5/10/10 or (5,5)/10/10-type bonuses can and have been written
why do you want to make my stats software cry
Conor Thompson (he/him)
Bangor HS (Maine) '16
Michigan '20

User avatar
Carlos Be
Wakka
Posts: 153
Joined: Sun Jun 25, 2017 11:34 pm

Re: Marking bonus parts as hard, medium, and easy

Post by Carlos Be »

Auks Ran Ova wrote:
Wed Feb 19, 2020 2:34 am
Cheynem wrote:
Tue Feb 18, 2020 12:13 pm
I think this would be an interesting thing, but I would urge people to treat the markings like the ordering of clues in tossups--they represent what the writers/editors think are the proper difficulty. They probably are "correct" for the most part, but not always. In other words, they do not represent the "objectively correct" ranking of difficulty and should not (in isolation) be used alone when considering difficulty for questions.
This is one reason I'm not a big fan of this proposal--it lends an "officialness" to what's ultimately the best guess of a writer/editor that can be misleading, especially given that people already cite things like "number of database hits (in a vacuum)" to justify or criticize difficulty.
The "officialness" still exists without the labels. You already see people asserting "this was a medium part at x tournament"; I don't think that's meaningfully different from "this was labelled M at x tournament."
Auks Ran Ova wrote:
Wed Feb 19, 2020 2:34 am
also, I wouldn't be me if I didn't point out that "Bonus parts that are not worth 10 points don't need to exist" is dogmatic and silly; perfectly reasonable 5/5/10/10 or (5,5)/10/10-type bonuses can and have been written, for example
There are many formats other than 10/10/10 that could be considered reasonable. However, there is never any need to write them— wacky formats have no measurable advantage over 10/10/10, and they make score-keeping unnecessarily complicated.
vinteuil wrote:
Wed Feb 19, 2020 3:05 am
justinfrench1728 wrote: ↑Sun Feb 16, 2020 10:51 pm
I think this would have a noticeable benefit for the version of the set read at a tournament or posted on the archives. Furthermore, I think it would have an enormous benefit for set production. On every set I've worked on, the most ubiquitous type of comment is stuff along the lines of "is this emh or meh?" It'd save a lot of time to have those questions answered by default.
This is just as easily solved by having people mark all bonuses as "h/m/e" "e/h/m" etc. in the answer spreadsheet (a practice I encourage).
This is true, but it leaves room for error if (for example) the order of two parts are exchanged and the spreadsheet isn't updated.
Justin/e French
UCLA

User avatar
vinteuil
Auron
Posts: 1421
Joined: Sun Oct 23, 2011 12:31 pm

Re: Marking bonus parts as hard, medium, and easy

Post by vinteuil »

I like this idea in theory and think it's worth experimenting with. I'll be trying it in future sets.
Jacob Reed (he/him/his)
Chicago ~'25 | Yale '19, '17 | East Chapel Hill '13
"...distant bayings from...the musicological mafia"―Denis Stevens

Post Reply