What Makes a Good Regular-Difficulty Set (comparatively)

Old college threads.
Locked
Urech hydantoin synthesis
Tidus
Posts: 525
Joined: Thu Dec 30, 2010 3:35 pm

What Makes a Good Regular-Difficulty Set (comparatively)

Post by Urech hydantoin synthesis »

This question is partly inspired by this post by John Lawrence, and Harry White's question regarding why no set from the past 2 years was on that list. What has ACF Regionals 2010 done to make it stand out from the other regular-difficulty sets in the past decade (or conversely, what have recent sets done that make them inferior to sets on that list)? Ike's praise of MAGNI's science in that thread seems like a good start, but I think it would be helpful to have a thread to discuss the positive aspects of recent tournaments and advice for what future writers should do or emulate.
Ben Zhang

Zucker School of Medicine at Hofstra/Northwell '23
Columbia University '18
Ladue Horton Watkins HS '14
User avatar
Cheynem
Sin
Posts: 7219
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Cheynem »

My four points for a good regular difficulty set:

1. Gradates. Does the set properly distinguish between all teams? Could, for example, Dickety Doo U and Maryland B (the two weakest teams in a field, let's say) have a meaningful game on it, but also Michigan A and Chicago A? I'm not saying there aren't a few muddy games here and there, but you also have lead-ins and hard parts that challenge your top teams, along with late clues that allow weak teams to gradate knowledge.

2. Accessible. I really respect sets that think of ways of writing questions that can reward knowledge of harder or deeper material but still make it accessible for average to weaker teams. Good sets I think can avoid the temptation to just write on individual Dziga Vertov films (or even Vertov himself!) and find ways of pumping out creative ways of writing on the Vertov oeuvre. Similarly, good bonuses have gettable easy parts that almost every team can get but are not just "find your ass," while hard parts are not "stump the chump" but rather flow intelligently from the material.

3. Interesting. As an Ike thread discussed, questions should be interesting. When I take out the set and read it or listen to it, I want to say "that's interesting" at a number of questions or clues. I don't want to play title bowl or operation bowl.

4. Care. This is hard to quantify, but I think great sets demonstrate clear care in being put together. You're not slapping this set together at 1 AM the night before. Independent of good packet feng shui or lack of typos or errors, though, there's just something about the great sets that...really feel like they show the work, empathy, and consideration that the editors put in. I'm not advocating for pretension here; sometimes the best sets are the ones that don't draw attention to themselves, but you get a feel like "yes, the editors really worked hard on this set."
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
naan/steak-holding toll
Auron
Posts: 2514
Joined: Mon Feb 28, 2011 11:53 pm
Location: New York, NY

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by naan/steak-holding toll »

I think the major things that distinguish good sets these days are constraints on difficulty, particularly in hard parts of bonuses. Beyond this, I think the usual markers of writing strength apply - a focus on rewarding descriptions over names, choosing clues that are actually important as opposed to trivial/not evocative/filler, etc.
Will Alston
Dartmouth College '16
Columbia Business School '21
bradleykirksey
Wakka
Posts: 187
Joined: Sat Nov 12, 2011 5:09 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by bradleykirksey »

"Interesting" and "care" cover what I was going to say pretty well, but I think another part of it is lead-ins and how unique your clues and answer lines are.

To pick an example about the lead-ins, there are 10 tossups on Esther in Quinterest. 2 of them mention Bigthana and Teresh in the first line and another 2 mention them in the second line. 9 of the 10 tossups include a clue on Vashti. That's cool, right? FTPs are all going to look the same. But actually, that's still a routinely in the middle of the tossup. 7 of the 8 tossups on the Pre-Raphaelite Brotherhood mentioned Elizabeth Siddal. 2 of the last 3 tossups on them put her in what would have definitely have been power range if there were powers, even though she's definitely in stock clue territory.

IMO, 2013 Dragoon's PRB tossup handles it properly. In addition to not mentioning Siddal at all, it described The Miracle of the Holy Fire in line 1 and The Scapegoat in line 2. It avoids the stock clues and puts in relevant, non-trampled-to-death clues in power before getting around to the "accessible" clues we've all come to known and buzz on. It gives a serious advantage to someone who actually knows art over someone who knows quinterest. Because if quiz bowl keeps re-using the same clues, I feel like it's easier for people with an internet connection and a little too much free time to beat people with real knowledge to tossups at regular difficulty. This goes away at the ICT/Nats level, but at "regular" level, this seems to be a pretty common trap. And I think that regular difficulty sets that are done well with writers who care about interesting clues tend to avoid doing that.

Also, some answer lines may get used a little too much at "regular" difficulty. ICT and Nats really open up the playbook of potential answers, and it's easier to avoid tossing up a 15th tossup on Jose Saramago, and the difficulty allows you to actually maybe toss up Blindness or a tougher writer like maybe Cecil Day-Lewis. I get that it's really hard to come up with new, relevant classic literature works that haven't been tossed up often before and are accessible. But at the same time, there are 4 tossups on Susan B Anthony to 34 on Emile Durkheim. Durkheim is important and good and all of that, but I think that an over-reliance on the same answer-lines while eschewing other important things really hurts. And personally, I feel like that good sets make an attempt to prevent that and find good, important answer lines that haven’t been tossed up 2 or 3 times in the last year. For my two cents, in addition to Mike Cheyne’s points, those are the things that stick out to me as a well written and enjoyable set.

TL;DR Some guy on the fringe of quiz bowl thinks you should stop putting “monopoly on the legitimate use of force” as a near-power clue in every regular difficulty tournament please.
Bradley Kirksey
Mayor of quiz bowl at the University of Central Florida (2010-2015)
The club at Reformed Theology Seminary Orlando (2017 - 2021)
User avatar
Adventure Temple Trail
Auron
Posts: 2751
Joined: Tue Jul 15, 2008 9:52 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Adventure Temple Trail »

bradleykirksey wrote:To pick an example about the lead-ins, there are 10 tossups on Esther in Quinterest. 2 of them mention Bigthana and Teresh in the first line and another 2 mention them in the second line. 9 of the 10 tossups include a clue on Vashti. That's cool, right? FTPs are all going to look the same. But actually, that's still a routinely in the middle of the tossup. 7 of the 8 tossups on the Pre-Raphaelite Brotherhood mentioned Elizabeth Siddal. 2 of the last 3 tossups on them put her in what would have definitely have been power range if there were powers, even though she's definitely in stock clue territory.
Piping in quickly to say as someone who's written at least three tossups about Purim/the book of Esther in my time, that the Book of Esther really isn't that long and doesn't have very many salient details to draw on. So tossups on it/her are going to draw on those same details as clues over and over for lack of any genuine alternative.
Matt Jackson
University of Chicago '24
Yale '14, Georgetown Day School '10
member emeritus, ACF
User avatar
Cheynem
Sin
Posts: 7219
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Cheynem »

I would say that it's not so much the answerlines, it's the clues, or I guess more accurately, sometimes all it takes is a little tweaking of an answerline to take a stale answerline and make it more interesting. Writing on the "Valley of Ashes" instead of The Great Gatsby, for example, or "Gatsby's shirts," or "Gatsby's house," etc.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Auroni
Auron
Posts: 3145
Joined: Thu Nov 15, 2007 6:23 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Auroni »

Or, since not every answerline can be creative, taking a fresh approach or choosing a self-evident theme with your standard answerlines goes a long way in improving the quality of your set. For instance, at MLK this year, Will wrote a tossup on Picasso focusing on his sculptures, which was unquantifiably better than all past tossups on Picasso at regular difficulty that went with a random, unconnected sample of his artworks.
Auroni Gupta (she/her)
User avatar
Mike Bentley
Sin
Posts: 6461
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Mike Bentley »

Auroni wrote:Or, since not every answerline can be creative, taking a fresh approach or choosing a self-evident theme with your standard answerlines goes a long way in improving the quality of your set. For instance, at MLK this year, Will wrote a tossup on Picasso focusing on his sculptures, which was unquantifiably better than all past tossups on Picasso at regular difficulty that went with a random, unconnected sample of his artworks.
I think theme questions certainly can and do work well, but I'd caution writers not to go too far in the direction of every "regular" tossup needs a theme. I've seen writers try to cram ill-advised themes into questions when a more conventional approach to writing the question would be better.
Mike Bentley
Treasurer, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
User avatar
Cheynem
Sin
Posts: 7219
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Cheynem »

I would say actually it's frequently better to use a standard answerline with thematic, organized clues on a specific aspect or trait rather than writing the tossup on the aspect or trait itself. For example, I think a tossup on Picasso using sculpture clues is better than a tossup on "Picasso's sculptures" (of course, I wrote a tossup on "photographs of Bourke-White" once so there you go). I also like really writing tossups on things that are basically "Abraham Lincoln, before he was President" or "Winston Churchill's 2nd stint as Prime Minister" or what have you because the vast majority of the clues are about that.

I would also agree with Mike that you don't have to do this with every tossup and that if you're not sure how to do it, then don't.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Auroni
Auron
Posts: 3145
Joined: Thu Nov 15, 2007 6:23 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Auroni »

Mike Bentley wrote:
Auroni wrote:Or, since not every answerline can be creative, taking a fresh approach or choosing a self-evident theme with your standard answerlines goes a long way in improving the quality of your set. For instance, at MLK this year, Will wrote a tossup on Picasso focusing on his sculptures, which was unquantifiably better than all past tossups on Picasso at regular difficulty that went with a random, unconnected sample of his artworks.
I think theme questions certainly can and do work well, but I'd caution writers not to go too far in the direction of every "regular" tossup needs a theme.
Right, I wasn't saying that every question needs to be like this. (In fact, if you are writing on an answer that can't be tossed up at a lower difficulty, then you should not use this approach, since you need to use all the easiest clues, whatever they may be.) But little things like this can make questions more enjoyable to hear and play for a lot of people.
Auroni Gupta (she/her)
User avatar
Corry
Rikku
Posts: 331
Joined: Fri Feb 10, 2012 11:54 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Corry »

I wholly support writing themed tossups on standard answer lines. In fact, I personally have a bit of a distaste for most tossups on "creative" answer lines, e.g. tossing up random aspects of the Great Gatsby. Although these tossups can occasionally work well, more often than not, they tend to just be confusing, especially for newer players.

From my personal experience: most of the (non-Arcadia) people on the Amherst team had relatively little/zero quiz bowl experience before they came to college. All of them have read the Great Gatsby high school. For them, however, tossups on things like "the green light in Gatsby" tend to come off as hopelessly confusing and obfuscated, even if they have much better knowledge of the book than I do (which is definitely the case). I think this speaks to a wider problem, in which tossups on creative answerlines can unintentionally turn quiz bowl into a game of "insider baseball" that only veterans will understand (e.g. "Oh, THAT'S what they were asking for?!").

When I edit for NAQT, one of my preferred practices is to convert regular tossups with creative (but confusing) answer lines into themed tossups with standard answer lines. In my opinion, a speeches-themed tossup on Otto von Bismarck works better than a tossup with the literal answerline "speeches of Otto von Bismarck." Themed tossups are great. More people should write them.
Corry Wang
Arcadia High School 2013
Amherst College 2017
NAQT Writer and Subject Editor
bradleykirksey
Wakka
Posts: 187
Joined: Sat Nov 12, 2011 5:09 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by bradleykirksey »

Adventure Temple Trail wrote: Piping in quickly to say as someone who's written at least three tossups about Purim/the book of Esther in my time, that the Book of Esther really isn't that long and doesn't have very many salient details to draw on. So tossups on it/her are going to draw on those same details as clues over and over for lack of any genuine alternative.
Haha, sorry to throw you under the bus then. I get your point, it's a short book and the writer probably didn't have quiz bowl in mind when he wrote it. But maybe in that case, it shouldn't be tossed up more often than Exodus or Psalms or Ruth or some of those other equally relevant OT books with more clues to pull from that don't get tossed up as often. Or, maybe it would be better to explore something like heaven across religions or hell across religions or creation stories in religion, or views of the end of the world. All of these are tossed up combined less than Esther and they have a wide breadth of clues available so you can avoid using the same three lead-ins.


Or, maybe the better example would have been Max Weber. The tossups on him have slowed down since his 2012-2014 heyday, which I think is an example of editors being responsive and caring. But Weber wrote lots of books and did a lot of very good things with his life. He helped write the Weimar Constitution, and he wrote a ton about religion. 2 of the 31 tossups reference his book about the religions of India and China, even though it was a major work. But 26 of the 31 tossups have the monopoly on the legitimate use of force clue, and more often than not in the top half of the clue. That might be a better example of my point. People are tossing up that clue around the power marker even though I feel like everyone who's been in quiz bowl for a little while should be able to buzz on it.
Bradley Kirksey
Mayor of quiz bowl at the University of Central Florida (2010-2015)
The club at Reformed Theology Seminary Orlando (2017 - 2021)
User avatar
Adventure Temple Trail
Auron
Posts: 2751
Joined: Tue Jul 15, 2008 9:52 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Adventure Temple Trail »

Bradley: One thing missing from your post is some reference to how significant such people as Elizabeth Siddall and such ideas as "the state is that which establishes a monopoly on the legitimate use of force" actually are outside of quizbowl. As such, it strikes me that you are inadvertently conflating several critiques of typical quizbowl writing. It's my hunch that all of the following are things you believe, and I agree with all of them, but does it make sense to consider them as separate claims?:
  • Clues of extreme real-world importance and fame are being often used too early in tossups, and should come later if they are used at all. (E.g.: Weber's definition of the state is probably the first or second most important thing he's written in actual reality; it should be at earliest a pre-giveaway clue at regular difficulty)
  • We have to find tossup topics that have a lot more viable clues than any individual tossup on that topic can contain; if it's not possible to toss something up with lots of new clues every time, it's not a good tossup idea for that difficulty level, period. (e.g.: toss up Exodus instead of Esther)
  • The fact that there have been a lot of tossups on some topic in a short span of time, independent of any concerns about its real-world fame or importance, is reason to cool off on writing tossups on that topic for a few years and uncover other stuff of similar importance that hasn't been asked about enough yet. (e.g. Even though Max Weber and Durkheim are probably the most important sociologists ever, quizbowl ought to find other sociology topics to do for a while)
It also seems like you may be implying that people such as Elizabeth Siddall and ideas such as Weber's definition of the state are not actually important to the Pre-Raphaelite Brotherhood's art or to Max Weber's theorizing respectively, which is incorrect. We've come a long way since the days in which stock clues were biographical chestnuts such as "Apprenticed to a bookbinder, he..."
Matt Jackson
University of Chicago '24
Yale '14, Georgetown Day School '10
member emeritus, ACF
Urech hydantoin synthesis
Tidus
Posts: 525
Joined: Thu Dec 30, 2010 3:35 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Urech hydantoin synthesis »

Let's apply this argument to its furthest extent, and imagine a hypothetical tournament in which none of these "stale" clues are used in any question. Max Weber tossups do not talk about Politics as a Vocation, and PRB tossups do not mention Elizabeth Siddall or any other clue that's been used more than X number of times before in tossups. Such a set would be far from good. An important aspect of a good "regular-difficulty" set is its ability to inspire, or maintain, in players the idea that if you study and put some time in quizbowl, you will get better at the game. I'm not saying there shouldn't be tossups on Picasso that focus on his sculpture, but if the majority of questions in a set were of that type, a new player who has put in the time to study the basics (e.g. Picasso's paintings) can easily get the impression that studying doesn't really pay off, and that getting good at this game is something that is very out of reach.
Ben Zhang

Zucker School of Medicine at Hofstra/Northwell '23
Columbia University '18
Ladue Horton Watkins HS '14
User avatar
naan/steak-holding toll
Auron
Posts: 2514
Joined: Mon Feb 28, 2011 11:53 pm
Location: New York, NY

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by naan/steak-holding toll »

It's also possible to write on a "common" topic with "common" clues but choose a new answerline; this might be thought of as the quizbowl analogue of the formalist concept of "defamiliarization" in which familiar material is presented in an unfamiliar way in order to enhance the audience's perspective. Some examples I wrote for Missouri Open:
Missouri Open 2015 wrote:Only three of these characters are women, one of whom owns the Black Inn where some of them meet. One of these people is a monk with a heavy staff who is nicknamed for his flowery tattoos, while another dies of poisoned wine and is persuaded by a mother to make a concubine of her daughter (*) Yan Poxi. Chao Gai is the first leader of these people, who oppose the corrupt official Gao Qiu and are called the 108 Stars of Destiny. Mount Liang is the base of - for 10 points - what group of people led by Song Jiang in a classic Chinese novel?
This is basically a pretty standard Water Margin question, but just written with an answerline I hadn't seen used before (though admittedly I had to make the answerline at least three lines long to make sure most reasonable possible answers were covered).
Missouri Open 2015 wrote:In this empire, concerns about the taxation of women prompted the Aba Women’s Riot. Under this empire, Ibadan became a trading city of over 100,000. A Muslim leader in this empire named Ahmadu Bello distrusted mercantile southern Christian ethnicities so much that he called them Jews. This empire let the (*) Sokoto sultan and local emirs govern cities like Kano, but also appointed District Commissioners to govern native peoples such as the Yoruba and Igbo. For 10 points, name this empire which controlled Nigeria before independence.
I don't know if Nigerian colonial policy has come up a lot before, but it's definitely something people know in real life - Nigeria's a populous and important country that a lot of people read about, including AP Comparative Government students (unless they've changed the curriculum recently). Rather than tossing up "Nigeria" with "this modern-day country" (an approach that would be guessable with some of the clues) I decided to use the answer "British empire" and clue the tossup with a bunch of African names, so the list of "empires" that people are thinking about is thrown off a bit. This approach of being "cute" or "misleading" shouldn't be used too often, but having some questions like this helps mix things up a bit and keep people honest/on their toes.
Last edited by naan/steak-holding toll on Sun May 08, 2016 4:14 pm, edited 1 time in total.
Will Alston
Dartmouth College '16
Columbia Business School '21
User avatar
Auroni
Auron
Posts: 3145
Joined: Thu Nov 15, 2007 6:23 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Auroni »

Urech hydantoin synthesis wrote:An important aspect of a good "regular-difficulty" set is its ability to inspire, or maintain, in players the idea that if you study and put some time in quizbowl, you will get better at the game. I'm not saying there shouldn't be tossups on Picasso that focus on his sculpture, but if the majority of questions in a set were of that type, a new player who has put in the time to study the basics (e.g. Picasso's paintings) can easily get the impression that studying doesn't really pay off, and that getting good at this game is something that is very out of reach.
This post seems rather orthogonal to the arguments presented in this thread, which propose means to improve the quality of regular difficulty set. Nobody has argued that standard questions should be thrown out and replaced wholesale with creative questions. The act of including more creative questions in tournaments doesn't render studying for quizbowl meaningless; in fact, the two goals may even be synergistic, as in some cases, the creative question taps into a wellspring of material that has been neglected by quizbowl, and that people actually do know, as evinced by Will's excellent British Empire tossup mentioned upthread.
Auroni Gupta (she/her)
User avatar
Adventure Temple Trail
Auron
Posts: 2751
Joined: Tue Jul 15, 2008 9:52 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Adventure Temple Trail »

Urech hydantoin synthesis wrote:Let's apply this argument to its furthest extent, and imagine a hypothetical tournament in which none of these "stale" clues are used in any question. Max Weber tossups do not talk about Politics as a Vocation, and PRB tossups do not mention Elizabeth Siddall or any other clue that's been used more than X number of times before in tossups. Such a set would be far from good. An important aspect of a good "regular-difficulty" set is its ability to inspire, or maintain, in players the idea that if you study and put some time in quizbowl, you will get better at the game. I'm not saying there shouldn't be tossups on Picasso that focus on his sculpture, but if the majority of questions in a set were of that type, a new player who has put in the time to study the basics (e.g. Picasso's paintings) can easily get the impression that studying doesn't really pay off, and that getting good at this game is something that is very out of reach.
I can't speak for anybody else, but I definitely believe that any particular difficulty level can (and must!) strike A Delicate Balance between breaking new ground, approaching existent topics in new ways, and rewarding people for knowing the basics about a lot of basics (which will not change very much from set to set or year to year). Indeed, the beauty of pyramidality is that it's not particularly onerous to strike such a thing!
Matt Jackson
University of Chicago '24
Yale '14, Georgetown Day School '10
member emeritus, ACF
User avatar
Corry
Rikku
Posts: 331
Joined: Fri Feb 10, 2012 11:54 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Corry »

Will's tossup on the British Empire was great (even though both Amherst and Penn totally bailed on that question when we played it at MO). In my opinion, it's actually emblematic of a "themed tossup on a standard line"; I personally think of it as a creatively themed tossup with a common answer line (in this case, Britain), rather than a common tossup with a creative answer line. Such a difference in views is not necessarily contradictory. I write tossups like this all the time.

I'd have a bigger problem with a tossup on, say, "British governors of Nigeria."
Last edited by Corry on Sun May 08, 2016 4:25 pm, edited 1 time in total.
Corry Wang
Arcadia High School 2013
Amherst College 2017
NAQT Writer and Subject Editor
User avatar
Sima Guang Hater
Auron
Posts: 1957
Joined: Mon Feb 05, 2007 1:43 pm
Location: Nashville, TN

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Sima Guang Hater »

Whenever this topic is discussed, it always goes off into left field with discussions about creativity. Creativity is important, no doubt, but I think "care" (to use Mike's word) is a much more important thing - making sure your clues are unique, your sentences grammatical and clear, your bonuses are without "fuck you" third parts that are left in due to laziness, etc.

In fact, it's that last thing I think is kind of understated. Too many times, I've found myself (and others) brushing over changing questions that don't appear to be overtly "bad", but are still not suitable for whatever reason. Tossups that are stretched using vague clues, bonus parts that are too hard, etc; the best sets I've seen tend to be free of these clunkers.
Eric Mukherjee, MD PhD
Brown 2009, Penn Med 2018
Instructor/Attending Physician/Postdoctoral Fellow, Vanderbilt University Medical Center
Coach, University School of Nashville

“The next generation will always surpass the previous one. It’s one of the never-ending cycles in life.”
Support the Stevens-Johnson Syndrome Foundation
User avatar
Ike
Auron
Posts: 1063
Joined: Sat Jul 26, 2008 5:01 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Ike »

I don't know what's going in on in this thread at all anymore, everyone's kind of incomprehensible after about the 7th post, so I'll just toss my two cents and move on:

I support Mike Cheyne's list. I question the ability of any regular difficulty set being able to gradate across teams - otherwise we would not be making nationals any harder, but I get it's more of an ideal to strive for rather than execute every time.

I agree with Bradley's examples, but I want to reword his idea and Matt Jackson's idea into what I think he is trying to articulate. I think what Bradley is saying is that quizbowl tends to highlight weird pockets of knowledge, and we should try to spread that out across all categories. To use the Weber example: the state as a monopoly of force clue is important in the realm of sociology - we covered it in Sociology 101, but there were also dozens of equally fundamental concepts that are also as important that just don't come up (as often) because they aren't as easy to incorporate into a tossup, and they are facts that aren't closely connected to a major thinker or some other salient things.
Missouri Open 2015 wrote:In this empire, concerns about the taxation of women prompted the Aba Women’s Riot. Under this empire, Ibadan became a trading city of over 100,000. A Muslim leader in this empire named Ahmadu Bello distrusted mercantile southern Christian ethnicities so much that he called them Jews. This empire let the (*) Sokoto sultan and local emirs govern cities like Kano, but also appointed District Commissioners to govern native peoples such as the Yoruba and Igbo. For 10 points, name this empire which controlled Nigeria before independence.
I don't know if Nigerian colonial policy has come up a lot before, but it's definitely something people know in real life - Nigeria's a populous and important country that a lot of people read about, including AP Comparative Government students (unless they've changed the curriculum recently). Rather than tossing up "Nigeria" with "this modern-day country" (an approach that would be guessable with some of the clues) I decided to use the answer "British empire" and clue the tossup with a bunch of African names, so the list of "empires" that people are thinking about is thrown off a bit. This approach of being "cute" or "misleading" shouldn't be used too often, but having some questions like this helps mix things up a bit and keep people honest/on their toes.
Actually, this is the kind of question I would like not to see at regular difficulty, and here's my reasoning why: I get this is important and that we all know the British Empire controlled Nigeria, but it's going to be confusing to people who have played fewer than like 3 quizbowl tournaments. Or at the very least, there are going to be a large amount of well-meaning, imperfect-knowledge possessing players who just don't know how to process information in quizbowl and won't know exactly what's going on and will say Nigeria anyways. The point is, you don't want to be scaring those new players / teams away with what, in their untrained opinion, sure feels and smells like a fuck you. Why you don't just write it on Nigeria saying "This colony" is beyond me. Anyway, regular difficulty is NOT the place to be doing this kind of mental gymnastics, save it for Nationals if you really want.

EDIT: if both Amherst and Penn are getting boned by this tossup, this tossup is playing much worse than you actually think, whether or not Corry thinks its fair.
Ike
UIUC 13
User avatar
Cheynem
Sin
Posts: 7219
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Cheynem »

I'm not sure why it's a fuck you (other than the fact, that perhaps a slamdunk giveaway could be appended, saying it's an European empire that also owned, I dunno, India). It seems pretty clear what the question is asking for (the empire that controlled Nigeria and Africa in its colonial days). I might guess that actually inexperienced quizbowl players might do better on it than not, seeing as how they're not automatically making mental gyrations that "African names + empire = something ancient."
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Corry
Rikku
Posts: 331
Joined: Fri Feb 10, 2012 11:54 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Corry »

If we're going to fixate on that particular question on the British Empire, then I'll throw in one more thought: the question would've worked better (aka been less confusing) if it had said "this European empire" at some point.
Corry Wang
Arcadia High School 2013
Amherst College 2017
NAQT Writer and Subject Editor
User avatar
Corry
Rikku
Posts: 331
Joined: Fri Feb 10, 2012 11:54 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Corry »

The Quest for the Historical Mukherjesus wrote:Whenever this topic is discussed, it always goes off into left field with discussions about creativity. Creativity is important, no doubt, but I think "care" (to use Mike's word) is a much more important thing - making sure your clues are unique, your sentences grammatical and clear, your bonuses are without "fuck you" third parts that are left in due to laziness, etc.

In fact, it's that last thing I think is kind of understated. Too many times, I've found myself (and others) brushing over changing questions that don't appear to be overtly "bad", but are still not suitable for whatever reason. Tossups that are stretched using vague clues, bonus parts that are too hard, etc; the best sets I've seen tend to be free of these clunkers.
Returning to the original question of the thread, this is an important point. When I play a set that I particularly enjoy, it really feels like the set's writers spent a lot of time in the editing process. Good sets aren't just free of "bad ideas"; they also have fewer "mediocre" or "so-so" ideas that editors were unwilling to fix/replace due to time constraints.

Speaking from my own experience, I edit a ton of geography for NAQT. Many of the submissions I get (probably around a third?) are not great, for a variety of reasons including poor difficulty gradation, uninteresting clues, or confusing wording. However, they're not necessarily bad either, so I can't really send them back as rewrites. Therefore, it's on me as the editor to transform an okay tossup into a "good" one.

Fortunately, due to the way Ginseng works (as well as my own lack of other commitments), I've basically had unlimited time to do this. I recognize that this is not always the case, especially when editors face a time crunch in packet-submission tournaments and the like.
Corry Wang
Arcadia High School 2013
Amherst College 2017
NAQT Writer and Subject Editor
User avatar
Ike
Auron
Posts: 1063
Joined: Sat Jul 26, 2008 5:01 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Ike »

Cheynem wrote:I'm not sure why it's a fuck you (other than the fact, that perhaps a slamdunk giveaway could be appended, saying it's an European empire that also owned, I dunno, India). It seems pretty clear what the question is asking for (the empire that controlled Nigeria and Africa in its colonial days). I might guess that actually inexperienced quizbowl players might do better on it than not, seeing as how they're not automatically making mental gyrations that "African names + empire = something ancient."
Well as someone who saw this particular question go dead after my teammate negged it against a pretty bad team, I do think it's neither straightforward and feels needlessly coy. This tossup would have worked on colonial _Nigeria_, and I think for regular difficulty tournaments where we're supposed to be encouraging more people to understand why quizbowl is a worthwhile activity, I would like to see people who probably haven't flashcarded anything for quizbowl being able to apply their incredibly superficial knowledge to the game and get points, and not get (reasonably) mad when they buzz in, say Nigeria, and don't understand exactly how this game works.

Let me talk a bit about empathy and regular difficulty. The very first collegiate tournament I played was the Matt Cvanovich Memorial Tournament back in 08. Those questions were long (six lines 10 TNR!), and tough (tossups on Ghana, which I barely knew existed!). Often times I couldn't bring myself to care / focus on a lot of them, especially science, and somewhat with history. It took me quite a bit of time to get acclimatized to hearing mACF tossups. The last thing I wanted back then, was to feel like tossups were taunting me or trying to preach to me about something. Obviously, here I am today, but like I can imagine a non-zero group of people getting angered because they feel like they are getting tricked: remember, a lot of the bottom half of teams that goes to a tournament like MUT or whatever can't even focus through many tossups, and only tune in at the giveaway! Of course it is their fault they aren't playing the game optimally, but why not be a bit more empathetic, write straightforward questions, so they can later potentially get the "aha moment" that MattBo has talked about, and they'll stick around more?

Again, this is only at intro and reg difficulty - once we get to Regs+ and Nationals, go wild. Seriously.
Ike
UIUC 13
User avatar
naan/steak-holding toll
Auron
Posts: 2514
Joined: Mon Feb 28, 2011 11:53 pm
Location: New York, NY

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by naan/steak-holding toll »

Corry wrote:If we're going to fixate on that particular question on the British Empire, then I'll throw in one more thought: the question would've worked better (aka been less confusing) if it had said "this European empire" at some point.
Cheynem wrote:I'm not sure why it's a fuck you (other than the fact, that perhaps a slamdunk giveaway could be appended, saying it's an European empire that also owned, I dunno, India). It seems pretty clear what the question is asking for (the empire that controlled Nigeria and Africa in its colonial days). I might guess that actually inexperienced quizbowl players might do better on it than not, seeing as how they're not automatically making mental gyrations that "African names + empire = something ancient."
Mike is right on point. People often write questions that enable skilled history players to buzz based off figuring out geography and a reasonable time frame. This question gives you a piece of that (West Africa for geography), so the piece of the puzzle here is "when did these things happen?" If you know this is the late 19th century, then the Ashanti, maybe the Sokoto caliphate (which I hadn't even heard called an empire ever before "stanford housewrite") and the British empire are your "reasonable" answers. All the clues apply to the British empire and not to any of the others, making it not only "reasonable" but "correct."
Ike wrote:I would like to see people who probably haven't flashcarded anything for quizbowl being able to apply their incredibly superficial knowledge to the game and get points, and not get (reasonably) mad when they buzz in, say Nigeria, and don't understand exactly how this game works.
Nigeria isn't an empire and your knowledge does not even pass for "superficial" if you think that is the case. Quizbowl is a worthwhile activity because you can get rewarded for genuine subject understanding, as well as knowledge of names (and opposed to knowledge of trivia) and achieve points. This tossup rewards understanding that is gained by knowing basic facts about history, and privileges people with this understanding over people without it.
Will Alston
Dartmouth College '16
Columbia Business School '21
User avatar
vinteuil
Auron
Posts: 1454
Joined: Sun Oct 23, 2011 12:31 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by vinteuil »

Ike wrote: Let me talk a bit about empathy and regular difficulty. The very first collegiate tournament I played was the Matt Cvanovich Memorial Tournament back in 08. Those questions were long (six lines 10 TNR!), and tough (tossups on Ghana, which I barely knew existed!). Often times I couldn't bring myself to care / focus on a lot of them, especially science, and somewhat with history. It took me quite a bit of time to get acclimatized to hearing mACF tossups. The last thing I wanted back then, was to feel like tossups were taunting me or trying to preach to me about something. Obviously, here I am today, but like I can imagine a non-zero group of people getting angered because they feel like they are getting tricked: remember, a lot of the bottom half of teams that goes to a tournament like MUT or whatever can't even focus through many tossups, and only tune in at the giveaway! Of course it is their fault they aren't playing the game optimally, but why not be a bit more empathetic, write straightforward questions, so they can later potentially get the "aha moment" that MattBo has talked about, and they'll stick around more?
I like this post a lot, especially the end of this paragraph. Maybe it doesn't have to be our responsibility as question writers to make the giveaways (no reason to stop there either) crystal-clear for teams who weren't paying attention, but I think it could only have a positive impact.
Periplus of the Erythraean Sea wrote:
Ike wrote:I would like to see people who probably haven't flashcarded anything for quizbowl being able to apply their incredibly superficial knowledge to the game and get points, and not get (reasonably) mad when they buzz in, say Nigeria, and don't understand exactly how this game works.
Nigeria isn't an empire and your knowledge does not even pass for "superficial" if you think that is the case. Quizbowl is a worthwhile activity because you can get rewarded for genuine subject understanding, as well as knowledge of names (and opposed to knowledge of trivia) and achieve points. This tossup rewards understanding.
I also like the "quizbowl rewards understanding" idea too. But I think, if you prioritize empathy at all, you have to acknowledge that the act of listening can be difficult, and players just might not hear or process the word they're supposed to understand. (Like Ike said: this is their fault, but the ending of the quesiton/the answerline should have this in mind.) On the other hand, not every question has to pass the "maximum easiness for a new player" test, and I don't think that this question absolutely had to be on "Nigeria" just because that might be the easiest answer to come up with.

[This is as someone who loved that question!]
Jacob R., ex-Chicago
User avatar
naan/steak-holding toll
Auron
Posts: 2514
Joined: Mon Feb 28, 2011 11:53 pm
Location: New York, NY

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by naan/steak-holding toll »

I will concede the "better giveaway" argument - I should've said European for the giveaway and maybe mentioned India. Before that, I don't really think there's a reason to say "European." In general, I do think tossups should be conscious of how solid their last two sentences are in particular, since that's where a plurality of buzzes are going to be happening (including tossups that get negged).
Will Alston
Dartmouth College '16
Columbia Business School '21
User avatar
Cheynem
Sin
Posts: 7219
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Cheynem »

I don't really know anything about African history colonial or otherwise, but I feel like the empathy question is perhaps taking us down the wrong path. Without knowing how good the clues are, my only critique of such a tossup (and I hate to keep talking about one question over and over, but I think it encapsulates some of the arguments here) is that it might be too hard. People playing the set might not know that the British controlled Nigeria during that period. The giveaway could thus be easier and maybe some of the clues also made easier.

But I am somewhat skeptical that you would have good knowledge of the period and events in the question and yet also believe that Nigeria was an empire. I guess I could believe you heard Nigerian things or things in Nigeria and said Nigeria, but that is not the right answer. Aside from the lack of giveaway, I do not think the question is coy (it is very clear it wants an empire all through the question). Again, I could easily see a team not knowing what empire controlled Nigeria during this period, or confusing it with another empire, but that's a question of difficulty, not confusion. If the question was on colonial India and again the question was on the British Empire, I'm not sure if we would be having this conversation.

To sum up, the tossup is not the easiest tossup in the world, sure. But that to me is very different than "it's not straightforward enough and is confusing."
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Ike
Auron
Posts: 1063
Joined: Sat Jul 26, 2008 5:01 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Ike »

Periplus of the Erythraean Sea wrote:Nigeria isn't an empire and your knowledge does not even pass for "superficial" if you think that is the case. Quizbowl is a worthwhile activity because you can get rewarded for genuine subject understanding, as well as knowledge of names (and opposed to knowledge of trivia) and achieve points. This tossup rewards understanding that is gained by knowing basic facts about history, and privileges people with this understanding over people without it.
I think you're missing my point. Like no one who is reasonably intelligent when you talk to them outside of quizbowl actually thinks "Nigeria is definitely an empire!" Even the people who neg the question with Nigeria! But Will, I think you're being a bit silly-- somehow you fail to see that for a lot of bad teams, they are not going to parse what's going on because they've never done anything like this before at game speed. Furthermore, I think you're being ridiculous that by assuming they have negged the tossup they don't have a basic knowledge of history . Much in the same way you flipped out at Stanford Housewrite because you haven't conceived of the Sokoto Caliphate as an Empire before in quizbowl, 90-99% of new teams have never conceived of Britain as the British Empire in this way for quizbowl, possibly ever, and we should keep that in mind as question writers.
Mike Cheyne, not a fiend in this thread wrote:But that to me is very different than "it's not straightforward enough and is confusing."
Here's why I use the word straightforward: To me, this tossup appears to me stump-the-chump. To use investipedia's definition, stump-the-chump means "The act of challenging a person in the spotlight in an attempt to make he or she appear foolish." I hate such questions at this difficulty because of the collateral damage of stump-the-chump type of questions - people who are clueless just get washed up in them as well. I mean yeah, by defintion Mike, stump-the-chump questions are inherently more difficult.

Here's a hypothetical experiment: Let's take 100 random students from universities who know a smidge to a lot of African history. And let's give them five seconds to answer the following question: The Yoruba were once part of what 20th century empire? I bet you that a non-zero amount of them are going to say Nigeria. Maybe some of them have imperfect knowledge, maybe some of them are tired that day because they woke up at 5am and misparsed your question, either way, you sure showed them and taught them how ignorant they are of African history! (It sure is a good thing this is purely hypothetical and didn't actually happen, otherwise why would they play quizbowl?)

And that's the bigger problem: the reason why quizbowl got realer was that you could start to fraud things by packet studying, Byrneboting, whatever you want to call it. But when you have a bunch of people who have never played quizbowl before playing your set, there is no reason to protect against that since they've never played quizbowl before! Will, you write:
People often write questions that enable skilled history players to buzz based off figuring out geography and a reasonable time frame. This question gives you a piece of that (West Africa for geography), so the piece of the puzzle here is "when did these things happen?" If you know this is the late 19th century, then the Ashanti, maybe the Sokoto caliphate (which I hadn't even heard called an empire ever before "stanford housewrite") and the British empire are your "reasonable" answers. All the clues apply to the British empire and not to any of the others, making it not only "reasonable" but "correct."
and that's fine. But when you have people who are really new to quizbowl, this is completely flibbertigibbet-gobbledygook to them.
Ike
UIUC 13
User avatar
naan/steak-holding toll
Auron
Posts: 2514
Joined: Mon Feb 28, 2011 11:53 pm
Location: New York, NY

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by naan/steak-holding toll »

Ike wrote:Much in the same way you flipped out at Stanford Housewrite because you haven't conceived of the Sokoto Caliphate as an Empire before in quizbowl
Yeah, I don't think the two are comparable. My confusion/bad reaction to the Sokoto question was, ultimately, a product of my ignorance: I'd read about it for class and in a book of military history and had literally only ever heard it called the "Sokoto Caliphate" - to be entirely honest I really didn't know a ton beyond "it's the state founded by Muhammad Bello of the Fulani" and "there are still Sokoto caliphs today." So yeah, I failed because I didn't know stuff and I reacted badly, especially since it was an important game. I don't see how that would happen here - I have difficulty thinking that people have never thought of the British Empire as an empire, because it's literally called the British Empire in everyday speech!

In any case, the people posting about needing a more player-sympathetic giveaway are correct. In general, referents should be player-friendly as well, as long as this doesn't result in saying things that are literally wrong (i.e. medieval Bulgaria is not a "country" and neither, for that matter, is the Byzantine Empire!)
Will Alston
Dartmouth College '16
Columbia Business School '21
User avatar
Cheynem
Sin
Posts: 7219
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Cheynem »

Wouldn't the more accurate thought experiment in regards to this question be "What 20th century empire appointed District Commissioners to govern native peoples such as the Yoruba and Igbo?"
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
1.82
Rikku
Posts: 398
Joined: Thu Feb 05, 2015 9:35 pm
Location: a vibrant metropolis, the equal of Paris or New York

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by 1.82 »

Ike, I disagree pretty strongly with the conclusions you've put forth in this thread. I played ACF Regionals 2015 on a two-person Maryland B team on which neither player had ever played any sort of quizbowl before that year's ACF Fall. Naturally we ended up in the bottom bracket because we had no idea what the answers to questions like this one were:
This thinker's distinction between "the possible" and "the real" was given a new third term, the "virtual," in a 1996 book about his –ism by Gilles Deleuze, who also built on this thinker's ideas in essays about cinema. This thinker distinguished between qualitative and quantitative types of multiplicity. This man linked emotional detachment to the "meaning of the comic" in three articles on laughter. This man used the image of a tape running between two spools to represent the flow of aging and the growth of memory. This author of Matter and Memory described the succession of conscious states as durée, or "duration." For 10 points, name this author of Time and Free Will, a French philosopher who discussed a non-mechanistic life force called élan vital in Creative Evolution.
ANSWER: Henri[-Louis] Bergson
Regular difficulty is really hard for people who have no experience playing quizbowl. This is something that everyone in quizbowl is aware of, which is why we have discussions regularly about whether regular difficulty should be made easier. This is why it's absurd to use "random college student" as a litmus of whether something is appropriate to be asked at regular difficulty; if we applied that test uniformly, it would certainly rule out tossups like this one far sooner than it would rule out mention that the British Empire ruled Nigeria. It's reasonable to expect that an intellectually curious person might know about basic details of the Scramble for Africa, since it is mentioned in high school history classes; I have never taken a class in my life (at any level of education) that mentioned Henri Bergson.

Last year, having never played quizbowl in my life, I could have gotten the question on Nigeria on the basis of the most passing knowledge of Nigerian history. I can't fathom how that's somehow inappropriate at regular difficulty compared to so many of the answerlines at that level.
Naveed Chowdhury
Maryland '16
Georgia Tech '17
User avatar
Ike
Auron
Posts: 1063
Joined: Sat Jul 26, 2008 5:01 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Ike »

Our Lady Peace wrote: stuff
Naveed, I find it hard to believe you've read my post at all. None of my post is about difficulty. If the point that I'm trying to make about empathy is so subtle that it needs its own forum topic, I will gladly do so, but I'm going to be a bit gauche and quote myself:
Let me talk a bit about empathy and regular difficulty. The very first collegiate tournament I played was the Matt Cvanovich Memorial Tournament back in 08. Those questions were long (six lines 10 TNR!), and tough (tossups on Ghana, which I barely knew existed!). Often times I couldn't bring myself to care / focus on a lot of them, especially science, and somewhat with history. It took me quite a bit of time to get acclimatized to hearing mACF tossups. The last thing I wanted back then, was to feel like tossups were taunting me or trying to preach to me about something. Obviously, here I am today, but like I can imagine a non-zero group of people getting angered because they feel like they are getting tricked: remember, a lot of the bottom half of teams that goes to a tournament like MUT or whatever can't even focus through many tossups, and only tune in at the giveaway! Of course it is their fault they aren't playing the game optimally, but why not be a bit more empathetic, write straightforward questions, so they can later potentially get the "aha moment" that MattBo has talked about, and they'll stick around more?
If everyone keeps that in mind for the next regular difficulty tournament they write, I'll be happy.
Ike
UIUC 13
User avatar
1.82
Rikku
Posts: 398
Joined: Thu Feb 05, 2015 9:35 pm
Location: a vibrant metropolis, the equal of Paris or New York

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by 1.82 »

Let me, if I may, quote the next line:
Ike wrote: Again, this is only at intro and reg difficulty - once we get to Regs+ and Nationals, go wild. Seriously.
I find it very odd that regular difficulty and novice difficulty are being grouped together, since regular difficulty is much more like those harder difficulties than it is like an introductory set. As I said, regular-difficulty questions are not meant for people who have never played quizbowl before, so I can't understand why those people are being used as a benchmark for what is or is not appropriate.

More to the point, a year ago I was someone who was completely new to quizbowl, and so I can attest to what was and wasn't a problem for me. I never at any point had an issue with hearing a bunch of names, buzzing, and being negged because I didn't know what the question was asking about; people who are beginners in quizbowl won't reflex-buzz on the basis of names they've heard, because they haven't studied for quizbowl and so haven't been led to believe that "Ibadan" = it's time to buzz with "Nigeria". To the extent that I neg questions like that, it's on things that I've studied for quizbowl, not things I actually know, because nobody outside quizbowl thinks like that. If anything, a person who knows things about Nigeria but not about how quizbowl works will benefit from opponents blindly guessing "Nigeria" instead of the right answer.

If we're saying that concerns of empathy prevent us from writing questions like this that experienced players might find more interesting, we have to consider whether anyone is actually benefiting from this empathy. I don't believe that people who are new to quizbowl would.
Naveed Chowdhury
Maryland '16
Georgia Tech '17
Strongside
Rikku
Posts: 475
Joined: Thu Apr 14, 2005 8:03 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Strongside »

I read that British Empire tossup to two experienced teams at the Minnesota mirror of MO, and it went dead.
Brendan Byrne

Drake University, 2006-2008
University of Minnesota, 2008-2010
touchpack
Rikku
Posts: 450
Joined: Tue Sep 20, 2011 12:25 am

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by touchpack »

Preface to this post: I'm terrible at history questions.

I think the main pitfall of the British Empire question is it's very difficult to parse. Sure, in retrospect the answer makes complete sense, but the way the question is worded doesn't point the player towards the answer unless they know the clues down COLD. Like, I can totally see a player knowing about the Scramble for Africa, and perhaps even knowing that Britain colonized Nigeria, and still not getting the question even at the end, since the phrasing doesn't point towards a non-African answer. I certainly had just assumed it was some obscure African thing I had never heard of before scrolling farther down Will's post. This is also exacerbated by the fact that the question was in NAQT format, which has lightning-fast game speed and very little time to make the lateral connections necessary to get the answer. I'm not saying that this is a bad question (it perhaps is even a very good question!), but I certainly think it's much, much harder than regular difficulty, and certainly should not be held up as an example of what regular difficulty should look like. I'm sure most of you don't care about science but the electric current tossup I wrote for the playoffs of ACF Nationals is very similar in style to this question--it approaches the answer from an unusual context (analytical chemistry rather than electrical engineering or physics!), and has a coy giveaway. In fact, both Chicago A and Michigan A failed to get this question at the end (perhaps I should have made the giveaway less coy). I guess my point with this analogy is to demonstrate anecdotally that even very, very, very good teams are going to get tripped up by questions like this, and I can't help but feel like the question(s) are almost deliberately misleading the player. (Determining whether that misleading effect is due to some inherent property of the question itself or a product of its deviation from the standard quizbowl style is left as an exercise to the reader)
Billy Busse
University of Illinois, B.S. '14
Rosalind Franklin University, M.S. '21, M.D. Candidate '25
Emeritus Member, ACF
Writer/Subject Editor/Set Editor, NAQT
User avatar
ThisIsMyUsername
Auron
Posts: 1005
Joined: Wed Jul 15, 2009 11:36 am
Location: New York, NY

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by ThisIsMyUsername »

There's too much here that is provocative and interesting for me to possibly respond to, but I'd like to respond to some of it:

It should be noted that when I made the original post of mine that Ben Zhang cited, I expected other people to respond with their Top 10 lists, as they would in a player poll discussion. It is not my actual position (at this point in time) that the 2011-2012 and 2012-2013 seasons were drought years for good tournaments, nor did I mean to suggest so even then. In retrospect, ACF Nationals 2012 should have been on my Top 10 (with more distance from it, I now regard it as the first fully contemporary ACF Nationals). I also refrained (out of decorum) from ranking any tournament that I myself worked on (except MAGNI, but even then I cited only Matt Jackson's editorship). And I say that not primarily to suggest that my own work from those eras is good, but also to acknowledge the hard work of many of my co-editors on those projects, which goes unacknowledged in my original post.

The other point I want to address is the one about "creativity". We have thankfully come to the point where we recognize that creativity is as much a function of clue selection as it is of answer choice. A "themed" tossup like the all-sculpture tossup on Picasso counts for most people, even though the answer-line is just Picasso, because the tossup's focus distinguishes it from other Picasso tossups. Thank goodness for that. What I'd like to point out, though, is that the qualities that make those sculpture clues a good idea--that the sculpture in question are important, well-known, under-asked, and that the tournament's audience might be expected to possess that latent knowledge, but to have gone unrewarded for it--would continue to exist even if they were transplanted to a tossup that was not themed.

Don't get me wrong. "Themed" tossups are cool. The theme can grant cohesion, and a sense of freshness to the topic. But a danger of over-rating this kind of creativity is that we start treating regular old unthemed "tossups on Important Work X" as "stale" or "uncreative". The worst thing that can happen is that we can stop noticing and valuing fresh clues in traditional tossups. And it's the clues that count most.

Penn Bowl 2012 had a tossup on the General Prologue from The Canterbury Tales. (I believe Magin wrote this tossup.) This was close to my definitional idea of what a good idea for a tossup is. We had been tossing up individual tales from the Canterbury Tales for ages, but I'm not sure anyone had tossed up the Prologue yet. Yet, it is the most widely read part of the text! Now, if someone tried to toss up the General Prologue, it wouldn't be as fresh an idea as that one was, because it has already been done. But there are many more clues from the General Prologue left to mine, which should be used: passages that are exactly the sort of stuff that people talk about in literature classes, would probably remember from reading the prologue on their own, etc. If someone included those clues in the lead-in to an unthemed, regular-difficulty tossup whose answer was just The Canterbury Tales, they might not get notice for doing so. And yet those clues might be among the freshest, buzzable lead-in clues available for a Canterbury Tales question. Who knows, it might even be--in a practical, create-a-nice-pyramid-of-buzzes sense--be the best Canterbury Tales-based tossup you could write.

Back in 2008-2011, Ted Gioia was maybe the most anti-"creative" partisan imaginable. He wrote only non-"creative" questions, and he chastised other people for being "creative" (sometimes entirely fairly, because their pursuit of "creativity" came at the cost of good tossup craftsmanship; sometimes unfairly, as Ted was overly suspicious of "creativity"). But even opponents of his positions acknowledge that he wrote some of the best literature questions of that era, and he did that by finding untapped clues that readers remembered, even though he almost exclusively framed them with very standard, straightforward answer-lines. The work that finding those clues involves is not less than the work involved in coming up with a fun way of linking together clues. And I say this as someone who has tried to write plenty of both sorts of questions.

Since Will Alston started the trend in this thread of citing one's own work, I'll cite one example from my own. For PADAWAN one-and-a-half years ago, I added this lead-in to a tossup on Notes from Underground that I was editing, written by another writer: "This work claims that only people of limited intellect can be men of action, since only they will have the lack of doubt needed to act". When we play-tested it, Matt Jackson buzzed in on that clue, and at first expressed surprise that his buzz was uncontested. Apparently, his professor had given some attention to this passage in class, treated it as quite important to the book. [EDIT: Matt Jackson has informed me that I'm misremembering one aspect of this: it wasn't a professor who had drawn attention to this; rather, he had independently come to the same conclusion: that this is basically the philosophical thesis of the first part of the book!] I entirely agree with this assessment: I happen to think that if you haven't thought about that claim, then you don't properly understand the first part of the novella, and probably have shortchanged the philosophical (as opposed to the fictional) dimension of the work. But my guess was that the number of people who could buzz on that clue was low enough that I could keep it as a first clue. (I don't actually know how it played at most sites, but I suspect that I guessed right.)

The rest of the tossup did not stick with the philosophical portions of the novella, because I didn't think there were enough fresh clues of the correct difficulty to sustain that approach. The next sentence of the tossup starts with words that make it very clear that this a work of fiction, so that someone who recognizes the opening clue, but resists buzzing because they're unsure whether this is actually a philosophy tossup, then has immediate motivation to take the plunge. And that sentence uses a plot moment that I felt was both memorable in and of itself, and connected to other ideas that I think are important to the overall theme of the book. I also don't waste valuable clue space preaching to you in the tossup itself why these moments are important. I trust that if you choose to engage with the actual work, you will hopefully discover this.

I'm not saying all this to celebrate my own choice of clues in that tossup. (You may disagree that this was a good clue, or dislike the tossup on other grounds; I'm not claiming it's a masterpiece.) But I'm doing this to explain a thought process that I think we should engage in more in our own writing and appreciate more in the writing of others. I'll have more to say about this in a future post, hopefully; though not in this thread.

So, my advice to writers: Search for the lead-in and early-middle-clue level knowledge that you think resides in the heads of your players, but is going untapped. Think of the possible answer-lines you could write to access that knowledge. Before deciding which answer-line to choose, think how the rest of the tossup will pan out. If a more "creative" one will work at your chosen difficulty: if it won't confuse players and will have enough middle and late clues produce a nice spread of buzzes, then go for it! All practical concerns being equal, the "creative" answer-line is always more fun! But always consider whether something tamer might lead to a more solid execution. First and foremost pursue the rewarding of worthwhile knowledge.
John Lawrence
Yale University '12
King's College London '13
University of Chicago '20

“I am not absentminded. It is the presence of mind that makes me unaware of everything else.” - G.K. Chesterton
bradleykirksey
Wakka
Posts: 187
Joined: Sat Nov 12, 2011 5:09 pm

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by bradleykirksey »

Sorry I may be a bit late getting back to this, and I see the conversation's moved on a bit, but I just got back from work.
Adventure Temple Trail wrote:
  • Clues of extreme real-world importance and fame are being often used too early in tossups, and should come later if they are used at all. (E.g.: Weber's definition of the state is probably the first or second most important thing he's written in actual reality; it should be at earliest a pre-giveaway clue at regular difficulty)
  • We have to find tossup topics that have a lot more viable clues than any individual tossup on that topic can contain; if it's not possible to toss something up with lots of new clues every time, it's not a good tossup idea for that difficulty level, period. (e.g.: toss up Exodus instead of Esther)
  • The fact that there have been a lot of tossups on some topic in a short span of time, independent of any concerns about its real-world fame or importance, is reason to cool off on writing tossups on that topic for a few years and uncover other stuff of similar importance that hasn't been asked about enough yet. (e.g. Even though Max Weber and Durkheim are probably the most important sociologists ever, quizbowl ought to find other sociology topics to do for a while)
Thanks, Matt. This is what I was trying to say, more eloquently than I was saying it. Obviously. after Protestant Ethic, Politics as a Vocation is Weber's most important work, and I'm not saying we should pretend those important works don't exist. FTPs can, and probably should, be stale. But it's very frustrating, personally, to see that in power, especially multiple times.

I think the reason I combined those points is that each ones make the other so much more frustrating. OK, we can toss up Esther *again*, because I guess it's important. Well, now it's even more frustrating that it's the same clues we heard last time. But it feels like I'm being cheated if someone buzzes in on the lead-in because it's been a lead in twice in the last two years. Those are separate points, but they work together to make some really frustrating tossups, I think, and I appreciate that, at least to me, it seems like ACF has done a good job of avoiding that lately, or at least in the narrow part of the distribution where I'd realize it. And I think that that helps makes for much better regular difficulty sets.

Urech hydantoin synthesis wrote:Let's apply this argument to its furthest extent, and imagine a hypothetical tournament in which none of these "stale" clues are used in any question. Max Weber tossups do not talk about Politics as a Vocation, and PRB tossups do not mention Elizabeth Siddall or any other clue that's been used more than X number of times before in tossups. Such a set would be far from good. An important aspect of a good "regular-difficulty" set is its ability to inspire, or maintain, in players the idea that if you study and put some time in quizbowl, you will get better at the game. I'm not saying there shouldn't be tossups on Picasso that focus on his sculpture, but if the majority of questions in a set were of that type, a new player who has put in the time to study the basics (e.g. Picasso's paintings) can easily get the impression that studying doesn't really pay off, and that getting good at this game is something that is very out of reach.
I don't think anyone wants that taken to the furthest extent and I hope that I haven't been implying that all clues used more than X times should be retired always. Every giveaway for John Wilkes Booth should mention Lincoln, every giveaway for Gabriel Garcia-Marques should mention One Hundred Years of Solitude, and every giveaway for Nixon should mention his time as the robot president in Futurama.

But certain answerlines, which have a knack for popping up a lot, often include the same clues really early. I don't think that there's a problem with Siddal appearing in PRB tossups, I have a problem with that answerline coming up 4 times in 6 months and her name appearing in the first half 3 of those times. You'd think that the PRB was the only art movement there's ever been and you'd be very confused about whether Siddal was extremely important or extremely obscure.

And I'm definitely not saying studying shouldn't make you better. I'm saying that studying past clues shouldn't give you a leg up on a chem tossup over someone who's taking Organic II or reading a book about Picasso should get you the tossup a few lines before someone who's just heard a bunch of tossups on Picasso.
Bradley Kirksey
Mayor of quiz bowl at the University of Central Florida (2010-2015)
The club at Reformed Theology Seminary Orlando (2017 - 2021)
Rococo A Go Go
Auron
Posts: 2248
Joined: Sat Jan 10, 2009 1:08 am
Location: Kentucky

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Rococo A Go Go »

Cheynem wrote:1. Gradates. Does the set properly distinguish between all teams? Could, for example, Dickety Doo U and Maryland B (the two weakest teams in a field, let's say) have a meaningful game on it, but also Michigan A and Chicago A? I'm not saying there aren't a few muddy games here and there, but you also have lead-ins and hard parts that challenge your top teams, along with late clues that allow weak teams to gradate knowledge.
We have a standard in this community that pretty much every existing team should try to attend regular difficulty tournaments, with ACF Regionals being the most important of the bunch. About 70% of the intended market for regular difficulty are teams like Dickety Doo U and Maryland B, plenty of whom believe that despite community pronouncements, regular difficulty is not meant for them. If we actually care about getting every team to attend regular difficulty tournaments, the questions for those tournaments have to be meaningful for them, which is something we seem to agree on here.

However, I don't think regular difficulty is meant to determine whether Michigan or Chicago is better, because we have national tournaments (and usually at least one regular season open) for that. I don't think the questions should be so easy that Michigan and Chicago are racing to the buzzer within power of every tossup, but I worry about what question writers sacrifice in an effort to make games meaningful for top teams. I think this has contributed to a situation where regular difficulty works harder to please the top teams than it does to make the game accessible for everybody else.

I do think that most people agree that regular difficulty is not meant to crown a national champion (again, that's what nationals are for) or to introduce quizbowl to novices (that's why we have novice tournaments). I believe that regular difficulty should strike a utilitarian balance that falls somewhere between those two goals, but doing so requires more attention to the latter for two reasons: 1) There are simply way more teams who struggle with harder questions. 2) Top teams should know better than to quit going to tournaments if they don't fit them perfectly.
Nicholas C
KQBA member
User avatar
Cheynem
Sin
Posts: 7219
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by Cheynem »

Yeah, that's a fair point. By "gradate Michigan A and Chicago A," I mean, it's not like giving them a packet of Collegiate Novice where most questions will be buzzer races on the first line. I just meant that for the most part, players with more knowledge will be able to buzz quicker--are we going to gradate a Jacob Reed and a John Lawrence on a music tossup? Possibly depending on knowledge gaps, but it also might be a first clue buzzer race in that particular case. You're right that the question writers for regular difficulty sets should not try to finely gradate the top teams--similarly, the vast majority of bonuses read to top teams are going to be 20'ed and many will be 30'ed--that's okay at this level, more so than it is for Nats. My general belief though is that intelligently challenging lead-ins and hard parts should be able to (sort of) gradate the top teams, or at the very least, produce non-bizarre results.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: What Makes a Good Regular-Difficulty Set (comparatively)

Post by grapesmoker »

Cheynem wrote:Yeah, that's a fair point. By "gradate Michigan A and Chicago A," I mean, it's not like giving them a packet of Collegiate Novice where most questions will be buzzer races on the first line. I just meant that for the most part, players with more knowledge will be able to buzz quicker--are we going to gradate a Jacob Reed and a John Lawrence on a music tossup? Possibly depending on knowledge gaps, but it also might be a first clue buzzer race in that particular case. You're right that the question writers for regular difficulty sets should not try to finely gradate the top teams--similarly, the vast majority of bonuses read to top teams are going to be 20'ed and many will be 30'ed--that's okay at this level, more so than it is for Nats. My general belief though is that intelligently challenging lead-ins and hard parts should be able to (sort of) gradate the top teams, or at the very least, produce non-bizarre results.
I guess since I edited a regular difficulty tournament that people really liked (Regs 2010), I'll just lean on that authority to encourage people to read what I posted in the tournament announcement: namely, that it's ok if Seth Teitler answers a science question early, because he's an expert and that's what should happen at these tournaments. Splitting hairs between top-level specialists is not what Regionals or any comparable set is about; consequently, you're going to get a lot of clues at those events that have come up before, even if the leadins are difficult.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
Locked