Non-question-specific criticisms of IS-80A

Dormant threads from the high school sections are preserved here.
rhentzel
Rikku
Posts: 264
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by rhentzel »

Jeremy Gibbs Free Energy wrote:And on that topic, because there is only so much someone can ask about high school basics, it seems to me in order to keep the sets kind of fresh they should be cut down by at least 1 A set per year.
I can see why you might think that, but the fact is that NAQT's fourth A-level set takes very little marginal work since it is based on questions that are adapted from those that we write for various television programs. Dropping it from the production schedule would not, in fact, free up that much time for other initiatives.
User avatar
Sen. Estes Kefauver (D-TN)
Chairman of Anti-Music Mafia Committee
Posts: 5640
Joined: Wed Jul 26, 2006 11:46 pm
Location: Columbia, MO

Re: Non-question-specific criticisms of IS-80A

Post by Sen. Estes Kefauver (D-TN) »

Something to chew on: I would say that a tossup that has some literature (or arts, or history) clues and ends with a geography giveaway is not a true literature (or arts, or history etc.) tossup, and should not be counted as one.
Charlie Dees, North Kansas City HS '08
"I won't say more because I know some of you parse everything I say." - Jeremy Gibbs

"At one TJ tournament the neg prize was the Hampshire College ultimate frisbee team (nude) calender featuring one Evan Silberman. In retrospect that could have been a disaster." - Harry White
User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: Non-question-specific criticisms of IS-80A

Post by Mechanical Beasts »

rhentzel wrote:
Jeremy Gibbs Free Energy wrote:And on that topic, because there is only so much someone can ask about high school basics, it seems to me in order to keep the sets kind of fresh they should be cut down by at least 1 A set per year.
I can see why you might think that, but the fact is that NAQT's fourth A-level set takes very little marginal work since it is based on questions that are adapted from those that we write for various television programs. Dropping it from the production schedule would not, in fact, free up that much time for other initiatives.
Then drop one of the ones that does take work!
Andrew Watkins
rhentzel
Rikku
Posts: 264
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by rhentzel »

I would say because math is "lucky" enough that some part of it *can* actually be tested in the quiz bowl format. You can't ask people to write essays, do proofs, or compose music in the quiz bowl format, but you can ask them to apply mathematical concepts. So we do. If it were possible to ask players to perform a piece of music in the tossup-bonus, buzzer-based format, we would.
This is only the "quiz bowl format" in that people are touching a button and then saying something and being judged right or wrong. Pyramidality, et cetera, are out the window.
NAQT doesn't consider "pyramidality" to be the defining element of what makes a good tossup question. We consider the critical element to whether or not the player with more knowledge is likely to get the question first.

Pyramidality is an excellent (really, the only) approach for typical quiz bowl questions, but it doesn't really apply to computation questions. On the other hand, NAQT believes that computation questions are compatible with the overarching goal of being able to be written in such a way that more knowledgeable players answer them correctly more often than less knowledgeable players.
If "speed of calculation" isn't valuable, then why is "speed of recall"? I could answer most quiz bowl questions given 60 seconds and access to Google.
We'd go too far afield if we were to start discussing why it's more important to know things about their world than for people to be calculators, and it's not an argument I obviously win, so I'll kick this point. Whatever, maybe they're both pretty dumb. We're not testing the set of all valuable skills; we're testing those that conform to quizbowl.
Hey, I agree with that!

Actually, here's a question for you: Would you accept a tossup that consisted of the zoomed-in display of a work of art that gradually pulled back revealing more and more of the painting? Would you consider it to be "pyramidal"?
I guess that's not quite a perfect analogy; one would say that quiz bowl tries to avoid questions revolving around "speed of recall," but I think that's also true of (most of) NAQT's computation questions. The vast majority of what we ask isn't "multiply 2.65333 by 7.2244," but something that involves understanding the problem and seeing the "trick" or the proper technique to apply. You might be able to solve them in 10 seconds by brute computation, but you'll be beaten by the person who understands the problem. I see this as the clear rewarding of knowledge (of mathematics).
The issue is that there are like fifteen tricks, and so after maybe two IS sets you've mastered all of quizbowl math.
I don't think that's empirically true; if it were, I would expect that we would see very, very high conversion of our computation questions at high-level tournaments (imagine a "normal" subject that only have 15 possible answers!). But, in fact, we don't. We see relatively low conversions, which suggests to me that NAQT does an effective (I'll avoid the word "good," here) job at finding new tricks and problems to ask about.

I'm more amenable to the argument that "computation just isn't fun" than to the argument that "there's really only 15 computation questions."
rhentzel
Rikku
Posts: 264
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by rhentzel »

Jeremy Gibbs Free Energy wrote:Something to chew on: I would say that a tossup that has some literature (or arts, or history) clues and ends with a geography giveaway is not a true literature (or arts, or history etc.) tossup, and should not be counted as one.
So one drop of geography turns any question into a geography question? Into a "mixed topic" question?

Would you accept a sports question that ended with a literature clue as a literature question?
rhentzel
Rikku
Posts: 264
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by rhentzel »

everyday847 wrote:
rhentzel wrote:
Jeremy Gibbs Free Energy wrote:And on that topic, because there is only so much someone can ask about high school basics, it seems to me in order to keep the sets kind of fresh they should be cut down by at least 1 A set per year.
I can see why you might think that, but the fact is that NAQT's fourth A-level set takes very little marginal work since it is based on questions that are adapted from those that we write for various television programs. Dropping it from the production schedule would not, in fact, free up that much time for other initiatives.
Then drop one of the ones that does take work!
I apologize; my reference to "the fourth" somewhat obfuscated my point.

All of NAQT's A-level sets involve a high degree of question reuse with television shows and/or our collegiate IMs. None of them is particularly high-effort (assuming that the other sets still need to be written).
User avatar
Sen. Estes Kefauver (D-TN)
Chairman of Anti-Music Mafia Committee
Posts: 5640
Joined: Wed Jul 26, 2006 11:46 pm
Location: Columbia, MO

Re: Non-question-specific criticisms of IS-80A

Post by Sen. Estes Kefauver (D-TN) »

I never said it automatically became geography either, and of course that sports/lit question is not literature. It should not be counted as really anything but maybe general knowledge, because a literature tossup should consist entirely of lit clues, and a geography tossup should consist entirely of geography clues, etc.
Charlie Dees, North Kansas City HS '08
"I won't say more because I know some of you parse everything I say." - Jeremy Gibbs

"At one TJ tournament the neg prize was the Hampshire College ultimate frisbee team (nude) calender featuring one Evan Silberman. In retrospect that could have been a disaster." - Harry White
User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: Non-question-specific criticisms of IS-80A

Post by Mechanical Beasts »

rhentzel wrote:On the other hand, NAQT believes that computation questions are compatible with the overarching goal of being able to be written in such a way that more knowledgeable players answer them correctly more often than less knowledgeable players.
The frequency with which two teams figure out how to solve the problem at very different times is low enough that it boils down to computation speed a nontrivial amount of the time.
Actually, here's a question for you: Would you accept a tossup that consisted of the zoomed-in display of a work of art that gradually pulled back revealing more and more of the painting? Would you consider it to be "pyramidal"?
Not really. To keep going with the visual tossup hypothetical, one that successively showed better and better known symbols from and features of said painting, or something, would be pyramidal. But that really has to do with, like, how the physical reality of a painting maps to the idea of pyramidality; I don't see what this has to do with anything.
I don't think that's empirically true; if it were, I would expect that we would see very, very high conversion of our computation questions at high-level tournaments (imagine a "normal" subject that only have 15 possible answers!). But, in fact, we don't. We see relatively low conversions, which suggests to me that NAQT does an effective (I'll avoid the word "good," here) job at finding new tricks and problems to ask about.

I'm more amenable to the argument that "computation just isn't fun" than to the argument that "there's really only 15 computation questions."
So, like, if you actively wanted to learn how to do them, I guarantee that that's exactly how you could. I was always a crappy player in high school, but junior year I could do the math. If you don't actively learn them, you sit there disinterested and guess a nice number like 24 or 8/3 or something. Also, the highest levels nowadays care the least about computation, right? Also, if it is, in fact, objectively true that there are only fifteen computation questions, then who cares if what players do on them suggests that there are more? It's still a poor idea. But sure, I'll also advance the argument that computation isn't fun.
Andrew Watkins
rhentzel
Rikku
Posts: 264
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by rhentzel »

Caesar Rodney HS wrote:Where did the 9 minute half come from anyway? Why such an arbitrary number? I read as fast as i possibly could and still at times barely made it through 20 questions, and i consider myself an excellent speed reader. An extra 2 minutes for the game would improve its quality immensely.
I wasn't a member of NAQT when the 9-minute half was decided upon, but it's my understanding that the thought was that College Bowl's 7-minute halves were too short; 9 minutes seemed like it would allow for more (and longer) questions while still allowing tournaments to keep to a 30-minutes-per-round schedule.

Hosts are more than welcome to institute different time limits (or to run untimed) at their tournaments if they believe doing so would improve the game. NAQT would welcome statistics on how this affected tournament length or surveys from those tournaments on how the change was perceived.
rhentzel
Rikku
Posts: 264
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by rhentzel »

Jeremy Gibbs Free Energy wrote:I never said it automatically became geography either, and of course that sports/lit question is not literature. It should not be counted as really anything but maybe general knowledge, because a literature tossup should consist entirely of lit clues, and a geography tossup should consist entirely of geography clues, etc.
There's not much I can say to this except that NAQT doesn't use this approach to categorization. If a question has at least two-thirds of its material from a subject area, we assign it there. Otherwise it goes to GK/mixed (unless it's more than two-third sports and popular culture, but not two-thirds of either alone. In that case we send it to sports or popular culture.)
User avatar
Gautam
Auron
Posts: 1413
Joined: Sun Feb 11, 2007 7:28 pm
Location: Zone of Avoidance
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by Gautam »

rhentzel wrote:If NAQT had more writers, we'd produce more regular Invitational Series.
I take an issue with this. If I am not mistaken, NAQT has 100+ writers; is that not correct? How are these many writers not enough to be able to produce several quality tournaments every year?

I suppose I can offer my own experiences with writing tournaments to show that more writers is not all that is necessary to produce better tournaments. I've been involved with producing 2 high school tournaments so far, both in quick succession last spring. In the middle of those two tournaments, my teammates and I worked on a college tournament. Only a handful (10 at the most) were involved in the production of each high school set I speak of. These sets were done on time and were praised by people for their quality.

I think this is a powerful testament to the fact that having better question writers rather than more question writers is the solution to that problem.

Also, I hope this is not a futile attempt but: can we please refrain from making this another thread on mathcomp? We've hammered that discussion out quite thoroughly, and we are really getting nowhere.
Gautam - ACF
Currently tending to the 'quizbowl hobo' persuasion.
rhentzel
Rikku
Posts: 264
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by rhentzel »

everyday847 wrote:
Actually, here's a question for you: Would you accept a tossup that consisted of the zoomed-in display of a work of art that gradually pulled back revealing more and more of the painting? Would you consider it to be "pyramidal"?
Not really. To keep going with the visual tossup hypothetical, one that successively showed better and better known symbols from and features of said painting, or something, would be pyramidal. But that really has to do with, like, how the physical reality of a painting maps to the idea of pyramidality; I don't see what this has to do with anything.
Oh, it was certainly something of a tangential question. I was casting about for an example of a question in a different field that didn't clearly fit the idea of pyramidality but yet could be argued would preferentially reward the most knowledgeable player.

Another example: Throw up a transparency with pseudocode and ask players to buzz with the name of the algorithm that it implements. Pyramidal? Acceptable tossup?
rhentzel
Rikku
Posts: 264
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by rhentzel »

gkandlikar wrote:
rhentzel wrote:If NAQT had more writers, we'd produce more regular Invitational Series.
I take an issue with this. If I am not mistaken, NAQT has 100+ writers; is that not correct? How are these many writers not enough to be able to produce several quality tournaments every year?
I guess I thought this was implicit in my original statement, but I'll revise it as follows: If NAQT had more good, high-school-level questions coming in, we would produce more regular Invitational Series. We anticipate that the most likely way to make that happen is to recruit new writers, since our existing crew is working near its maximum capacity.

NAQT does have 100+ writers who have produced questions in the past year, but their output varies by over three orders of magnitude. In addition, some prefer to only write television questions, some prefer to only write HSNCT or college questions, and we also have "imbalances" that result in surpluses of some categories and shortages of others. So not every question written produces 1/720 of a new Invitational Series.
I suppose I can offer my own experiences with writing tournaments to show that more writers is not all that is necessary to produce better tournaments. I've been involved with producing 2 high school tournaments so far, both in quick succession last spring. In the middle of those two tournaments, my teammates and I worked on a college tournament. Only a handful (10 at the most) were involved in the production of each high school set I speak of. These sets were done on time and were praised by people for their quality.

I think this is a powerful testament to the fact that having better question writers rather than more question writers is the solution to that problem.
I'll concede the point that NAQT would need to have "good writers" for this to work rather than simply "more writers." I'm sorry that wasn't clear.
rhentzel
Rikku
Posts: 264
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by rhentzel »

closesesame wrote:Let's consider NAQT "SCIENCE!". Most science questions written by HSAPQ did have good opening clues (what a surprise...) There are plenty of hard science clues out there for relatively simple topics that NAQT could use. Let's say the answer to a science TU were linear momentum. The tossup could open by talking about how Noether's Theorem guarantees its conservation if a system is translationally invariant.
Could we really have done this without triggering criticism? We used this clue (more-or-less verbatim) in tossup #213200, which is less than six months old. In fact, we also used it (worded somewhat differently) in last year's #198644 (an A-level set, no less). In fact, I would have considered this to be too much of a chestnut to use as a lead-in.
A TU about specific heat could have early clues about phonon modes and the work of Peter Debye. Instead, we are subjected to tossups about the personal life of the ideal gas constant, R.
This was used more-or-less verbatim as the lead-in to tossup #211809 which, admittedly, was in our ICT and so probably not heard by the vast majority of high school teams . . . but do you want us to consider it perfectly acceptable to reuse lead-ins form our college sets in our high school sets?

Could you expand on your comment about the "personal life of a constant" in private e-mail? I think I'm looking at the the question to which you refer, but I don't see any content in it that isn't actual science, so I might be misunderstanding your complaint.
User avatar
Sen. Estes Kefauver (D-TN)
Chairman of Anti-Music Mafia Committee
Posts: 5640
Joined: Wed Jul 26, 2006 11:46 pm
Location: Columbia, MO

Re: Non-question-specific criticisms of IS-80A

Post by Sen. Estes Kefauver (D-TN) »

Well, correct me if I'm wrong but the most trained scientists like Jerry V. and Seth T. seemed to agree that science was the weakest part of the ICT, so it actually seems like it might be plausible that your leadins to ICT science are too easy for that level and could work better in HS instead.
Charlie Dees, North Kansas City HS '08
"I won't say more because I know some of you parse everything I say." - Jeremy Gibbs

"At one TJ tournament the neg prize was the Hampshire College ultimate frisbee team (nude) calender featuring one Evan Silberman. In retrospect that could have been a disaster." - Harry White
User avatar
Stained Diviner
Auron
Posts: 4830
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by Stained Diviner »

Thanks for your responses, R.

Is there any chance that NAQT will take a look at its distribution in the near future and adjust it to make it more academic? I wouldn't suggest that you should ban Pop Culture/Miscellaneous, but I would like to see more Literature and Fine Arts, especially well-known serious works, and it would be fine with me if there was less Pop Culture/Miscellaneous. I know that a fair number of such questions already are in your packets, but I'd like to see more.

Also, in regards to an academic tossup with a Pop Culture giveaway, I think you need to consider that a lot of teams know a lot more about Pop Culture than academic material, which is why a lot of those questions end up being decided on the Pop Culture clue. If you're not going to ban those questions outright and want to keep counting them as academic, then please make every effort to give a very easy academic clue before heading in the Pop Culture direction. And I would consider that a big If.

For the rest of the people in this thread: I think you need to see the questions for what they are. If you are an elite player, and you play on an A Set, you are going to see a fair number of what you consider to be stock clues in early leadins and a fair number of easy 30s on bonuses. If you are surprised by this, stop being surprised by this. Even with IS Sets, you are going to get some frustrating situations when two Top 20 teams play each other. The questions are generally written so that, when the 10th best team in the country plays the 200th best team in the country, the 10th best team dominates by getting a decent number of powers and 30s. When the 10th best team plays the best team on questions at the same level, the 10th best team is still going to get some powers and 30s. We can argue that, by adding an extra line or two at the beginning of questions, NAQT could better differentiate between two elite teams, but it doesn't sound like that's going to happen. If you want more perspective on how NAQT questions play out with average-to-weaker teams, then the next time you are at an IS tournament and have a bye round, go watch a match between weaker teams. Also, don't be surprised when there are some clunkers in an IS Set--you should expect to hear about one question you truly hate per packet at any tournament you go to, whether or not the questions are by NAQT.
David Reinstein
PACE VP of Outreach, Head Writer and Editor for Scobol Solo and Masonics (Illinois), TD for New Trier Scobol Solo and New Trier Varsity, Writer for NAQT (2011-2017), IHSSBCA Board Member, IHSSBCA Chair (2004-2014), PACE Member, PACE President (2016-2018), New Trier Coach (1994-2011)
User avatar
AKKOLADE
Sin
Posts: 15570
Joined: Thu Apr 24, 2003 8:08 am

Re: Non-question-specific criticisms of IS-80A

Post by AKKOLADE »

I write most of a post I want to make on this issue before going to sleep, wake up and this thread has done blew up. So this may not all be fresh stuff here, but you're stuck with it.

I have not been able to see an A-set from this year yet, but given what I saw last year coupled with the comments so far in this thread, I feel like this is a fair statement to make.

When I first heard of the A-set concept last year, I was hopeful. The idea of creating a set of questions based around answer selections that are, on average, easier than your average IS seemed like a decent idea for recruiting new teams and giving freshmen and the like experience. I expected a slight decrease in length, but given how NAQT questions are typically about five sentences each, I wasn’t expecting much to change as that’s a decent length to combine a good number of clues with accommodating a faster pace of game.

The first tournament I read at in the ’07-’08 season was at Parkersburg Catholic in my hometown. I wasn’t able to get the questions until late on the Friday before the event. Not having much time to look over them, I thought that they looked shorter than usual, but did not note anything else.

The day of the event, when I actually read the questions, I was astonished at what had happened. I can honestly say that I had never been so disappointed in anything involving quiz bowl before in my life. The set contained an ungodly amount of cute giveaways, abhorrent clue density, buzzer races and, worst of all, trivial subjects – not even trash, per se, but just things that are not of importance great enough to warrant inclusion in quiz bowl. As an introduction to quiz bowl, it was utterly insufficient. I now regret that I never did what I wished to do, which is to go page-by-page through the set and critique it until my fingers bled – an action that I could have easily done.

I tried to dismiss it as a one-set issue, as NAQT has had a couple of clunkers in the past set wise – these things happen. But I read at two more A-set events and each one was just as bad.

Worse yet is that these sets are being used as sets for varsity tournaments and are expected to provide reliable differentiation between top teams. I cannot fathom how this could be accomplished with the sets I have seen to date. They just do not do enough to differentiate between both depth and breadth of knowledge to reliably determine the best team between, say, Hunter & State College.

Maybe there is still hope for these sets, but it’ll take a good bit of work to get the necessary improvement going.
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, co-owner
PACE
former (?) hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
User avatar
Matt Weiner
Sin
Posts: 8414
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Re: Non-question-specific criticisms of IS-80A

Post by Matt Weiner »

Lime, Self and Society wrote:For the rest of the people in this thread: I think you need to see the questions for what they are. If you are an elite player, and you play on an A Set, you are going to see a fair number of what you consider to be stock clues in early leadins and a fair number of easy 30s on bonuses. If you are surprised by this, stop being surprised by this. Even with IS Sets, you are going to get some frustrating situations when two Top 20 teams play each other.
This statement, and equivalent statements by R, seem to demonstrate a fundamental misunderstanding of how quizbowl works. I'm not sure if some people who say things like this really mean it or are putting it up as a smokescreen for other interests. In any case, taking it at face value, I pose you this question: Why do we bother with the enormous effort required to write substantive, well-ordered pyramidal tossups, on which a small but nonzero amount of teams will buzz on the first clue, more on the second, etc until as many teams as possible know the question on the end? Why do we write progressive-difficulty bonuses, with clear easy, middle, and hard parts? These things take literally a hundred times as long as just writing one-clue tossups of whatever difficulty and throwing together three questions on any topic without regard to conversion rates. The reason we do these things is so that all teams can play on the same questions. If it was just about the lower-end teams, we could give them 20 or more "Who wrote For Whom the Bell Tolls?" type questions, and we could probably distinguish their knowledge because one team will know a number of questions that the other does not. If it was just about the higher-end high school teams, we could give them 20 questions like "In what work does Hannah Arendt analyze the trial of a Nazi?" and again, there would be usually one team who knows 0<x<20 of the tossups, and another who knows x<y<20, and the team who knows Y would win, and we would have rewarded better knowledge. However, the reason we use pyramidal questions is so that we can test the head-to-head depth of knowledge of any pair of teams on the same questions. There are clues at the start of the tossups hard enough to distinguish Charter from Walter Johnson, clues in the middle that can separate Douglas Freeman from Clover Hill, and clues at the end that can separate two entirely new teams from each other. Bonuses have one part that almost everyone knows (but that will distinguish a team in the eighty-fifth percentile from one that is almost at the bottom), one part that about half of teams know (now making a 50-50 cut among all teams), and one part that fifteen to twenty percent of teams will know (separating the elites from everyone else, and making thorough knowledge of these 30-point parts with no subject holes, and of tossup play, the key factor in games between top teams).

If NAQT or any other group is just going to say that they can't write questions like the above, then they are by definition not writing good questions at all. Questions that distinguish any reasonably predictable combination of teams in the field, without alienating the lower-end teams through their absolute difficulty, are what good quizbowl is. A tossup which can't be played in a Hunter-Kellenberg match because even the leadin clues are more the speed of Example Bad Team C is, ipso facto, a bad tossup, and a set full of such tossups is a bad set. Conversely, if it is true that the regular IS set tossups are rampantly going dead in games between non-elite teams, then they are also bad for the same reason. I don't think NAQT or anyone else should be in the business of producing bad question sets. I also don't think that "just don't play A-sets if you're any good" is an acceptable response from R on this, since R himself has been aggressively marketing A-sets for use, including as replacements for tournaments that previously used questions of more general interest. High school is not college; the top players aren't going to just stay home or staff things while their freshmen play, and be satisfied with two or three tournaments before nationals. The best high school teams expect to send their real lineups to tournaments nearly every weekend. It is not NAQT's place to tell them that this long-established practice is wrong.
Matt Weiner
Founder of hsquizbowl.org
User avatar
aestheteboy
Tidus
Posts: 570
Joined: Sat May 13, 2006 5:07 pm

Re: Non-question-specific criticisms of IS-80A

Post by aestheteboy »

In response to Matt: I don't think there's anything wrong if A-set questions didn't have enough clues to be playable for the top teams. I guess you could criticize R's marketing strategy, but I really don't see why there can't be novice-level tournaments at HS level; I don't think that the players in HS circuit are somehow less flexible than those in college circuit. It's theoretically possible to write tossups on Twain and Melville with really hard clues, but I think that would make the production of A-sets a lot more time consuming than it already is, and it still wouldn't be that satisfying for some teams - personally, I'd much prefer to learn something important about someone obscure than learn something really obscure about someone famous.

In response to R (his first post): Let's not lose our common sense here. It would take many buzzer races (say 10 per game) to consistently affect the outcome of the game because if one team is clearly better, that team will get more tossups in the other 10 questions. If neither team is better than the other, then obviously there will be no noticeable statistical impact because the outcome would be similar with or without any buzzer races (assuming that buzzer races award points randomly). It takes, however, far fewer buzzer races to leave players angry and discontent. We don't need any statistical evidence to see that there is a problem: the problem is that practically everyone who cares about good quizbowl is unhappy with the situation!
Really, we aren't just talking about winning and losing here. In terms of impact on the outcome, transparency isn't a huge issue because generally teams that are good at real quizbowl are good at frauding; that doesn't mean that we won't be upset and frustrated when we fraud tossups.
Daichi - Walter Johnson; Vanderbilt; U of Chicago.
Daichi's Law of High School Quizbowl: the frequency of posting in the Quizbowl Resource Center is proportional to the likelihood of being overrated.
User avatar
at your pleasure
Auron
Posts: 1712
Joined: Sun Aug 03, 2008 7:56 pm

Re: Non-question-specific criticisms of IS-80A

Post by at your pleasure »

While there is an element of speed to any computation question, NAQT strives to avoid asking questions that are merely "divide x by y" or "factor z." We try to focus our computation questions on tricks/techniques that exploit short-cuts, formulas, or symmetries that can take most of the work out of the problem
This has probably been said, but my point was that probably 95% of the teams figure out how to solve the problem by the time the first clue is read.
The basic answer is that, historically, there are lots of geography answers that have proven answerable by quiz bowl teams (as a whole) and fewer answers from the fine arts that fit the bill. Including more fine arts questions in our sets would either require using the same answers more frequently or asking about things that are too hard for a majority of teams.
Since certain paintings have a tendancy to come up very freqently,the first alternative that you outline as being necessary to increase the fine arts distribution is happening anyhow. I can think of quite a few paintings and/or artists that are well-known enough that anyone who reads a "intro to art history" book or takes a survey class(if the high school they go to offers one) would have at least heard of. While we're at it, since you defend polity questions on the grounds of "they make more difficult subjects acessible", why not write polity questions on, say, Flemish art? That would probably be gettable by most novice teams on the givaway, but should still distinguish between teams that have deeper knowlege of that aspect of art history.
Douglas Graebner, Walt Whitman HS 10, Uchicago 14
"... imagination acts upon man as really as does gravitation, and may kill him as certainly as a dose of prussic acid."-Sir James Frazer,The Golden Bough

http://avorticistking.wordpress.com/
User avatar
ClemsonQB
Tidus
Posts: 511
Joined: Tue Dec 04, 2007 11:12 pm
Location: Clemson, SC
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by ClemsonQB »

It would be a stretch to say that 95% of teams can answer a tossup on William Shakespeare, much less get a math comp question from the first clue. Also, you extremely overestimate the number of people who have even heard of Flanders.
George Stevens

Dorman High School 2008
Clemson University 2012
User avatar
Stained Diviner
Auron
Posts: 4830
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by Stained Diviner »

Matt--
Quizbowl packets generally have target audiences. You yourself write different questions for strong college teams, average college teams, novice college teams, strong high school teams, and average high school teams. You are correct that good questions can appeal to a wide audience, but that does not mean that the same good questions appeal to everybody. If they did, then you and your cohorts would spend less time writing and editing.

I've never seen an A Set, but I know that it is inappropriate for my team. I knew that before this thread was started, and my team is worse than many of the teams that are represented here. IS Sets are somewhat useful for my team--they are far superior to what other vendors produce (I'll see my first HSAPQ set Saturday, so I'll suspend judgment on them for a few more days) and have some good clues that my team does not already know, but there are also problems. When we play a team that is average or worse, we beat them by 300 points and it doesn't matter. When we play a team that is roughly at our level, however, the matches often get decided based on unacademic subjects, bonuses that either team would have gotten the same score off of, transparency issues caused in part by the limited answer space, and buzzer races due to easy powers or difficulty cliffs. Some of this could be fixed based on improving the distribution or including more tossup clues, but part of it is what we'd expect when playing on questions that we are at the upper end of the target audience for. These problems are lessened at HSNCT, where our team is well within the target audience.

I am sure that when you play-tested the EFT questions you realized that, even though the questions were good, they would not be good for ACF Nationals or CO. For the same reason (in addition to the distribution problems and in some sets some editing problems), IS is not great for Charter vs Walter Johnson. I'm not saying at all that those teams should stop playing IS, and of course I don't speak for NAQT or those teams anyways, but we shouldn't pretend we are shocked when those teams prefer tournaments that are geared to the level of play more common in the DC Area than the rest of the country. Of course they prefer questions that are geared more to them.
David Reinstein
PACE VP of Outreach, Head Writer and Editor for Scobol Solo and Masonics (Illinois), TD for New Trier Scobol Solo and New Trier Varsity, Writer for NAQT (2011-2017), IHSSBCA Board Member, IHSSBCA Chair (2004-2014), PACE Member, PACE President (2016-2018), New Trier Coach (1994-2011)
User avatar
vcuEvan
Auron
Posts: 1087
Joined: Mon Feb 23, 2004 5:49 pm
Location: Richmond VA

Re: Non-question-specific criticisms of IS-80A

Post by vcuEvan »

Lime, Self and Society wrote: I am sure that when you play-tested the EFT questions you realized that, even though the questions were good, they would not be good for ACF Nationals or CO.
Although I do think the EFT set would have done a decent job of producing an ACF nationals champion, I do agree that good high school teams should not be playing A-sets.
Evan Adams
VCU '11, UVA '14, NYU '15
User avatar
Sir Thopas
Auron
Posts: 1330
Joined: Tue Mar 28, 2006 10:10 pm
Location: Hunter, NYC
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by Sir Thopas »

Lime, Self and Society wrote:I am sure that when you play-tested the EFT questions you realized that, even though the questions were good, they would not be good for ACF Nationals or CO. For the same reason (in addition to the distribution problems and in some sets some editing problems), IS is not great for Charter vs Walter Johnson. I'm not saying at all that those teams should stop playing IS, and of course I don't speak for NAQT or those teams anyways, but we shouldn't pretend we are shocked when those teams prefer tournaments that are geared to the level of play more common in the DC Area than the rest of the country. Of course they prefer questions that are geared more to them.
Of course you don't speak for NAQT. If you did, you would be espousing precisely the opposite viewpoint of what you are doing now.

For what it's worth, I'm done with A-sets for good, and would be reluctant to play even on IS-level sets in the future.
Guy Tabachnick
Hunter '09
Brown '13

http://memoryofthisimpertinence.blogspot.com/
cdcarter
Yuna
Posts: 945
Joined: Thu Nov 15, 2007 12:06 am
Location: Minneapolis, MN
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by cdcarter »

rhentzel wrote:All of NAQT's A-level sets involve a high degree of question reuse with television shows and/or our collegiate IMs. None of them is particularly high-effort (assuming that the other sets still need to be written).
1) You state NAQT A-sets are based on TV show questions, and Collegiate IM questions
2) You run A-set tournaments for colleges in MN
3) These colleges also purchase/use the IM questions
4) COLLEGIATE QUESTIONS ARE BEING USED TO CONSTRUCT A SETS!?
Christian Carter
Minneapolis South High School '09 | Emerson College '13
PACE Member (retired)
User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: Non-question-specific criticisms of IS-80A

Post by Mechanical Beasts »

cdcarter wrote:2) You run A-set tournaments for colleges in MN
Notably, a phenomenon that NAQT apparently wants to spread to regions that obviously have a deficiency of good schools, like the Northeast.

Harvard will not be hosting a college A-set (or IS) tournament, please and thank you.

This is a terrible idea.
Andrew Watkins
User avatar
at your pleasure
Auron
Posts: 1712
Joined: Sun Aug 03, 2008 7:56 pm

Re: Non-question-specific criticisms of IS-80A

Post by at your pleasure »

Yeah, what the heck was NAQT thinking when they decided that sets that high schoolers find painful to play would be good for college tournaments?
Douglas Graebner, Walt Whitman HS 10, Uchicago 14
"... imagination acts upon man as really as does gravitation, and may kill him as certainly as a dose of prussic acid."-Sir James Frazer,The Golden Bough

http://avorticistking.wordpress.com/
User avatar
Matt Weiner
Sin
Posts: 8414
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Re: Non-question-specific criticisms of IS-80A

Post by Matt Weiner »

Anti-Climacus wrote:Yeah, what the heck was NAQT thinking when they decided that sets that high schoolers find painful to play would be good for college tournaments?
"cha-ching"
Matt Weiner
Founder of hsquizbowl.org
rhentzel
Rikku
Posts: 264
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by rhentzel »

cdcarter wrote:
rhentzel wrote:All of NAQT's A-level sets involve a high degree of question reuse with television shows and/or our collegiate IMs. None of them is particularly high-effort (assuming that the other sets still need to be written).
1) You state NAQT A-sets are based on TV show questions, and Collegiate IM questions
2) You run A-set tournaments for colleges in MN
3) These colleges also purchase/use the IM questions
4) COLLEGIATE QUESTIONS ARE BEING USED TO CONSTRUCT A SETS!?
I don't think I understand the exact complaint here; all four points are basically true, though the reality of (4) is more that high-school questions are being used to construct collegiate intramural sets.

If the complaint is that the Tartan Tussle (my 11/1 event at Macalester College for new teams and freshmen from established programs that is using Invitational Series #82A) may be compromised because colleges will have seen the questions in their intramural sets, that isn't a problem: Only IS #78A and #80A share with the intramurals. NAQT has also worked with other hosts (e.g., Stanford) that want to run both intramurals and A-level tournaments to ensure that appropriate questions are available.

If the complaint is that essentially A-level questions are too easy for collegiate intramural tournaments, we haven't typically found that to be the case. In the past, we have happily sold some colleges regular invitational series for their intramurals if they thought a greater difficult was warranted.
User avatar
Matt Weiner
Sin
Posts: 8414
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Re: Non-question-specific criticisms of IS-80A

Post by Matt Weiner »

rhentzel wrote:Only IS #78A and #80A share with the intramurals.
rhentzel wrote:All of NAQT's A-level sets involve a high degree of question reuse with television shows and/or our collegiate IMs.
I guess people in college don't have televisions, either?
Matt Weiner
Founder of hsquizbowl.org
User avatar
btressler
Tidus
Posts: 620
Joined: Fri Mar 26, 2004 7:23 pm
Location: West Chester, PA
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by btressler »

Ike wrote:Now, not to call him out, but on Kidder's podcasts I've heard him say that you can easily trim down around a full sentence of verbiage from a tossup, and that it really doesn't matter if clue one or clue two are switched around as to create what I called "a tiny pocket of anti-pyramidality." I contend that it does matter, for every word and clue must count in a constrained format. Gustave Flaubert would have been the perfect NAQT writer if he were quizbowl writer because of le mot juste (or something like that.) I hope NAQT realizes that they should be more careful about their question writing because of their limit, not less careful, and keep that in mind when they write stuff.
I want to play devil's advocate for one of the things I think NAQT gets right.

Not to pick on Dennis Jang, but we read this tossup in practice yesterday:
Dennis Jang in EFT wrote:The ladies in this work do not believe that the protagonist of this work could forget “noblesse oblige,” even after the death of that character’s father. The title character, who is the subject of complaints by the Board of Aldermen, gives china-painting lessons, refuses to pay taxes, and has her servant Tobe run her errands. The purchase of a silver toilet set, a complete outfit of men’s clothing, and arsenic occur before the title character’s cousins leave town, after which the foreman Homer Barron returns to Jefferson before never being seen again. However, it is revealed that the title character has been sleeping with the corpse of her lover at the end of, FTP, which short story by William Faulkner?
What's wrong with this:
Bill Tressler wrote:Ladies in this work disbelieve its protagonist could forget “noblesse oblige,” even after the father's death. The Board of Aldermen level complaints against the title character, who gives china-painting lessons, refuses to pay taxes, and has her servant Tobe run errands. The purchase of a silver toilet set, a complete men’s clothing outfit, and arsenic occur before the title character’s cousins leave town, after which foreman Homer Barron returns to Jefferson before disappearing. FTP, the title character sleeps with the corpse of her lover in which William Faulkner short story?
My point, and perhaps Dwight Kidder's point is that there are a lot of questions out there that are "wordy". I knocked about a full sentence's worth of words out of the question. But I preserved all the clues and (I hope) the pyramidality.

Now if you are going to argue that we need the extra words to give players time to think, or to run the moderator out of breath, or whatever, I have no counterargument. But I personally much prefer the "less wordy" version. And when NAQT is written well, it generally has less verbiage.

EDIT: And if there's anyone out there who happens to agree with me and might like an editor willing to look at questions, I might be willing to volunteer.
Last edited by btressler on Wed Oct 29, 2008 8:09 pm, edited 1 time in total.
User avatar
Maxwell Sniffingwell
Auron
Posts: 2163
Joined: Sun Feb 12, 2006 3:22 pm
Location: Des Moines, IA

Re: Non-question-specific criticisms of IS-80A

Post by Maxwell Sniffingwell »

It's harder for a player to identify what is being asked for ("its" instead of "this work") in your tossup. Also, do you think that any player/moderator could actually recognize the difference between those tossups without seeing them side-by-side?
Greg Peterson

Northwestern University '18
Lawrence University '11
Maine South HS '07

"a decent player" - Mike Cheyne
User avatar
Maxwell Sniffingwell
Auron
Posts: 2163
Joined: Sun Feb 12, 2006 3:22 pm
Location: Des Moines, IA

Re: Non-question-specific criticisms of IS-80A

Post by Maxwell Sniffingwell »

cornfused wrote:It's harder for a player to identify what is being asked for ("its" instead of "this work") in your tossup. Also, do you think that any player/moderator could actually recognize the difference between those tossups without seeing them side-by-side?
By which I mean, if the biggest positive change you can make to a tournament set is elimination of wordiness (while preserving all of the clues,) then you're already producing a better product than do any of the best writers out there. Saying you'd like to eliminate extra wordiness is like saying you wish Barack Obama were more attractive - no one's going to deny that it would be a positive change, but it's an issue that just does not matter compared to his other merits/demerits.

One problem that I have with NAQT is that often, this desire to end "wordiness" cuts out not just filler but also specificity/clarity and actual clues.
Greg Peterson

Northwestern University '18
Lawrence University '11
Maine South HS '07

"a decent player" - Mike Cheyne
User avatar
Ike
Auron
Posts: 1037
Joined: Sat Jul 26, 2008 5:01 pm
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by Ike »

The problem is that when you do massive deletion - a sentence to me is massive, you aren't trimming, youre using a sledge hammer to remove a portion of the question. Maybe your question needs sledgehammering, and I know I've written some stuff that should be sledgehammered at times, But I'm finding it really tough to believe that questions already below 425 characters need sledgehammering; at the very least, you are certainly affecting the pyramidality of the question.

Also because of these artificial caps, you can't write tossups that contain substantive information other than buzzphrases. Well actually you can, and NAQT does at times, but they don't a lot.

For example, Looking back at the hsnct, the schopenhauer tossup literally began, "this author of on the basis of morality argued there are four types of objects of knowledge in his on the fourfold root of the principle of sufficient reason..." and it goes on. You might have studied Schopenhauer's philosophy, but you will get outbuzzed by a guy who studied a list of Schopenhauer's works only. Its questions like these that make me believe the desire for fewest characters as possible is a bad idea, simply because whether it was the editor or the question writer, they don't realize that they are creating an anti-pyramid.

The main reason (and correct me if I'm wrong) NAQT prefers shorter questions is that they keep tournaments running shorter, or they can spend time doing something else, like asking more tossups, which is perfectly legitimate. But when you compromise a question to do something else with your time, that's a problem.

Oh and Bill your tossup editing has a few problems:
1.) Whose father's death? (ladies or protagonist)
2.) Who gives China painting lessons? Who Refuses to pay taxes? (Board of Alderman or Title Character)
3.) (Maybe you've read A rose for emily, but I haven't so I will ask this) Are you sure Homer Baron disappears, instead of just deciding never to return?

I think you could be changing some of the details of the story as pointed out in #3, and unless you've read it (which you may very well have) but going through mounds of questions, you probably haven't read everything tasked about, and at the very least you create some ambiguity.
Ike
UIUC 13
User avatar
btressler
Tidus
Posts: 620
Joined: Fri Mar 26, 2004 7:23 pm
Location: West Chester, PA
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by btressler »

Please forgive the grouchy mood that follows:

I think this should help with the "disappearance" http://en.wikipedia.org/wiki/A_rose_for_emily

Of course I don't want NAQT to eliminate actual clues. And I've already said earlier in this thread that I support NAQT adding about a sentence worth of actual clues into tossups.

If nobody else but me cares about unnecessary question length, then I withdraw my remarks as a pointless effort to persuade.

EDIT: (and now less grouchy!) I agree with your #1 and would personally put "her father's death". But the original writer intended to keep the gender of the protagonist obscure. Since I'm in the process of writing a packet, must the gender of the protagonist not be revealed until the later clues?
User avatar
ClemsonQB
Tidus
Posts: 511
Joined: Tue Dec 04, 2007 11:12 pm
Location: Clemson, SC
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by ClemsonQB »

Bad Boy Bill wrote:If nobody else but me cares about unnecessary question length, then I withdraw my remarks as a pointless effort to persuade.
While I agree that unnecessary words should be taken out of questions, I believe Ike's point is that pyramidiality matters more.
George Stevens

Dorman High School 2008
Clemson University 2012
User avatar
Gautam
Auron
Posts: 1413
Joined: Sun Feb 11, 2007 7:28 pm
Location: Zone of Avoidance
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by Gautam »

Bad Boy Bill wrote:Since I'm in the process of writing a packet, must the gender of the protagonist not be revealed until the later clues?
I don't think so. Maybe saying "the protagonist" for the first clue is fine, but I think there are enough works with female protagonists that saying "she" pretty early (second line or so) is fine. You might want to avoid that, however, if there are only or two characters in the work (which is, if I'm not mistaken, the case in A Rose for Emily); from my experience, it has been difficult for me to write about works with fewer characters and I inevitably end up using a lot of space in an attempt to conceal the fact I'm talking bout just one or two characters.

I am sure some of the more experienced writers will have better advice regarding the latter case.
Gautam - ACF
Currently tending to the 'quizbowl hobo' persuasion.
User avatar
Auroni
Auron
Posts: 3104
Joined: Thu Nov 15, 2007 6:23 pm
Location: Urbana

Re: Non-question-specific criticisms of IS-80A

Post by Auroni »

I'm going to make the argument that IS sets are a more reasonable introduction to quizbowl for new teams than A-sets are. Something I've noticed is that they often cover the same answer selection, although the A sets are decidedly of much poorer quality (hurf I hear artemio cruz experienced "death" FIFTEEEEEEEN). With this being the case, why are A sets so in demand that one must be produced for each IS set? How many tournaments packed with new teams are played anyway, and how many tournaments with decent teams that have expected to run on IS sets ended up running on an A set because they're in a 1:1 ratio? (many)
Auroni Gupta
UIUC
ACF
User avatar
naturalistic phallacy
Auron
Posts: 1445
Joined: Tue May 01, 2007 12:03 am
Location: Minneapolis, MN
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by naturalistic phallacy »

How about instead of having absurdly easy questions on often random answers, A sets have good questions on easier answers?
Bernadette Spencer
University of Minnesota, MCTC
Member, ACF
Event Manager, PACE
Order Support and Administrative Assistant, NAQT
User avatar
theMoMA
Forums Staff: Administrator
Posts: 5786
Joined: Mon Oct 23, 2006 2:00 am

Re: Non-question-specific criticisms of IS-80A

Post by theMoMA »

GARBAGE DAY 183 wrote:I'm going to make the argument that IS sets are a more reasonable introduction to quizbowl for new teams than A-sets are. Something I've noticed is that they often cover the same answer selection, although the A sets are decidedly of much poorer quality (hurf I hear artemio cruz experienced "death" FIFTEEEEEEEN). With this being the case, why are A sets so in demand that one must be produced for each IS set? How many tournaments packed with new teams are played anyway, and how many tournaments with decent teams that have expected to run on IS sets ended up running on an A set because they're in a 1:1 ratio? (many)
While I think your general point is valid, I'm not sure the example of "rudimentary knowledge of Carlos Fuentes works" proves or even supports it.
Andrew Hart
Minnesota alum
User avatar
AlphaQuizBowler
Tidus
Posts: 695
Joined: Mon Dec 03, 2007 6:31 pm
Location: Alpharetta, GA

Re: Non-question-specific criticisms of IS-80A

Post by AlphaQuizBowler »

Ike wrote:The problem is that when you do massive deletion - a sentence to me is massive, you aren't trimming, youre using a sledge hammer to remove a portion of the question. Maybe your question needs sledgehammering, and I know I've written some stuff that should be sledgehammered at times, But I'm finding it really tough to believe that questions already below 425 characters need sledgehammering; at the very least, you are certainly affecting the pyramidality of the question.

Also because of these artificial caps, you can't write tossups that contain substantive information other than buzzphrases. Well actually you can, and NAQT does at times, but they don't a lot.

For example, Looking back at the hsnct, the schopenhauer tossup literally began, "this author of on the basis of morality argued there are four types of objects of knowledge in his on the fourfold root of the principle of sufficient reason..." and it goes on. You might have studied Schopenhauer's philosophy, but you will get outbuzzed by a guy who studied a list of Schopenhauer's works only. Its questions like these that make me believe the desire for fewest characters as possible is a bad idea, simply because whether it was the editor or the question writer, they don't realize that they are creating an anti-pyramid.

The main reason (and correct me if I'm wrong) NAQT prefers shorter questions is that they keep tournaments running shorter, or they can spend time doing something else, like asking more tossups, which is perfectly legitimate. But when you compromise a question to do something else with your time, that's a problem.

Oh and Bill your tossup editing has a few problems:
1.) Whose father's death? (ladies or protagonist)
2.) Who gives China painting lessons? Who Refuses to pay taxes? (Board of Alderman or Title Character)
3.) (Maybe you've read A rose for emily, but I haven't so I will ask this) Are you sure Homer Baron disappears, instead of just deciding never to return?

I think you could be changing some of the details of the story as pointed out in #3, and unless you've read it (which you may very well have) but going through mounds of questions, you probably haven't read everything tasked about, and at the very least you create some ambiguity.
I agree with Bill on this. I think that a lot of questions contain unneeded verbiage. Ike, you said that he was affecting the pyramidality, but outlined no example of where that occurred. The her is a problem, but does it really affect the pyramidality? The part about the Board of Aldermen has the who specifically after "the title character", so it's not really ambiguous. I think that he managed to effectively shorten the question without sacrificing pyramidality, and I think, as he says, that this could be done with many tossups written. To me, if you can "use a sledgehammer to take out a portion of the question" while retaining the same clues in the proper order, you should.
William
Alpharetta High School '11
Harvard '15
User avatar
Ike
Auron
Posts: 1037
Joined: Sat Jul 26, 2008 5:01 pm
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by Ike »

you said that he was affecting the pyramidality
I'm not sure where I said Bill was explicitly affecting the pyramidality. And he isn't. My main point is that when editing for length, 1.) Pyramidality shouldn't be sacrificed and 2.) Ambiguity and Factual correctness might be altered, without the editor even realizing it. The latter two cases, I think applies on a whole a lot more. I'ven't read A Rose for Emily or know too much about it, so that's why I put a little disclaimer before my third point.
I think, as he says, that this could be done with many tossups written.
I agree. When I made my post about question length I was more disgusted at the fact that I'm seeing questions that have clues out of order in what appears to me, an attempt to make a shorter sentence. I can think of examples from recent IS sets, and they don't even go over the limit. And when editing for length, people inadvertently alter something they probably don't mean to alter. I'm trying to illustrate that when you cut questions for length, something's probably going to happen that you don't intend; I'm not saying "don't cut question length."
Ike
UIUC 13
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 5657
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by Important Bird Area »

Ike wrote:For example, Looking back at the hsnct, the schopenhauer tossup literally began, "this author of on the basis of morality argued there are four types of objects of knowledge in his on the fourfold root of the principle of sufficient reason..." and it goes on. You might have studied Schopenhauer's philosophy, but you will get outbuzzed by a guy who studied a list of Schopenhauer's works only. Its questions like these that make me believe the desire for fewest characters as possible is a bad idea, simply because whether it was the editor or the question writer, they don't realize that they are creating an anti-pyramid.
Let's talk about the anti-pyramid. I've always defined an "anti-pyramidal" question as one that places easier clues in front of harder ones. It's possible to imagine a question that's non-pyramidal because it consists entirely of clues so easy that it is reasonable to expect a first-clue buzzer race in many or most rooms, but that's not what we're talking about here either. Here's the text (again, 2008 HSNCT, so clear for discussion):

"This author of ~On the Basis of Morality~ argued there were four kinds of "objects of knowledge" in his ~On the Fourfold Root of the Principle of Sufficient Reason~. He denounced Fichte in his ~Parerga and Paralipomena~, but is best known for a work based on Kant's idea of the "thing-in-itself" that praises asceticism as the denial of one of its title concepts. (*) For 10 points--name this author of ~The World as Will and Idea~."

"On the Basis of Morality" is fairly clearly the most obscure of the four works mentioned; unlike the other three, it does not appear in Schopenhauer's Britannica entry. "Parerga and Paralipomena" is a fairly well-known middle clue for Schopenhauer, and "The World as Will and Idea" is his best-known work, so it belongs at the end, for people who know only that one item.

Now then, let's look at the conditions for the buzzer race you identify. You need a population of players as follows:

-some players who have studied Schopenhauer's philosophy, but do not know the title of his fourth-most famous work (I have no idea of the relative fame of minor Schopenhauer works: but let's say fourth.)

-some players who have memorized a list of Schopenhauer's works, including the various minor ones, but who have not studied his philosophy

I would guess that both of these groups are small relative to the HSNCT field, and that the odds that more than one tournament room would contain members of both are tiny.
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 5657
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by Important Bird Area »

Ike wrote:I was more disgusted at the fact that I'm seeing questions that have clues out of order in what appears to me, an attempt to make a shorter sentence. I can think of examples from recent IS sets, and they don't even go over the limit. And when editing for length, people inadvertently alter something they probably don't mean to alter.
Examples welcome by private email, please.
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
User avatar
Auroni
Auron
Posts: 3104
Joined: Thu Nov 15, 2007 6:23 pm
Location: Urbana

Re: Non-question-specific criticisms of IS-80A

Post by Auroni »

theMoMA wrote:
GARBAGE DAY 183 wrote:I'm going to make the argument that IS sets are a more reasonable introduction to quizbowl for new teams than A-sets are. Something I've noticed is that they often cover the same answer selection, although the A sets are decidedly of much poorer quality (hurf I hear artemio cruz experienced "death" FIFTEEEEEEEN). With this being the case, why are A sets so in demand that one must be produced for each IS set? How many tournaments packed with new teams are played anyway, and how many tournaments with decent teams that have expected to run on IS sets ended up running on an A set because they're in a 1:1 ratio? (many)
While I think your general point is valid, I'm not sure the example of "rudimentary knowledge of Carlos Fuentes works" proves or even supports it.
yeah? there are a bunch of other examples from cleared sets in the past year that do (the martin luther king tossup mentioning the garbage workers' strike right away, to name just one). The most egregious example was the first thing that stuck. Also, by having tossups on Stendhal in A-sets where the main character of The Charterhouse of Parma is bizarrely not in power just proves that the answer selection and intended difficulty aren't much easier than those of IS sets.
Auroni Gupta
UIUC
ACF
User avatar
Sir Thopas
Auron
Posts: 1330
Joined: Tue Mar 28, 2006 10:10 pm
Location: Hunter, NYC
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by Sir Thopas »

bt_green_warbler wrote:Now then, let's look at the conditions for the buzzer race you identify. You need a population of players as follows:

-some players who have studied Schopenhauer's philosophy, but do not know the title of his fourth-most famous work (I have no idea of the relative fame of minor Schopenhauer works: but let's say fourth.)

-some players who have memorized a list of Schopenhauer's works, including the various minor ones, but who have not studied his philosophy

I would guess that both of these groups are small relative to the HSNCT field, and that the odds that more than one tournament room would contain members of both are tiny.
That's not true at all, a buzzer race is much more likely than you allow for it. If someone has studied Schopenhauer's philosophy and, as a consequence, knows all of his works, he'll buzz at the first title. Someone who has just memorized titles, on the other hand, will buzz at the same point. I'd put the place where they'd buzz like so, where ** is the first player and [] is the second:

"This author of ~On the Basis of Mor*[ality~]* argued there were four kinds of "objects of knowledge" in his ~On the Fourfold Root of the Principle of Sufficient Reason~."

"This author argued there were four kinds of "objects of knowledge"* in his ~On the Fourfold Root* [of the Principle of Sufficient Reason~.]"

In the first instance, the two players will buzz at the same time, which leads to a buzzer race. This cripples the advantage of the player who clearly has more knowledge, as opposed to just list knowledge. This goes against the reasons for pyramidality, and is thus fast approaching antipyramidality (even if it doesn't quite hit it). In fact, I'd give the list player a bit of an advantage, as list players tend to know how to buzz on list clues better than others. The second lead-in, however, clearly gives advantage to the player with more knowledge. Even if he doesn't buzz immediately on "objects of knowledge", he'll be thinking in the right direction, and will almost certainly buzz earlier on in that long title than the list player will.

There are many list players, especially in high school, so the odds aren't really that tiny at all. What bothers me, though, is that the fact that odds of a bad question's critical flaw being exposed are pretty small, is used as justification for leaving the flaw in.
Guy Tabachnick
Hunter '09
Brown '13

http://memoryofthisimpertinence.blogspot.com/
User avatar
at your pleasure
Auron
Posts: 1712
Joined: Sun Aug 03, 2008 7:56 pm

Re: Non-question-specific criticisms of IS-80A

Post by at your pleasure »

There are many list players, especially in high school, so the odds aren't really that tiny at all. What bothers me, though, is that the fact that odds of a bad question's critical flaw being exposed are pretty small, is used as justification for leaving the flaw in.
I'm not even sure that the odds of a flawed question's flaw being exposed are terribly low to begin with.
Also, a considerable number of untimed tournaments that are run on longer questions, which have the rooms hearing about the same number of questions as they would in timed NAQT round, finish at a perfectly reasonable time. This badly undercuts the efficiency argument(which NAQT seems to have made central to its justification of timers and short questions) for timed rounds and short questions.
Douglas Graebner, Walt Whitman HS 10, Uchicago 14
"... imagination acts upon man as really as does gravitation, and may kill him as certainly as a dose of prussic acid."-Sir James Frazer,The Golden Bough

http://avorticistking.wordpress.com/
User avatar
Ike
Auron
Posts: 1037
Joined: Sat Jul 26, 2008 5:01 pm
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by Ike »

Even though I used the word outbuzzed, my intention is more about the fact of knowledge in itself.
I'm no Schopenhauer fanatic, but I'm willing to wager "four objects of knowledge" is more obscure than On the Basis of Morality.

And on another note as illustrated earlier:
this is a question that is a perfect example of consisting mostly of buzz-phrases:

Phrases You Can Buzz
"On The Basis of Morality"
"4 Objects of Knowledge"
"On The Fourfold Root of the Principal of Sufficient Reason"
"Parerga and Paralipomena"
"thing in itself"
"World as Will and Representation"

Clues that near substantive knowledge:
"denounced fichte"
"praises ascetism as the denial..."

I'm willing to wager that a good bit of the time this tossup does a good job at distinguishing teams (but I don't think it does when it counts - HSNCT), but my bigger complaint is that I really don't see any in depth knowledge on Schopenhauer's concepts. This is probably the biggest reason I cringe at attempts to trim down questions, it's because it puts no value on substantive information. That's my personal reason for being against NAQT's short questions and trimmings.

Bill's examples of trimmings aren't anywhere close to what I'm trying to go against.
Ike
UIUC 13
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 5657
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by Important Bird Area »

Sir Thopas wrote:In the first instance, the two players will buzz at the same time, which leads to a buzzer race. This cripples the advantage of the player who clearly has more knowledge, as opposed to just list knowledge. This goes against the reasons for pyramidality, and is thus fast approaching antipyramidality (even if it doesn't quite hit it). In fact, I'd give the list player a bit of an advantage, as list players tend to know how to buzz on list clues better than others. The second lead-in, however, clearly gives advantage to the player with more knowledge. Even if he doesn't buzz immediately on "objects of knowledge", he'll be thinking in the right direction, and will almost certainly buzz earlier on in that long title than the list player will.
This is an entirely legitimate criticism that does, I believe, produce a better question.
Sir Thopas wrote:There are many list players, especially in high school, so the odds aren't really that tiny at all. What bothers me, though, is that the fact that odds of a bad question's critical flaw being exposed are pretty small, is used as justification for leaving the flaw in.
This on the other hand misrepresents what I'm trying to say. This is only a bad question with a critical flaw, if in fact many high school players have memorized a lengthy list of Schopenhauer's works. If (as I suspect) the overwhelming majority of players will not reflex buzz on this clue, it will work just fine.

This discussion leads me on to two larger questions. Not directed at any particular poster here, but intended to gauge community standards of good quizbowl.

1. Is it ever ok to use the title of a minor work as a leadin clue, or should we assume that most teams will have at least one player who has memorized the listed works of all canonically-tossupable answers? (I would suggest that while the example above might be a mediocre question, it's not a bad question. A bad question is "This author of ~The World as Will and Idea~...")

2. Is it possible to entirely eliminate the buzzer race? My belief has always been: no, sometimes two teams will happen to know the same clue at the same time. Fundamentally, this is why we play entire packets and not individual tossups. A question that generates a buzzer race in one or two rooms at a tournament the size of the HSNCT could easily be good quizbowl, assuming it is pyramidal and enjoyable for the other 150-some teams that don't know that particular leadin clue. A tendency I've seen in recent question criticism (this is also true of the "high school bonus difficulty and team sorting" thread) is to focus in on the flaws of individual questions as perfect sorting mechanisms, and I think at some point this misses the forest for the trees. (Note: this is not, at all, intended to discourage criticism and commentary about individual questions, produced by NAQT or any other provider. Also: not speaking for NAQT, so if you think the above is a recipe for bad quizbowl and the doom of the circuit, come talk to me and don't take it out on R.)
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 5657
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: Non-question-specific criticisms of IS-80A

Post by Important Bird Area »

Ike wrote:I'm willing to wager that a good bit of the time this tossup does a good job at distinguishing teams (but I don't think it does when it counts - HSNCT), but my bigger complaint is that I really don't see any in depth knowledge on Schopenhauer's concepts.
Is this one better? (2002 HSNCT)

"An opponent of the German metaphysicians of his era, he called Hegel a "charlatan" and blasted their work in his 1813 "On the Fourfold Root of the Principle of Sufficient Reason." His (*) pessimistic outlook stemmed from his belief that people strive for the unattainable because the fundamental feature of the universe is will. For 10 points--name this German philosopher of ~The World as Will and Idea~."

Observationally, I think the underlying tension here is that it is more difficult to write uniquely identifying clues based on the substantive contents of works (here: of the various other people who have opposed German metaphysics over the years, how many of them hated Hegel?). Therefore writers have an incentive to put the names of minor works early in questions, giving teams an incentive to memorize progressively more obscure lists of works in order to earn points. Whether it's possible to stop that cycle I'll leave as an exercise to the reader.
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
Locked