I think this view overlooks the obvious fact that “difficulty” is both an objective and subjective category. It’s determined by concrete stats measuring the number of questions answered and people’s subjective sense for how the questions “felt” to play. Numerical data and personal experience are both valid criteria for appraising difficulty that deserve respectful consideration. While upper-level editors might favor statistics as a metric, I think less experienced teams largely tend to judge difficulty on subjective “feel” more than empirical data. Now, a third-bracket team might answer the same number of tossups in packet six of 2014 Nats and 2015 Nats, but there’s a massive difference in how a dead tossup on Mary Tyrone feels compared to a dead tossup on The Fifth Column. Statistically, they are equivalent in difficulty, but psychologically feel quite different to play. Lower-bracket teams don’t care so much about converting one extra bonus part as much as feeling they have a chance to know some of the answers. This general sense makes even the worst players engaged with a tournament because they know there will be some answers that they’ll recognize. When you don’t know a single answer to a lit tossup for three straight packets, it’s crushing and only natural for a player to disengage. Let’s look at the lit tossups for editors 4, 5, and 6: My Kinsman, Major Molineux, Between the Acts, Leopardi, Makioka Sisters / Dictionary of the Khazars, V, Buchan, Ariel Dorfman’s play / Fifth Column, Candida, Baudolino, and Malaria (from ungettable clues). At this point, the collective effect of so many upper-level difficulty questions “feels” overwhelming to the average player—and apparently several top players as well.theMoMA wrote: I'd like to offer the following argument: claims about how hard a tournament was are empirical, and depend on hard data about how many questions were answered and when the buzzes occurred; they are not about how important the topics asked about were, and do not depend on how much importance any particular expert attaches to a particular answer, and similarly, they do not depend on one person's universalization of their subjective feeling of playing the question.
This begs the question: How can an editor predict the subjective “feel” of a tournament? There’s a group of esteemed senior editors who seem to dismiss all personal complaints of this sort about difficulty as entirely arbitrary—outside their control and their responsibility. Editors like Ryan Westbrook, Jerry, and Ike seem irritated that audiences would even deem to judge their questions on this criterion. Andrew Hart offers the most eloquent summary of this view:
Conversely, I think it’s possible for editors to predict the general difficulty of packets and think it’s important, even essential, for editors of marquee tournaments like ACF Nats and ICT to put premeditated thought into planning the arrangement of answers in the editors packets to have consistent difficulty. Evan has already called attention to the concrete difficulty scale I used to balance the difficulty of my categories at last year’s Nats, but I also took specific measures to program the subjective “feel” of playing the editors packets.theMoMA wrote: I'd venture that, if you posted similar lists of answers from one editors' packet to the next, it would be close to impossible to pick out which ones actually turned out to be more difficult. From a practical standpoint, as I'm sure everyone knows, editors' questions get written in chunks, and there's a lot of chance that goes into how exactly they get clumped together into packets. A particularly hard editors' packet is almost certainly just a difficult-to-foresee outlier. So while the effect of Editors' 6 being particularly tough may have been large, it's hard to blame the editors too much for not being able to predict that that would be the case.
The reason last year’s tournament felt less difficult was because I fastidiously planned the answer selection—and answer arrangement—across the editors packet to create a well-balanced range of answers. I appointed myself as “managing editor” because I wanted ton devote a large chunk of my editing time to molding the “feel” of the tournament. For example, Jerry initially had slated the first three philosophy tossups in the editors packets to be on Hubert Dreyfus, Gettier Problems, and Jerry Fodor's language of thought. Regardless of the other answers in the philosophy distribution, after starting the playoffs with a murder’s row of 20th century philosophy most people’s impression of the category as impossible will already be cemented. Accordingly, I worked with Jerry to replace two of the hardest answers with two easier concept tossups and rearranged the order of the tossups across the first six editors packets so Dreyfus-Gettier-mentalese-ressentiment-master/slave dialectic-thing in itself became Nagel-ressentiment-history-immortality of the soul-mentalese-master/slave dialectic. Now, the second group of answers certainly isn’t easy and perhaps is statistically equivalent in difficulty, but I would argue it feels more accessible since it has tossups on general concepts such as “history” and “the soul” with clues quoting people like Plato rather than hardcore analytic philosophers.
My methodology was to adjust my answer selection to create a semi-consistent difficulty in each packet. In packet five, I saw Jerry was tossing up both Walter Lippmann and Mentalese, so I adjusted the distribution to have three easier lit answers (Stephen Dedalus, Ovid, Enemy of the People) to balance the “feel” of the packet’s humanities. After being introduced to Water Lippmann and Jerry Fodor, I thought many teams might not be in the mood to meet Ryszard Kapuscinski in the same packet. So I switched some answers to bring in Ovid and Ibsen. Moreover, I wanted to avoid the fashionable bias toward obscure, experimental 20th century writers that tends to permeate upper-level tournaments. At the same time, I intentionally placed my hardest painting tossup Diebenkorn in that round, after specifically placing easy artist tossups on Picasso and Velazquez in the previous rounds. I wanted the painting distribution to take a violent shift from Velazquez to Diebenkorn. I wanted to create a sense of dissonance to combat transparency, so players would think “This could be a tossup on Picasso or someone who’s never come up before. This could be Ovid or it could be the Roman-poet-version of Richard Diebenkorn.”
Now, I’m not saying that every editorial decision I made was infallible or that I'm more responsible for the positive reception of last year's ACF Nats than the other editors. But I think last year's set benefited from the fact I spent time thinking about the overall experience of playing the editors packets as a whole rather than just focusing on the discrete experience of playing each question in isolation. I tried to adopt the mindset of someone who is sitting down to play the editors packets straight through, and adjusted my distribution to make this experience more balanced. You have a year to write the editors packets for ACF Nationals. It’s entirely reasonable to ask the editors to put some thought into planning the overall distribution.
