How many "cooks" is "too many cooks" (when producing a set)?

[Adventure Temple Trail]

In January, when CO 2023 did not yet have editors, I posted:

Stray observation: It seems like, in recent years, the number of editors per tournament has sharply increased (10+ is now normal) and the typical number of (sub)categories edited per editor has decreased proportionally. Does the fact that most subject editor jobs are smaller increase anxiety about the prospect of head editing? Does this make it harder or easier to assemble writing teams? Do people tend to feel they've hit their limit for the ~year editing 1/1 or 2/2 on one set, or more able to do several small editing jobs?

Thankfully, CO 2023 did assemble a team. With fifteen editors, two logistics persons, and about a dozen freelance writers, it had the largest production team of any CO to date. It's not alone in that trend -- every ACF event this year has 10+ people on the editing team alone. It's not unusual for some editors to do onlyone sub-subcategory (e.g. "earth science" or "opera"). Add dedicated proofreaders, playtesters, freelancers, advisors, etc. and sometimes thecredits roll of a set reaches 30+ people. (You do still see some feats by one writer (e.g. DMA) or a very small handful, but they're getting rarer.)

I'd like to start some broader discussion of the pluses and minuses of this change. Some things that appear to me as tradeoffs/axes include:

• Manageable per-person workloads vs. coordination problems. It feels much more reasonable to ask a person to edit 1/1 or 2/2 than it is to ask the same person to edit 6/6 or 7/7, and that's because it is. That said, more people causes more tracking problems for those in charge. While raising the size of the production team reduces the impact of one person flaking, it also may make such flaking more likely due to diffusion of responsibility; it may feel "less bad" to flake on a task that didn't feel very large to begin with.
• More editing opportunities for more people vs. less of a "resume to show for it" from each opportunity. It's easier now than it used to be to apply to do some work on a set in a category without having social ties to the existing in-crowd. I do worry about it being less easy to demonstrate a body of good work over time if editing jobs are smaller and more people have them. (Relatedly, I do sometimes wonder about question sets presented as written "by" a high school team or collection of novice writers, with high-caliber upperclass collegiate and open-level "editors" brought in. It seems highly likely that almost all the newcomers' output is discarded or near-totally rewritten. At that point, how honest is it for one of those newcomers to present themselves as qualified to take on more responsibility in the future?)
• Improvement of (hyper-)specialized editing vs. decline in generalist oversight. Some categories, most notoriously science and auditory fine arts, are often prone to subtle factual errors that generalists are less likely to detect, and the standards for accuracy are (rightly, for a game about facts) higher than they once were. Specialized subject editors can take the time they need, and have the knowledge base needed, to weed those out. I do fear that if most editing jobs are expected to be small in size and specialized in subject matter, it will end up making the idea of head-editing or overseeing an entire set look too daunting for most experienced subject editors to want to take up. This seems to be occurring in tandem with a decline in subject-spanning generalist players in the current game (most of the ones that remain are, well... from before the current game).
• Non-editing tasks getting the attention they need vs. non-editing tasks interfering with editing. It's good when big editors can be freed from also handling mirror logistics, proofreading, document styling, and the like, because other capable people are handling them. That said, I do worry about nice-to-have tasks, such adding pronunciation guides, scrunching the timeline for editing even further, which to some extent puts the cart before the horse.

It's a good development that sets are, mostly, produced in a more professional manner, and with more institutional quality-checks than there used to be. I would not return to the Bad Old Days where sets were routinely "finished" the night before the first mirror (and occasionally being written during the first mirror). I'm glad that we no longer shrug off persistent flaking and/or last-minute work as inevitable aspects of quizbowler culture.

From my experience, my hunch has long been that about 10 people can realistically contribute a "significant chunk" to an independent question set before coordination problems set in, and/or quality starts to dip, and/or someone in the top 10 contributors starts doing most of the work the 11th person was supposed to do anyway. I have an additional hunch that the ideal distribution of work on an n-person production team is not an even division among the n people; typically, the top few contributors will do a lot more than the next few, and so on, and that's not an accident.

A potential happy medium I've seen some people discuss is a model in which a few relatively co-equal "core editors" each have a pretty big chunk of the set (say, 3 to 5 of them splitting up ~50-75% of the set), with additional specialized subject editors brought in to fill gaps in their competencies where needed. Does there really need to be exactly one "head editor" all the time? (And is there really one at all if the nominal head editor is only doing 2/2 as other subject editors act autonomously?) In A World where few to no players can purport to supervise almost all the categories, it might make more sense to have a small "brain trust" at the top anyways.

Thoughts?

[Cheynem]

This is a good post, and something I've thought about in recent years.

First, there is a key difference between editors and contributors. In general, barring crazy exceptions, I don't think there's an issue in having a lot of contributors, as long as they are contributing decent work. After all, technically any packet submission tournament probably has 50+ contributors. Sometimes tournaments just need people chipping in questions, particularly if they are a tournament that has a lot of needs.

Secondly, I would say some of this is contextual. I do not mind a tournament like ACF Nationals or even Chicago Open from having a lot of editors because these are some of the most difficult questions for a best of the best field. You want your editors working on categories they feel comfortable with, especially if they may not necessarily feel experienced. I would though raise an eyebrow at some regular or easy difficulty tournament sets having so many editors unless there was a good reason. When I was working on MUT, it would have felt strange and unwieldy to have tons of sub editors.

Thirdly, you don't necessarily need to bring in an outside editor for some specialized topics. While I'm old and have an okay generalist baseline for editing, there are some topics I know nothing about, like economics or linguistics. If I were editing social science (and I feel okay doing that at lower levels), I wouldn't bring in an "economics" editor, I'd just ask someone I trust to chip in a question or look over my question. Obviously that's easier for a small topic than something like "chemistry," but it's a possibility.

Fourthly, I do think we should gently encourage a certain amount of "grip it and rip it," especially for lower level collegiate tournaments. I remember my second MUT, I was told I was going to be the literature editor. Surprise! And some of my questions were originally pretty poor. Now, I had the advantage of being able to get help/feedback from very experienced people, but I would also say I got a lot of long-term growth from that. In my time working on MUT, I think I would end up editing basically everything but science and maybe some fine arts stuff at one point or another.

Finally, I don't know if Matt remembers the CO we worked together on (2016), but while it had a lot of editors, I think it was a good model overall. I don't know what Matt and John were actually designated as (head editors? supervising editors?), but I liked that while Matt deferred to my choices as co-history editor, he also offered some good feedback--both broad and specific. I also liked that for the few sub-genres I didn't know very well, he'd chip in questions or look things over. I'm a big fan of having, as Matt suggests, head or supervising editors who are very active and busy but also deferential when appropriate. You need that sort of supervisory hand rather than every editor running willy nilly amok.

[Gene Harrogate]

Since you bring up CO, I do think the size of the team created some issues that came through in the final set. When Young, Alex, and I were planning back in February, we tried to make up for the very short amount of time we had to write the thing by taking up as many people as possible on their offers to edit. Working on CO is a big commitment to make in peak burnout season, so this was usually some amount of 2/2 or less. Some slices of 1/1 even ended up the responsility of three different editors, which was helpful at the initial writing stage. But when it came time to packetize editor questions and combine submissions, it was exceptionally difficult for even the most hardworking and present subcategory editors to coordinate what questions to keep or where to put them. I ended up making many of the calls regarding packetization, often in the hours before the packet was due to be playtested -- hardly an ideal state of affairs considering the category in question was usually some science. The rushed production schedule, the packet sub nature of the tournament, and my own inexperience of course exacerbated this problem. But in any case, I would suggest that future sets -- unless they're produced by coherent teams that will communicate exceedingly well with each other, such as individual clubs or long-standing friend groups -- have a single person with the responsibility to make the final editorial decisions regarding a particular 1/1, leaving any "sub-subcategory" roles either advisory or entirely at the level of individual questions.

Mike's distinction between editors and contributors is helpful. It has not really been my experience on any set that there were too many people writing questions.

[The King's Flight to the Scots]

This is an insightful and well-reasoned post. In the past, posts on this topic have had a bit of a "kids these days" feel to them; this one seems more balanced, identifying the real trade-offs entailed when you increase the size of the editing team. From personal experience, 10 seems like an appropriate number of editors per set to me as well.

The one area where I may want to push back a bit is Matt's fourth point. I actually think most sets don't devote enough time to "postproduction" tasks, given how much they impact the players' experience. Missing words, misleading constructions, and incomplete answerlines ruin lots of otherwise well-edited questions. As an editor, there are few feelings worse than seeing your question ruined by an easily-fixable mistake; to avoid that, it's reasonable to have some form of proofreading taking place for the last two weeks of editing.

It's possible Matt intended this point more narrowly than I'm construing it, though. I do think pronunciation guides are essential aids for moderators, but I agree that mispronounced words are usually less disastrous than missing answers. More generally, we should at least be strategic about how we spend our time in proofreading so that we're rooting out the most consequential issues.

[naan/steak-holding toll]

Fourthly, I do think we should gently encourage a certain amount of "grip it and rip it," especially for lower level collegiate tournaments. I remember my second MUT, I was told I was going to be the literature editor. Surprise! And some of my questions were originally pretty poor. Now, I had the advantage of being able to get help/feedback from very experienced people, but I would also say I got a lot of long-term growth from that. In my time working on MUT, I think I would end up editing basically everything but science and maybe some fine arts stuff at one point or another.

Seconded. As long as you have some canon sense, are willing to put in work to do real research (as opposed to just ripping clues from old questions wholesale), and have someone more experienced looking over your shoulder, editing a new category and making a decent product isn't that hard. It's going to take you a while, of course, but will also be a good learning experience!

[theMoMA]

I generally think seven or eight editors is an appropriate higher-end number for most sets, although larger or more challenging sets (like Nationals or CO) might warrant more. Editing teams that have a degree of comfort and familiarity with each other might be able to make larger (or indeed smaller) numbers work; in my experience, people who have worked together before and can trust one another to get things done are usually able to handle coordination and division of labor in an effective and agreeable way. I agree that having contributors beyond those on the editorial masthead (such as freelancers, mirror coordinators, logistical support, proofreaders, or pronunciation-guide adders) is not a problem and is indeed desirable.

For what it's worth, I generally think the slide toward bigger editing teams has been a negative development. Although smaller editor assignments obviously carry smaller individual burdens, I think it actually increases the total amount of work by an appreciable amount, and places a lot of the extra work on the head editor, who has to coordinate a large number of category editors and harmonize many different (and possibly disparate) visions into a cohesive whole. I think the smaller assignments themselves are also detrimental to development of the category-level editors; people may get scared into believing that they can't edit beyond a narrow speciality area, may not have enough responsibilty to grow and flourish in their roles and take on larger future responsibilities, and may be more likely to procrastinate if they think their assignment is small enough to knock out quickly. Having more editors also increases the number of people who might flake out on a set or have other major problems during editing.

My suspicion is that the ever-increasing size of editorial slates is tied to a few factors.

First, editors may think that standards are so high that players will react poorly to work outside of the editor's narrow speciality area. I don't think this is really true; most editors do not have strong speciality knowledge of individual question topics, even in subject areas where they might have strong overall knowledge, and this is generally fine. Furthermore, most of the strongest editors have a general set of editing skills and principles that they apply (coupled with diligent, question-specific factual research) to different areas of the distribution, and this (rather than application of existing technical knowledge) is generally how good questions get produced. All categories have a baseline level of requisite knowledge, and in some categories (like science) that knowledge is relatively technical and hard to come by, but I think it's largely a myth that you need to have formal training or existing expert-level subject-matter knowledge to edit most, if not all, categories.

Second, I think editorial application processes play a role. It's good that a lot of events have open editing calls, but I think it's harder to stand out as the right editor for the job across multiple categories when you don't have existing subject-matter knowledge in those categories, even if you're actually the best candidate based on general editing skill that can be applied to all of those categories. I think this encourages people to apply for smaller roles where they can stand out; equally importantly, I think decisionmakers are steered toward editor candidates based on the same criteria, and may needlessly (or even detrimentally) expand the editor team based on the notion that it's best to have credentialed experts in every category editing those categories. For example, an art history major who is a strong general editor and would be actually the best candidate to handle visual arts and portions of history and literature might only apply for the visual arts categories; even if that person applied for multiple categories, they might not seem like the best person for all of the jobs (even if they actually are), especially if literature and history majors are also applying to edit.

Third, I think people generally have the false notion that it's easier to edit a tournament when there are more editors who all have smaller assignments. I don't think this is true on the whole; as I said above, the total amount of work generally goes up with the number of editors, because coordination and harmonization become much more difficult. I also think it's not as true as people might expect when it comes to individual assignments. Having significant responsibilities often means being more deliberate about working on the set and more collaborative with other editors. Even though editing 4/4 might seem like four times the amount of work as editing 1/1, I think it's actually a much softer multiplier, because more responsibility usually means planning your time more responsibly, and there are also efficiences that come with working on larger parts of a set as you learn what works and what doesn't and can apply those better processes to writing and editing down the line. (In other words, you get better and better at knowing how to produce questions for a particular event, and the 100th question is usually going to be easier than the 3rd, so bigger assignments are usually going to be more efficient.)