ACF Nationals Impressions or Commentary Redux

[salamanca]

Before too much time passes, I just wanted to share my impressions about the questions at ACF Nationals 2009, as well as the events leading up to the tournament.

I want to begin with a discussion of the logistics and what Matt or Sean, or the rest of the staff, could or could not have done differently on the day(s) of the tournament. In general, I thought that certain annoying logistical issues that plague many tournaments (e.g., some slower readers, games being split in different buildings, initial schedules being messed up, etc.) just happened to hit all at one event, which made the situation seem that much worse than it was.
That does not mean that I wasn’t sympathetic to the frustrations of all the players who showed up and had to deal with the various snafus over the course of the two days. In fact, as I told, I think it was, Andrew Hart of the Minnesota team, when they decided to postpone their playoff game with Chicago A, so that a working buzzer could be found, if I was still playing and this stuff would have occurred, I would have done the same thing and been damn angry about the workings of the tournament for longer than just a hot second.

Moving on to specifics:
1. buzzers- I must admit that I shared Matt’s initial wonder at the few *working* buzzers that showed up at the site on Saturday morning—I have never seen such a situation arise at a prestigious college tournament and, although I was not aware that Sean approached Matt in advance about procuring buzzers from high schools, I have a lot of empathy for the situation. I mean, people used to bring their buzzers to quizbowl tournaments. What the fuck happened? Also, mad props to Sorice and the rest of the Illinois team for showing up with 4(?) sets—everyone needs to take those boys out for a drink sometime in the near future.

2. schedules- The slightly messed up schedule thing was also unfortunate, but again that happens at tournaments all the time—the bigger issue was that we had no immediate access to a compatible printer, so that we could fix the mistakes immediately. I don’t know whether or not this was something Matt and Sean discussed beforehand, but that did strike me as a notable oversight.

3. distribution of questions- first off, I would have preferred to print the whole damn tournament out, but I realize that folks have been running paperless tournaments for some time now, without major hitches, and it seems reasonable to me that if WUSTL explained to Matt that most of the rooms would be equipped with computers that had internet access, that he would have trusted that representation. It is true that Matt probably should have requested that everyone bring their flash drives before the first round and loaded the whole set on them as a back up, but that didn’t happen and so we did waste some time running up and down the stairs trying to get everyone the questions.

4. readers- this is one of my own biggest issues and I must say that even though the assembled staff tried gamely to work with both the length and the sheer volume of the question set, there was a marked disparity in the quality and the speed of the readers at the event. Of course that happens to a certain extent at every tournament, but I do think that where Matt and Sean dropped the ball, at least initially, was assigning certain folks to read dense and deep ACF Nationals level questions when those persons were more accustomed to NAQT or HS questions. Though I do want to assert that by the end of the tournament, much of this was rectified, I think future directors of this tournament have to recognize that reading at ACF Nationals is a different animal from your regular QB event and ACF should strive to host the tournament in locations where more experienced moderators are available (e.g., the DC area, Chicago, Michigan, etc.) or should find some way of enticing experienced moderators to fly in.

I am sure I missed some stuff with regard to the logistics and I must beg forgiveness if I mistakenly misrepresented anything I said above, but at the end of the day, there is nothing to do but grin, bear it, and do it right the next time. In that spirit I would also like to commend the teams, who remained incredibly patient throughout most of the delays.

Anyways, moving on to the other major issue discussed after the tournament, the state and quality of the questions:

Were they complete?
The set, as it was eventually played, was all but finished when I got on the plane to come to St. Louis. There was some late night editing and writing going on Friday before the tournament to fill out a category or two, and there was some randomization and the odd question here or there written on Saturday night for the editor’s rounds that were to be played on Sunday, but 21 rounds is a shit-load of rounds and, at least from my point of view the set was completed in time. Moreover, Matt did not have to exhort folks to finish the set, people volunteered to pitch in and I think the circuit and ACF is better for having such dedicated persons involved.

However, I would say that Matt was probably overextended heading into Nationals. He has edited a ton of tournaments this year and should have given ACF Nationals more attention at an earlier date, particularly since he initially planned to edit/write every category except for science and RMP/Fine Arts. I think he still did a good job with the questions he managed to get to, but I think he was a bit too confident in his abilities to finish the questions that remained to be written in the final two weeks. Unfortunately, I feel as though I should have been more assertive and insisted to Matt, two weeks before the tournament, that I was going to write all the lit that needed to be finished. That would have freed Matt up to finish other categories and I am confident that we would have had the time we needed to polish the set. Instead, I was told that things were alright as the backlog of questions grew, and when we finally made the switch, it was too late to build a review of the set into the week before the tournament. Now, on some level, this turn of events did not surprise me one bit and I think part of the problem was what I would like to term “editorial hubris.”

If you have ever sought to be the chief editor of a Nationals level tournament and you think of yourself as a good editor/writer you know what I am getting at. The hubris I am talking about occurs when you want all of the questions to be touched by you and you want them all to be good, so you take your time writing them, crafting them, but the clock just keeps on ticking and soon you are struggling to finish, yet you still refuse to ask for help because it is your tournament. Let me tell you something, that shit is deadly. In addition, to necessitating the addition of folks at the last minute to the process, whose questions may or may not jibe with the majority of the set you or your editing team have already produced, more importantly, it costs your tournament that critical week where you can sit down and read the entire set to get a global perspective on the questions. It has certainly happened to me before (most notably with Manu), and I’d wager it has probably happened to every major editor of the past 15 years at some point in their careers. So what can we do to avoid this? Start early and ask for help early.

If there is one thing I learned from editing/writing tournaments over the years, is that you have to get your stuff done far enough in advance to be able to let the questions marinate and then take a look at them, and I think that is the biggest issue I had with this set. Moreover, I think that it was this lack of review time that gave birth to the contention that this tournament was inconsistent about difficulty, especially with regard to the bonuses in the playoffs.

Question quality and difficulty in general-

To begin, I would like to say that I thought this question set, as a whole, lived up to the standards of ACF Nationals of the past. A large majority of the questions had interesting, non-fraudable clues, and worked the canon in novel ways. And although Matt and I don’t always agree on question writing philosophies, I thought the history was good and, with the exception of some odd answer selection, I thought most of his literature questions were also sound. I thought the SS suffered from Jonathan being told very close to the date of the tournament that he would have to take over, but it was certainly solid and The Civilizing Process TU in the second finals was really cool. Obviously, I can’t speak authoritatively on the science, but I thought the editors worked their tails off to get a competent set of questions out to the public.

As far as claims that the bonus difficulty in the playoffs was wildly askew—I kind of buy that, but I kind of don’t. For sure, the play-in finals round that Andrew has focused on in his discussion of the set had some serious issues (e.g., no one disputes that the Browning bonus, and the Kranz anatomy bonus were both invitational level and should have been revised), but I think that much of the rest of the packet (non-science) was pitched at an appropriate level. And I certainly think that the playoffs as a whole, though they fluctuated somewhat internally per packet, were consistent enough. I mean, I certainly sympathize with Andrew’s position and while I was reading I noted that Chicago got both of the super easy (at this level) bonuses, which helped them to make up for the initial disparity in tossup conversion, but I think that the packet and the playoffs as a whole were less up and down than some folks have implied. Indeed, Andrew himself bemoaned only getting 10 on the Man Ray bonus, in effect admitting that he should have gotten more, as his team was playing it. I think this may also have happened with the Dutch Landscape Artists, but I can’t recall exactly.

What is more frustrating and, I take it, is Andrew’s main point was that bonus equalization should’ve been built into the editing process as part of a one week review before the tournament. In fact, evidence of how much good even a two day review would have done exists in nearly every packet that has been released. All of the editor’s rounds had extra bonuses that could have replaced some of the weaker bonuses that made the top 20 during the initial randomization. Just take a look at the extras in the round that Andrew and Seth refer to in their general discussion of the tournament, we can see that Browning could have been replaced by a Cather bonus that was written at a more appropriate level and the C4 plants could have been replaced by a Markov Chains bonus (though I don’t know if that would have been better). I mean, even if folks don’t like those particular questions, each of the editor’s packets had extra bonuses that could’ve been switched around from packet to packet to ensure that the difficulty was equalized. Unfortunately, there was no time for this.

In addition, I think that one of the issues that has gotten overlooked in the previous discussion about variable bonus difficulty, though it was touched upon tangentially by Mike Bentley in his initial posts, is to what extent the questions at an event like this are affected by different understandings of acceptable difficulty among the many editors/reviewers of the set? For example, both Mike and Ryan seemed to think this set was mad hard and certainly when they, or some other folk, contributed questions to fill out the set in the final days leading up to the tournament, that may have played into their answer selection. This matters a lot, especially when a set that began as the vision of one, maybe two, editors, with their understanding of what appropriate difficulty should be, is altered at the last minute with no time to review. Maybe, I am overstating this somewhat as this was my first time really working with Google Docs and an army of reviewers, but I was occasionally taken aback at how different my understanding of a question’s difficulty was when compared to the opinion of other folks, including, but not limited to that of Jonathan M., Mike B., and Matt.

Continuing on the subject of difficulty, I would also like to assert that this set was nowhere near as hard as ACF Nationals 2005. Do you recall the bonus conversion from the 2005 playoffs amongst the top teams? It was crazy low. We aren’t even in the same ball park. Though I will admit that a week before the tournament I told both Dwight and Matt that I had some misgivings about the overall tenor of the questions I was editing/writing in terms of difficulty. This was not because I thought they were too hard individually, but as a collective set of questions they were works heavy (e.g., Concertos of Rachmaninoff, Church Dogmatics, Science and Health) and that can occasionally lead to problems with conversion. I think it is especially hard to hit that difficulty sweet spot with the minor categories, but that may be a discussion for another day.

Finally, although I generally do not pay much attention to criticism that seeks to make proclamations about the overall difficulty of a tournament by taking one question and going on and on, I was so amused by what some folks were saying that I have to rant a little bit… so here goes: Hold up, people are cool with Lyric Pieces, but Concierto de Aranjuez is way too hard. Are you serious? Is Joaquin Rodrigo not in the canon anymore? Wait, are you going to tell me how many times it shows up in one of the various databases—great! That really doesn’t mean shit to me. Those data bases aren’t comprehensive and moreover ACF Nationals is a tournament where new topics deserve to come up and should come up a few times per packet. I submit that Rodrigo’s most famous work makes that list, just as Albeniz’s Iberia once did, and just as Villa-Lobos’ Bachianas once did. Also please point me to the question that began “it may have been inspired by Michelangelo’s work in the Sistine Chapel” and is purportedly in one of the Finals Packets, because I don’t see it? As Matt noted in his earlier posts, I’m cool if the criticism of the questions is prefaced by the fact that they weren’t up your particular alley or you don’t prefer the topic, but let us not call the shit impossible just because you don’t happen to know it. On the other hand, I love constructive criticism, so if you have some other comments about the RMP/Fine Arts hit me up at eberdich at gmail dot com and we can discuss.

To sum up, I really enjoyed working on this set. I stand by my questions and am willing to discuss them with folks at any time if they want to email me, that is, save for my brain fart on who composed The Miraculous Mandarin; I apologize profusely for that miscue and am sad that it ruined such an amusing question to write on. On a less confrontational note, it was really fun to meet a whole new generation of players who appreciate learning and, as always, it was cool to see great players competing on challenging questions in games that really mattered.

Until next time, peace,
Ezequiel

[Mechanical Beasts]

Thanks for the post, Zeke.

Also please point me to the question that began “it may have been inspired by Michelangelo’s work in the Sistine Chapel” and is purportedly in one of the Finals Packets, because I don’t see it? As Matt noted in his earlier posts, I’m cool if the criticism of the questions is prefaced by the fact that they weren’t up your particular alley or you don’t prefer the topic, but let us not call the shit impossible just because you don’t happen to know it.

I think Hannah was referring to tossup 15 in Finals 2, which begins "Its subject may have been inspired by Michelangelo’s last paintings in the Vatican." I'll preface my comment by saying that I know diddly about fine arts. Does "Michelangelo's last paintings in the Vatican" refer to, like, a certain group of paintings? Otherwise I'd be hesitant to buzz--what if his third-to-last painting is intended? Or his seventh-to-last? And do they all refer to the same subject? The Alibris result I got after a Google search seems to indicate that the Crucifixion of St. Peter is also depicted in at least one author's impression of "last paintings." But then again, I don't have the knowledge; I'm just researching.

[salamanca]

Re: the Michelangelo clue... aha, we are talking about the The Conversion of St. Paul TU. Duly noted. Ok. I will definitely look into this. I guess that is what happens when you post at 2 in the morning...

Ezequiel

[at your pleasure]

That's definitely talking about the Conversion of Paul in the Paoline Chapel. Anyhow, the famous Caravaggio Coversion doesn't really look much like the Michelangelo-prehaps the source for the Michelangelo clue was referring to a less famous Caravaggio. For that matter, the sentence sounds like Paul being inspired by frescos painted some centuries after his death.

[No Rules Westbrook]

For the record, I'm not a huge fan of searching the databases to determine difficulty, and it's not how I do it. It's okay for less-experienced players and editors to get a very rough feel of what has or hasn't come up very recently, but it can also skew your feel for the canon.

I usually just rely on an intuitive impression of how many times I've seen an answer/clue/whatever come up before. I don't recall ever encountering Joaquin Rodrigo or Concierto de Aranjuez, which usually would make me automatically think it's really hard, because I have a high degree of confidence in my packet and canon memory. But, I'm guessing Zeke is drawing on an earlier memory bank than I have, so I'll buy his call (together with his call on, perhaps, Sir Godfrey Kneller - cause I know the Kit-Cat Club used to be all the rage back in the wet wild days). As I've said ad nauseum, I think there's a funny rare phenomenon in qb where some things tend to fall into a "lost canon" (that is, things which are important enough to come up and haven't been phased out purposely like science bio - but just have somehow fallen off the map such that noone writes on them anymore)...and it's cool to be able to explore those things.

As for the editorial hubris business - well, sure, noone disagrees that you should seek help as early as possible when it becomes clear that you're not going to be able to adequately finish what you promised to do (particularly at ACF Nats - since its a pretty important event). The trouble is, that happens way way too often these days, particularly with some people. Everyone (and those people in particular) needs to stop having patently fictional ideas of how long it takes to satsifactorily edit/write a tournament. Be realistic about the time frames from the get go. I'm not sure it's hubris a lot of times; I think it's more that some people truly believe you can satisfactorily edit 100 questions, write 4 packets, and randomize a tournament in 4 and a half hours.

[Ethnic history of the Vilnius region]

3. distribution of questions- first off, I would have preferred to print the whole damn tournament out, but I realize that folks have been running paperless tournaments for some time now, without major hitches, and it seems reasonable to me that if WUSTL explained to Matt that most of the rooms would be equipped with computers that had internet access, that he would have trusted that representation. It is true that Matt probably should have requested that everyone bring their flash drives before the first round and loaded the whole set on them as a back up, but that didn’t happen and so we did waste some time running up and down the stairs trying to get everyone the questions.

I'm glad you brought this up, because it did cause a decent delay and it's something to learn from at future tournaments. I've been to a number of paperless tournaments, and most have gone without incident. However, I can think of at least 3 paperless events I've been to in the last 2 years not including Nats that had delays due to the problems that seem to come up in paperless events. Lack of flash drives, people's crappy laptops, people going home early and taking their laptops with them, software compatibility issues, file corruptions, and lousy wireless access are the main culprits that come to mind. Even without major delays, I can't remember a paperless event in which something didn't go wrong.

Tournament directors, especially directors at events with a bunch of teams, can continue to rely solely on having a theoretically seamless chain of computers being usable and available, files being compatible and clean, and campus internet not sucking to have paperless tournaments go without a hitch. However, I submit that, until that seamless chain always becomes reality instead of theory, tournaments should at least prepare for having a hard copy-based tournament.

My half-baked proposal would be to have, for at least a larger tournament, at least 3 hard copies of the full set printed out (or, say, the first 12 rounds when you have a 21 round event), and the first 5 rounds or so printed out for every room at the tournament. Yeah, it would be kind of a waste to print out a bunch of rounds if those rounds would never be used, but such a contingency would have been well worth it at this year's Nats. With this plan, there would have been no delay at all in distributing the rounds to the rooms, and five rounds would have been ample time for some dude with a flash drive to visit every room to get the files to the laptops. Then you would have had a paper-free tournament for the next 13 rounds or whatever. Other contingencies would be covered by this plan as well. A complete disaster necessitating paper for every round would be easily met by having lunch after the 5th round and printing the rest of the packets at lunchtime. If 1 or 2 rooms had laptops that just couldn't handle things or caused unreasonable delays because of their shittiness, those rooms would have packets despite the technical problems. Etc.

I don't expect this proposal to ever be done at many, if any, events beyond the meager number of tournaments I direct. The incentive to rely solely on paperless technology is probably too great a lure for budget conscious TDs who don't want to have to go to Kinko's and waste paper. It's a powerful incentive indeed, and hopefully in the next few years the paperless delays I have seen firsthand over the years will be a thing of the past. However, as I said earlier, I will have my contingency plan in effect for all events I direct in the years to come, and I guarantee that distributing the questions to the readers will not be the cause for delays at those events.

[No Rules Westbrook]

I don't know, I usually just bring enough 5 1/4-inch floppy disks for everyone, and things go fine.

[Ethnic history of the Vilnius region]

I don't know, I usually just bring enough 5 1/4-inch floppy disks for everyone, and things go fine.

Problem: SOLVED

[Mike Bentley]

I agree that printing out at least the first round of a tournament, espcially for a national tournament, seems like a good idea when feasible. This will help minimize delays, as the TD can spend the 30-40 minutes in which Round 1 is being read (and all the time before the round starts) getting questions on to computers.

[grapesmoker]

I don't know, I usually just bring enough stone tablets and chisels for everyone, and things go fine.

[naturalistic phallacy]

I don't know, I usually just bring enough stone tablets and chisels for everyone, and things go fine.

Next time, I'll just clone Ryan and have him recite the questions in every room.