Page 1 of 1

Fuck it, let's rank the sets

Posted: Wed Aug 01, 2018 4:29 pm
by vinteuil
As Jonathan Suh recently pointed out, we haven't done this in forever, and it's actually a highly valuable exercise. (How else will posterity know what to read first?)

I'm not going to produce a numerical ranking here, and I'm not going to plug any sets I've worked on (although I think 2014 Fall and 2016 MYSTERIUM were truly great sets, especially the latter; I'm very proud of the 2017 and 2018 SCTs, especially the former; and I think 2014 CO was excellent despite my contributions; 2018 is a little too fresh to rank). But in ascending order of difficulty/chronology, I think the best all-subject sets I've heard or played since the last thread were:
  • 2013 DRAGOON
  • 2015 Regionals
  • 2016 MLK
  • 2016 Terrapin
  • 2017 Regionals
  • 2017 ICT
  • 2018 CMST
  • 2017 Nationals
  • 2018 Nationals
  • 2015 CO
I mainly picked sets that I thought nailed their difficulty target (sorry 2018 Regionals or 2016 CO! I like you a lot more than several of these sets!) in addition to having rock-solid questions and innovative or exciting content. I apologize for not having more easy sets; for all I know 2016 MUT and SMT really might belong up here (I do think that none of the ACF Falls since 2014 have measured up to these sets, for a variety of reasons).

I could also be convinced that this kind of ranking/discussion is best done in difficulty tiers. In that case, we could also open the discussion up for subject tournaments; obviously I'm a big fan of VICO, but I think that It's Lit, Jordaens, 2015 CO Visual Arts, and SHEIKH really set a high standard for "standard quizbowl fare" side events, and the Mike Bentley and Will Nediger events have been just awesome for their well-built questions on not-at-all-standard quizbowl fare.

Finally: how do the older sets match up? In practice, I've been stunned at how well both Regionals and Fall from 2008 have stood up. I think MAGNI, and 2011/12 Regionals all would play pretty well now. (At least for me, 2010 Regionals may have been a victim of its own success—too many of the tossup clues seem to have become hyper-canonical, so it's hard for me to judge. The same might be true of the 2011 VCU Open.) 2010 MO was probably the best tournament of its kind until CMST, and I could be easily be convinced that it tops the latter. And 2012 Nationals could hold its own against a tournament as good as 2015, I think. (2005 Nationals remains humanities-impossible.)

Some honorable mentions:
  • 2014 Nationals. Aside from some wonkiness in the lit and visual arts, and some problems in the music, the rest of this tournament really was stellar.
  • 2015 VCU Open. Mostly fantastic, but the science could be all over the place.
EDIT: I'm hoping to generate discussion with this thread, not to impose my opinion. I've almost definitely forgotten something or had some blind spots! So please point those out.

Re: Fuck it, let's rank the sets

Posted: Wed Aug 01, 2018 4:34 pm
by Cheynem
I really enjoyed Kenji's GEODUCK set for one-subject events.

I feel like This Tournament is a Crime might deserve some respect. I liked that better than CMST I think.

Re: Fuck it, let's rank the sets

Posted: Wed Aug 01, 2018 4:39 pm
by naan/steak-holding toll
TTIAC is the best open tournament I've had the pleasure of playing. Bruce did a fantastic job for his first time trying at that level, and Ike, Auroni, and Eddie all wrote delightful questions. Auroni in particular deserves credit for learning from his (well-done, but far too hard) previous execution of George Oppen and leading the production of an innovative tournament that hit an al dente balance of canonical and underasked content. I can't comment on Wang's stuff bcz I can't science, but I'm sure it was great as well.

"stanford housewrite" also deserves credit as a tournament that was a bit rough around the edges, but nonetheless had a ton of great ideas. It did a good job proving that you can write an open set that's not oppressive in difficulty, and is probably my second-favorite open tournament. I think having more sets like "stanford housewrite" would be great.

I'd second the selection of DRAGOON and Terrapin as ideals for the upper end of the new paradigm of "medium difficulty" that has been promoted recently.

Re: Fuck it, let's rank the sets

Posted: Wed Aug 01, 2018 4:40 pm
by Ewan MacAulay
I'll jump on the Terrapin 2016 train - a really solid regular-difficulty set which still managed to feel fresh. Penn Bowl 2015 was another fun set, although alas the last PB set that have remembered that time difference means that UK sites need packets earlier than American ones.

On either side of regular difficulty, EFT 2016 and Stanford Housewrite stuck in my mind as being very good fun to play.*

As for side events, I say it all the time, but GEODUCK and Stock Clues were some of the best questions I've ever played and showed how rich the geo/food distributions can be. Would be very excited to see sequels for them.

*my opinions on all-subject sets should be read while remembering that I only understand like 3-4 questions per packet

Re: Fuck it, let's rank the sets

Posted: Wed Aug 01, 2018 4:44 pm
by Ye Unfeeling Romeo
vinteuil wrote:I apologize for not having more easy sets; for all I know SMT really might belong up here
:mad: :mad: :mad:

On the other hand, if you're talking about "hitting their difficulty target," it might be less fair to talk about SMT or MUT, since I think they both tended harder than what their ideal audience would have liked. SMT definitely improved a lot as the mirrors progressed, but I can't really be bothered to go through a retrospective analysis of the set. I don't think anyone cares too much, anyway.

Re: Fuck it, let's rank the sets

Posted: Wed Aug 01, 2018 4:51 pm
by Red Panda Cub
Stock Clues remains the best quizbowl tournament.

Re: Fuck it, let's rank the sets

Posted: Wed Aug 01, 2018 5:12 pm
by ryanrosenberg
Red Panda Cub wrote:Stock Clues remains the best quizbowl tournament.
Yes, this.

CO Trash 2016 is the best full tournament I've played, and ACF Nationals 2018 the best I've read.

Re: Fuck it, let's rank the sets

Posted: Wed Aug 01, 2018 5:46 pm
by touchpack
I know this is tooting my own horn a bit, but for easier tournaments, I was really proud of the work my co-editors and I did on MUT 2014, and I remember that set being very well-received as well.

Re: Fuck it, let's rank the sets

Posted: Wed Aug 01, 2018 7:13 pm
by The Bold Ideas of Bernie Sanders (I-VT)
touchpack wrote:I know this is tooting my own horn a bit, but for easier tournaments, I was really proud of the work my co-editors and I did on MUT 2014, and I remember that set being very well-received as well.
It was a very fun and good set.

Re: Fuck it, let's rank the sets

Posted: Wed Aug 01, 2018 7:16 pm
by Banana Stand
TTIAC and Stanford Housewrite were both better than CMST.

Re: Fuck it, let's rank the sets

Posted: Wed Aug 01, 2018 8:23 pm
by vinteuil
Banana Stand wrote:TTIAC and Stanford Housewrite were both better than CMST.
The science in Stanford Housewrite had a lot of problems; TTIAC and CMST were both more consistent in that regard.

Re: Fuck it, let's rank the sets

Posted: Wed Aug 01, 2018 8:34 pm
by Doga (Dog Yoga)
1) 2018 CO
2) 2016 ACF Fall

Re: Fuck it, let's rank the sets

Posted: Wed Aug 01, 2018 9:11 pm
by a named reaction
IMO 2017 ACF Fall was quite a bit better than 2016 (or at least didn’t have any bonuses on impossibly hard Tibetan rivers). I’d tentatively say without having thought about it much that 2017 Fall was one of the best recent easy sets.

Re: Fuck it, let's rank the sets

Posted: Thu Aug 02, 2018 2:34 am
by Corry
Relatively few college sets have stood out in my memory, but Terrapin 2016 was definitely one of them - the questions just felt "fresh" in a way that I can't quite articulate. Still have no idea how Jordan and co. pulled it off.

Regarding side events, I too have recently enjoyed GEODUCK and Stock Clues.

Re: Fuck it, let's rank the sets

Posted: Thu Aug 02, 2018 11:51 am
by ThisIsMyUsername
I agree with a lot of the tournaments that have been praised so far. To this I might add a couple of things:
- I think Jordaens is one of the best academic side events ever written, from a technical standpoint. It is not merely the model of how to write a good academic side event, but also of how to include a healthy amount of canon expansion while still keeping questions very playable, using smart common-links and good sub-distribution. Even if we isolate individual best-edited categories in most all-subject tournaments of the past few years, very few of them would surpass Jordaens.
- I think the past few ICTs (2016-2018) are the best ICTs I've played. 2017 was maybe the strongest, but the general trend is what I find more noteworthy. At the time of the past list, I think the general impression was that NAQT still often operated below the collegiate circuit norms. I don't think that is the case any longer.
- I think the best Nats since 2012 is actually ACF Nats 2015. 2014, 2017, and 2018 attract a lot of attention because of the assertive philosophical underpinnings of their editors' questions. But I thought the general execution of 2015 across the tournament displayed the best control (finals packet excepted), while still having interesting answerlines.
- Perhaps we're deliberately limiting ourselves to academic or academic-proximate sets, but CO Trash 2016 needs to be up here among the top. (EDIT: Ah, I see Ryan mentioned it.)

Stepping back, though: I feel like we've undergone a big shift since the time of the previous list. All four Regionals from 2010 to 2013 were well received, and all of the most praised hard tournaments from the previous era (MO 2010, CO 2011, and ACF Nats 2012) were packet submission. MAGNI is one of the few housewritten tournaments that is mentioned alongside the other great tournaments of that era.

Now, the situation has reversed. I think the best regular-difficulty tournament (Terrapin 2016) and the best hard tournaments (stanford housewrite, TTIAC) have been housewrites. (It seems like many others agree with this general assessment.) And I think that these are on the whole better than the flagship core events (ACF Regionals, ACF Nats, and CO) have been over the past few years. This should make us wonder whether we've gotten to the point where the community at large is markedly better at writing than it is at editing, and whether we should self-examine why that is and how to alter that.

A fun exercise to do, perhaps, in addition to listing events that are great as a whole: Can one imagine great "Frankensets" that could be made out of recents sets (i.e. the Lit from Tournament X, the Science from Tournament Y, etc.)? That is, are there also standout individual categories within sets that otherwise were not as strong across the board?

Re: Fuck it, let's rank the sets

Posted: Thu Aug 02, 2018 12:17 pm
by vinteuil
ThisIsMyUsername wrote:This should make us wonder whether we've gotten to the point where the community at large is markedly better at writing than it is at editing, and whether we should self-examine why that is and how to alter that.
I'd love to see you expand on this. I'd think that the difference comes from housewrites' opportunity to avoid the compounded problems of "unknown topics" and "have to write/edit quickly" that arise from the inevitable late packets.

Re: Fuck it, let's rank the sets

Posted: Thu Aug 02, 2018 12:22 pm
by Cheynem
Some of this is also to blame on the submissions too, no?

Re: Fuck it, let's rank the sets

Posted: Thu Aug 02, 2018 12:37 pm
by rylltraka
As someone who doesn't really have a lot to add to this discussion, but was interested, I found it rather difficult to find the previous thread on this topic (addressing 2010-2012 or so; I did eventually find it). Might be just me.

What might be useful to the average player or enthusiast would perhaps be an easily accessible "trophy room" section of HSQB or perhaps through the forums, where anyone could take a look at the "best sets of all time", perhaps with a blurb describing difficulty, or at least some expert feedback. While it's fun to rank them, I wouldn't advise doing it there, but to simply have a collection of 20 or so sets where the community has said "this are examples of what excellent quizbowl sets look like".

Because, in this day and age, we have an embarrassment of riches when it comes to choice. Even subject tournaments are in such abundance that it's hard to keep track; when I was a wee lad, history-only tournaments were a thing only of the imagination - now there are tons. So perhaps subsections of "best subject-only sets" or "best specialty tournaments", with explanation, would be useful for the wider community.

Re: Fuck it, let's rank the sets

Posted: Thu Aug 02, 2018 12:38 pm
by vinteuil
rylltraka wrote:As someone who doesn't really have a lot to add to this discussion, but was interested, I found it rather difficult to find the previous thread on this topic (addressing 2010-2012 or so; I did eventually find it). Might be just me.
I literally linked to it in the first clause of my post.

Re: Fuck it, let's rank the sets

Posted: Thu Aug 02, 2018 12:56 pm
by rylltraka
Yes, you did. I didn't realize it and tried to use the site's search function.

Re: Fuck it, let's rank the sets

Posted: Thu Aug 02, 2018 12:56 pm
by UlyssesInvictus
rylltraka wrote:What might be useful to the average player or enthusiast would perhaps be an easily accessible "trophy room" section of HSQB or perhaps through the forums, where anyone could take a look at the "best sets of all time", perhaps with a blurb describing difficulty, or at least some expert feedback. While it's fun to rank them, I wouldn't advise doing it there, but to simply have a collection of 20 or so sets where the community has said "this are examples of what excellent quizbowl sets look like".
QuizDB does have a "quality" attribute on the sets in its database, and I've long contemplated outsourcing this to a Google Poll where people can simply rank sets and I'll take that data and apply it to the database. (Thus making this, and prior threads, in effect a nominating thread.) It's a 5 point scale right now, but I'll probably change that to a 7 point scale and reset all the current values I have (since they're just subjectively being set by the uploaders right now i.e. mainly me).

If people have recommendations on how to conduct such a poll (the most glaring problem is that ranking 50+ sets over the past 5 years would just turn into a name recognition slogfest, as often happens in similar polls), or don't think such a venture would be helpful, I'd love feedback.

Re: Fuck it, let's rank the sets

Posted: Thu Aug 02, 2018 12:57 pm
by UlyssesInvictus
Oh, and I'd probably run separate HS and College/Open polls.

Re: Fuck it, let's rank the sets

Posted: Thu Aug 02, 2018 1:03 pm
by theMoMA
Of tournaments not yet mentioned, Cane Ridge Revival was an excellent set with innovative and interesting humanities and social science questions (with the caveat that I don't remember the science and wouldn't be able to comment on it even if I could). That more than any other set influenced how I look at clues, especially drawn from contemporary scholarship, in history and social science questions.

Re: Fuck it, let's rank the sets

Posted: Thu Aug 02, 2018 1:05 pm
by ErikC
I'd like to throw my lot in with 2016 EFT for the easier side of collegiate quizbowl. I was a bit outside the target demographic when I played it but there was still a good amount of challenging clues in power for the regionals/regionals+ crowd. It also had some really fresh things I was excited to hear clued in Quizbowl.

Re: Fuck it, let's rank the sets

Posted: Thu Aug 02, 2018 1:06 pm
by hydrocephalitic listlessness
theMoMA wrote:Of tournaments not yet mentioned, Cane Ridge Revival was an excellent set with innovative and interesting humanities and social science questions (with the caveat that I don't remember the science and wouldn't be able to comment on it even if I could). That more than any other set influenced how I look at clues, especially drawn from contemporary scholarship, in history and social science questions.
Seconded!

Re: Fuck it, let's rank the sets

Posted: Thu Aug 02, 2018 1:17 pm
by gyre and gimble
ThisIsMyUsername wrote:Now, the situation has reversed. I think the best regular-difficulty tournament (Terrapin 2016) and the best hard tournaments (stanford housewrite, TTIAC) have been housewrites. (It seems like many others agree with this general assessment.) And I think that these are on the whole better than the flagship core events (ACF Regionals, ACF Nats, and CO) have been over the past few years. This should make us wonder whether we've gotten to the point where the community at large is markedly better at writing than it is at editing, and whether we should self-examine why that is and how to alter that.
I'm not so sure about this. Just because the best housewrites have been better than the best ACF events doesn't mean that on the whole the community is better at writing than editing. (For example, how many regular-difficulty events besides Terrapin 2016 have been better than, say 2015 or 2017 ACF Regionals? I can't really think of any clearly better ones.)

The question you raise is pretty interesting, though. Are writing and editing skills very distinct? I ask this as an editor who tends to just write all my questions from scratch rather than editing submissions. The reason I do so is often not because the submissions are poor, but because I am adhering to a carefully-curated subdistribution that makes most submitted question topics unworkable. By choosing my own topics I can also manage how much novel/"fresh" content I can put into a set. I wonder whether the perceived difference in quality between the best housewrites and the best packet-sub tournaments can be attributed to the lack of control an editor has over distributional feng shui, relative to a subject writer? Has enough of the community become competent at writing, such that it's this difference--one of distribution, novelty, freshness--rather than a difference in average question quality, that stands out to us now?

Re: Fuck it, let's rank the sets

Posted: Thu Aug 02, 2018 11:06 pm
by ErikC
gyre and gimble wrote:The question you raise is pretty interesting, though. Are writing and editing skills very distinct? I ask this as an editor who tends to just write all my questions from scratch rather than editing submissions. The reason I do so is often not because the submissions are poor, but because I am adhering to a carefully-curated subdistribution that makes most submitted question topics unworkable. By choosing my own topics I can also manage how much novel/"fresh" content I can put into a set. I wonder whether the perceived difference in quality between the best housewrites and the best packet-sub tournaments can be attributed to the lack of control an editor has over distributional feng shui, relative to a subject writer? Has enough of the community become competent at writing, such that it's this difference--one of distribution, novelty, freshness--rather than a difference in average question quality, that stands out to us now?
I've noticed that at submission tournaments, especially the ones that use most submitted questions, that my individual performance spikes. I think that's because the approach taken by people to the same subjects or even answerlines can be very different. Sometimes the packet is written by people who like the exact same things you like about certain subjects, sometimes they don't - I'm sure a least a few people have noticed this.

At housewrites, I generally find myself having a much more uniform experience because it's only written by a few writers. Good writers will usually write questions using a variety of clues and answerlines (like tossing up a balanced mix of places, people, and things in history, for example) over the course of the tournament - which I think most non-sub tournament that I've played have done. I think this helps their reception, besides just the distributional feng shui.

Re: Fuck it, let's rank the sets

Posted: Fri Aug 03, 2018 9:04 am
by vinteuil
ThisIsMyUsername wrote:
- I think the best Nats since 2012 is actually ACF Nats 2015. 2014, 2017, and 2018 attract a lot of attention because of the assertive philosophical underpinnings of their editors' questions. But I thought the general execution of 2015 across the tournament displayed the best control (finals packet excepted), while still having interesting answerlines.
I can't say that I'm initially inclined to agree with this characterization of either 2015 or 2017: I don't remember the former as having great "control" (the range of easy parts, some of the whackier tossup answerlines. I was under the impression that, by consensus, 2016 was widely agreed to be better "controlled" in many ways---not that that consensus is necessarily correct), while the latter didn't strike me at all as having a major divide between the submitted and editors' questions. I'd be happy to be convinced of the opposite about either though.

On the other hand, I completely agree with you that the trend toward really phenomenal ICTs has been a really heartening phenomenon; I look forward to this year's, and I can't say I did even in 2016.

Re: Fuck it, let's rank the sets

Posted: Fri Aug 03, 2018 9:56 am
by Cheynem
The decline in the numbers of packet submission tournaments probably plays some role in lowering people's experience in editing submissions. In many years, the ACF tournaments, Chicago Open, and maybe a random tournament here and there are the only packet submission sets, so people turning in submissions and the people editing them don't have a lot of experience at doing both.

Now, I am not of the Matt Weiner school which asserts that almost all tournaments should be packet submission, and I don't think *inherently* this is a problem, but certainly submitting packets and editing them are becoming less frequent, and I think that plays a role in what happens in packet submission tournaments (for the record, I think most of the last few COs have been fine and I don't think the problems of them can be blamed on submissions).