Ophir was kind enough to share some the conversion rates for these bonuses across sites. Even accounting for the small sample sizes, I think there are some useful conclusions to draw from this data. For simplicity, I am going to aggregate the data across all sites provided that there was not a "major" change in question content.Smuttynose Island wrote: ↑Sun Mar 29, 2020 6:29 pm Some miscellaneous thoughts on various VisArts bonuses (based on the 2/29 packets that were posted):
Overall, I thought that the visual arts questions were well executed. I also don't mind an arts distribution that skews modern. Although this tournament seemed to go just a bit too far in that direction.
- Steyerl/Goldin/Washington: I liked the concept behind this bonus, but both Steyerl and Goldin seem like hard parts. Plus while "Misty and Jimmy Paulette in a taxi, NYC" has come up more in quizbowl, but The Ballad of Sexual Dependency is probably Nan Goldin's most notable work.
- Jafa/Beyonce/Venice: I thought that both Beyonce and Venice were easy parts. Golden Lion+Film Festival strikes me as too generous at this level.
- Cuba/Frankenthaler/Pollock: This hard part on "Cuba" struck me as very hard. We got a description of some Carmen Herrera works (who has unfortunately never come up before?) and then not the most famous work by Ana Mendieta. Mentioning her Silueta series or even one of the more famous photos from that series would have been nice.
Conversion Rates (Gets/Rooms by part):
Steyerl/Goldin/Washington: (5/22) / (3/12) / (21/22)
Jafa/Beyonce/Venice: (1/23) / (23/23) / (18/23)
Cuba/Frankenthaler/Pollock: (4/23) / (13/23) / (19/23) [I was told that the "Cuba" part was made harder after the first mirrors. Specifically Spanish sounding names were removed]
With the usual caveat of small sample sizes, I think there are some decent conclusions that we can draw from this conversion data. Both the Steyerl and Goldin bonus parts sat in between being "easyish hard parts" and "hard middle parts". Each of these results (or getting worse conversion) are pretty unlikely (<10%) if the Steyerl or Goldin bonus had a "true conversion" percentage of 50%.*
The Cuba bonus part played more like a hard part. Even if two teams from the first mirror sites frauded, the part has a 8.6% conversion rate. I'd have to see which teams were answering the bonus to get a better sense of if this is due to chance (good teams got the bonus and either knew it or figured it out) or if the bonus was a good hard part.
Stepping back a bit, I think this is illustrative of not only how set editors can use advance stats to improve future mirrors, but also how we as a community can use advance stats to write better questions. For instance, the Joker clue and/or the Golden Lion clue in the Venice bonus look to be fairly well-known by players at this level. Well-known enough that they should be used in an easy part on Venice or as late clues in a film TU on Venice. While not perfect, a fair bit of conversion numbers comes from context, these advance stats can hopefully serve as a much better guide for future clue placement than hits on the archives or gut feeling, especially for newer writers.
*Yes, I'm using a significance test. No, I don't have time to do something better.