Regarding BBB and Community Norms

[Dantooine is Big!]

The primary context required for this post is this thread.

I think I had a strong desire to post something really aggressive on the forum thread for BBB for a while. I decided not to because I just looked at the situation externally and felt it would both seem and be pretty silly for some grad student to just yell at some extremely young kids over this. Given everything I saw in the thread, and I guess what resulted in retrospect too lmao, it seems unclear that such a post would have had any utility, as the kids running the tournament were, by all accounts, exceedingly unresponsive (both in terms of answering questions and providing information in the thread as well as in terms of any sort of response or adjustment based on any feedback at all), so like I guess they probably wouldn't have cared. More importantly, no one (i.e. not me or any other quizbowl community member) besides these kids is inherently responsible for this tournament's outcomes, and thus, no one could really take responsibility for guiding the project to a better outcome, or at least, no one can anymore now since indeed no one took responsibility to try to help the project, though someone theoretically could have (but that seems unlikely to have worked out anyway given the aforementioned unresponsiveness of those running the tournament).

At the same time, I wonder where we are, in the current quizbowl climate, with regards to community response to bad tournaments. Like if this tournament was (hypothetically, since online quizbowl has only been super commonplace for so long) run 10+ years ago, these kids would have gotten flamed, right? In the same way that lots of old hs quizbowl sets have gotten flamed online for being bad. Obviously, such an aggressive response has not been made yet, at least publicly — maybe it’s public if you’re under 18 / in the hsquizbowl discord? So maybe I should say, no older quizbowlers have posted any sort of super scathing response (besides Eric, who was pretty benign and honestly relatively tactful in his evaluation of what to take away from the outcome of the tournament). Actually, there was also Josh Xu’s post, who relayed issues with the tournament that I had basically already expected to hear about, and which seemed… to me, frankly, dishonestly positive in its evaluation of what happened. Do I believe Josh sensed there was a “massive amount of effort that went into this tournament”? Maybe. Josh also said that “the lesson from this tournament is not that enterprising quiz bowlers, especially middle schoolers, need to stay out of event organization.” I both think, regarding the latter, that’s not entirely true (in that I do think quizbowlers under the described demographic should see the outcome of this tournament and evaluate, in the future, whether they feel a particular project they’re considering working on may lead to similar outcomes), and, regarding the former, that there actually was a very low amount of effort that went into this tournament. Maybe I’m wrong, actually, and those kids did actually put a lot of time and effort into this. Regardless, I can very confidently say that this tournament was very low quality, and there’s nothing that can really sugarcoat that. I don't think flaming would have been justified, obviously, and the point is not to return to the state of the discourse 10+ years ago.

A large amount of people seem to have played this tournament (14 teams). Obviously, now that all the mirrors are cancelled, that is the permanent limit on how many people played the question set. It was stated that all registration fees were either going to go to staff or be donated to, er, charity (but which specific charity seems unclear, since different charities were announced at different times), so as long as all the profit after compensating staff did indeed go to some ostensibly good cause, part of me feels this somewhat nullifies any real reason to complain about players getting ripped off or not getting back the value that they paid for. My reasoning being, I guess, that they could have just sent that money to charity directly anyway without playing a tournament, and compensating staff seems only fair. But for the flipside of that reasoning, part of me still honestly feels like those players did, indeed, get ripped off, and that playing a tournament, even if all the money is going straight to charity, implies some reasonable expectations, and that it’s not the same thing as if you just donated some money to charity with no further expectations. I’m not honestly sure if either of those perspectives are entirely true. I would bet that if BBB charged more for registration and did not say they would be donating the entirety of the proceeds (minus staff $) to charity, someone would have said something very aggressive much earlier in the life cycle of this tournament.

I think I mostly agree with Josh in that there’s no real point in admonishing these kids at this point, so it’s mostly important to just be constructive about the outcome and help people in the community get their own takeaways. But my personal takeaway is honestly that BBB was a bad tournament, that we should expect much, much more of tournaments than how BBB played and ran, and that out of a few options, some sort of intervention was possible that would have either improved the tournament or just convinced these kids that they shouldn’t run the tournament if they were going to inevitably half-ass it like they did.

The real question here is what the community norm should be, specifically from older community members. Is there some sort of responsibility on the part of the greater community to intervene when a project seems like it’s obviously going to result in a bad tournament? There’s honestly no inherent reason why middle schoolers and high schoolers have to necessarily be like babied through the steps of running a housewrite, and successful such tournaments have existed — and even if you contest that those tournaments were “actually successful,” there’s certainly some hypothetical group of competent high schoolers who could, at any point in the future, write and run a good housewrite entirely by themselves. So maybe there’s not a “real need” for any of us to actually intervene. There’s also not a whole lot any of us can reasonably do to stop anyone who wants to run a(n, unbeknownst to them, probably doomed to be bad) tournament, provided interest and a large enough field. Similarly, no one can reasonably stop players from registering for an event, since if there is actual interest in the event that seems pointless.

I hope that had BBB charged a higher base registration fee, either someone would have dissuaded them from running the tournament at some point or no one would have registered.

[Halinaxus]

Regarding the former, that there actually was a very low amount of effort that went into this tournament.

But my personal takeaway is honestly that BBB was a bad tournament, that we should expect much, much more of tournaments than how BBB played and ran, and that out of a few options, some sort of intervention was possible that would have either improved the tournament or just convinced these kids that they shouldn’t run the tournament if they were going to inevitably half-ass it like they did.

Just because a tournament is bad does not mean that there was not effort put into it. Indeed, it's actually quite easy to put a ton of effort into a tournament that turns out terribly if you are eager to run a tournament but lack tournament-hosting experience! I have no knowledge of BBB other than what's been posted on the forums, but it seems like Josh does, and I have no problem believing his take, glass-half full as it may be.

When I was in high school, I (and my teammates and coach, but mostly me) ran a couple of Science Bowl tournaments. I can confidently state that neither one was particularly well-run; among a myriad of other issues (including question-acquisition practices so shady that anyone found to be using them for a quiz bowl tournament would rightly be tarred, feathered, and ridden off the forums on a rail), we used extremely suspicious formats that led to wacky results. This despite me being a massive bracket nerd who spent at least five hours researching formats and manually crafting schedules; my status as a clueless, inexperienced high schooler simply trumped those exertions.

That work was not in vain, however. I've since been able to apply some of what I learned from running those Science Bowl tournaments to properly run mainstream quiz bowl events (having neg 5 explode in my face when I attempted to use it for stats helped us avoid it at last year's Purdue Boilermaker Buzzathon, for example).

Returning to the issue at hand: While I do agree that it would be ideal for tournaments like BBB to have some "adult supervision" to keep things from falling apart (and heartily encourage any would-be new TDs reading this post to ask for help), I don't believe it's either feasible for this to happen or proper for older community members to assert some type of jurisdiction over independent quiz bowl tournaments. I would advocate for gently pointing out problems and offering suggestions, as Darryl did, and then warning potential players if those issues are not resolved.

P. S. Let's also remember that most bad quiz bowl tournaments (many far worse than BBB, which I gather for all its faults was at least pyramidal, of roughly appropriate difficulty, and guaranteed teams a fair number of games) are not run by students, but rather by adults, some of them with decades of experience, who are either ignorant of better practices or simply don't care. If the amount of ire that is currently being directed at the BBB organizers was directed at Minnesota Knowledge Bowl, Indiana's White River Academic League, the Arkansas Governor's Quiz Bowl Association, or one of the many other formats out there that blatantly shun the above ideals that BBB attempted to follow, I bet some real progress might be made.

[Votre Kickstarter Est Nul]

Without really wading further into this overall—I know nothing but what was in the forum post—I don’t think the existence of organizations which put on “bad Quizbowl” (non pyramidal, etc) invalidates criticizing orgs who put out a bad version of good Quizbowl (like a bad tournament). At minimum, the people running Arkansas Governor Bowl or whatnot probably aren’t reading this, and probably don’t care, but people putting on our version of Quizbowl are. “Running a tournament using our Quizbowl rules” is also a very low bar, and we ought to hold ourselves to a higher bar regardless of whether outside organizations that don’t even meet that bar exist.

[halle]

I would advocate for gently pointing out problems and offering suggestions, as Darryl did, and then warning potential players if those issues are not resolved.

I want to highlight this sentiment, because I think it is a productive direction for this conversation. I'd been keeping an eye on the BBB threads for awhile, and I was definitely inclined towards making the kinds of interventions you suggest here. Because I'm not experienced with middle school quizbowl, I had asked some friends who know more about it to keep an eye on the threads and make sure the right questions were being asked, and I also offered suggestions on how to make the announcement posts adhere better to the norms of the forums, including making some edits myself, like changing the thread title to have the new date (since that is part of my responsibility in moderating the forums). As the tournament approached, I was still weighing whether there was a way to warn potential players without coming across as disrespectful to the organizers. I only stopped thinking about this when the plagiarism accusations were raised, because I thought that settled things and that the set would not be played. I'm still not sure, though, what I would have said, and whether I was truly in a position to offer a warning without doing something to confirm my suspicions about the tournament (which were pretty much exactly aligned with what ended up happening, but were based on the evasiveness about sharing information on the set and tournament plans rather than any actual knowledge of the quality of those things). I'd love to hear thoughts on the right way to communicate these worries without condemning newer community members who are trying their best. My current thought is that something like "If you are reading these updates and are not sure how to interpret them, it would not be unwarranted to be very concerned" would avoid insinuating that the set is definitely bad, and would achieve the goal of letting people who don't know what red flags to look for know that there are, in fact, red flags, but it still sounds really shady to me, not to mention kind of stilted! What is a reasonable way of warning potential players, and what is enough evidence that there are unresolved issues?

[Stained Diviner]

Reilly's quote cited by Halle is good and relevant. Flames should be addressed to bad people, and snark should be addressed to friends and leaders who mess up. None of that applies to BBB.

It is appropriate for one or two people to ask questions, and that includes follow up questions and pointing out when questions have not been answered or have been answered poorly, and it is appropriate for one or two more to add another perspective to that conversation if they think doing so would be productive. A pile on doesn't really help things, but a couple of people asking questions and pointing out obvious truths is not a pile on.

It is common for threads about tournaments or sets to get missed by a lot of people who are very active on these forums because there are many such threads and reading them is usually a waste of time if you have no personal involvement with the tournament or set. In those threads, it is good to bring it to the attention of other people if you see something wrong, as Halle did. It would have been fine for Halle or somebody she contacted to post something--quizbowl standards are not radically different across levels once you account for the fact that high school and middle school teams have coaches, and those coaches often are not present at online tournaments anyways. It also would have been appropriate for a staffer to post something a few days before the tournament saying they have not been contacted or for somebody working on the set to give an honest update on the state of the set. Additionally, the issues with this tournament were difficult to follow because there were parallel threads in the high school and middle school fora. In hindsight, somebody should have said something in the high school thread pointing to the problems that came up in the middle school thread.

Quizbowl is decentralized, which means we should expect overall improvement over time along with cases of bad tournaments and bad questions. We should strive for better communication and offers of assistance. Middle schoolers can run a tournament if they want to, but we should make them aware that help is available and should make participants aware of whether that help is being used and whether it seems necessary.

[Skepticism and Animal Feed]

I wonder how many people in the last 20 years have dismissed and ignored good advice about quizbowl because the manner in which the advice was delivered was insensitive or offensive. Probably a very large number of people, some of whom left the game because of it.

Humans are emotional animals and good advice delivered poorly will actually lead to people feeling defensive and rejecting your advice. You may actually end up breeding advocates of bad quizbowl.

[meebles127]

I wonder how many people in the last 20 years have dismissed and ignored good advice about quizbowl because the manner in which the advice was delivered was insensitive or offensive. Probably a very large number of people, some of whom left the game because of it.

Humans are emotional animals and good advice delivered poorly will actually lead to people feeling defensive and rejecting your advice. You may actually end up breeding advocates of bad quizbowl.

This is something that took me time to learn and I'm so glad I did. Coming from a place of empathy and compassion goes so far.

[etotheipi]

P. S. Let's also remember that most bad quiz bowl tournaments (many far worse than BBB, which I gather for all its faults was at least pyramidal, of roughly appropriate difficulty, and guaranteed teams a fair number of games) are not run by students, but rather by adults, some of them with decades of experience, who are either ignorant of better practices or simply don't care. If the amount of ire that is currently being directed at the BBB organizers was directed at Minnesota Knowledge Bowl, Indiana's White River Academic League, the Arkansas Governor's Quiz Bowl Association, or one of the many other formats out there that blatantly shun the above ideals that BBB attempted to follow, I bet some real progress might be made.

Without really wading further into this overall—I know nothing but what was in the forum post—I don’t think the existence of organizations which put on “bad Quizbowl” (non pyramidal, etc) invalidates criticizing orgs who put out a bad version of good Quizbowl (like a bad tournament). At minimum, the people running Arkansas Governor Bowl or whatnot probably aren’t reading this, and probably don’t care, but people putting on our version of Quizbowl are. “Running a tournament using our Quizbowl rules” is also a very low bar, and we ought to hold ourselves to a higher bar regardless of whether outside organizations that don’t even meet that bar exist.

I'll note that there are organizations out there, consisting of adults with decades of experience, who consistently put on "bad versions of good quizbowl" and receive little to no attention to the community despite this. I can't speak to what GATA has done this year, but as recently as last year (2021-2022, that is) it was still running single elimination playoff tournaments with five guaranteed games (on good-QB questions, to be clear - IS sets and the like).

I didn't have anything to do with BBB, but I do think it important that (a) we don't forget that institutions like GATA still exist, and (b) we realize that, in the scheme of things, a couple of middle schoolers running a poorly-executed charity tournament is not nearly as destructive as the above (in my [one-year-old] experience, plenty of high schools in Georgia have become so used to the five guaranteed prelim games -> single-elim playoffs structure that they would chafe at playing a properly-structured tournament that guaranteed them nine or ten games).

The above is a bit off-topic but I still thought it an important point to be raised in this context.

[Cheynem]

I think part of the difference is that hopefully, the people who ran BBB would be receptive to (polite, instructive) feedback and criticism (and it seems like in some ways they have been!) and the people who run stuff like GATA or whatever have demonstrated over the years to be completely un-receptive to any sort of feedback or criticism. So most people on these forums or in Discord are aware that nothing they can say can at the moment make those organizations change their ways, as opposed to individuals like the BBB organizers. That doesn't mean they should be put on 24/7 full blast, but that why they would seem to be receiving more attention perhaps than far worse quizbowl offenders.

[Votre Kickstarter Est Nul]

Insofar as it matters if I respond, given that mike said what I was intending to convey better than I had: Basically, I don’t think the existence of bad orgs that ignore us invalidates criticizing attempts to run quizbowl by people who would listen and/or are broadly speaking part of our community (I don’t really consider orgs that run bad qb deliberately and in spite of better quiz bowl part of the qb community). Can’t stress how much I don’t know the extent or tone of the criticism of BBB and thus am not really commenting on that specifically.

[halle]

Recently, on the Discord, I've been finding myself reminding or informing people that the quizbowl community that exists on the forums and Discord is centered around, at at times fiercely partisan about, good quizbowl as opposed to other quiz activities, especially ones that call themselves quizbowl. I'm hugely in favor of newer members of the community and people who are branching out from playing into some sort of set or tournament organization role for the first time taking the time to familiarize themselves with the differences between good and bad quizbowl that have been hashed out after years of discussion, some of it highly contentious, to avoid rehashing those same discussions in the future. I don't think quizbowl is necessarily perfect in its current form, but if you're going to try something different from the general precepts of good quizbowl, you should probably know if it's been tried before, and if that was successful, or at least be able to articulate why the ways you're diverging from the standards of good quizbowl will not devolve into bad quizbowl. I'm pretty firm in this belief.

I don't think this point about knowing the difference between good and bad quizbowl is all that relevant to BBB and tournaments like it, but it is relevant to the discussion that has arisen, so I wanted to make my stance on it clear, because I'm honestly on the hardline side of wanting to eliminate widespread bad quizbowl. I do think, though, that we're at a point where almost everyone on the forums knows, generally, what good quizbowl is, and that, since there are countless ways that good quizbowl can be done badly, it is worth pointing out when that's happened, or working to prevent it from happening. I haven't been putting quotes around "good" and "bad" quizbowl in this post because I think that these terms have firm enough meanings within this community that I'm resistant to giving the impression that I need to hedge when I use them, but I do want to be clear that I'm not using these terms subjectively--in this post at least, I mean by "good quizbowl" questions that are pyramidal, don't have hoses, are accurate, and so on--essentially the qbwiki definition. The fight against bad quizbowl is certainly not completely over. But it doesn't serve the community very well to have the lingering wartime mindset of letting anything slide besides that main enemy.

After a few days of reflection, I don't think it crosses the line to say that a tournament entirely written and planned by middle schoolers, with absolutely zero oversight, is not a good idea. It isn't bad quizbowl! But it's very rarely successful, at least by the standards that the community tends to hold itself to. There haven't been many open articulations of the norms surrounding whether middle schoolers can handle an entire set themselves because it hasn't been tried many times in recent years, at least as far as I know, but it doesn't seem too harsh to take this opportunity to say that it is, or should be, considered against best practices to have a writing and editing team composed entirely of middle schoolers. It is not the same kind of firm, unbreakable community norm--the kind of norm that approaches a hard and fast rule--as questions needing to be pyramidal or anything else that defines good quizbowl. No one is doing anything *wrong* by trying to write and host a set without adult (or even high schooler) supervision, but it is not a recipe for success. It's genuinely nothing personal about the middle schoolers who worked on BBB, which is I think what was making me uncomfortable when I was trying to think of ways to suggest BBB wasn't looking promising without suggesting the individual organizers and writers were particularly incompetent. As a community, though, there is a level of collective knowledge that something like that wasn't a good idea, even if not every single member of the community realizes it. There are some other norms that BBB skirted as well, like announcing the writers and editors and (from what I could see last week) creating an entry on the tournament database (the latter is something I've forgotten to do in the past as well, in fairness).

I'm not sure what, if anything, to do about the fact that there is a body of norms and best practices that are not clearly stated anywhere for those who might not know them. On the one hand, this is just how community norms work. They are very rarely codified, because then they risk becoming rules, and lead to debates over exact wording and carving out room for exceptions. On the other hand, it sucks to find out you've repeated a mistake that has already been made many times just because you didn't know about those examples, or to disappoint people because there was an expectation that hadn't been made clear ahead of time. For now, I think my idea of the solution is for more experienced community members to keep an eye on things when they can, and to not be afraid to point out the best practices that aren't being followed as they arise. Asking why there aren't writers and subject editors listed in an announcement post is important, but when the response is that the set's writing team prefers not to be named, responding by pointing out how unusual that is, and asking if there's a reason for that, might be better than just reading the response and thinking "well, that's weird and not good" and hoping everyone else who reads it picks up on that too. I had the opportunity to do this with BBB, and there are probably many other sets and tournaments that I might've had the opportunity to do something similar with if I'd been reading their threads more closely, but I didn't take it upon myself to say anything because I didn't feel it was my place. I guess this post, which has turned out to be much, much longer than I anticipated, is in part an opportunity to encourage myself and others to feel empowered to share their own thoughts about problems they foresee in threads on the forums, by offering the somewhat more experienced members of the community a reminder that it isn't always just personal opinion and judgment that lead to those conclusions, but experience, and experience is a valuable thing to pass down.

[cchiego]

Halle's entire post is superb. I'd like to especially endorse this point:

...I don't think it crosses the line to say that a tournament entirely written and planned by middle schoolers, with absolutely zero oversight, is not a good idea. It isn't bad quizbowl! But it's very rarely successful, at least by the standards that the community tends to hold itself to.

The whole quizbowl community benefits from well-run tournaments and suffers from poorly-run tournaments. Poorly-run tournaments make quizbowl look amateurish and disorganized, particularly next to other extracurricular opportunities. Yes, more-experienced teams and players will often roll with logistical punches and even badly-written questions (though arbitrary/unfair/incorrect[!] protest resolution is particularly bad), but not so much the less-experienced ones. If you want to run a low-key fun event, scrimmage, or league with your friends that's a bit less-than-ideally organized, go for it. But official announcements (on multiple subfora, no less) and especially money changing hands should imply some level of standards.

I would also note that, having spoken with a number of non-pyramidal quizbowl coaches over the years, many of them seem to have had some negative experience with logistics at a previous pyramidal event that soured them on pyramidal quizbowl. I certainly don't think that's a particularly compelling reason to not play using pyramidal questions or avoid best practices in tournament formatting and scheduling. But how well an event runs definitely matters for not just that event but the perception of quizbowl as a whole. If quizbowl wants to be a wide-scale activity instead of a niche one and if pyramidal quizbowl wants to continue to expand to places where it's rare or nonexistant, logistics and responsibility cannot be an afterthought or a "nice-to-have"; they're essential.

In this case, I think it would have been fine for more people to note explicitly that it seemed like this tournament was likely running into trouble. I also thought that the tournament had been cancelled based on the posts in the forum, so I was very surprised to see that it actually took place (with the expected issues).

I'm not sure what, if anything, to do about the fact that there is a body of norms and best practices that are not clearly stated anywhere for those who might not know them.

Perhaps having model templates for set and tournament announcements that are not per se "required" but could be stickied and directly linked in a thread and/or sent (in a friendly manner) to those posting any vague announcements as a model for the information to provide and with links to the best practices for writing sets and running events. Such a template could include things like "who will the editors be [it may help to describe recent sets that they have worked on before or who is being consulted for advice]" and "current progress on the set [what percentage of written + edited questions produced so far, with a specific date noted]" that might also help make the expectations/norms more clear. A similar one for tournament announcements would be possible as well.

Maybe the forum moderators could also look to see if there's any new accounts posting these kinds of events (are new accounts still required to get their posts reviewed before being approved?) and ask them to fill those out before approving the posts.