A Proposal to Deal with the issue of cheating

Elaborate on the merits of specific tournaments or have general theoretical discussion here.
Post Reply
User avatar
Ike
Auron
Posts: 1050
Joined: Sat Jul 26, 2008 5:01 pm
Contact:

A Proposal to Deal with the issue of cheating

Post by Ike »

Speaking for not any organization:

This is a topic that has been plaguing quizbowl in the era of COVID, and what follows is basically my thoughts on how to handle the issue of cheating. The problem as I see it, is that up to now, there is a need to quickly address serious accusations of cheating in a way so that the tournament's logistics do not get severely impacted, as well as the need to treat the accusations with the gravity that is warranted. I think these are two separate concerns, and what I propose is basically a procedure to separate the concerns.

In my opinion, the best way to resolve the first concern of "how TDs can address accusations in a timely manner," I would propose that TDs of online tournaments reserve the option to "refuse service for any reason." In this case, the reason would be suspicion of cheating. Basically, TDs have the right to say "I'm sorry, you can't play the remainder of the games," much in the same way that businesses have the right to refuse service to anyone who isn't a protected class. his "right of refusal" actually isn't that uncommon of a practice in quizbowl; a while back when a tournament submitted results to NAQT, NAQT chose not to accept the results of the tournament. That is NAQT's prerogative, and I suggest that TDs be given a similar right. At this point, the TD should look into whether or not it makes sense to keep the player out for the remainder of matches.

Granted TDs can be unreasonable or poorly trained, but hopefully with proper guidance, I think its an acceptable power to give them to solve cheating. I realize that the above proposal is fraught with complications, a TD may be overzealous or somewhat more unsavory and ruin things. Hopefully in an age where TDs are supposed to be taught to deal with misconduct, we will eventually reach a place where this kind of judgment is few and far between. think this can be ameliorated by all stats reports / tournament writeups including why in the TD's judgment they chose to remove a player from competition, etc.

The other issue is addressing the actual accusation of cheating itself. I think it's pretty rare that we will get a straight up smoking gun in many cases. It would be nice if everyone just owned up to their mistake, but historically it has been very, very, rare that the parties involved do. Therefore, I think the best we're going to get in most cases is circumstantial evidence. So any policy that has a chance of being effective in my opinion needs to acknowledge this. Again, I think NAQT's actions in that thread show the way forward: players who put up a statistically anomalous performance should be given a chance to verify their results. If they choose not to cooperate, that alone could be grounds for some kind of ban. If they do replicate the results, the organization investigating the claims could say "we did not find grounds for suspicion" and thus will not be instituting a ban. If the player is unable to replicate their performance to the satisfaction of the investigating organization the organization could say "the results submitted are inconsistent, at this time, we will be instituting a ban." NOTE: Organizations don't need to say whether or not they think a party cheated.

Note, how this second concern is something that I've kind of shifted over to other quizbowl organizations. Given that when we talk bans, they have to be implemented by some kind of organization, since even if the TD were to investigate, in order for there to be any lasting consequences, I feel that they have to be passed on to some kind of central organization; the end result is that invitational tournaments also can choose whether or not to follow an organization's ban.

Anyway those are my thoughts, I'm sure others have more enlightened ideas, feel free to discuss, especially if there's some part I've overlooked, or really didn't leave fleshed out.
Ike
UIUC 13
User avatar
Cheynem
Sin
Posts: 6837
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: A Proposal to Deal with the issue of cheating

Post by Cheynem »

Thanks for starting this thread, Ike. I had been meaning to but kept forgetting.

Caveat: I wasn't at the LIT mirror. I don't know any details and this thread isn't meant to litigate the cheating accusations there.

There are three things that I think should be codified, some of which Ike is alluding to.

1. The process of raising a cheating accusation. Tournaments should indicate who (presumably the TD, but perhaps the TD designates like an ombudsman) should receive such cheating accusations and how. This prevents things like the TD only hearing from whisper networks, people throwing out accusations in very informal settings like a DM or a group chat message, etc.

2. Who, if anyone, should publicly reveal the accusation. Obviously if the TD or whomever concludes the accusation holds no water, the accusation should remain private (they should, of course, reveal to the accusers why they reject their accusation). But if the accusation is determined to be with merit (see below), then the process of a public revelation should be codified. I would say something like the "TD must publicly reveal this within a 24-hour span after the tournament is over" (even if all the details are not revealed) makes sense and would prevent things like players or third parties revealing this information at haphazard times.

3. What the process of determining guilt looks like. Barring a confession or Hines-esque super ludicrous stats, in most cases cheating accusations are going to feature a lot of circumstantial evidence--some suspicious stats, reports of suspicious behavior, suspicious behavior on camera (or not being on camera), etc. The TD and perhaps some third parties are going to have to look through the evidence, hear from the accused and their teammates, and reach a conclusion. It would be good to figure out--who is going to make the call, what the judgment process looks like (do they talk to the accused team? Does the accused get to make a statement? Does the accused get a chance to prove themselves?), and perhaps as Ike alludes to, is there a distinguishing from the short term form of this process and the long term (i.e. an accused cheater, if the evidence seems suspicious, might be removed from the tournament before a fuller investigation can begin just to avoid further delaying the tournament). This process should be codified as well. This would avoid seemingly what happens after every accusation, in which the accused, their team, and the TD all have different ideas of what a truly fair process looked like.

I think each tournament should codify these processes and include them in a code of conduct. It doesn't involve anything complex--I expect in most cases, a boilerplate series of paragraphs could be produced and used by most tournaments if they feel comfortable doing so. But this would result in stopping what seems to be a wild wild west-esque response to any cheating accusations. A TD can say--follow this process, this is the process of public revelation, this is the process of determining guilt (you might not AGREE with any of the decisions, but that's another story).

The other thing that needs codification is "what is the punishment for someone judged to have cheated"--this is beyond the scope of this post, as it would require multiple community groups to sign on.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Cheynem
Sin
Posts: 6837
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: A Proposal to Deal with the issue of cheating

Post by Cheynem »

One other thing I'd like to see tournaments codify:

a policy on using things like computers and cell phones *mid rounds*. I know many of us don't have great attention spans (including yours truly), but for the sake of security, I think tournaments should explicitly forbid looking at your phone or your computer *mid round* besides playing the rounds (i.e., looking at other tabs, texting, chatting) or at least forbid it when live questions for your team are in play (i.e., not enforced when the other team has a bonus). I would assume most people already do this, but I assume in this multi-tasking generation, some don't (I've certainly fiddled with other tabs mid-tournament, although almost always not during live questions). I would assume most people would not be cool with someone texting or on their phone/tablet/computer during an in person tournament during tossups or their team's bonus, so the same logic should apply here.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
mgysac15
Kimahri
Posts: 5
Joined: Sat Nov 05, 2011 9:26 pm
Location: CT

Re: A Proposal to Deal with the issue of cheating

Post by mgysac15 »

I've only recently started browsing this forum again, having been away for a couple years, and even at the best of times I was more a lurker than a poster, but I've read up on some of the current situations.

When I was active in in-person tournaments, I often kept a sheet of paper next to me to jot down words or phrases I either wanted to remember or look up more about afterwards. For online tournaments, I can easily imagine many competitors wanting to do the same, but using a word document to do so - given that they are already on their computer.

How can a moderator easily distinguish the "innocent case" of someone taking notes for later from the cheating case of someone googling?
Michael Garn
St. Ignatius (OH) '11
Yale '15
User avatar
Ike
Auron
Posts: 1050
Joined: Sat Jul 26, 2008 5:01 pm
Contact:

Re: A Proposal to Deal with the issue of cheating

Post by Ike »

mgysac15 wrote: Wed Sep 09, 2020 2:43 am I've only recently started browsing this forum again, having been away for a couple years, and even at the best of times I was more a lurker than a poster, but I've read up on some of the current situations.

When I was active in in-person tournaments, I often kept a sheet of paper next to me to jot down words or phrases I either wanted to remember or look up more about afterwards. For online tournaments, I can easily imagine many competitors wanting to do the same, but using a word document to do so - given that they are already on their computer.

How can a moderator easily distinguish the "innocent case" of someone taking notes for later from the cheating case of someone googling?
My recommendation is to make things as analog as possible; teams should be allowed to take notes in a notebook, but not type on word docs, for example. As much as possible, the only part that should be digital is the actual mechanics of playing the game itself.
Ike
UIUC 13
User avatar
mgysac15
Kimahri
Posts: 5
Joined: Sat Nov 05, 2011 9:26 pm
Location: CT

Re: A Proposal to Deal with the issue of cheating

Post by mgysac15 »

Ike wrote: Wed Sep 09, 2020 4:22 am
mgysac15 wrote: Wed Sep 09, 2020 2:43 am I've only recently started browsing this forum again, having been away for a couple years, and even at the best of times I was more a lurker than a poster, but I've read up on some of the current situations.

When I was active in in-person tournaments, I often kept a sheet of paper next to me to jot down words or phrases I either wanted to remember or look up more about afterwards. For online tournaments, I can easily imagine many competitors wanting to do the same, but using a word document to do so - given that they are already on their computer.

How can a moderator easily distinguish the "innocent case" of someone taking notes for later from the cheating case of someone googling?
My recommendation is to make things as analog as possible; teams should be allowed to take notes in a notebook, but not type on word docs, for example. As much as possible, the only part that should be digital is the actual mechanics of playing the game itself.
That seems like a fair and reasonable requirement - but one that TDs should probably explicitly state. The idea mostly came to me when reading the Bayesian probability survey on the discord and thinking about what legitimate reasons someone could have to be typing, and I realized what a huge potential problem that could be if such a rule weren't explicitly stated beforehand. I do not intend for this to be used to exonerate any recently accused cheaters - if this were their real defense, they would have stated as much already, and should they state it only after seeing me describe it, it would be highly implausible.

I'm bringing this up to help avoid future ambiguous cases.
Michael Garn
St. Ignatius (OH) '11
Yale '15
User avatar
Ike
Auron
Posts: 1050
Joined: Sat Jul 26, 2008 5:01 pm
Contact:

Re: A Proposal to Deal with the issue of cheating

Post by Ike »

One more note:

Just because a player has been sidelined doesn't mean they're guilty. Hypothetically, let's consider the 2014 case of Jordan Brownstein, who went from being a pretty good scorer at ICT and Nationals, to a very good scorer at 2014 Chicago Open. If COVID had happened, I think people would be naturally suspicious when he surfaced at his first online tournament. Under my proposed protocol, there is a very high chance that Jordan would have been sequestered from the tournament, and asked to demonstrate his skill on separate packets. And under my proposed system that's going to happen: just because a player has been sequestered does not indicate guilt. Presumably, after they see 2014 Jordan put up 4 powers no matter the packet, there should be no issue. I say this to emphasize that if this practice does become standard, that we as a community do not rush to judgment based on that fact alone; the follow-up is the more important factor in ascertaining guilt.

On the other hand, I hope no one uses the above reasoning as the basis for allowing tournament directors sequestering players randomly. Based on Brad's post in the Princeton thread, Duke's TD consulted other folks for the best judgments -- and that feels appropriate to me.
Ike
UIUC 13
User avatar
Ike
Auron
Posts: 1050
Joined: Sat Jul 26, 2008 5:01 pm
Contact:

Re: A Proposal to Deal with the issue of cheating

Post by Ike »

And double posting, because it's a new idea that came about after reading Brad's post in the Princeton thread, and sitting on it.

If you need to tab out of the Zoom/Discord tournament, look something up on a computer, make a phone text, or do something that otherwise takes focus away from listening to the tossup, that should render you automatically ineligible to answer the current tossup / answer or help collaborate on the current bonus (all three parts of the bonus.) This is basically the equivalent of "tuning out" during a tossup, with the caveat that even if you do know the answer at some point, you've forfeited your right to answer it. This idea might solve some of the problems of the online play, and allow people to check their phone if they need be, or allow them to take a mental break during a question outside of their wheelhouse. Perhaps this is cumbersome to implement, I don't really know, but it might be something worth thinking about.
Ike
UIUC 13
Post Reply