Statement on LIT Cheating Allegation: Princeton A

Elaborate on the merits of specific tournaments or have general theoretical discussion here.
Post Reply
efleisig
Kimahri
Posts: 3
Joined: Sun Nov 11, 2018 10:39 pm

Statement on LIT Cheating Allegation: Princeton A

Post by efleisig »

Princeton Quiz Bowl has now investigated the allegations concerning Chase Chiang during LIT.

We were confronted mid-round by the accusation that, after some rounds of observation, the tournament organizers had concluded that Chase was cheating. Immediately after the accusation, a team member in the same physical location as Chase took a picture of his search histories, and we removed him from all subsequent rounds. We then proceeded to gather all possible evidence about his behavior, including his text message histories, his explanations for his movements and for each buzz, and his performance at past tournaments and practices (both in-person and online). We established an internal committee consisting of the current and former officers to deliberate this evidence.


We found that:
- Chase was dealing with a serious family incident (of which we were then unaware) that resulted in him texting during the tournament. He acknowledges that he should have informed us at the time.

In the haste to give Chase’s text histories to the tournament organizers immediately, we also included some that were not from that day by mistake. When we were informed of this, we confronted Chase, who explained what had occurred and reasonably did not want to provide his entire text history from a private and painful incident. He eventually agreed to provide part of it to us as evidence that he was in fact dealing with a serious matter; we provided everything he gave us to the tournament director. The timestamps for these texts fall within the times when he was supposedly cheating.

- Chase’s search histories, on all browsers and devices, include no searches for information during the tournament. We acknowledge that he could have deleted them in the two minutes between when he was accused and when the search histories were examined.

- Chase’s performance is in line with his steady improvement in performance at in-person practices up through March, at online practices since March, and at in-person practices during the past week. It is also consistent with his performance at his most recent online tournament, Oxford Open, and his knowledge in person outside of practice. In other words, if he cheated at this tournament, he must have been cheating increasingly more for the past six months, including while under in-person observation, in a way that is consistent with normal growth as a player.


The tournament organizers held that:
- Chase was “extensively typing only during a tossup or during [Princeton]’s bonuses (specifically on questions with particularly google-able quotes and titles, and sometimes turning his face away from the camera to do so).” We have not been told why his texting does not reasonably explain this, given that:
(1) Chase was only visible from the mid-chest upwards, making it near-impossible to distinguish between typing on a keyboard and typing on a phone over the keyboard
(2) Chase has a long history of mannerisms that include shaking his head, looking away, and making a twitching hand movement while thinking. We provided the tournament organizers the opportunity to observe these tics in a controlled setting, but they refused.
(3) We were not told what the “google-able” words were, but given that quiz bowl questions consist primarily of such clues, we find it difficult to believe that it is distinguishable from typing at random or reacting to recognizing a notable word.

- The tournament director noted that Chase made a “shaky/rapid hand movement” that was perceived as Chase trying to disguise typing but is, as mentioned, a natural tic.

- Chase was pursuing this behavior “only during questions that specifically would only impact your team’s scoring.”
We have not been told what those questions are, nor have we been given any documentation of this assertion.

- Chase would not know “an easier part of a bonus but then answer a subsequent harder part with the correct answer without knowing how to pronounce it.”
It is unclear how this view distinguishes someone who engages in frequent packet study and online reading from someone who had just Googled the name.

- Observers saw “the lighting changing on [Chase’s] face as he was clicking/typing, indicating that the screen had likely changed to a different tab.”
This is dependent on screen and tab settings and room lighting. We tested this and could not distinguish between the light change from Discord to a new tab on a laptop and from Discord to a light-themed texting screen on a phone held over the laptop keyboard.

- Observers felt that “the amount of typing observed was more than what is required to type ‘buzz.’”
It is however, consistent with texting.


When we provided our evidence, the tournament organizers responded that:

- We should have disclosed the family emergency immediately.
Chase was facing a serious personal matter. It is wholly reasonable for him to be hesitant about sharing such information with strangers. We on the team did not disclose it because we did not know until after the tournament. Furthermore, tournament organizers repeatedly tried to prevent us from understanding what was going on and pressured us to continue playing, resulting in accidentally sending some texts from the wrong day in our haste to document the evidence while being rushed to keep playing and delaying us from learning of what was happening to Chase. When Chase told us, we immediately disclosed everything that Chase was comfortable sharing.

- The tournament organizers replied that this delay was “questionable” and that “features of the screenshot bring its authenticity into question.” We were not told what the “features” are. They further claimed that a single screenshot could not prove his innocence over every possible round. We acknowledge that more screenshots would help to understand the situation, but we deem it inappropriate to force Chase to reveal something so deeply personal on a public forum, and it is perfectly reasonable for him to be reluctant to do so.


We further requested that Chase be read a packet under controlled circumstances so that his mannerisms and performance could be observed, which we believe would be consistent with the tournament. This request was denied.

We also requested that we be given any video or text evidence that Chase's behavior was suspicious beyond general impressions. This request was ignored.

We understand the necessity of rapid action on the tournament organizers’ part and appreciate that they made a concerted effort to determine whether Chase was cheating using their qualitative evidence at the time. We also wholeheartedly agree that the preventive measures they took during the tournament out of an abundance of caution were justified and reasonable.


However, we hold that there is not sufficient evidence to conclude that Chase was cheating during LIT. Nor can we condone how the incident was handled: namely,
- the lack of documented evidence given by the accusers after repeated requests,
- the refusal to allow us to demonstrate his innocence at the time, calling it “no time for litigation,” then asserting that we should have defended him earlier when we attempted to do so later on;
- the refusal to let us present evidence that could provide much better context on Chase’s behavior; i.e., having him play in a controlled setting--which we hold would demonstrate that Chase’s mannerisms and performance during the tournament were wholly in line with his typical behavior;
- the assertion that our lack of an immediate, convincing defense was due to an assumption of Chase’s guilt rather than the lack of evidence and communication on their end; and
- the pervasive “guilty until proven innocent” attitude that has allowed a poorly documented accusation to be taken as conclusive proof of guilt.


It is unjust to assert that Chase must prove his innocence under assumption of guilt, even revealing extensive personal messages about a painful situation, to refute an assertion of cheating based on brief observation and what under further scrutiny amounts to speculative evidence.

Accusations of cheating are a serious matter that must be investigated thoroughly. Because of this, we were meticulous about documenting evidence in order to form as clear a picture as possible of what actually happened. By contrast, the tournament organizers provided us at most with informal impressions that Chase’s behavior was suspicious, giving us little to no chance to provide explanations. For these reasons, we hold that the tournament organizers’ decision was neither fair nor balanced.

Chase has expressed great hesitation about participating in more tournaments for the time being, given what he has been through. In any future tournaments, we will have Chase’s screen and movements be recorded by the team members who live with him at the discretion of the tournament director: not because we believe he has done anything wrong, but rather to further an environment of trust that is vital to online quiz bowl.

[Edited to remove uncleared set content]

Eve Fleisig
Princeton '21 (club president)
Wootton HS '18 (club president)
User avatar
Ike
Auron
Posts: 1047
Joined: Sat Jul 26, 2008 5:01 pm
Contact:

Re: Statement on LIT Cheating Allegation: Princeton A

Post by Ike »

Speaking entirely as a disinterested observer, and not for any quizbowl organization:

I find the way many aspects of this report worth discussing, perhaps if only for the future:
- Chase was dealing with a serious family incident (of which we were then unaware) that resulted in him texting during the tournament. He acknowledges that he should have informed us at the time.
How and why is this particularly relevant? I feel that one of the most common tactics used by cheaters is appealing to sympathy that is not relevant to the facts at hand. I think that one area that Duke could have improved upon was leaving a lot of the personal stuff out of official club responses with regards to their own cheater. It's one of thing if the accused posts it, but in my opinion, this kind of information appearing in a report makes the process appear less impartial.
The timestamps for these texts fall within the times when he was supposedly cheating.
Information like this doesn't really prove or disprove anything, partly because we--the readers of this report--can't determine whether the accused party copied the texts into a word file and sent them to you, (in which case it's easy to tamper), or he actually sent you logs from his ISP, service provider etc.
- Chase’s search histories, on all browsers and devices, include no searches for information during the tournament. We acknowledge that he could have deleted them in the two minutes between when he was accused and when the search histories were examined.
Aside from the fact that it's possible he could be running an obscure browser like Opera, or be searching things on a handheld device that you may not know the existence of, it's super easy to just go into incognito mode and no search histories are recorded.
We provided the tournament organizers the opportunity to observe these tics in a controlled setting, but they refused.
What does a controlled setting mean? Speaking personally, if such a request happened at my tournament, I would refuse such a request. Evidence of these tics isn't all that exculpatory, and at best is a dog-and-pony show that doesn't prove or disprove anything. If however, the request was "hey I'd like these packets read to us because we can demonstrate that"the accused has been rapidly improving and thus show that the accusations are unfounded," I think it would be foolish of Duke to not* honor the request.
- We should have disclosed the family emergency immediately.
Chase was facing a serious personal matter. It is wholly reasonable for him to be hesitant about sharing such information with strangers. We on the team did not disclose it because we did not know until after the tournament. Furthermore, tournament organizers repeatedly tried to prevent us from understanding what was going on and pressured us to continue playing, resulting in accidentally sending some texts from the wrong day in our haste to document the evidence while being rushed to keep playing and delaying us from learning of what was happening to Chase. When Chase told us, we immediately disclosed everything that Chase was comfortable sharing.
There is nothing wrong with a player saying before an online tournament (or one in real life) "hey, I need to respond to my texts during the match, I have a personal matter, it's quite serious. Is that okay?" I don't think you need to disclose particulars to every moderator. I think if you're going to be texting, or otherwise communicating frequently during a game, let the TDs know beforehand! To be honest, I really feel that Duke is in the right on this one, if a player is going to be using a cell phone or something during a game, they should let the TDs know beforehand.
This request was ignored.
One of the few things I disagreed in regards to Caleb's post was that in my opinion, Duke is pretty new to running an online tournament and have lives that necessitated them being slow. The TD only got back to address some of the concerns a few hours ago. And here you folks at Princeton made judgment quite fast. When the Matt Bruce story broke, NAQT took a long time to get their ducks in a row to discharge him. (IIRC, it was a several weeks decision.) Look, maybe you guys were very thorough, but from reading this report, I feel that many aspects of this process were incredibly lacking, including this write-up. At the very least, I think that if Duke doesn't get back to you with information in two days, wait a little longer; everyone has lives outside of quizbowl. In the end Duke was forthcoming in the thread!
the refusal to allow us to demonstrate his innocence at the time,
Can you explain how you would demonstrate his innocence? I find this to be poor wording. There's no way you can possibly prove he didn't cheat right, just that the accusations are unfounded.

I say all this because with the way this report is written, and the way aspects of the investigation played out as described in the report, I had more questions than answers. Again, I don't know any of you, but I think the process and the optics of the process could have been handled a bit better. Even the mention of including past club officers just reeks of the insiderism we've been so desperately trying to combat.

Look here's my two cents. Based on the numbers I saw from what Chase put up at Oxford Open, and the numbers he put up here, he _vastly_ improved. He put up 14 gets and 3 negs in the first five rounds of Oxford Open prelims (there were no powers there) and went 13/15/3 in the first five rounds of this tournament. That's an enormous improvement! My suggestion is to have someone* read him five rounds and see if he puts up similar numbers. If he does half as well as he did on LIT, I think people will realize it's pretty plausible he just improved. That to me is the best way to demonstrate plausibility.

*someone from outside of Princeton

EDIT: missed a not
Last edited by Ike on Tue Sep 08, 2020 9:54 am, edited 1 time in total.
Ike
UIUC 13
User avatar
Cheynem
Sin
Posts: 6809
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Statement on LIT Cheating Allegation: Princeton A

Post by Cheynem »

Ike wrote: Tue Sep 08, 2020 7:15 am
We provided the tournament organizers the opportunity to observe these tics in a controlled setting, but they refused.
What does a controlled setting mean? Speaking personally, if such a request happened at my tournament, I would refuse such a request. Evidence of these tics isn't all that exculpatory, and at best is a dog-and-pony show that doesn't prove or disprove anything. If however, the request was "hey I'd like these packets read to us because we can demonstrate that"the accused has been rapidly improving and thus show that the accusations are unfounded," I think it would be foolish of Duke to honor the request.
Seeing as how you eventually suggest someone read packets to Princeton, do you mean it would be foolish "not" to honor the request?
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Ike
Auron
Posts: 1047
Joined: Sat Jul 26, 2008 5:01 pm
Contact:

Re: Statement on LIT Cheating Allegation: Princeton A

Post by Ike »

Cheynem wrote: Tue Sep 08, 2020 9:44 am
Ike wrote: Tue Sep 08, 2020 7:15 am
We provided the tournament organizers the opportunity to observe these tics in a controlled setting, but they refused.
What does a controlled setting mean? Speaking personally, if such a request happened at my tournament, I would refuse such a request. Evidence of these tics isn't all that exculpatory, and at best is a dog-and-pony show that doesn't prove or disprove anything. If however, the request was "hey I'd like these packets read to us because we can demonstrate that"the accused has been rapidly improving and thus show that the accusations are unfounded," I think it would be foolish of Duke to honor the request.
Seeing as how you eventually suggest someone read packets to Princeton, do you mean it would be foolish "not" to honor the request?
Correct, my bad!
Ike
UIUC 13
touchpack
Rikku
Posts: 394
Joined: Tue Sep 20, 2011 12:25 am

Re: Statement on LIT Cheating Allegation: Princeton A

Post by touchpack »

Speaking as an outside observer, not any organizations, I find the "family emergency" story to be extremely implausible. If someone truly were having a family emergency that required them to be constantly texting during rounds, you would expect their performance to be at least mildly, if not significantly impaired by the distraction. On the contrary, in this case Chase put up numbers significantly higher than he has ever put up at an in-person tournament. (References to other online tournaments are not a defense, as he could have cheated at those as well).

I think Ike's idea is a good one here, but I would add the caveat that performance playing vs empty chairs will be significantly better than performance in a room of 8 people, so results should not be compared 1:1.
Billy Busse
Illinois '14
Full Member, ACF
Writer/Subject Editor/Set Editor, NAQT
Borrowing 100,000 Arrows
Wakka
Posts: 103
Joined: Sun Feb 19, 2017 11:29 pm

Re: Statement on LIT Cheating Allegation: Princeton A

Post by Borrowing 100,000 Arrows »

One thing I found very telling from the stats is the difference in Princeton A's PPB with and without Chase. Before Chase was removed from the tournament, Princeton A's PPB was a very solid 21.56. After Chase was removed, Princeton A's PPB dropped precipitously to 15.42. That meant that Chase was doing *a lot* of heavy lifting on the bonuses. Last year's Princeton A, which seems to have been this lineup with Ryan Golant instead of Chase, was a very good team (they even beat us pretty handily at Regionals). Going from the third or fourth wheel on a middling team like Princeton B to single-handedly carrying a borderline top 25 team like Princeton A, that seems like a pretty massive jump to me. Maybe it's possible, but I can't think of anyone improving this much this quickly.
Caleb K.
Maryland '24, Oklahoma '18, Norman North '15
User avatar
ThisIsMyUsername
Yuna
Posts: 880
Joined: Wed Jul 15, 2009 11:36 am
Location: New York, NY

Re: Statement on LIT Cheating Allegation: Princeton A

Post by ThisIsMyUsername »

touchpack wrote: Tue Sep 08, 2020 11:23 am I think Ike's idea but I would add the caveat that performance playing vs empty chairs will be significantly better than performance in a room of 8 people, so results should not be compared 1:1.
Given that we'd expect the performances to be radically different, isn't this a pretty useless exercise? We have no baseline for what this "controlled" performance should look like in either the fair-play-throughout or cheating-the-first-time cases. The only piece of potentially useful additional evidence is if someone's performance is significantly worse against empty chairs. To get remotely comparable data, the accused would need to play with the same teammates against the same opponents from before on very similar questions, and it would have to be multiple rounds (to reduce packet-to-packet variance), which seems like an impractical set-up.
Last edited by ThisIsMyUsername on Tue Sep 08, 2020 1:11 pm, edited 1 time in total.
John Lawrence
Yale University '12
King's College London '13
University of Chicago '20

“I am not absentminded. It is the presence of mind that makes me unaware of everything else.” - G.K. Chesterton
User avatar
Illinois Admin
Wakka
Posts: 176
Joined: Mon Apr 03, 2017 11:11 pm
Location: Austin, Texas

Re: Statement on LIT Cheating Allegation: Princeton A

Post by Illinois Admin »

Speaking as one of the people whose opinion was sought with regards to whether Chase should be given the opportunity to play more packets as a comparison, my opinion was no. The reason for this is that the "counterevidence" provided did not actually address the evidence presented (more on this below) and (more importantly) no result from such an exercise could adequately explain what was observed. It could only serve to muddy the waters: if Chase performed poorly an excuse of high pressure could be proffered, and if he continued to play at a high level it still would not address the suspicious behavior that was observed. In my undergrad courses (which carry much greater weight than this) we were told that if we behaved in a manner that would convince a reasonable person that we were cheating, we would be considered to be cheating unless we could provide compelling evidence to the contrary. Given that three people came to the same conclusion about his behavior, I would say the reasonable observer standard was met. As for compelling evidence to the contrary, I do not believe such evidence has been provided.

To that point, I will say that I was one of the witnesses who reported Chase's behavior. Speaking as a witness (and not in any official capacity with my connection to the set), I find the narrative constructed in this report by Princeton Quiz Bowl to be misleading, and in some cases, flatly false. Most notably:

1. Chase was observed texting. This was a distinct phenomenon from what we (the witnesses) believed was clearly Chase reading from a computer screen. When on his cell phone, Chase had his head bent downward and did not experience flashes of light across his face. He also did not exhibit obvious eye saccades while on his cell phone.

2. Chase was also observed typing on a keyboard. When typing, Chase faced the computer screen, exhibited eye saccades consistent with reading text from a screen, and had light flash across his face repeatedly in a manner consistent with changing tabs. Additionally, and perhaps most importantly, on a few occasions the sound of his mouse and keyboard were picked up by the microphone during his team's bonuses to an extent not consistent with typing his answers (he was giving his answers verbally anyway).

3. The amount that Chase was typing would not be consistent with sending text messages, unless literally hundreds of messages were being sent. As Billy noted, if he was sending that many messages over text, it is unclear why he would also improve given the distraction.

4. Additionally, a consistent pattern emerged in Chase's behavior on bonuses, where he would start the bonus with his hand on his chin, then as soon as a noun or title was read he would begin to type and exhibited wide eye saccades consistent with reading from a computer screen as light flashed across his face. Chase would then pull his hand back and "guess" the answer. This pattern was generally not followed during opponents' bonuses. Additionally, as noted above, the sound of Chase's keyboard and mouse was occasionally picked up during his bonuses when he was observed typing and reading from computer screens.

5. I also contest that you were not informed what "googleable words" were. You were given several examples from the very packet you had just played as to what was being referred to. I will also contest that quiz bowl questions are primarily composed of easily Googleable facts, at least at the speeds needed to cheat. Scientific formulas and descriptions of historical or literary events that do not give names would be examples where it is much harder to quickly use Google to identify the answer. With these Googleable words, I went through a packet ahead of time and determined which words and phrases were most likely to be searched by someone cheating. With very few exceptions, Chase was observed beginning to type when those words and phrases were read. More notably, this behavior was mostly absent when such words and phrases had not just been read. This continued on bonuses where Chase was observed typing on nearly every one of his teams' bonuses but on nearly none of his opponents' bonuses.

At the very least, all of this is not consistent with the claim that texting explained all the observed suspicious behavior and, in my opinion, it is extremely strong evidence that he was using his computer during questions (at the absolute least). Combined with the very strongly improved stat-line, I believe this is a compelling case that Chase cheated.
Bradley McLain
2015- Rockford Auburn High School
2019 - BS in Psychology - University of Illinois at Urbana-Champaign
2021 - MS in Social Work - University of Texas at Austin
jaredlockwood
Lulu
Posts: 29
Joined: Sat Sep 17, 2016 3:54 pm

Re: Statement on LIT Cheating Allegation: Princeton A

Post by jaredlockwood »

On behalf of the Princeton University Quiz Bowl team:
Ike wrote: Tue Sep 08, 2020 7:15 am
- Chase was dealing with a serious family incident (of which we were then unaware) that resulted in him texting during the tournament. He acknowledges that he should have informed us at the time.
How and why is this particularly relevant? I feel that one of the most common tactics used by cheaters is appealing to sympathy that is not relevant to the facts at hand. I think that one area that Duke could have improved upon was leaving a lot of the personal stuff out of official club responses with regards to their own cheater. It's one of thing if the accused posts it, but in my opinion, this kind of information appearing in a report makes the process appear less impartial.
The timestamps for these texts fall within the times when he was supposedly cheating.
Information like this doesn't really prove or disprove anything, partly because we--the readers of this report--can't determine whether the accused party copied the texts into a word file and sent them to you, (in which case it's easy to tamper), or he actually sent you logs from his ISP, service provider etc.
This was poorly communicated; thanks for pointing it out. Our intent here was not to appeal to pathos. It had been pointed out to us that Chase was on his phone throughout the tournament, and that it was being used as evidence against him. When he was accused in the moment, Chase claimed that he was texting his family. This was meant to verify that claim.

For full transparency, a member of our team who resides with Chase used their own phone to take photos of two of the text threads on Chase’s phone immediately after the accusations were levied. The individual who took the photos told us that they felt rushed and flustered in the moment, and that because of this they did not verify that the threads they photographed were Chase’s most recent texts and/or dated from the day of the tournament. When questioned further, they indicated that they had been so rattled by the the situation--the suddenness of the accusation, lack of explanation for it, insistence that the determination of guilt had already been made (the individual repeatedly referred back to a message from Brad in the Discord channel: “lets be clear, this is not a trial and this is not being litigated”), and being told that the next round was to begin immediately--that they were feeling overwhelmed and were too hasty.

After Jacob alerted us the next evening that one of the text threads was a week old, a member of the team who resides with Chase physically obtained his phone and verified that Chase had, in fact, been texting his family during the tournament by viewing a series of texts. On Chase’s phone, they screenshotted two sections of the text conversation showing timestamps during the tournament (12:17PM and 4:11PM) and forwarded them to Jacob.

Ike wrote: Tue Sep 08, 2020 7:15 am
- Chase’s search histories, on all browsers and devices, include no searches for information during the tournament. We acknowledge that he could have deleted them in the two minutes between when he was accused and when the search histories were examined.
Aside from the fact that it's possible he could be running an obscure browser like Opera, or be searching things on a handheld device that you may not know the existence of, it's super easy to just go into incognito mode and no search histories are recorded.
You’re absolutely right. The point we are trying to make here is not “Chase’s browser history proves his innocence” – which would be a ridiculous claim – but rather “Chase’s browser history turns up nothing that can be used to find him innocent or guilty.”

Ike wrote: Tue Sep 08, 2020 7:15 am
We provided the tournament organizers the opportunity to observe these tics in a controlled setting, but they refused.
What does a controlled setting mean? Speaking personally, if such a request happened at my tournament, I would refuse such a request. Evidence of these tics isn't all that exculpatory, and at best is a dog-and-pony show that doesn't prove or disprove anything. If however, the request was "hey I'd like these packets read to us because we can demonstrate that"the accused has been rapidly improving and thus show that the accusations are unfounded," I think it would be foolish of Duke to not* honor the request.
The request we have made to Duke is the latter: Chase could have several LIT packets read to him by someone not affiliated with Princeton in a setting that they deem appropriate (alone, with his teammates, both; against empty chairs or a team) where his behavior and conduct can be observed to a degree that meets their satisfaction. As we would like to make abundantly clear, we recognize that this may not be considered entirely exculpatory, but it would at least provide a setting in which the observers can determine whether his tics are consistent with the behavior they observed during the tournament, and, more importantly, some form of substantive evidence to support claims one way or the other; we believe that a demonstratively weaker performance would be a considerable indication of misconduct.

When we proposed a packet reading to Duke, we were told that “it does not make sense for that to happen. The behavior that was observed on Saturday could not be explained or disproved by the outcome of reading a fresh packet.” We agree that a packet reading doesn’t directly address the observed behaviors at the tournament, but it certainly does provide some indication as to whether or not these behaviors were linked to Chase’s performance (i.e., whether the observed behaviors can be confidently attributed to Chase looking up answers). We think this provides important information for other organizations and clubs who may be considering barring Chase from their events.

Ike wrote: Tue Sep 08, 2020 7:15 am
- We should have disclosed the family emergency immediately.
Chase was facing a serious personal matter. It is wholly reasonable for him to be hesitant about sharing such information with strangers. We on the team did not disclose it because we did not know until after the tournament. Furthermore, tournament organizers repeatedly tried to prevent us from understanding what was going on and pressured us to continue playing, resulting in accidentally sending some texts from the wrong day in our haste to document the evidence while being rushed to keep playing and delaying us from learning of what was happening to Chase. When Chase told us, we immediately disclosed everything that Chase was comfortable sharing.
There is nothing wrong with a player saying before an online tournament (or one in real life) "hey, I need to respond to my texts during the match, I have a personal matter, it's quite serious. Is that okay?" I don't think you need to disclose particulars to every moderator. I think if you're going to be texting, or otherwise communicating frequently during a game, let the TDs know beforehand! To be honest, I really feel that Duke is in the right on this one, if a player is going to be using a cell phone or something during a game, they should let the TDs know beforehand.
We agree; Chase exercised poor judgment in not indicating to anyone that he would be using his phone during the tournament and why. We also concur with the tournament director that using a phone during matches without express authorization is reasonable cause for suspicion and legitimate grounds for Chase’s removal from the tournament.

We recognize that some people also question whether Chase was even experiencing an urgent personal matter during the tournament. To that end, we have physically observed on his phone select portions of Chase’s conversation with timestamps that overlap with the tournament, including the strings of messages stamped beginning 12:17PM and 4:11PM that were sent to Duke demonstrating the nature of Chase’s situation (it is worth noting that Chase was not playing at 12:17). The first of the string of messages stamped 4:11PM overlaps with a tossup that Chase heard, buzzed in on, and answered correctly while the two editors who observed him were present (during round 6, which began at 3:54PM). According to Discord, tossup 11 begins at 4:11:03 PM, Chase buzzes at 4:11:38 PM but is beat by the other team who negs. He buzzes in again with the correct answer at 4:12:10 PM.

We do think it pertinent to point out that at the time of our first post it was not clear to us that the two editors who observed Chase appear to be able to distinguish between when he was using his phone versus when they believed him to be on his computer, and that their accusations of cheating were distinct from and largely unrelated to their observation that Chase was on his phone during matches. At the time of our post, we believed that the accusations followed from Chase’s use of his phone. Regardless, unless the editors did see Chase use his phone during live play and haven't made that explicit, which is possible (and if it is the case we would welcome their clarification), we are under the impression that they saw Chase use his phone, but not during live play--or at the very least not before answering a tossup-- and that the actions of typing on his phone and typing on his laptop were clear and distinct in their eyes. However, as this text record shows, Chase did use his phone during live play before buzzing; if our assumptions about what the editors saw are correct, then it means at least one of the instances that they believed Chase was typing was not so, and, from our perspective, raises questions about the confidence one can have in their observations.

Ike wrote: Tue Sep 08, 2020 7:15 am
This request was ignored.
One of the few things I disagreed in regards to Caleb's post was that in my opinion, Duke is pretty new to running an online tournament and have lives that necessitated them being slow. The TD only got back to address some of the concerns a few hours ago. And here you folks at Princeton made judgment quite fast. When the Matt Bruce story broke, NAQT took a long time to get their ducks in a row to discharge him. (IIRC, it was a several weeks decision.) Look, maybe you guys were very thorough, but from reading this report, I feel that many aspects of this process were incredibly lacking, including this write-up. At the very least, I think that if Duke doesn't get back to you with information in two days, wait a little longer; everyone has lives outside of quizbowl. In the end Duke was forthcoming in the thread!
To be clear it’s not that Duke didn’t get back to us, it’s that when they did respond they didn’t acknowledge our request at all, or seemed dismissive of the points we raised as if to brush off the problem and move past it. They did not share any documentation that they had collected during their two-game observation period, such as recordings, screenshots, or notes that they had taken down to record exact instances of alleged misconduct beyond the single instance referenced in our original post*, or even clarify whether such documentation exists. You can probably imagine our frustration when Chase’s accusers failed to provide us with specific instances to substantiate their allegations and yet projected a dismissive attitude towards any effort we made to contribute evidence.

We don’t have the same amount of expertise or experience as NAQT has in adjudicating allegations of misconduct, nor do we claim to. Our knowledge of the Matt Bruce case is elementary at best, though we understand it to have involved a search for evidence of misconduct that would have taken place around 20 years ago and which was different in nature from what is alleged to have occurred here. You would be correct to deduce that we felt pressed for time to put out a public statement, given that Chase’s name was prematurely dropped in the forums and that readers were led to believe that there was utterly irrefutable evidence against him. As a result, our response was absolutely less thorough and detailed than it should have been. We appreciate you pointing this out, and wish to make clear that we are more than willing to consider any new evidence that presents itself, and will revise our decision if we deem it correct to do so.



*This is the example that was edited out of our original post because it contained uncleared set content. To the writers and editors, and anyone else who may have enjoyed the opportunity to play that bonus later on, we are sincerely sorry. Including that content here was an utterly embarrassing oversight on our part, and we apologize for the inconvenience it caused. Thank you to the individual who immediately pointed it out to us and to the editors for acting quickly to address its impact on the set.

Ike wrote: Tue Sep 08, 2020 7:15 am
the refusal to allow us to demonstrate his innocence at the time,
Can you explain how you would demonstrate his innocence? I find this to be poor wording. There's no way you can possibly prove he didn't cheat right, just that the accusations are unfounded.
This was poorly communicated: we indeed cannot definitively prove he didn’t cheat without, as has been suggested elsewhere, videographic evidence showing Chase’s full body, screens, and surroundings during the seven matches he played. The point we wished to make here is that Chase was deemed guilty in the same moment that the accusations were made, and that Princeton A’s efforts to address or appeal this were not taken seriously, as far as we could tell, both immediately after and the day following the tournament.

We wouldn’t even go so far as to say that the accusations are entirely unfounded. Rather, we would contend that while the observation of suspicious behavior may be sufficient for removing a player from a tournament, the threshold for a “conviction” with implications for a player’s future eligibility should be higher. Though it is certainly cause for suspicion, wariness, and increased scrutiny, entirely circumstantial evidence is, in our opinion, not sufficient to establish that misconduct has occurred. We recognize that others may reasonably disagree with us here, but this is the standard we use within our club and the basis from which we reach our internal decision.

Ike wrote: Tue Sep 08, 2020 7:15 am Even the mention of including past club officers just reeks of the insiderism we've been so desperately trying to combat.
Our intent here was to indicate that we are doing our best to make our internal investigation as objective as we can; all but one of the current officers were members of the Princeton A team, and we certainly didn’t want this to be that team simply attempting to vindicate itself.

It was brought to our attention that this statement may have been interpreted as us including in our investigation former members of the Princeton team who have since graduated. We will clarify that because the investigation was internal, it consisted only of current members of the Princeton quiz bowl team.

Ike wrote: Tue Sep 08, 2020 7:15 am Look here's my two cents. Based on the numbers I saw from what Chase put up at Oxford Open, and the numbers he put up here, he _vastly_ improved. He put up 14 gets and 3 negs in the first five rounds of Oxford Open prelims (there were no powers there) and went 13/15/3 in the first five rounds of this tournament.
It’s worth pointing out that the team compositions at these two events are different in a significant way. At Oxford Open, Chase and Ryan Golant were competing together, and they have a high degree of overlap in their knowledge base (e.g., both are primarily science players with a good degree of expertise in Greco-Roman content), and it’s natural to expect that Chase’s performance would be significantly affected by this. If there’s reason to believe that Chase and Ryan perform similarly across most subjects, it makes sense for Chase to get more buzzes when they’re not competing together. None of us are professional statisticians, but perhaps a more fitting methodology would take this into account and involve comparing Chase and Ryan’s relative performances at these two events (though we imagine this is no trivial task, given the conditions under which Ryan and Chase were playing at LIT differed considerably from each other).

To the point made elsewhere by Caleb about Chase doing heavy lifting on bonuses, the Princeton A team sans Ryan or Chase is severely handicapped on most science in addition to taking hits in other subject areas where Chase has consistently demonstrated deeper knowledge than his teammates, like Greco-Roman content and much Asian content. Absent a good metric for quantifying what the size of the difference should be, we’re cautious about reaching any conclusions from this, but would welcome analyses by trained statisticians.

As has been mentioned, the more pertinent question, perhaps, concerns the degree by which Chase improved during the six months between his last in-person tournament on February 8 and LIT. As teammates who have practiced with Chase multiple times every week since February, we don’t really find Chase’s performance out of line with the development he’d begun showing in the first half of our spring semester (which was still in person). Obviously, though, Chase’s growth trajectory isn’t observable to anyone who wasn’t present at the time; we can only use stats to see where he is now and where he was then. Given the question here is whether Chase could have legitimately achieved the statline he did at LIT, we again suggest that a controlled packet reading could be helpful. If Chase is able to replicate his performance at LIT, it should add credibility to his statline. If he can’t, it should raise serious questions that can’t be ignored.

touchpack wrote: Tue Sep 08, 2020 11:23 am Speaking as an outside observer, not any organizations, I find the "family emergency" story to be extremely implausible. If someone truly were having a family emergency that required them to be constantly texting during rounds, you would expect their performance to be at least mildly, if not significantly impaired by the distraction. On the contrary, in this case Chase put up numbers significantly higher than he has ever put up at an in-person tournament. (References to other online tournaments are not a defense, as he could have cheated at those as well).
Chase’s personal situation has already been addressed previously in this post, but we would add that determining whether Chase performed better/worse than he otherwise would have requires some known baseline of how Chase would perform if he hadn’t been experiencing a family emergency. We think everyone would agree that the best available baseline right now is his performance at SCT. As we have already emphasized, however, we don’t believe that Chase’s SCT stats reflect how well we would expect him to perform six months later.

Borrowing 100,000 Arrows wrote: Tue Sep 08, 2020 12:47 pm One thing I found very telling from the stats is the difference in Princeton A's PPB with and without Chase. Before Chase was removed from the tournament, Princeton A's PPB was a very solid 21.56. After Chase was removed, Princeton A's PPB dropped precipitously to 15.42. That meant that Chase was doing *a lot* of heavy lifting on the bonuses. Last year's Princeton A, which seems to have been this lineup with Ryan Golant instead of Chase, was a very good team (they even beat us pretty handily at Regionals). Going from the third or fourth wheel on a middling team like Princeton B to single-handedly carrying a borderline top 25 team like Princeton A, that seems like a pretty massive jump to me. Maybe it's possible, but I can't think of anyone improving this much this quickly.
This was mostly addressed above, but actually Princeton A’s PPB went from 21.58 to 17.08 without Chase. It appears you forgot to factor in that Chase did not play during Princeton A’s match against Penn B, during which the team averaged 21.54 PPB over 13 bonuses (we would also point out that it isn’t reflected in the stats, but Alice didn’t play in the last two games of the day, so Princeton A was even more shorthanded and the observed PPB gap is artificially widened a bit further). This implies that Chase was uniquely contributing about 15% of Princeton A’s bonus answers while he was playing. Given that, among other things, Chase has consistently shown deep knowledge in science and how weak the other members of Princeton A are on that subject (which notably makes up 20% of the overall set distribution), it’s really not surprising to us that Chase could contribute significantly to the team’s PPB.

ThisIsMyUsername wrote: Tue Sep 08, 2020 1:03 pm
touchpack wrote: Tue Sep 08, 2020 11:23 am I think Ike's idea but I would add the caveat that performance playing vs empty chairs will be significantly better than performance in a room of 8 people, so results should not be compared 1:1.
Given that we'd expect the performances to be radically different, isn't this a pretty useless exercise? We have no baseline for what this "controlled" performance should look like in either the fair-play-throughout or cheating-the-first-time cases. The only piece of potentially useful additional evidence is if someone's performance is significantly worse against empty chairs. To get remotely comparable data, the accused would need to play with the same teammates against the same opponents from before on very similar questions, and it would have to be multiple rounds (to reduce packet-to-packet variance), which seems like an impractical set-up.
Under the right set-up (we envision one similar to what you describe in the latter half of your post and don’t find it immediately apparent why this is impractical), we don’t see any reason to expect this to be useless. At the very least, a performance of the same Princeton A team against empty chairs could help determine whether Chase’s powers per game and the team’s PPB are meaningfully different from what was observed during the tournament. Particularly with PPB--which doesn’t depend much upon whether there is an opponent--since the team would be read all bonuses, each mock round is roughly equivalent to two tournament matches (the Princeton team averaged right around 10 TU/game). If there are three LIT packets remaining, that gets us to six tournament match-equivalents of sample data. This seems like fairly useful data, and requires few assumptions to make relevant comparisons. While powers in a reading against empty chairs would be intuitively and practically less useful, it would certainly be relevant if Chase gets far fewer powers against empty chairs while playing with the rest of the Princeton A team.

Illinois Admin wrote: Tue Sep 08, 2020 1:04 pm Speaking as one of the people whose opinion was sought with regards to whether Chase should be given the opportunity to play more packets as a comparison, my opinion was no. The reason for this is that the "counterevidence" provided did not actually address the evidence presented (more on this below) and (more importantly) no result from such an exercise could adequately explain what was observed. It could only serve to muddy the waters: if Chase performed poorly an excuse of high pressure could be proffered, and if he continued to play at a high level it still would not address the suspicious behavior that was observed.
While we respect your opinion, we strongly disagree with the claim that a controlled packet reading would be unhelpful. Packet reading may or may not directly address the “suspicious behavior” you observed, but it would certainly address the inferences you drew from it (i.e., that Chase was typing/cheating). If Chase continued to perform at a comparable level in the absence of the behaviors you described, it would weaken the link between the observations you made and the conclusions you drew from them; on the other hand, if Chase performed poorly, it would very much strengthen the conclusions you drew. In addition, if Chase had continued to exhibit the behaviors you describe seeing on screen (particular head, eye, and arm movements, etc.) while playing in a highly visible, controlled environment and while performing at a similar level, it casts doubt on the validity of the inference that those same behaviors during the tournament were indicators of cheating. You appear in your post to be quite confident in your ability to distinguish arm movements that indicate typing on a keyboard from other arm movements, so we trust that you would also be able to tell if Chase was attempting to replicate this sort of movement without typing while his hands are visible. If your concern is that Chase could make excuses post-facto to explain away a poor performance, there’s a pretty simple remedy to that: we can agree up front that no such excuse is valid! The performance is what it is, and excuses to rationalize it are just the kind of speculation we’re objecting to.

The specific claim that having more relevant information would somehow “muddy the waters” doesn’t seem helpful. Forgive us if we’re misinterpreting you, but it sounds to us like you’re arguing against collecting additional context and information because doing so would make it more difficult to attribute the behaviors you observed to cheating, and we find that troubling. At worst (though we think this unlikely), a controlled packet reading provides no new helpful information. On the other hand, it very well could provide concrete evidence one way or the other. Perhaps it’s not worth investigating further for the sake of litigating LIT, but (and we suspect you would agree with this, given you expressed interest in learning whether Chase’s case and your evidence for it would be passed to ACF and NAQT) third parties--particularly those at whose events Chase may wish to participate in the future--absolutely do have an interest in understanding this situation as clearly and fully as possible.

Illinois Admin wrote: Tue Sep 08, 2020 1:04 pm To that point, I will say that I was one of the witnesses who reported Chase's behavior. Speaking as a witness (and not in any official capacity with my connection to the set), I find the narrative constructed in this report by Princeton Quiz Bowl to be misleading, and in some cases, flatly false. Most notably:

1. Chase was observed texting. This was a distinct phenomenon from what we (the witnesses) believed was clearly Chase reading from a computer screen. When on his cell phone, Chase had his head bent downward and did not experience flashes of light across his face. He also did not exhibit obvious eye saccades while on his cell phone.

2. Chase was also observed typing on a keyboard. When typing, Chase faced the computer screen, exhibited eye saccades consistent with reading text from a screen, and had light flash across his face repeatedly in a manner consistent with changing tabs. Additionally, and perhaps most importantly, on a few occasions the sound of his mouse and keyboard were picked up by the microphone during his team's bonuses to an extent not consistent with typing his answers (he was giving his answers verbally anyway).

3. The amount that Chase was typing would not be consistent with sending text messages, unless literally hundreds of messages were being sent. As Billy noted, if he was sending that many messages over text, it is unclear why he would also improve given the distraction.

4. Additionally, a consistent pattern emerged in Chase's behavior on bonuses, where he would start the bonus with his hand on his chin, then as soon as a noun or title was read he would begin to type and exhibited wide eye saccades consistent with reading from a computer screen as light flashed across his face. Chase would then pull his hand back and "guess" the answer. This pattern was generally not followed during opponents' bonuses. Additionally, as noted above, the sound of Chase's keyboard and mouse was occasionally picked up during his bonuses when he was observed typing and reading from computer screens.

5. I also contest that you were not informed what "googleable words" were. You were given several examples from the very packet you had just played as to what was being referred to. I will also contest that quiz bowl questions are primarily composed of easily Googleable facts, at least at the speeds needed to cheat. Scientific formulas and descriptions of historical or literary events that do not give names would be examples where it is much harder to quickly use Google to identify the answer. With these Googleable words, I went through a packet ahead of time and determined which words and phrases were most likely to be searched by someone cheating. With very few exceptions, Chase was observed beginning to type when those words and phrases were read. More notably, this behavior was mostly absent when such words and phrases had not just been read. This continued on bonuses where Chase was observed typing on nearly every one of his teams' bonuses but on nearly none of his opponents' bonuses.
We regret that you believe the entire Princeton team is trying to mislead or lie to the public. We’re trying to be as clear and as fair as we can and to that end we’re including below exactly what was sent to us by Duke in the way of evidence (quoted from Jacob’s email):
“I reviewed the material that you forwarded to me as well as the editors’ allegations, and at this point I unfortunately do agree with the editors that Chase was cheating. The two text chains which he provided do not appear to be from yesterday (one of them is actually dated from last Sunday), and the content of both chains was in no way related to the “Coronavirus family emergency” that was described as the reason for texting during the competition, and instead seemed to be texts with friends about meeting up on campus. Additionally, there is no way of confirming that any browser history which he provided is a complete record. And then there is the behavior noted by several people that is extremely convincing. Some examples:
  • Extensively typing only during a tossup or during your team’s bonuses (specifically on questions with particularly google-able quotes and titles, and sometimes turning his face away from the camera to do so)— and not when the other team was answering their own bonuses (that would have been the more logical time to be texting one’s family)— and always returning his focus in time to buzz before FTP.
  • Not knowing an easier part of a bonus but then answering a subsequent harder part with the correct answer without knowing how to pronounce it (e.g. [redacted - uncleared set content]).
  • The lighting changing on his face as he was clicking/typing, indicating that the screen had likely changed to a different tab.
  • The amount of typing observed was more than what is required to type “buzz.” Not to mention that most people pre-type “buzz” before a tossup is even read and then press enter when they do want to buzz.
  • The statistical anomaly of his higher statline yesterday can’t fairly be compared to your club’s recent online practices where similar behavior could easily have occurred, rather than the most recent in-person competitions. I agree that his stats alone can’t prove cheating, but when combined with the above behavior, it seems to be more than just a coincidence”
After the correct screenshot of the texts from the family emergency were sent:
“I have reviewed and considered the new text screenshot that you forwarded to me yesterday evening. The determination that was previously made will not change for the following reasons:

1. If a family emergency had arisen, it seems odd that this was not disclosed immediately when the cheating allegations were brought to your team's attention, especially given what was at stake both for Chase individually and for your team. (Not to mention that if a true emergency arose, then Chase should have withdrawn to address the matter instead of continue to play additional rounds.) If there was an issue of sensitivity and he was reluctant to disclose the exact nature of the matter, that could easily have been discussed and dealt with at the time.

2. Regardless, we maintain that the behavior exhibited by Chase, and the timing of his behavior, clearly indicated activity that was not just texting.

3. It is questionable that this new screenshot was only forwarded to me many hours after you received my email yesterday– an email which revealed that the prior texts that Chase provided to prove that he wasn’t cheating were texts dated from a week earlier. Also, there are certain features of the new screenshot which bring its authenticity into question. Even if, for argument's sake, we assume that the new screenshot is authentic, a photo of one quick text does not explain Chase's behavior, why his behavior continued throughout multiple rounds, or why the typing was occurring only during questions that specifically would only impact your team’s scoring.

Regarding your club's request to read a new LIT packet to Chase– while your club may be interested in that kind of an exercise as part of your internal investigation, it does not make sense for that to happen. The behavior that was observed on Saturday could not be explained or disproved by the outcome of reading a fresh packet. The determination made on Saturday is reflective of his performance that day. His behavior was persistent and witnessed by several people, and was brought to our attention only after their careful consideration over several rounds of play. If the editors had not been completely certain that cheating was occurring, then the matter would not have been raised at the tournament.
It's unfortunate that all of this happened, but our previous conclusion stands– that Chase was cheating.”
As we hope is evident here, what you’ve included in your post differs to some extent in its precision of detail from the information we were given to work with and, in some cases, provides new or incongruent information, so we’re not sure how we were expected to consider that in our original post. We were given no specific examples of “googleable” words or phrases that Chase was suspected to have looked up beyond the one redacted example, and there was never any mention of eye saccades, or much indication that you distinguished between what you believe to be Chase looking at his phone vs. the computer screen. We’re not suggesting this information was intentionally withheld, but it certainly was not conveyed clearly to us.

We’re not disputing that Chase probably exhibited the behaviors you described (light flashes, eye saccades, shoulders/upper arms displaying some form of motion), but we do question whether your interpretation of these motions is the only reasonable explanation for them, and we further wonder whether the presumed statistical evidence combined with the suggestion that Chase was cheating--and your being dispatched to verify this--may have influenced your interpretation of the timing, frequency (including the frequency of the “very few” exceptions in tossups that you mention and patterns that were “generally not” followed during opponents’ bonuses), or assumed meaning of these observations. Chase has consistently maintained to us that he was not using his internet browsers or typing extensively on his laptop, but that he was frequently toggling back and forth between text and video channels on Discord and occasionally using iMessage to send his texts. It’s certainly possible he’s lying to cover up cheating, but at this time we also have no real evidence to show that he’s not. As we referenced before, as long as our understanding of what the editors observed is correct, we can also show that at least one instance that Chase was presumed to have been typing to Google an answer was actually due to him sending a text message. Maybe what you observed directly was enough for you to reach the conclusion that cheating was going on, but absent something more compelling than the accounts of two non-independent/collaborative observers that draws significantly upon inference we don’t feel that it’s appropriate for us to do the same.

Illinois Admin wrote: Tue Sep 08, 2020 1:04 pm 2. Chase was also observed typing on a keyboard.
According to your responses in the quiz bowl discord, and given Chase’s hands/lower arms were not visible on screen, by this comment we take it you mean that Chase was observed moving his shoulders in such a way that you inferred he was typing on a keyboard. How confident are you that you were able to accurately distinguish typing from other behaviors (e.g., fidgeting)? At the very least, your conclusions draw heavily on the ability to pick out typing from other behaviors, so we think this is pretty important to establish as a baseline.

Illinois Admin wrote: Tue Sep 08, 2020 1:04 pm Additionally, and perhaps most importantly, on a few occasions the sound of his mouse and keyboard were picked up by the microphone during his team's bonuses to an extent not consistent with typing his answers (he was giving his answers verbally anyway).
Given you believe this to be perhaps one of the most important bits of evidence, we think it pertinent to point out that the other editor-observer has stated elsewhere that they did not hear any typing picked up from Chase’s microphone, but rather took your word that it happened. We will also clarify for other readers that Brad observed Chase during round 6 and for the 10 questions of round 7 that were played. The other editor was present only for the 10 questions of round 7 (they did listen to the end of round 5 as well, but were not there to observe Chase and gave no indication that they suspected anything at the time). To our knowledge, this is the full scope of the “several” observers over “several” rounds.

Illinois Admin wrote: Tue Sep 08, 2020 1:04 pm 4. Additionally, a consistent pattern emerged in Chase's behavior on bonuses, where he would start the bonus with his hand on his chin, then as soon as a noun or title was read he would begin to type and exhibited wide eye saccades consistent with reading from a computer screen as light flashed across his face. Chase would then pull his hand back and "guess" the answer. This pattern was generally not followed during opponents' bonuses. Additionally, as noted above, the sound of Chase's keyboard and mouse was occasionally picked up during his bonuses when he was observed typing and reading from computer screens.

...

With these Googleable words, I went through a packet ahead of time and determined which words and phrases were most likely to be searched by someone cheating. With very few exceptions, Chase was observed beginning to type when those words and phrases were read. More notably, this behavior was mostly absent when such words and phrases had not just been read. This continued on bonuses where Chase was observed typing on nearly every one of his teams' bonuses but on nearly none of his opponents' bonuses.
These are certainly interesting observations that raise a lot of questions, and we’d love to dig into them more deeply with you. For instance, given that Chase’s explanation for his conduct was that he was alternating between the video and chat tabs in Discord, it seems plausible that some of this observed behavior could have been due to routine actions like switching to video on bonuses. We think it would be very edifying if you could share with us the packet that you annotated ahead of time. It would be quite helpful for answering questions like what the “googleable” words were that you identified, at which points in which questions Chase began showing presumed signs of typing (was it consistently after the first “googleable” word of the question, which would make the most sense? Was there variation?), and how rare the exceptions to the pattern that was “generally not” followed during the opponent’s bonuses and the “very few exceptions” on tossups are in your observations (particularly given your experiment’s sample size of either 20 toss-ups and 10 Princeton/10 non-Princeton bonuses, or 10 toss-ups and 8 Princeton/2 non-Princeton bonuses) among others. Frankly, this exercise would be most convincing to us, and would enhance our confidence in the precision of your observations, if both of the editor-observers could independently revisit and annotate the half-packet they heard to the best of their recollection, but we get that that’s probably asking a bit more than they’re willing to do.

We totally understand that you may not be interested in spending more time on the cheating allegations against Chase. Quite frankly, if the consequences were limited to Princeton A’s disqualification at LIT we, too, would be content to let the past be the past. But given that Chase’s status as a welcomed member of our quiz bowl community may be on the line, we think it critical that he be given a proper chance to defend himself and that any club or organization that is considering barring Chase from its events is able to make a determination that is informed by the best evidence possible. If that means sitting him down in a controlled environment and playing through the remaining LIT packets--which we believe could be very helpful in procuring stronger evidence for his guilt or innocence--we’re more than willing to comply. We’re also willing to comply with other requests or ideas that could provide clarity on the situation.



Though we wholeheartedly agree that the decision to disqualify Chase and his team at LIT was entirely justified given the circumstances and evidence available at the time, we simply don’t think it appropriate for us to bar Chase from competing with our team in future events until we can say with a high degree of certainty that misconduct has actually occurred. We are not convinced that the available evidence is sufficient to prove Chase cheated, particularly given that the entire case relies on circumstantial evidence put forth by two observers who were not acting independently of one another; that these observers were “primed” to view Chase’s actions with suspicion; that there remains substantial room for interpretation of the evidence; the existence of plausible alternatives that can explain, at the very least, a substantial portion of Chase’s conduct; and, most importantly, because there remains an opportunity to collect more evidence that could strengthen the case for or against him and therefore make any response from a third-party organization feel more just and appropriate. We recognize that one can reasonably disagree with us about this.

Let us be clear that we are not at all suggesting that the allegations against Chase should be dismissed outright; they absolutely should be taken seriously and warrant serious scrutiny of Chase’s behavior. To that end, you can rest assured that we will be closely monitoring Chase’s conduct in the coming months. We reiterate our willingness to take whatever measures are deemed necessary to ensure that TDs can feel comfortable with Chase competing at their events and other teams can be confident that their games against him are fair and free of external influence (including things like physically sitting in the room with him, confiscating his phone, installing and activating a keylogger on his computer, recording his screen and gameplay, requiring him to play in front of a mirror or other location where his computer screen is clearly visible, etc.. Recall that these are feasible options given that Chase currently resides with other members of the club.).

We would also point out that we place our full support behind the discussions that have been sparked by this incident about establishing clear, strictly-enforced rules to mitigate cheating at online events and developing a fair and formal procedure for addressing and adjudicating cheating allegations. We believe these discussions will help improve the state of online quiz bowl, and would encourage anyone who has thoughts or suggestions to contribute them. Additionally, nothing that we have said here or elsewhere should be construed as a direct attack on Jacob, the Duke quiz bowl team, or anyone else. Though we take issue with what we perceived in our private communications as a dismissive and uninterested response to our efforts to understand more completely what happened, we fully recognize that running LIT was by no means an easy undertaking and that expecting perfection is entirely unreasonable. We’re grateful to the Duke quiz bowl team for putting on a fun event and are looking forward to participating in future tournaments that are able to run more seamlessly thanks to LIT’s exposure of some of the technical challenges and other vulnerabilities of online quiz bowl.

Finally, we close by apologizing for this extremely lengthy response and giving our thanks to those who have read to this point. This entire situation has been incredibly taxing for us, and we imagine it is no less unpleasant or difficult for anyone else involved. We hope you can appreciate what we’ve put forward here, though we recognize that ultimately each club, organization, and member of the wider quizbowl community is best placed to determine its own best interest and that any decisions outside of our club’s own are out of our hands. If any organization interested in the circumstances of this incident would like any information from us, we welcome you to contact Eve.
Jared Lockwood

Hallsville High School 2015
Princeton University 2019, 2023
User avatar
TylerV
Wakka
Posts: 135
Joined: Sat Feb 16, 2013 7:58 pm

Re: Statement on LIT Cheating Allegation: Princeton A

Post by TylerV »

jaredlockwood wrote: Sun Sep 13, 2020 11:53 pm Though we wholeheartedly agree that the decision to disqualify Chase and his team at LIT was entirely justified given the circumstances and evidence available at the time, we simply don’t think it appropriate for us to bar Chase from competing with our team in future events until we can say with a high degree of certainty that misconduct has actually occurred.
Is your argument that the evidence is substantial enough to deserve disqualification, but not further consequences, or that the evidence was originally substantial enough but that evidence has since been questioned? This isn't meant to push any particular point I'm making, I am just genuinely interested.
jaredlockwood wrote: Sun Sep 13, 2020 11:53 pm We are not convinced that the available evidence is sufficient to prove Chase cheated, particularly given that the entire case relies on circumstantial evidence put forth by two observers who were not acting independently of one another;
I'm unsure what point you are trying to make.

From my, admittedly limited, knowledge of the situation, the timeline is as follows: An accusation is made. Observer 1 begins their investigation. Observer 1 then asks a trusted individual, Observer 2, to investigate as well. What is wrong with that exactly? With your given definition of "independent" observers, I am unsure if there exists a tournament that would be able to investigate a cheating allegation without "circumstantial evidence" collected by "independent" observers.

Furthermore, I am concerned that your wording is implying an accusation that you are not intending to make. For clarity, are you questioning the integrity of these specific observers? If so, that is an accusation far more serious than the one levied against Chase and is certainly something that should be discussed.

jaredlockwood wrote: Sun Sep 13, 2020 11:53 pm that these observers were “primed” to view Chase’s actions with suspicion;
I understand why this would be your initial reaction, but I find it to be a pretty disingenuous claim to make. As someone who has worked with all-levels of quizbowler, from all around the county, and dealt with several cheating incidents, what you've described has never been anyone's mindset going into an investigation.

In my experience, the individuals investigating such claims take their roles very seriously. In fact, it is far more often that the mindset is "I'm sure this person just caught a packet and their opponent has sour grapes" than "This person is cheating and I just need to find evidence to prove it."
jaredlockwood wrote: Sun Sep 13, 2020 11:53 pm that there remains substantial room for interpretation of the evidence; the existence of plausible alternatives that can explain, at the very least, a substantial portion of Chase’s conduct
Another honest question, has anyone unaffiliated with Princeton agreed that there is substantial room for interpretation? I'm not trying to accuse you of impropriety, but it is often easy in these cases to get tunnel-vision while attempting to defend a friend and teammate.
jaredlockwood wrote: Sun Sep 13, 2020 11:53 pm To that end, you can rest assured that we will be closely monitoring Chase’s conduct in the coming months. We reiterate our willingness to take whatever measures are deemed necessary to ensure that TDs can feel comfortable with Chase competing at their events and other teams can be confident that their games against him are fair and free of external influence (including things like physically sitting in the room with him, confiscating his phone, installing and activating a keylogger on his computer, recording his screen and gameplay, requiring him to play in front of a mirror or other location where his computer screen is clearly visible, etc.. Recall that these are feasible options given that Chase currently resides with other members of the club.).
If Chase is allowed to play additional tournaments under abnormal conditions, and performs significantly worse, will that be used as evidence that he cheated at LIT? If so, will that be substantial enough evidence to result in a ban?

I ask for two main reasons.
  • I do not believe playing under abnormal conditions can be used to provide any negative evidence. If Chase performs poorly it is easy, and more importantly fair, to claim the pressure of having someone else watch him or feeling that he had to prove himself contributed to this poor performance.
  • If such an experiment cannot produce negative evidence, it side-steps the point of such an experiment. It can only serve to either kick the can down the road indefinitely or exonerate him.
I would also like to point out that assuring opponents and TDs that he is playing fair is only one part of the issue that comes with a cheating accusation. The other is that if Chase did cheat, punitive action needs to be taken. While I fully believe a person who has cheated can regret their actions and earn their way back into the community, they should also be deprived of the activity for some period of time.

Overall, what I am trying to say here is that I am concerned that in your pursuit to defend a team member that you have made several dangerous assumptions that call into question the integrity, or at very least the competency, of at least 2 members of the community.

As I've said previously, I am not trying to accuse you or anybody on the Princeton team of malice or of acting in bad faith, I would just like clarification on my above points.
Tyler Vaughan
UW-Platteville, Rock Valley, UIUC 2014-2017
User avatar
Carlos Be
Wakka
Posts: 193
Joined: Sun Jun 25, 2017 11:34 pm

Re: Statement on LIT Cheating Allegation: Princeton A

Post by Carlos Be »

TylerV wrote: Mon Sep 14, 2020 3:02 am Another honest question, has anyone unaffiliated with Princeton agreed that there is substantial room for interpretation? I'm not trying to accuse you of impropriety, but it is often easy in these cases to get tunnel-vision while attempting to defend a friend and teammate.
I am not affiliated with Princeton and I believe there is a lot of room for interpretation. In particular, I am skeptical that it is possible to observe typing over Discord to the accuracy necessary to conclude that Chase was typing after google-able words. I am also unconvinced that eye saccades can be identified with reading text on a screen, since I have observed players making various eye movements in person. To my knowledge, there has been no demonstration of these capabilities outside of the Duke LIT mirror. In my opinion, if this sort of behavioral observation is to be used as evidence, then there should be some sort of demonstration that it is, in fact, possible to consistently observe these behaviors over Discord.
TylerV wrote: Mon Sep 14, 2020 3:02 am If such an experiment cannot produce negative evidence, it side-steps the point of such an experiment. It can only serve to either kick the can down the road indefinitely or exonerate him.
Are you suggesting that evidence is only useful if it can be used to prove Chase's guilt? If so, I find that attitude disturbing. Given the severity of the accusation, I certainly think that Chase deserves a chance to exonerate himself, even if the exonerating evidence cannot be turned into convicting evidence.
Justine French
UCLA

Packet submission must be destroyed.
User avatar
Auks Ran Ova
Forums Staff: Chief Administrator
Posts: 4136
Joined: Sun Apr 30, 2006 10:28 pm
Location: Minneapolis
Contact:

Re: Statement on LIT Cheating Allegation: Princeton A

Post by Auks Ran Ova »

Carlos Be wrote: Mon Sep 14, 2020 1:02 pm
TylerV wrote: Mon Sep 14, 2020 3:02 am If such an experiment cannot produce negative evidence, it side-steps the point of such an experiment. It can only serve to either kick the can down the road indefinitely or exonerate him.
Are you suggesting that evidence is only useful if it can be used to prove Chase's guilt? If so, I find that attitude disturbing. Given the severity of the accusation, I certainly think that Chase deserves a chance to exonerate himself, even if the exonerating evidence cannot be turned into convicting evidence.
I don't think that's the right framing--I think he's saying an experiment is only useful if it could potentially produce evidence of a player's guilt. If a positive result is exonerating but all possible negative results could be explained away, the experiment doesn't serve much of a purpose.
Rob Carson
University of Minnesota '11, MCTC '??
Member, ACF
Member, PACE
Writer and Editor, NAQT
RexSueciae
Tidus
Posts: 515
Joined: Sun Dec 29, 2013 12:24 am

Re: Statement on LIT Cheating Allegation: Princeton A

Post by RexSueciae »

Auks Ran Ova wrote: Mon Sep 14, 2020 7:01 pm
Carlos Be wrote: Mon Sep 14, 2020 1:02 pm
TylerV wrote: Mon Sep 14, 2020 3:02 am If such an experiment cannot produce negative evidence, it side-steps the point of such an experiment. It can only serve to either kick the can down the road indefinitely or exonerate him.
Are you suggesting that evidence is only useful if it can be used to prove Chase's guilt? If so, I find that attitude disturbing. Given the severity of the accusation, I certainly think that Chase deserves a chance to exonerate himself, even if the exonerating evidence cannot be turned into convicting evidence.
I don't think that's the right framing--I think he's saying an experiment is only useful if it could potentially produce evidence of a player's guilt. If a positive result is exonerating but all possible negative results could be explained away, the experiment doesn't serve much of a purpose.
What? That's still a strange position to hold -- if a positive result is exonerating, and a positive result would have been reached, but the experiment is not held, that sounds like a grave error. I'm personally skeptical of these sorts of experiments anyways -- if I were ever accused of cheating and was ordered to take part in an experiment, I'd probably say no out of spite -- but if I specifically requested some experiment which could clear my name, and it was denied, I'd be quite irritated. And unless I misunderstand, that is approximately what happened here?
Vasa Clarke [he/they]

Maggie Walker '14
Virginia '18
William and Mary '21
User avatar
Santa Claus
Wakka
Posts: 189
Joined: Fri Aug 23, 2013 10:58 pm

Re: Statement on LIT Cheating Allegation: Princeton A

Post by Santa Claus »

Carlos Be wrote: Mon Sep 14, 2020 1:02 pm I am not affiliated with Princeton and I believe there is a lot of room for interpretation. In particular, I am skeptical that it is possible to observe typing over Discord to the accuracy necessary to conclude that Chase was typing after google-able words.
I would consider hearing "the sound of [...] mouse and key board [...] picked up by the microphone" as one method of accurately determining when someone is typing, and "[having] light flash across [the] face in a manner consistent with changing tabs" as another. Nevertheless, it seems immaterial whether typing happened exactly at the moment at which aforementioned "googleable words" came up so long as it could be reasonably assessed to have occurred during tossups and friendly bonuses.
Carlos Be wrote: I am also unconvinced that eye saccades can be identified with reading text on a screen, since I have observed players making various eye movements in person.
Brad, in his post wrote: 2. Chase was also observed typing on a keyboard. When typing, Chase faced the computer screen, exhibited eye saccades consistent with reading text from a screen, and had light flash across his face repeatedly in a manner consistent with changing tabs. Additionally, and perhaps most importantly, on a few occasions the sound of his mouse and keyboard were picked up by the microphone during his team's bonuses to an extent not consistent with typing his answers (he was giving his answers verbally anyway).
Brad chose to describe his observations as "consistent with" reading text, changing tabs, etc.
I think that, by the nature of saccades and the physical layout of text in English, one can make reasonable assumptions that certain eye movements are "consistent with" reading text. While I can think of innocuous situations in which one would (appear to) be doing one or several of the mentioned actions, I struggle to think of one in which you would (appear to) do all of them at the same time, and even fewer that would bring one to (appear to) do them repeatedly and in a manner that appeared to sync up with an external stimulus. Something, something, quack quack duck.


It seems that there were some problems in communication between Duke, Princeton, and the independent observers over this last week - might I recommend that you guys arrange a time to have a Zoom call and discuss this in greater depth? Typing forum posts is all well and good but that takes forever; it'd be much faster to hash this out directly.
Kevin Wang
Arcadia High School 2015
Amherst College 2019

2018 PACE NSC Champion
2019 PACE NSC Champion
touchpack
Rikku
Posts: 394
Joined: Tue Sep 20, 2011 12:25 am

Re: Statement on LIT Cheating Allegation: Princeton A

Post by touchpack »

Auks Ran Ova wrote: Mon Sep 14, 2020 7:01 pm
Carlos Be wrote: Mon Sep 14, 2020 1:02 pm
TylerV wrote: Mon Sep 14, 2020 3:02 am If such an experiment cannot produce negative evidence, it side-steps the point of such an experiment. It can only serve to either kick the can down the road indefinitely or exonerate him.
Are you suggesting that evidence is only useful if it can be used to prove Chase's guilt? If so, I find that attitude disturbing. Given the severity of the accusation, I certainly think that Chase deserves a chance to exonerate himself, even if the exonerating evidence cannot be turned into convicting evidence.
I don't think that's the right framing--I think he's saying an experiment is only useful if it could potentially produce evidence of a player's guilt. If a positive result is exonerating but all possible negative results could be explained away, the experiment doesn't serve much of a purpose.
Let's define a "positive" as the suspected cheater performing poorly on the controlled packet reading (indicating cheating), and a "negative" as the suspected cheater performing well on the controlled packet reading (indicating genuine improvement). We can assume this test has a low false negative rate, but an appreciable false positive rate (since poor performance could be attributed to nerves, difference in the testing conditions, etc). This test is thus said to have high sensitivity but low specificity.

Another example of a test with high sensitivity but low specificity is the mammogram. Around 50% of women that get an annual mammogram for 10 years will experience at least one false positive. So why do we perform this test? Because it's a relatively easy/quick/cheap test to perform, and the negative result saves lots of time and energy--no need to have the patient go through anything more invasive or expensive like an MRI/biopsy/etc. The "controlled packet reading" test, if administered immediately, could also save lots of time and energy--a negative result means no one's name gets posted on the forums, no one has to ask the suspected cheater to hand over their personal texts/search history/etc. While it may no longer be useful in this specific case, I think it has value as a test and should be performed in the future, before things blow up publicly.
Billy Busse
Illinois '14
Full Member, ACF
Writer/Subject Editor/Set Editor, NAQT
User avatar
TylerV
Wakka
Posts: 135
Joined: Sat Feb 16, 2013 7:58 pm

Re: Statement on LIT Cheating Allegation: Princeton A

Post by TylerV »

touchpack wrote: Wed Sep 16, 2020 1:54 am
Auks Ran Ova wrote: Mon Sep 14, 2020 7:01 pm
Carlos Be wrote: Mon Sep 14, 2020 1:02 pm
TylerV wrote: Mon Sep 14, 2020 3:02 am If such an experiment cannot produce negative evidence, it side-steps the point of such an experiment. It can only serve to either kick the can down the road indefinitely or exonerate him.
Are you suggesting that evidence is only useful if it can be used to prove Chase's guilt? If so, I find that attitude disturbing. Given the severity of the accusation, I certainly think that Chase deserves a chance to exonerate himself, even if the exonerating evidence cannot be turned into convicting evidence.
I don't think that's the right framing--I think he's saying an experiment is only useful if it could potentially produce evidence of a player's guilt. If a positive result is exonerating but all possible negative results could be explained away, the experiment doesn't serve much of a purpose.
Let's define a "positive" as the suspected cheater performing poorly on the controlled packet reading (indicating cheating), and a "negative" as the suspected cheater performing well on the controlled packet reading (indicating genuine improvement). We can assume this test has a low false negative rate, but an appreciable false positive rate (since poor performance could be attributed to nerves, difference in the testing conditions, etc). This test is thus said to have high sensitivity but low specificity.

Another example of a test with high sensitivity but low specificity is the mammogram. Around 50% of women that get an annual mammogram for 10 years will experience at least one false positive. So why do we perform this test? Because it's a relatively easy/quick/cheap test to perform, and the negative result saves lots of time and energy--no need to have the patient go through anything more invasive or expensive like an MRI/biopsy/etc. The "controlled packet reading" test, if administered immediately, could also save lots of time and energy--a negative result means no one's name gets posted on the forums, no one has to ask the suspected cheater to hand over their personal texts/search history/etc. While it may no longer be useful in this specific case, I think it has value as a test and should be performed in the future, before things blow up publicly.
This is a very good point that I did not originally consider. I'm not sure how this could be executed in a timely manner in a way that would satisfy everyone, especially since the idea of playing against empty chairs is, at the very least, not 1:1, but I agree with the sentiment.
Tyler Vaughan
UW-Platteville, Rock Valley, UIUC 2014-2017
User avatar
Auks Ran Ova
Forums Staff: Chief Administrator
Posts: 4136
Joined: Sun Apr 30, 2006 10:28 pm
Location: Minneapolis
Contact:

Re: Statement on LIT Cheating Allegation: Princeton A

Post by Auks Ran Ova »

touchpack wrote: Wed Sep 16, 2020 1:54 am
Auks Ran Ova wrote: Mon Sep 14, 2020 7:01 pm
Carlos Be wrote: Mon Sep 14, 2020 1:02 pm
TylerV wrote: Mon Sep 14, 2020 3:02 am If such an experiment cannot produce negative evidence, it side-steps the point of such an experiment. It can only serve to either kick the can down the road indefinitely or exonerate him.
Are you suggesting that evidence is only useful if it can be used to prove Chase's guilt? If so, I find that attitude disturbing. Given the severity of the accusation, I certainly think that Chase deserves a chance to exonerate himself, even if the exonerating evidence cannot be turned into convicting evidence.
I don't think that's the right framing--I think he's saying an experiment is only useful if it could potentially produce evidence of a player's guilt. If a positive result is exonerating but all possible negative results could be explained away, the experiment doesn't serve much of a purpose.
Let's define a "positive" as the suspected cheater performing poorly on the controlled packet reading (indicating cheating), and a "negative" as the suspected cheater performing well on the controlled packet reading (indicating genuine improvement). We can assume this test has a low false negative rate, but an appreciable false positive rate (since poor performance could be attributed to nerves, difference in the testing conditions, etc). This test is thus said to have high sensitivity but low specificity.

Another example of a test with high sensitivity but low specificity is the mammogram. Around 50% of women that get an annual mammogram for 10 years will experience at least one false positive. So why do we perform this test? Because it's a relatively easy/quick/cheap test to perform, and the negative result saves lots of time and energy--no need to have the patient go through anything more invasive or expensive like an MRI/biopsy/etc. The "controlled packet reading" test, if administered immediately, could also save lots of time and energy--a negative result means no one's name gets posted on the forums, no one has to ask the suspected cheater to hand over their personal texts/search history/etc. While it may no longer be useful in this specific case, I think it has value as a test and should be performed in the future, before things blow up publicly.
Sure, I buy this. (also, sorry for using "positive" and "negative" in the exact opposite way they'd ordinarily be used!)
Rob Carson
University of Minnesota '11, MCTC '??
Member, ACF
Member, PACE
Writer and Editor, NAQT
Post Reply