The Quizbowl Resource Center

naqt.com wrote:The initial results submitted from the St. Mark's School of Texas Invitational (held on February 25) documented a statistically anomalous performance by the Windhaven Park E team given its members' previous tournament experience.

After reviewing comments from coaches, moderators, and players involved with the matches and soliciting statements from the Windhaven Park E team and coach, NAQT extended an offer to the team to validate the results by demonstrating a similar proficiency at quiz bowl under the controlled circumstances of a Skype session. The team declined this offer.

As a result of that decision, NAQT chose not to accept the submitted results and has instead vacated all wins by Windhaven Park teams at the tournament, rescinded invitations to the HSNCT earned at earlier events, and banned two of the players on Windhaven Park E from further competition at NAQT events. In addition, NAQT has issued nationals invitations to teams at the tournament that had a plausible chance to earn them if their games against Windhaven Park teams had gone the other way.

Despite the exceptional nature of this action, NAQT does not consider performances by Windhaven Park students at other tournaments or by other Windhaven Park players at the St. Mark's School of Texas Invitational to be in any way questionable.

complete statement

Has NAQT, TQBA, or St. John's informed anyone in the Plano West faculty or administration about this incident? This is a somewhat ambiguous case because the students were not representing the school, but they were allowed to play as a team because they all attend the school.

Also, has anyone determined whether access to the questions was gained through the tournament site, by question sharing across states, or through the NAQT website?

The TQBA announcement seems a bit incongruent with NAQT's. Was the alleged actions just due to the two students, or was it more systemic?

Edit: also, this happens again in February.

TQBA does not allege a systemic issue.

Big Y wrote:Has NAQT, TQBA, or St. John's informed anyone in the Plano West faculty or administration about this incident? This is a somewhat ambiguous case because the students were not representing the school, but they were allowed to play as a team because they all attend the school.

NAQT has not contacted the school's administration.

Big Y wrote:Also, has anyone determined whether access to the questions was gained through the tournament site, by question sharing across states, or through the NAQT website?

We don't know the precise details of what happened in this case. (Note that we have not conclusively determined that there was prior access, which is why we were willing to offer the option of cross-checking the tournament performance with the results from a Skype session.) At this time we have no reason to believe that this incident resulted from inappropriate access to the NAQT website.

Stephen: Why is the entire Windhaven Park program banned from TQBA? Why is their coach specifically banned as well? I know both are stated as indefinitely, in contrast with two players' permanent bans, so there is some differentiation there, but I'm wondering why there is a lack of a definite length.

AKKOLADE wrote:The TQBA announcement seems a bit incongruent with NAQT's. Was the alleged actions just due to the two students, or was it more systemic?

Note that the most serious sanctions here are in fact limited to two players:

naqt.com wrote:banned two of the players on Windhaven Park E from further competition at NAQT events.

TQBA is an independent organization and has issued its own statement about the events that took place at this tournament.

Just wanted to say, NAQT has been very helpful in bringing justice to Ethan and Basel. I spent many hours organizing the tournament, so I am very pleased that NAQT has taken this investigation as seriously as we at St. Mark's have. They have been very professional, thorough, and committed to getting to the bottom of the issue, and for that I am very thankful.

bird bird bird bird bird wrote:

naqt.com wrote:The initial results submitted from the St. Mark's School of Texas Invitational (held on February 25) documented a statistically anomalous performance by the Windhaven Park E team given its members' previous tournament experience.

After reviewing comments from coaches, moderators, and players involved with the matches and soliciting statements from the Windhaven Park E team and coach, NAQT extended an offer to the team to validate the results by demonstrating a similar proficiency at quiz bowl under the controlled circumstances of a Skype session. The team declined this offer.

As a result of that decision, NAQT chose not to accept the submitted results and has instead vacated all wins by Windhaven Park teams at the tournament, rescinded invitations to the HSNCT earned at earlier events, and banned two of the players on Windhaven Park E from further competition at NAQT events. In addition, NAQT has issued nationals invitations to teams at the tournament that had a plausible chance to earn them if their games against Windhaven Park teams had gone the other way.

Despite the exceptional nature of this action, NAQT does not consider performances by Windhaven Park students at other tournaments or by other Windhaven Park players at the St. Mark's School of Texas Invitational to be in any way questionable.

complete statement

Could someone please elaborate on "statistically anomalous"? I've seen the results, and if it's simply the fact that these two players scored at abnormally high levels throughout the tournament, that's a problem. It seems wrong that statistical anomaly and failure to corroborate tournament results over Skype establish grounds for vacating an entire school's wins for an entire tournament, but at the same time, I feel like NAQT is euphemizing a deeper problem. Clarification would be appreciated.

dhumphreys17 wrote:
Could someone please elaborate on "statistically anomalous"? I've seen the results, and if it's simply the fact that these two players scored at abnormally high levels throughout the tournament, that's a problem. It seems wrong that statistical anomaly and failure to corroborate tournament results over Skype establish grounds for vacating an entire school's wins for an entire tournament, but at the same time, I feel like NAQT is euphemizing a deeper problem. Clarification would be appreciated.

I totally agree with this. It doesn't really seem right to penalize an entire program for an action that appears to be committed by only two members for which no absolutely conclusive evidence has been found. I personally believe that the team should be allowed to continue to compete until the is evidence that the players cheated, e.g. those two players regress back to the level of past performance, though I do understand why one might impose these sanctions upon the program. If this is the only issue, I would like to see these sanctions lifted. If this is not it, I beg you for a better explanation to this.

I've been in this storm for weeks now - and I'm sick of the entire thing.

I think it's safe to say Cistercian was one of the aggrieved parties in this
whole mess, but, HEY, at a certain point, you either move on, and practice more,
and get better --- or wallow in a trough of "what-might-have-been's" . . . and I don't
have time for that.

More info? That would be nice.
Will it happen? Probably not.
Move on!

dhumphreys17 wrote:Could someone please elaborate on "statistically anomalous"? I've seen the results, and if it's simply the fact that these two players scored at abnormally high levels throughout the tournament, that's a problem. It seems wrong that statistical anomaly and failure to corroborate tournament results over Skype establish grounds for vacating an entire school's wins for an entire tournament, but at the same time, I feel like NAQT is euphemizing a deeper problem. Clarification would be appreciated.

For the record: both NAQT's members and staff on-site at the tournament considered the statistical record in this case to be not just "abnormally high," but so far out of the ordinary that it indicated some sort of advance access to question content.

bird bird bird bird bird wrote:

dhumphreys17 wrote:Could someone please elaborate on "statistically anomalous"? I've seen the results, and if it's simply the fact that these two players scored at abnormally high levels throughout the tournament, that's a problem. It seems wrong that statistical anomaly and failure to corroborate tournament results over Skype establish grounds for vacating an entire school's wins for an entire tournament, but at the same time, I feel like NAQT is euphemizing a deeper problem. Clarification would be appreciated.

For the record: both NAQT's members and staff on-site at the tournament considered the statistical record in this case to be not just "abnormally high," but so far out of the ordinary that it indicated some sort of advance access to question content.

Has it been determined how that happened and what steps can be taken to prevent such "advance access" from becoming more pandemic? I know that here in Alabama, we are somewhat concerned about the security of our State Championship Tournament coming up on April 7....

quizbowllee wrote:

bird bird bird bird bird wrote:

dhumphreys17 wrote:Could someone please elaborate on "statistically anomalous"? I've seen the results, and if it's simply the fact that these two players scored at abnormally high levels throughout the tournament, that's a problem. It seems wrong that statistical anomaly and failure to corroborate tournament results over Skype establish grounds for vacating an entire school's wins for an entire tournament, but at the same time, I feel like NAQT is euphemizing a deeper problem. Clarification would be appreciated.

For the record: both NAQT's members and staff on-site at the tournament considered the statistical record in this case to be not just "abnormally high," but so far out of the ordinary that it indicated some sort of advance access to question content.

Has it been determined how that happened and what steps can be taken to prevent such "advance access" from becoming more pandemic?

We do not know the details of what happened in the weeks leading up to this tournament.

quizbowllee wrote:I know that here in Alabama, we are somewhat concerned about the security of our State Championship Tournament coming up on April 7....

Lee, I'll contact you privately about this.

Oh wow, another scandal in February.

In my opinion, the most important thing is to try to figure out how the tournament's integrity was ruined - obviously it's beyond NAQT's direct control if a student snuck through a window to get access to the questions, but if there is a writer who is untrustworthy or one of the students has inappropriately accessed NAQT's administrative website, NAQT should be seriously concerned -- which I imagine NAQT is.

I also think that NAQT should try to explain the "statistical" argument here a bit better. Don't get me wrong, if anyone put up these numbers at my tournament, I would immediately suspect that they were cheating, but to many people who do not know any better it may just look like "hey dude, this guy had a really good tournament, and you (wrongly, perhaps) said he cheated." And to add my two cents: I don't think it looks good to offer a Skype tournament as a way of clearing someone's name -- while it's immediately clear to me that results should be reproducable over a Skype session, I suspect that many people wouldn't be convinced - I can see someone failing to reproduce their results over Skype weaseling their way out of it* with statements like "I wasn't being as aggressive since I was playing against no one, etc." And it just looks tacky to offer something to the extent of "put up 80ppg or more on this Skype packet and we'll clear you." Even if the players had a Skype session and had met some threshold, was NAQT really going to clear them on that basis alone?

*to the general public or even a court of law.

Another question I have: what data was used to determine "statistically anomalous"? Because I just went through a bunch of TQBA tournament results and saw no one who ever played for Windhaven Park named Basel Cheema, though I might have missed something. Or did NAQT remove them completely from the site?

EDIT: I previously named Ethan McBride but after realizing that the school name was actually Plano West, I found info in him. My bad.

NAQT probably has to be careful about what they say, but people interested in whether or not a statistical argument has merit can do some research.

I'll add that, while it's easy to point out that NAQT's and TQBA's solution is not a perfect solution that solves every problem real and imagined, I would be interested in hearing from anybody who can suggest how this situation could have been handled better. Obviously, most of us do not know all of the details, but it does seem like we have a decent overall sense of what the primary evidence is. As somebody who oversees a lot of tournaments, I would love to know if there is a better approach than what has been done here.

I think NAQT is a great organization!
They have worked with me on many issues - sometimes we agree; at other times, not!
But always with respect and attention -- or so I thought.

Having said that: isn't there an inherent vulnerability when tournament packets
are used in various parts of the country? -- and not at the same exact time and date?

Doesn't that admit of the possibility of 'sharing' what should NEVER be 'shared'?
I don't know.

My real fear is not the case at hand -- "it" was so obvious that the matter was
identified amazingly fast, and then dealt with. I was there, folks; they handled
it pretty well, IMHO!

Rather, my real fear is that the "good team" or even the "really good team" needs
only 'a little help from one's friends' to push them over the line to victory.

How in the world would anyone be able to statistically analyze that sort of "micro-advantage"
and unambiguously identify it? People smarter than I might be able to help here.

In closing, I only point this out because Cistercian has a long record of our students
enjoying this wonderful activity - and many people work selflessly long hours to make
tournaments run as smoothly as they do - both here in Texas and nationally.

THEY, too, deserve to have their efforts respected, not circumvented by a few who
think winning is the only thing
- it ain't!