The Quizbowl Resource Center

Okay, so I rarely post on the forums and don't want to become a central part of the discussion and fallout from the recent cheating scandal, but I felt I had to post and start a new thread on this topic since the existing ones were NAQT-specific or just an outlet to vent.

My question is, what is quizbowl as a whole doing to improve question security and the integrity of tournament results? Shortly after the Josh Alman revelation, I dug up the thread from Penn-ance at MIT, specifically viewtopic.php?f=21&t=13594&p=250886&hil ... an#p250775. I think his statline at that tournament, considered in light of the eventual revelation, would suggest he cheated there as well. It is not implausible that he had access to the set before the tournament since the mirror was hosted at MIT.

How exactly did Penn send their packets to the various hosts? I am assuming by emailing them to a club/institutional account. Since (according to the Boston Globe article) Josh Alman was club president at MIT, he would have had access to that email account, and given his tendencies to cheat, the temptation to look at the questions he planned to play on would have been incredibly high.

I apologize if I am wrong here and he accessed the questions by other, nefarious means (hacking into another MIT person's email? breaking into a room with printed packets?). However, this does not obviate the need for caution when transmitting packets, and from while I do not wish to implicate foul play on the part of any other MIT player, there is hardly a guarantee that other teams won't be more sloppy with their procedures/ethical standards (e.g. Cam from Harvey Mudd/Claremont Colleges).

What I intend to say is this: saying "NAQT's website sucks" is not going to solve the problem of question security. Unless other fundamental changes are made to the way information is transmitted on the quizbowl circuit, I am afraid the opportunity will always exist for less-scrupulous persons to take advantage of.

Yeah, a similar thing happened at SCT a few years ago when the UGA club treasurer who was playing the tournament had access to the club email account that the packets were sent to. Tournament sets should, whenever possible, be sent to the TD's personal address to prevent this sort of thing from happening.

Not many people remember this, but the cheating incident that occurred at Harvey Mudd years ago was due to a guy who was dating the Scripps College club president when the latter institution hosted ACF Regionals. We can and should take a lot of precautions, but at some point, unless you're doing nationals and control all the questions all the time, you'll be sending your questions to someone and trusting them to keep them secure.

grapesmoker wrote:Not many people remember this, but the cheating incident that occurred at Harvey Mudd years ago was due to a guy who was dating the Scripps College club president when the latter institution hosted ACF Regionals. We can and should take a lot of precautions, but at some point, unless you're doing nationals and control all the questions all the time, you'll be sending your questions to someone and trusting them to keep them secure.

Can there not be a way in which questions are held on a central server at the main site (the college doing the housewrite, or NAQT/ACF), and password-protected, allowing only the TD to access them. Somehow, the general practice of sending sets by email or on a flash drive, with passwords available to anyone with access to the email account or in charge of printing, and expecting the club leaders to exercise control over who sees what, seems to be a less secure alternative.

EDIT: Of course, this is assuming that NAQT/ACF/host school's servers themselves are secure, but in addition to other security patches to the NAQT website, that can presumably be taken care of? It would certainly allow players to be reasonably certain that no one is cheating at NAQT or ACF tournaments, unlike both the Cam and Tim Baughey cases, where the players' girlfriends had access to the set.

If he did simply get these questions from the club account, then that's an issue that needs to be addressed. If not, then we may need to ask whether he had an (perhaps unknowing?) accomplice.

On the broader point about club accounts, I would propose a policy of never sending questions to an account that more that one person can access. If you're not sending questions directly to a tournament director's personal email, then you're basically just hoping everyone is competing in "good faith." Not only can people like Josh Alman simply read an email in an account he has the login information for, but when multiple people can access the account then that multiplies the chances that somebody divulges information to the wrong person.

Hosting the packets and sending a password has almost the exact same security risk as just sending the packets. If somebody can access the e-mail to get the packets, they can access the e-mail to get the password, and then get the packets.

The only benefit is possibly some sort of trace or IP tracking or something, but that may not be true of all methods of storing packets centrally.

Also it adds another thing that could go wrong, like if the writer's server goes down.

I have attended tournaments where each individual packet was password-protected, though I can't say if it affected the flow of the tournament or not.

1971 Hamilton Tiger-Cats season wrote:

grapesmoker wrote:Not many people remember this, but the cheating incident that occurred at Harvey Mudd years ago was due to a guy who was dating the Scripps College club president when the latter institution hosted ACF Regionals. We can and should take a lot of precautions, but at some point, unless you're doing nationals and control all the questions all the time, you'll be sending your questions to someone and trusting them to keep them secure.

Can there not be a way in which questions are held on a central server at the main site (the college doing the housewrite, or NAQT/ACF), and password-protected, allowing only the TD to access them. Somehow, the general practice of sending sets by email or on a flash drive, with passwords available to anyone with access to the email account or in charge of printing, and expecting the club leaders to exercise control over who sees what, seems to be a less secure alternative.

EDIT: Of course, this is assuming that NAQT/ACF/host school's servers themselves are secure, but in addition to other security patches to the NAQT website, that can presumably be taken care of? It would certainly allow players to be reasonably certain that no one is cheating at NAQT or ACF tournaments, unlike both the Cam and Tim Baughey cases, where the players' girlfriends had access to the set.

We(HSAPQ) actually created such a distribution system that logs downloads and sets limits on how long a person has to retrieve the packets. We haven't actually used the system, but we might in the future.

I can't speak about the question security at the MIT Penn-ance mirror in particular but at other MIT tournaments that I've staffed each packet has been password protected.

tiwonge wrote:Hosting the packets and sending a password has almost the exact same security risk as just sending the packets. If somebody can access the e-mail to get the packets, they can access the e-mail to get the password, and then get the packets.

The only benefit is possibly some sort of trace or IP tracking or something, but that may not be true of all methods of storing packets centrally.

How about having a setup where the TD only gets access to the packets (hosted centrally) at 6am or later on the morning of the tournament, whether by sending the password(s) then or another means of preventing access to the server earlier. No one really needs to have packets (password-protected or otherwise) in hand a day or two before a tournament. It seems to me that late delivery of packets would prevent cheating where someone decides to play on a house team after initially committing to staffing, by which time they may have had access to packets, and for all cases where a person with access to the club email chooses to play instead of staffing.

Of course, the packets themselves could simply be emailed on the morning of the tournament, but I am still tempted to believe that central hosting of packets will afford an added layer of security, and that means of sending passwords/access information to TDs other than club emails should be used (e.g. having TDs set up a special account to access packets on the central server, instead of passwords being given out via electronic means).

The Toad to Wigan Pier wrote:We(HSAPQ) actually created such a distribution system that logs downloads and sets limits on how long a person has to retrieve the packets. We haven't actually used the system, but we might in the future.

I guess something like that would work, especially if access can be restricted to TDs who set up an account at HSAPQ for the purpose, and provide specific information so you can verify their identity.

Running a paperless tournament, if possible, seems to solve many of these problems. The questions can be sent to the TD who emails it out to the readers right before the tournament.

Marble-faced Bristle Tyrant wrote:Also it adds another thing that could go wrong, like if the writer's server goes down.

I have attended tournaments where each individual packet was password-protected, though I can't say if it affected the flow of the tournament or not.

Yes, there may be server issues, but I would hope that there would be some backup plan, say, the head editor keeps a copy of the set on his/her own computer and emails them as a last resort, or even some antiquated method such as sending packets by fax.

mtimmons wrote:I can't speak about the question security at the MIT Penn-ance mirror in particular but at other MIT tournaments that I've staffed each packet has been password protected.

I staffed Penn-ance at the main site and packets were password-protected (also to answer Farrah, mostly this does not affect the flow of the tournament). However, packets would also have been password-protected at MIT, and this obviously did not stop Josh from gaining access. As Nick Conder mentioned, he may have had an unwitting accomplice, though I do not wish to implicate anyone.

1971 Hamilton Tiger-Cats season wrote:How about having a setup where the TD only gets access to the packets (hosted centrally) at 6am or later on the morning of the tournament, whether by sending the password(s) then or another means of preventing access to the server earlier. No one really needs to have packets (password-protected or otherwise) in hand a day or two before a tournament.

This seems like a horrible idea.

Don't send packets to a club account that someone playing the tournament might have access to. I don't think there is any way that using a central question server is going to have enough benefits to outweigh the potential issues. At some point, we have to be able to trust TDs to handle question security themselves.

Communi-Bear Silo State wrote:

1971 Hamilton Tiger-Cats season wrote:How about having a setup where the TD only gets access to the packets (hosted centrally) at 6am or later on the morning of the tournament, whether by sending the password(s) then or another means of preventing access to the server earlier. No one really needs to have packets (password-protected or otherwise) in hand a day or two before a tournament.

This seems like a horrible idea.

Yeah, I've been burned too many times by packets coming in the night before the tournament to trust a system where that's the norm. NIU hosts tournaments in a building without accessible WiFi -- flash drives are just something we have to live with, and we can't be the last school left in that position.

Writers: E-mail the set to an e-mail address confirmed to have only one user -- the TD of the tournament. This is the weakest link, as Amit, Dmitri, and UGA will attest.
TD: Password protect every round individually, and don't put the set on a flash drive until the night before/morning of. Send out passwords via e-mail or on the paper scoresheet for the next round.

So this is the way it worked. On the morning of the 20th, I sent the packets to all of the mirrors running that day. For MIT, I sent them to both Stephen Eltinge (at his gmail) and Olivia Murton (at her MIT email). The email contained a list of passwords in its body, and both a locked and unlocked zip file of packets. Immediately after that, I discovered a mistake in a question. I sent another email to the same people alerting them of the mistake (in the body of the email), with a new attachment containing an unlocked, corrected version of the packets.

It seems most likely to me that Josh managed to somehow sneak an unlocked copy of the packets from a moderator's computer.

in on these shenanigans wrote:Amit

For that matter, if someone involved with the tournament changes their email address while the event is still in production, confirm the change via some other line of communication-- the old address, hsqb profiles, AIM, whatever.

There is an ideal situation in terms of security. In that ideal situation, the Head Editor (and the Head Editor's assistants who played a role in finalizing the packets) is the only one with full access to the questions. Everybody else, including the TDs at various sites, only gets access to each round when it is just about time to start that round. Because there is a risk of emails sometimes getting held up by servers, that probably means that the moderators get the password-protected rounds the morning of the tournament and then get the passwords throughout the day. (If there is an email problem, passwords can be exchanged over the phone, whereas packets cannot be.) If TDs have a reason to get password-protected questions without the passwords the day before, that can be done. The TDs give the moderator email addresses to the Head Editor, and the Head Editor, or somebody they trust, spends the day sending out passwords to individual moderators, communicating with the TDs so that the passwords are getting sent out at a reasonable time.

If you have a situation like a building with bad wireless, then the TD can call the Head Editor (or vice versa) every half hour for passwords, and you tell moderators the passwords each round when they turn in their scoresheets. If you have a moderator or two who insist on paper copies, then the Head Editor decides how to handle that situation, though it does lessen security.

SCT and ACF Regs in particular, and other tournaments that use mirrors in general, probably should be headed more in this direction. Something like this more practical now than it would have been a few years ago.

NAQT's policy is (as many of you know) to prefer paper tournaments, both to avoid questions about the availability of tournament-site wireless access and because we believe paper packets help prevent "moderator reads the wrong round" problems.

We do normally email out packets (to a specific td, not a club email- as has been alluded to in this thread, we learned our lesson about that a few years ago) shortly before SCT (and quite often for the first use of a particular high school set). However, in an ideal world our policy about emailing sets would be "Never. If you're the TD, expect NAQT to ship you a box containing a physical dead-tree question set."

Leucippe and Clitophon wrote:There is an ideal situation in terms of security. In that ideal situation, the Head Editor (and the Head Editor's assistants who played a role in finalizing the packets) is the only one with full access to the questions. Everybody else, including the TDs at various sites, only gets access to each round when it is just about time to start that round. Because there is a risk of emails sometimes getting held up by servers, that probably means that the moderators get the password-protected rounds the morning of the tournament and then get the passwords throughout the day. (If there is an email problem, passwords can be exchanged over the phone, whereas packets cannot be.) If TDs have a reason to get password-protected questions without the passwords the day before, that can be done. The TDs give the moderator email addresses to the Head Editor, and the Head Editor, or somebody they trust, spends the day sending out passwords to individual moderators, communicating with the TDs so that the passwords are getting sent out at a reasonable time.

If you have a situation like a building with bad wireless, then the TD can call the Head Editor (or vice versa) every half hour for passwords, and you tell moderators the passwords each round when they turn in their scoresheets. If you have a moderator or two who insist on paper copies, then the Head Editor decides how to handle that situation, though it does lessen security.

SCT and ACF Regs in particular, and other tournaments that use mirrors in general, probably should be headed more in this direction. Something like this more practical now than it would have been a few years ago.

This seems to me to be a good practical solution. In general I find the idea of emailing passwords in advance a dubious one, since emails can be hacked/accidentally forwarded/printed and left lying around etc. Sending moderators the passwords just before the previous round is completed, or telephoning the TD to notify them of the password just seems more secure by virtue of ensuring that nobody, including the TD, can open the packets until the round they are supposed to be used in.

Of course, this has the disadvantage that each password would have to be transmitted four or more times in order to account for tournaments in different time zones.

The Quest for the Historical Mukherjesus wrote:I sent another email to the same people alerting them of the mistake (in the body of the email), with a new attachment containing an unlocked, corrected version of the packets.

Was there a reason that the entire, unprotected set was sent, as opposed to just the packet in which there was an error? Two unlocked sets may well be more vulnerable than one or none at all, and I imagine the TDs downloaded the locked set to their computers the first time round, but then downloaded the entire unlocked set only when it was sent the second time.

bt_green_warbler wrote:NAQT's policy is (as many of you know) to prefer paper tournaments, both to avoid questions about the availability of tournament-site wireless access and because we believe paper packets help prevent "moderator reads the wrong round" problems.

We do normally email out packets (to a specific td, not a club email- as has been alluded to in this thread, we learned our lesson about that a few years ago) shortly before SCT (and quite often for the first use of a particular high school set). However, in an ideal world our policy about emailing sets would be "Never. If you're the TD, expect NAQT to ship you a box containing a physical dead-tree question set."

I haven't waded into this morass and don't have much desire to do so, but this is what NHBB has done this year in most cases and it has worked well. Priority mail flat rate boxes are great for this; it saves us $ money on printing, it saves the TD time, and seems to me to be a reasonably secure way of handling it. Any thoughts from anyone regarding moving in the direction of not allowing packets to be distributed post-tournament to participating teams if mirrors are still to occur? Seems like a good idea to me.

nationalhistorybeeandbowl wrote:Any thoughts from anyone regarding moving in the direction of not allowing packets to be distributed post-tournament to participating teams if mirrors are still to occur? Seems like a good idea to me.

It would be nice, especially for new circuits, to be able to give teams a copy of the questions. It's good to practice on for the future*, and I think it's fun for everybody (or gratifying for the coach) to have immediate access to the questions they didn't see during the tournament. In far-flung areas (like Idaho) and with paper copies, the question security isn't too big a concern.

NAQT has an advantage of more tournaments written, so it's a bit easier to release copies to participating schools.

*Edit: I mean, I know there are questions online for practice, but a lot of times new teams aren't aware of it, don't bother looking for it, or don't realize the importance of practice and don't take the trouble to download them online. I saw this with the local Science Bowl earlier this month. A lot of teams played and were completely unfamiliar with the format and the type of questions.

NAQT is fine with distributing paper packets to teams at the conclusion of a tournament. (Notably: it is *much* more effort on the part of potential cheaters to mail physical copies of sets around the country than it is to forward an email.)

One thing I kind of get paranoid about when editing and rewriting NAQT questions (though this could apply to any organization writing a set, which is why I'm putting it here) is discussing question content via email. Sometimes it isn't worth it to recode and reupload questions back and forth with the relevant writer or editor; email is much more efficient when you aren't sure what a rewrite's note meant. Aside from general email privacy measures, writers should be careful about leaving their email open in a quizbowl setting.

Regarding this specific discussion Josh/MIT/Penn-ance mirror discussion.
I moderated the MIT Penn-ance mirror at MIT, based on my email records, packets were distributed to the moderators at 8:50 am in a .zip. The passwords were only distributed to the moderators round-by-round on the scoresheet for the corresponding round.

In light of the recent developments regarding Josh's performance, MIT has decided to vacate its Penn-ance win. While we don't currently have any definitive proof of misconduct on Josh's part, we think the situation with NAQT raises enough questions about Josh's playing at Penn-ance to justify vacating the win. We'll be making another announcement with more details within a few days. Congratulations to Yale A on its retroactive win!

Whoa, whoa. We had the same record as Yale! We beat them but lost to Brown.

(Perhaps we can split this off, but) As I recall, when we thought we both finished 2nd, it had been a long day, we asked whether you guys wanted to play a match for 2nd, everyone wanted to leave, and Yale was decided to be in 2nd place due to PPG. You obviously wouldn't have declined to play if this were actually a final, so I'm fine with declaring this a first-place tie if that's what you guys want.

RyuAqua wrote:(Perhaps we can split this off, but) As I recall, when we thought we both finished 2nd, it had been a long day, we asked whether you guys wanted to play a match for 2nd, everyone wanted to leave, and Yale was decided to be in 2nd place due to PPG. You obviously wouldn't have declined to play if this were actually a final, so I'm fine with declaring this a first-place tie if that's what you guys want.

Sorry, I was just joking in my last post. I guess we would have played a final but yeah, I don't actually care if you guys claim first.

I also wanted to gloat about the last time I might ever beat you at quizbowl.

Marble-faced Bristle Tyrant wrote:One thing I kind of get paranoid about when editing and rewriting NAQT questions (though this could apply to any organization writing a set, which is why I'm putting it here) is discussing question content via email.

Actually I just realized that worrying about merely discussing questions is kind of silly when we already get files of whole questions via email. Concerns about email privacy still apply, though.

Marble-faced Bristle Tyrant wrote:

Marble-faced Bristle Tyrant wrote:One thing I kind of get paranoid about when editing and rewriting NAQT questions (though this could apply to any organization writing a set, which is why I'm putting it here) is discussing question content via email.

Actually I just realized that worrying about merely discussing questions is kind of silly when we already get files of whole questions via email. Concerns about email privacy still apply, though.

Changing this (both sending and discussing questions by email) will be under discussion this summer.