Questions About NAQT Packet Usage

Anything that's on topic but doesn't fit elsewhere, including related events that might be of interest to quizbowl players.
Post Reply
User avatar
Cheynem
Sin
Posts: 6836
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Questions About NAQT Packet Usage

Post by Cheynem »

Today, TJ's tournament ran into an issue where a player participated in the tournament despite having purchased the question set from NAQT. This generated a number of questions on Discord, and I think it's best to collect all the questions and separate them from this specific tournament.

First, the set in question, IS-193A, was used going back to around February 2020.

https://www.naqt.com/schedule/assigned- ... et_id=3260 As you can see, it was booked to use this weekend and is still being booked to use at some future events. This means the set kind of "straddles" the 2019-2020 and 2020-2021 seasons (which makes some sense, as the set was written about February 2020).

IS-193A is currently available to purchase on NAQT's website. I am not sure when it was first opened up for purchase.

(I have been told that similar things happened for sets in previous years, but I'll just focus on this for now.)

That opens up a number of questions:

1. Was this an intentional decision by NAQT to actively use IS-193A for tournaments even though it was available for purchase? (By this, I mean this was not the result of the tournament inadvertently being included in the 2019-2020 clump of tournaments added for purchase even though it was still being mirrored).

[I am assuming if it was not intentional, then the rest of the questions are moot. I will assume that if the set was not meant to be available for purchase, the fix going forward would be to not sell it and change the wording on the packet to indicate that it isn't clear until the end of the following year]

2. If this was intentional, then what can be done to improve tournament situations going forward and to avoid repeats of this weekend? It seems that just eyeballing it, some obvious fixes include--changing the wording on the packet to indicate that it isn't clear until the end of the following year, finding a way to link the people downloading the set from NAQT to online rosters for tournaments, and giving clear warnings to people purchasing, potential players, and TD's that sets for purchase might still be being mirrored in the future at tournaments.

I think it might be best for NAQT to respond first before people throw in their opinions (after all, if it was an inadvertent oversight to sell the set while it was still being used, there's no point in really throwing out lots of discussion on the subject).

I'm posting this because I think these are good questions to be asked. I'm not accusing anyone of any wrongdoing or malfeasance--we all want good quizbowl and we all want good tournaments. I'm confident that we can work out a good solution or fix to this issue so that future tournaments won't have any such confusion or problems.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
CPiGuy
Yuna
Posts: 913
Joined: Wed Nov 16, 2016 8:19 pm
Location: Ames, Iowa

Re: Questions About NAQT Packet Usage

Post by CPiGuy »

An important additional note: Further discussion of this incident in the Discord noted that all NAQT sets for a given school year are marked as being clear on July 1st following that school year -- nobody in the discussion had a copy of IS-193a but it was noted that one student's copy of IS-184a (which they bought for practice) said it was clear for discussion after July 1st, 2019, but several hosts were allowed to mirror it in the fall of 2019 (after the student bought it). So, this is not the first time an NAQT set has been available for purchase and clear for discussion but still used for a tournament.

In my opinion, it is unacceptable for NAQT to sell hosts a set which is available for purchase and which can be freely discussed in public spaces without, at the very least, making the hosts explicitly clear of this. They probably should not do it at all. It is, in my opinion, an implicit condition of the mirroring arrangement that, in exchange for money, the set producers are providing the hosts a set that is not clear -- otherwise hosts would just use sets off the packet archive for free!

It's quite disappointing that an organization that takes question security so seriously in almost every other aspect (and generally does a very good job of it -- keeping questions secure when they're played by thousands of high schoolers every year is not easy!) could suffer such a major institutional failure. I hope NAQT can speak as to what they will do to ensure that it does not happen again (including to the four other tournaments currently scheduled to use IS-193a!), in order to maintain the competitive integrity of tournaments using NAQT's questions (which I think we all want to see maintained).

If Mike's suggestion that the "clear date" for sets released late in the year be moved forward is adopted, I would encourage NAQT to adopt a public listing of which sets are clear / available for public purchase on their website to avoid confusion -- right now all non-nationals sets used in a given year are cleared at the same time; if this stops being the case it would probably be good for NAQT to have an easily-referrable list so that the community can avoid screwing up.

Like Mike, I don't want to accuse NAQT of any sort of malfeasance or intentional wrongdoing. But, this is a serious problem that has apparently persisted for multiple years, and it's important that NAQT fix it so that we can continue to have the trust in their tournaments that they've earned through the years.
Conor Thompson (he/him)
Bangor High School '16
University of Michigan '20
Iowa State University '25
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 5674
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: Questions About NAQT Packet Usage

Post by Important Bird Area »

We're in the process of investigating what happened here, and we'll have more to say about the details in the next few days.
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 5674
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: Questions About NAQT Packet Usage

Post by Important Bird Area »

NAQT's long-standing practice has been that some of our question sets are made available for tournament hosting in the competition year following the year in which they were produced. (For example, it is common for tournament hosts to use the previous spring's SSNCT set at events in the fall and winter whose audience is primarily large schools.)

Our policy for hosting events using older question sets has been as follows:

-We notify the tournament host at the time of the hosting request that the set was completed in the previous year. (It's important that the host agree to this for reasons beyond security- for instance, the presence of time-sensitive questions in categories like current events, sports, and popular culture.)

-NAQT prohibits the sale of practice materials to teams that might potentially attend a tournament in their region using the same question set. Our usual practice is to suggest that the order placer substitute a different set of equivalent difficulty that poses no possible security issue. Obviously, in the case under discussion from Saturday's event at Thomas Jefferson, our usual checks failed due to a clerical error, and we inadvertently sold practice material to players who would end up playing the very same set at a future tournament. This was a mistake with a direct impact on the integrity of competition that makes quiz bowl possible, and we would once again like to apologize to the community for our error.

NAQT has investigated what went wrong in this case, and we will be considering potential changes to our policies and safeguards going forward.

We welcome questions and comments from the community about our policies for tournament hosting and practice material sales, either in this thread, on social media, or by email ([email protected]).
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
Post Reply