HSNCT and its Problems

Dormant threads from the high school sections are preserved here.
Locked
User avatar
etotheipi
Lulu
Posts: 78
Joined: Thu Jun 13, 2019 9:13 pm

HSNCT and its Problems

Post by etotheipi »

This post is long overdue.

I will preface this post by noting that, while it was written by me and me only, and no one else should be flamed for it, the sentiments in it are not at all unique to me. Most, if not all, of the criticisms here have been levied at the HSNCT and NAQT in general by many elite players from elite teams (I'll define "elite" as "having a reasonable chance to make HSNCT playoffs") on the hsquizbowl discord server. I have made an effort to avoid listing reasons why I in particular don't like HSNCT/NAQT, and instead to attempt to explain why much of the elite high school quizbowl community holds such negative views of both. Recognizing that elite high school teams are not the target audience of most of NAQT's sets (in particular IS and IS-A sets), I have made an effort to restrict my criticisms to just HSNCT-adjacent topics.

This post will be divided into four overarching criticisms of HSNCT, specifically this year's HSNCT. I will try to provide ways NAQT can address these criticisms, whether now or for next year or maybe beyond that.

1. Cheating

I include this first because it is the most obvious problem with this year's HSNCT, as well as the easiest one to fix.

It's clear that cheating (in the form of using a search engine to illicitly obtain information during a game, or "biking" as it is sometimes called) is a problem that faces online quizbowl, and a nationals at which there is cheating, or at which there is even the possibility of cheating, is not particularly legitimate. Unfortunately, NAQT does not appear to be doing anything to prevent cheating other than requiring a camera, which is known to be an ineffective measure.

There have been multiple cases of people cheating on camera this year, sometimes even while showing one of their hands on camera. While many of these people have been found and called out, (1) we do not know how many people get away with cheating and (2) even those cheaters who were caught managed to ruin multiple games. For example, take Joe, a cheater on the quizbowl team of City High. Joe cheats his way to the quarterfinals, where he is caught. Now, Joe might have beaten the team from Lambert in the previous round. Should Lambert now be able to play the quarterfinals? What about North High, who lost their last match of the prelims to City High, knocking them out of the playoffs? Even if cheaters are caught, they can ruin a tournament.

Even if, by some miracle, no one ends up cheating at this hypothetical online HSNCT, the tournament is still ruined. Let us say Jim has been studying extremely hard over the past semester, and ends up putting up extremely good stats at the tournament, leading his team to a reasonably high finish. Instead of being rewarded for his hard work, Jim is now faced with suspicion from the entire quizbowl community, because it seems that he cheated (and the statistics may well point to this). There is no way to distinguish people who have improved from cheaters without camera evidence, no way to prevent "microcheating" (maybe someone only cheats on the last tossup of important matches; they've probably only cheated 3-4 times throughout the day but have significantly influenced the course of the tournament).

What can NAQT do to prevent this? It's as easy as mandating that both hands of every player are shown on camera the entire time. PACE has done a very good job of instituting anti-cheating measures for its national tournament; those procedures could serve as a guideline for NAQT to establish its own. This would only require a minor change to the rules, and I urge that, despite one's opinion on the rest of these criticisms, NAQT take a stronger stance against cheating.

[edit: after I'd written this section, I was informed that NAQT does intend to institute greater anti-cheating policies for the HSNCT. I applaud this decision, and hope that NAQT considers the rest of the criticism I've presented.]

2. Cost/Structure.

HSNCT is far too expensive, and it is far too short.

HSNCT's registration fee for this year is $650. This is virtually unchanged from last year's fee, despite the online setting. Compare this to the NSC, which costs $450 this year, down from over $700 last year. The ONCT, meanwhile, will only cost $250.

NAQT will only guarantee eight games. Compare this to, say, the SMH Southeast Mirror (the last tournament I was involved in), which guaranteed nine games on a similar-difficulty set and cost $70, about 10% of the price. Sure, SMH is being mirrored in multiple places as opposed to HSNCT, which is grounds for an increased price. But ten times the price? I think not.

One could bring up length considerations. But consider that SMH Southeast ran 9 rounds from 9 AM to 5 PM, with a 45 minute lunch break in between, and one staffer in each room, with generous timing rules (more on this later) and longer questions (more on this later as well). Within the thirteen-hour period NAQT has designated, it is certainly possible to run 15 or more rounds, especially if lunch breaks are staggered.

In fact, I know of multiple teams who are either strongly reconsidering their decision to attend HSNCT, or who have decided not to attend HSNCT whatsoever, due to the high cost. Especially in this year, when school administrations may be (understandably) more reluctant to cover expenses, and when many teams are paying out of pocket, charging the same amount for less games is unfortunate.
An elite player who is not me wrote:My team's attended HSNCT almost every year we have qualified for as long as we have existed, but we're very unlikely to this year. The price is a big deal since we don't have school funding, but it's also the late announcement and a general sense than NAQT has been a lot less prepared for online than the other main quiz bowl groups, although Buzzword is of high quality.
In general, HSNCT's format is incredibly outdated. Very few other tournaments use double elimination. Most, including the two other national tournaments set to run this year, use a rebracketing format. Why does HSNCT not use rebracketing, and if necessary, restrict its field size to make this possible? This would guarantee most teams more games (certainly more than the abysmal eight that is currently guaranteed). Furthermore, double elimination does not guarantee games against teams of the same level in the way that rebracketing does; for example, the top team in the prelims will spend much of the playoffs racking up easy wins, instead of playing against every team of similar calibre. Why use a convoluted power-matching system/double elimination if better alternatives exist?

What can NAQT do to fix this? Unfortunately, I am not privy to NAQT's finances. But if other organizations, like PACE, are able to significantly cut their prices and simultaneously guarantee thirteen games, I do not think it is unreasonable to expect NAQT to do the same. I am sure many of NAQT's employees have substantially more experience designing tournament structures than I do, and could thus come up with a structure that guaranteed more (and more even) games to everyone. Just eight guaranteed games is unreasonable, and charging $650 for the privilege of playing them is even more so.

3. Timing Rules

NAQT's timing rules, while not-great before online play, have not been optimized for online play. I am sure many others have encountered the following scenario:

I buzz in, and I answer, not noting that I am muted. Someone notices that I am muted; I am reminded to unmute and immediately give my answer, which is then evaluated by the moderator.

Using a timing system like 8/8/8 (which I support for online play) allows this to happen, allows for lag, allows for one to collect oneself before one answers a question. Under NAQT's extraordinarily strict 2-second policy, many people who know the answer are going to be cheated out of points for either technical difficulties or normal behavior, or a combination of both. I don't think this is ideal at all.

Add to this the fact that some mods are going to be generous to people with technical difficulties, and others are going to be extremely strict (and some moderators do not know what one second is), and timing will just be a mess.

This is also easily fixed; NAQT should just amend its timing rules to something closer to 8/8/8 for HSNCT. Certainly more than 2 seconds after buzzing are necessary for proper play.

4. Questions

NAQT's questions are low-quality. This is a big one.

Throughout this section, I will use questions from NAQT's sample HSNCT packet (2019 HSNCT Round 31, I believe) as evidence. I would use other packets, but for whatever reason, NAQT does not believe in adhering to the quizbowl norm of releasing a set to the public after it is done being mirrored. I certainly do not want to be sued by NAQT, and anyways do not have any copies of any HSNCT packets. This is also something NAQT should change (its policy of not releasing packets to the public after they are clear), if it is not obvious from my tone.

I will first note that I have yet to find a high schooler who supports NAQT's distributional variance. There are certainly some who support the increased geography/current events/trash and decreased fine arts/pss. There are some who go the other way. This is not an argument I am interested in having now. But NAQT ensures that luck plays an even larger role than normal because it does not distribute its questions properly (per packet, not per tournament; frankly, I don't care about per tournament distribution).

NAQT's tossups are short. Even including powermarking/bolding and the copious pronunciation guides NAQT puts in, many tossups are only four lines long, and none over five. Question 6 in the sample packet is just three sentences long.

There are clear problems with questions of this length being used in national tournaments. The first is simply the lack of places to buzz. If there are only four or so buzzable clues, compared to the eight or nine of standard questions, there is that much more potential for a buzzer race.

The second is the impact of lag. If questions are short, the buzz distribution is more compact, leading to even one second of lag making a large impact on one's playing.

A third is that less clues means more cliffs. Instead of a nice smooth pyramid of clues, we get a ziggurat.

HSNCT tossups are shorter than those of every other non-novice high school set. For example, in last year's CALISTO set, which was pretty much universally lauded, tossups ranged from six to sometimes seven lines. The same was true of last year's BHSAT and LOGIC. BLAST was seven lines. PACE is generally around eight, as is SMH. There is a reason that every single other set writes longer questions than NAQT, and that is because longer questions are better at differentiating teams and providing a positive playing experience.

This makes NAQT's structure/cost decisions even more absurd. Seeing as HSNCT tossups are quite literally half of the length of standard nationals tossups, teams are in effect paying for four rounds' worth of tossups, not eight. If NAQT wants to pay its writers more, maybe it should ask them to actually write questions, not to string a three-sentence stub together and call it quits.

Now, moving on from NAQT's question size, we encounter their quality. Unfortunately, there are no redeeming factors here.

Let's start with difficulty variation. Some difficulty variation in every set is desirable, at least in tossups. Bonuses should be as uniform as possible. It is obvious that NAQT does not try to control the difficulty of its bonuses. I point to the following bonuses:
NAQT wrote:It takes two values to specify the location of an object in the sky. For 10 points each—
A. One value, the declination [dek-luh-NAY-shun], measures an object's angle north or south of the “celestial” feature
named for this great circle on Earth.
answer: (celestial) equator
B. This is the second coordinate. It measures the distance eastward, in hours, from the Sun's location at the vernal
equinox to the great circle containing the object.
answer: right ascension (prompt on “RA”; do not accept or prompt on partial answers)
C. This is an arrangement of two or more objects in the sky with the same right ascension. If the objects also have
matching declination, a syzygy [SIH-zuh-jee] results.
answer: conjunction(s) (in right ascension)
NAQT wrote:This work's composer noted that he had created many performances of it while walking alone in the woods.
For 10 points each—
A. Name this work premiered by pianist David Tudor at the Maverick Concert Hall in Woodstock, New York.
answer: 4'33” [“four minutes thirty-three seconds”]
B. This American composed 4'33”.
answer: John Cage (or John Milton Cage Jr.)
C. Cage's early works—such as Sonatas and Interludes—called for screws, paper, and other objects to be inserted
between a piano's strings, a technique known as this type of ”piano.”
answer: prepared piano (or preparing the piano; accept all forms of prepare or prepared or preparation(s))
I think it's pretty clear that the first bonus is significantly harder than the second. Consider, for one, that the medium and hard parts of the first bonus have been asked about a combined two times (and indeed, conjunction has never actually been asked about in quizbowl before). On the other hand, the second bonus has a medium part (4'33" I assume) that, with roughly the same information, appeared as a medium at HFT 2016, noted non-nationals set. Its hard part can barely even be called a hard part; I do think that prepared piano is a reasonable hard part for a high school regular set, and it has come up as the middle part in a recent PACE.

There are several other bonuses in this packet that are either far too easy or far too hard. Entire categories seem more difficult or more easy than others.

Other examples include rather hard answerlines (Zadie Smith, trigonal planar geometry, multiple inheritance, and Norman Schwarzkopf stand out; the first has never been tossed up below Terrapin difficulty, the second has never been tossed up, nor has the third or fourth). The third of these answerlines led to an absurd powermark placement, two words before "for 10 points." This kind of powermark placement clearly means that no thought has gone into how the tossup will actually play; if there are virtually no post-powermark, pre-FTP clues, one can expect buzzer-races on the last line for sure (if the tossup does not go dead).

Meanwhile (and this is all in the same packet, mind you), we see a commonlink on souls, which attempts to clue a Sam & Dave song, whatever that is (seems to be more "boomer trash," as NAQT trash has often been described), along with the video game Dark Souls and DuBois' The Souls of Black Folk, before ending with a clue from Henley's Invictus and a weak description. Leaving aside the weird, certainly-not-pyramidal difficulty of this tossup, I'm curious to know what writer even thought this was a good idea and what editor approved this absurd trash-literature-philosophy common link, and why it is taking up space from, say, more fine arts content (another policy decision that makes no sense, NAQT, why no writer tags?).

This same packet contains a math tossup on the number two-pi. NAQT math has long been one of the main subjects of criticism from the high-school community, but I'd like to focus on one particular clue, the tau manifesto one. Instead of finding an actual math clue, the writer/editor of this question decided to insert (in a crucial place, a place where a lot of players would normally buzz) what amounts to a pop-culture clue.

This should be reasonably easy for NAQT to fix; after all, if every other set is able to produce a reasonable-quality product, why can't NAQT? If many of the people who work for NAQT have written and edited some very good sets, why can't they do the same for NAQT?

I think I've ranted enough about HSNCT question quality, but the fact is that in the end, NAQT is offering a more expensive and shorter national tournament, without adequate cheating precautions, and with low-quality questions (questions of such low quality that it's hard to call NAQT "pyramidal good quizbowl" anymore).

To NAQT, I ask, what gives? More importantly, how do you expect HSNCT to continue to be viable without doing anything to satisfy your customers? The revolt has already begun; multiple top teams have already decided against playing HSNCT this year.

And to coaches and players, consider not attending HSNCT this year. Both PACE and ONCT offer good sets, reasonable prices, and reasonable formats/guaranteed games.

Note: I am a writer for the set that ONCT is using, the Southeast-Midwest Housewrite. However, there exists no financial incentive for me to recommend ONCT, due to the details of our arrangement. The recommendation I make for ONCT is solely based on the format, question quality, etc, and has nothing to do with my affiliation with ONCT.
Arya Karthik (they)

Lambert HS, 2018-22
Georgia Tech, 2022-24
St. Catherine's College, Oxford, 2024-25

2022 PACE NSC; 2023 ACF Nationals
User avatar
CPiGuy
Auron
Posts: 1070
Joined: Wed Nov 16, 2016 8:19 pm
Location: Ames, Iowa

Re: HSNCT and its Problems

Post by CPiGuy »

I'm not going to bother responding to everything in this post but a lot of it strikes me as kind of hopelessly naive.
In general, HSNCT's format is incredibly outdated. Very few other tournaments use double elimination. Most, including the two other national tournaments set to run this year, use a rebracketing format. Why does HSNCT not use rebracketing, and if necessary, restrict its field size to make this possible?
Very few other tournaments have field sizes in excess of 200 teams. I'd love to see a proposal for a rebracketing format that can accommodate NAQT's huge field sizes and doesn't either include eleventy million games or eliminate teams unfairly early if there are any seeding mistakes (which are very easy to make at a national championship).

Changing the format to rebracketing would be cool but it's not worth excluding over a hundred teams from nationals to do it. Either qualification would have to get significantly harder or teams would be locked out because they didn't sign up fast enough. Neither is reasonable.
What can NAQT do to fix this? Unfortunately, I am not privy to NAQT's finances. But if other organizations, like PACE, are able to significantly cut their prices and simultaneously guarantee thirteen games, I do not think it is unreasonable to expect NAQT to do the same.
NAQT has gotten way less income this year from mirror fees than normal, in part due to their (extremely admirable) decision not to license their sets for in-person play. It's pretty clear to me they aren't money-grubbing. You are, in fact, not privy to NAQT's finances, and neither am I. But I'm pretty confident they wouldn't be charging this much if they didn't feel they had to.
One could bring up length considerations. But consider that SMH Southeast ran 9 rounds from 9 AM to 5 PM, with a 45 minute lunch break in between, and one staffer in each room, with generous timing rules (more on this later) and longer questions (more on this later as well). Within the thirteen-hour period NAQT has designated, it is certainly possible to run 15 or more rounds, especially if lunch breaks are staggered.
At in-person HSNCT, teams played 7 matches between 8am and 6:30pm. NAQT was explicit that teams would not necessarily be playing throughout the entire 13-hour period. Unless you want teams to be expected to wake up at 6am on the West Coast and be playing until 10pm on the East Coast, it may not be reasonable to guarantee 10 prelim games.

NAQT also explicitly said that more prelim games may be offered if the field size expands or the format permits it. They have not said that there will only be eight prelim games. They are, in fact, being deliberately conservative because there are so many unknowns and it's really goddamn hard to run a massive national championship during a global pandemic.
It is obvious that NAQT does not try to control the difficulty of its bonuses.
As someone who has dunked on NAQT quite hard in the past, what the hell? Of course they try. This is insulting and an unacceptable insinuation if you want to be taken seriously.

The question quality complaints are likely legitimate if overblown -- there have certainly been housewrites this year that, if they were produced by NAQT, would be held up as evidence that NAQT questions are irreparably bad. The timing complaints are extremely legitimate -- I think NAQT needs to change their timing rules for online play in order to avoid serious inconsistency and feelbad situations.

As a whole, though, this post generally strikes me as poorly thought out and failing to take into account any reasons for NAQT's decisions other than malice, which is wholly unreasonable and unfair. I trust that NAQT will run an excellent tournament, as they have in the past.

Look -- I am probably the poster child for making unreasonable and poorly-thought-out posts that complain about things without considering their underlying causes or giving the people in charge credit for doing their best. So you should probably listen to me when I tell you to stop doing those things.
Conor Thompson (he/it)
Bangor High School '16
University of Michigan '20
Iowa State University '25
Tournament Format Database
matthewspatrick
Lulu
Posts: 76
Joined: Tue Jul 15, 2014 11:30 am
Location: Wilmington, DE

Re: HSNCT and its Problems

Post by matthewspatrick »

etotheipi wrote: Thu Mar 04, 2021 6:27 pmTo NAQT, I ask, what gives? More importantly, how do you expect HSNCT to continue to be viable without doing anything to satisfy your customers? The revolt has already begun; multiple top teams have already decided against playing HSNCT this year.
The revolt has been so widespread and the customers so manifestly unsatisfied that a mere 91 teams have signed up in the four hours and change since HSNCT has been declared open for registration.

[Please note that while I am an NAQT Member Emeritus, and that I do perform some tasks for NAQT, I am not in any way involved in the company's management, nor am I privy to their financial information.]
Patrick Matthews
University of Pennsylvania 1989-94
NAQT Member Emeritus and co-founder
I do not speak for NAQT in any way, shape, or form.
User avatar
AKKOLADE
Sin
Posts: 15773
Joined: Thu Apr 24, 2003 8:08 am

Re: HSNCT and its Problems

Post by AKKOLADE »

Are the bonuses in the OP clear?
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
User avatar
Santa Claus
Rikku
Posts: 285
Joined: Fri Aug 23, 2013 10:58 pm

Re: HSNCT and its Problems

Post by Santa Claus »

AKKOLADE wrote: Fri Mar 05, 2021 1:45 am Are the bonuses in the OP clear?
I assume it was in a podcast.
Kevin Wang
Arcadia High School 2015
Amherst College 2019

2018 PACE NSC Champion
2019 PACE NSC Champion
User avatar
Whiter Hydra
Auron
Posts: 1418
Joined: Tue Dec 04, 2007 8:46 pm
Location: Fairfax, VA
Contact:

Re: HSNCT and its Problems

Post by Whiter Hydra »

AKKOLADE wrote: Fri Mar 05, 2021 1:45 am Are the bonuses in the OP clear?
Both bonuses in question come from the sample packet.
Harry White
TJHSST '09, Virginia Tech '13

Owner of Tournament Database Search and Quizbowl Schedule Generator
Will run stats for food
User avatar
AKKOLADE
Sin
Posts: 15773
Joined: Thu Apr 24, 2003 8:08 am

Re: HSNCT and its Problems

Post by AKKOLADE »

Good deal.
Fred Morlan
University of Kentucky CoP, 2017
International Quiz Bowl Tournaments, CEO, co-owner
former PACE member, president, etc.
former hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator
User avatar
Santa Claus
Rikku
Posts: 285
Joined: Fri Aug 23, 2013 10:58 pm

Re: HSNCT and its Problems

Post by Santa Claus »

These arguments purport to target this year’s HSNCT but many of them don’t really work unless you disagree with the way HSNCT and NAQT are run in general. That’s fine, I guess: I certainly don’t agree with that but it is a stance one can take.

The points about distro, format, and question quality are pretty obviously among the general complaints, but even cost, as impacted by the pandemic as it is, verges on being in this category too: as has been noted by other people, any team outside of the Atlanta area will pay much less to attend this HSNCT due to a lack of travel and hotel costs, so I think that teams that remain interested in attending this year will find the price the same level of onerous as they did in past years.

The only other issues which are relevant to this year specifically are the timing, which, fair point - they should do something about that - and cheating prevention, which they already are thinking about.

This leaves just a discussion about the pros and cons of HSNCT and more generally NAQT. These have been argued back-and-forth for a long time, but since you’ve brought them up again I’ll chip in my thoughts on these matters, and rebuttals to several of your points:
  • Quiz bowl is an activity that, like many other extracurriculars and hobbies, has been buoyed by countless people doing volunteer work for little or no pay. No housewrite that has ever been written has fairly compensated its writers or its editors, they have merely been closer to or further from reaching minimum wage. It’s probably true that NAQT doesn’t quite hit that mark either, but it’s certainly closer than most. The amount of time and effort that goes into writing a set is astronomical, and as someone who is a writer for other sets you should know that. This, of course, does not guarantee that the cost of HSNCT is justified, but it is a reminder that, typically, things cost a non-trivial amount of money and that it is altruism on the part of the community that prices are generally as low as they are.
  • Teams attending the 2019 HSNCT were only guaranteed 10 games - in light of the extenuating circumstances, I find that even the extreme case of lowering that number to 8 this year is not unreasonable.
  • I do not think it makes sense to compare the format of HSNCT to other tournaments and expect things to make sense. The reason other tournaments can guarantee more rounds or use other formats is because they do not operate at the same scale as HSNCT. Fundamentally there will always be a trade off between efficiency (in terms of games played) and fairness (in terms of choosing the “real” champion). Double elimination may be more variable than a round robin, but it takes literally orders of magnitude fewer rounds - for 128 teams, this is the difference between 7 rounds + advantaged finals and 127 (or as Conor put it, “eleventy million”).
  • I’m going to disagree with your subjective assessment of this first example bonus as harder than the second, because in my subjective assessment they are very closely matched in difficulty. As I believe Matt Jackson wrote, database hits != difficulty.
  • If you’re going to complain about questions cluing things people don’t care about like “boomer trash”, you don’t get to turn around and complain about “pop science” bringing things that people actually know about into science questions. I think there’s a word for information that people learn from encountering it in real life and not from packet study, but I can’t remember what it is.
  • I don’t know why NAQT doesn’t use writer tags, but I bet it’s to keep angry quiz bowlers from saying specific writers produce low-quality questions. Clearly that hasn’t worked though, so maybe they should use them anyways.
I make these arguments defending NAQT not because it’s an infallible entity, because it certainly isn’t - it has made and still does make mistakes, and this is simply the latest post in a decades-long tradition of people calling them out for that. But I do like HSNCT, bumps and all, and it’s certainly much better than you made it out to be - ‘it's hard to call NAQT "pyramidal good quizbowl" anymore’? Gimme a break! Including a few legitimate points in your post does not an argument make, and the incredible callousness of this post for seemingly no reason makes it really hard for me to care much about the same five anti-NAQT talking points that people have been using for ages.
Kevin Wang
Arcadia High School 2015
Amherst College 2019

2018 PACE NSC Champion
2019 PACE NSC Champion
User avatar
jonpin
Auron
Posts: 2266
Joined: Wed Feb 04, 2004 6:45 pm
Location: BCA NJ / WUSTL MO / Hackensack NJ

Re: HSNCT and its Problems

Post by jonpin »

etotheipi wrote: Thu Mar 04, 2021 6:27 pm 2. Cost/Structure.

HSNCT is far too expensive, and it is far too short.

[...]

In general, HSNCT's format is incredibly outdated. Very few other tournaments use double elimination. Most, including the two other national tournaments set to run this year, use a rebracketing format. Why does HSNCT not use rebracketing, and if necessary, restrict its field size to make this possible? This would guarantee most teams more games (certainly more than the abysmal eight that is currently guaranteed). Furthermore, double elimination does not guarantee games against teams of the same level in the way that rebracketing does; for example, the top team in the prelims will spend much of the playoffs racking up easy wins, instead of playing against every team of similar calibre. Why use a convoluted power-matching system/double elimination if better alternatives exist?

What can NAQT do to fix this? Unfortunately, I am not privy to NAQT's finances. But if other organizations, like PACE, are able to significantly cut their prices and simultaneously guarantee thirteen games, I do not think it is unreasonable to expect NAQT to do the same. I am sure many of NAQT's employees have substantially more experience designing tournament structures than I do, and could thus come up with a structure that guaranteed more (and more even) games to everyone. Just eight guaranteed games is unreasonable, and charging $650 for the privilege of playing them is even more so.
(disclaimer: While I'm a member of PACE, I am absolutely in no way speaking for them here)
One really important thing to recognize in reference to the two bolded passages above is that NAQT is a for-profit company with several full-time employees (by comparison PACE is a non-profit). As Conor mentioned above, NAQT has likely taken a substantial financial hit from the loss of revenue associated with regular season tournaments. I believe it's common wisdom that HSNCT is the moneymaker for NAQT, that it's where it makes a large chunk of its profit for the year, allowing it to exist in its current form. Lowering the number of teams is going to cut into that profit. Lowering the cost is going to cut into that profit. I'm not saying both of those are impossible, I'm just saying that they are not actions that NAQT can take lightly.
Jon Pinyan
Coach, Bergen County Academies (NJ); former player for BCA (2000-03) and WUSTL (2003-07)
HSQB forum mod, PACE member
Stat director for: NSC '13-'15, '17; ACF '14, '17, '19; NHBB '13-'15; NASAT '11

"A [...] wizard who controls the weather" - Jerry Vinokurov
Joshua Rutsky
Tidus
Posts: 663
Joined: Mon Oct 24, 2005 9:59 am
Location: Hoover, AL

Re: HSNCT and its Problems

Post by Joshua Rutsky »

Does anyone else get uncomfortable when we start having conversations about what "elite teams" are deciding should and should not be done as part of a national tournament, and when the threat that the elite teams will cease attending the event because it does not meet their standards is put out there? This is an ongoing issue in quizbowl, to be sure, but one that really should be addressed if a community is going to exist. Quizbowl is played by people at all levels of skill. Some teams are good, some teams are very good, and some teams are ungodly. Does being an "elite" team give you special status in terms of saying what should and should not be done in quizbowl nationally? Isn't that the same argument that breaks out once every few years about this board in general, and the "That's bad quizbowl and you shouldn't play it because you are encouraging bad quizbowl!" argument?

There's nothing wrong with critiquing an organization. There's nothing wrong with critiquing rule sets. When we start talking about what the elite demand, however, I think we are going in the wrong direction. Quizbowl isn't just for the elite. 280 teams don't come to Nats thinking they all have an equal shot at winning. They come because they want to play against other teams that love the game as much as they do. They come because they want to test themselves against strong competition. They come because it is a TREAT to take a big trip (not so much this year, obviously) to a major city, and a fun way to cap a year's worth of hard work. They come for reasons that are their own, and they don't all directly correspond with the needs of the top 10-20 teams at that event. They aren't less important for that reason.
Joshua Rutsky
VP for Curriculum and Camp Operations, Qwiz
ASCA Board Member
Hoover High School Coach (Retired)
User avatar
jonpin
Auron
Posts: 2266
Joined: Wed Feb 04, 2004 6:45 pm
Location: BCA NJ / WUSTL MO / Hackensack NJ

Re: HSNCT and its Problems

Post by jonpin »

Joshua Rutsky wrote: Fri Mar 05, 2021 3:21 pm Does anyone else get uncomfortable when we start having conversations about what "elite teams" are deciding should and should not be done as part of a national tournament, and when the threat that the elite teams will cease attending the event because it does not meet their standards is put out there? This is an ongoing issue in quizbowl, to be sure, but one that really should be addressed if a community is going to exist. Quizbowl is played by people at all levels of skill. Some teams are good, some teams are very good, and some teams are ungodly. Does being an "elite" team give you special status in terms of saying what should and should not be done in quizbowl nationally? Isn't that the same argument that breaks out once every few years about this board in general, and the "That's bad quizbowl and you shouldn't play it because you are encouraging bad quizbowl!" argument?

There's nothing wrong with critiquing an organization. There's nothing wrong with critiquing rule sets. When we start talking about what the elite demand, however, I think we are going in the wrong direction. Quizbowl isn't just for the elite. 280 teams don't come to Nats thinking they all have an equal shot at winning. They come because they want to play against other teams that love the game as much as they do. They come because they want to test themselves against strong competition. They come because it is a TREAT to take a big trip (not so much this year, obviously) to a major city, and a fun way to cap a year's worth of hard work. They come for reasons that are their own, and they don't all directly correspond with the needs of the top 10-20 teams at that event. They aren't less important for that reason.
I would not agree with this take. The reason modern quiz bowl exists in its current, not terrible, format is because the "elite" collegiate teams of the early 1990s looked at the product that College Bowl was putting out and said "this sucks, let's do better". Over the next 15 years, the reputation of CBI degraded to the point where it was no longer a legitimate national championship because enough of the really good collegiate teams didn't consider it one, and so stopped attending. At the high school level, the same thing happened to Questions Unlimited (though NAC didn't completely die out). Hundreds of teams travel to NAC to have a big ol' hurrah and play teams from across the nation, and almost nobody who posts on these boards considers that a national championship anymore. Heck, HSNCT itself no longer includes computational math tossups because a lot of strong teams thought they were antithetical to quiz bowl.
The point of a NATIONAL CHAMPIONSHIP should be, at least at the top of the list, to determine the national champion. If the teams that are reasonably described as contending for that title think a tournament has flaws that prevent it from doing so, that's a legitimate complaint.

This is not to say that the complaints registered in OP are accurate, are shared by a critical mass of elite teams, or are sufficiently damaging to render HSNCT a fake national championship. Merely to say that, a set of complaints that is accurate, is shared by a critical mass of elite teams, and is significant in scope would indeed damage a tournament's credentials as a national championship.
Jon Pinyan
Coach, Bergen County Academies (NJ); former player for BCA (2000-03) and WUSTL (2003-07)
HSQB forum mod, PACE member
Stat director for: NSC '13-'15, '17; ACF '14, '17, '19; NHBB '13-'15; NASAT '11

"A [...] wizard who controls the weather" - Jerry Vinokurov
Joshua Rutsky
Tidus
Posts: 663
Joined: Mon Oct 24, 2005 9:59 am
Location: Hoover, AL

Re: HSNCT and its Problems

Post by Joshua Rutsky »

Fair enough, I suppose. There's no question that the point of HSNCT is a national championship event, and as such it should be something that serves that purpose adequately. I am more concerned with the idea overall that a group of teams that are top performers should be the sole arbiters of what is or is not appropriate difficulty level, number of matches, format, etc. If that wasn't the intent behind the OP's reference to the elite, I stand corrected.
Joshua Rutsky
VP for Curriculum and Camp Operations, Qwiz
ASCA Board Member
Hoover High School Coach (Retired)
User avatar
aurochs-and-angels
Lulu
Posts: 10
Joined: Tue Apr 07, 2020 12:57 am

Re: HSNCT and its Problems

Post by aurochs-and-angels »

etotheipi wrote: Thu Mar 04, 2021 6:27 pm the first has never been tossed up below Terrapin difficulty, the second has never been tossed up, nor has the third or fourth
I imagine most people cringe reading this, but I remember after HSNCT in 2019, I too could only interpret the difficulty of questions (in fact, these same ones!) based on database hits. However, not only is “how many times has this been an answerline and at what level” a flawed criteria anyway, but it is most especially flawed in the context of HSNCT for a couple of reasons.

1. Shorter questions give more freedom to use certain answerlines.

Stormin’ Norman is very famous (and indeed comes up all the time at the HS level), but I imagine the reason he isn’t a frequent answerline (at any level) is because he doesn’t make for a great 7-8 line pyramidal question. With 4-5 lines though, there’s no trouble. Same with trigonal planar -- this might be conceptually confusing or need too difficult of early clues in a PACE tossup, but because of the NAQT format, it becomes workable (and certainly not difficult to convert; all of the post-power clues should be trivial for a team with someone who has taken a chemistry class).

2. HSNCT and PACE questions aim to uniquely reward in-depth knowledge of topics that high schoolers learn frequently independent of Quizbowl, and thus are in a way removed from the larger difficulty-continuum of high-level questions.


It is worth quoting in full this comment by PACE editor Will Alston in 2019:
naan/steak-holding toll wrote:My aim was to emulate the best parts of previous NSCs, which wrote questions on deep cut on something that appears in high school curricula, or topics that an intellectually curious high schooler might know a lot about if they "cared about" a subject or, in the case of music, were a practicing musician; this being contrasted with shallowly mining the college canon and transporting it down. I also tried to work more "world" content into the fine arts than in previous sets. So, when I heard a comment that this set felt like it was testing a much more expanded canon, my thought was "that's because I didn't really care about the canon when writing this set" - I think it'd be pretty hard to power my tossups on Arabic numerals, speed limits, canoes, world drums, or musical cadences from reading old packets.
This really caught me off guard in 2019 after my first nationals. Because I'm not sure if I can reference HSNCT questions, I'll use equivalent examples from PACE instead. Questions on "the feud between the Grangerfords and Shepherdsons," "the Nurse [in Romeo and Juliet]," "Shinichi Suzuki," to list a few, are supposed to reward a certain form of engagement with the material -- just because they haven't "been answerlines," that doesn't make them harder or easier necessarily.
Kris
User avatar
naan/steak-holding toll
Auron
Posts: 2514
Joined: Mon Feb 28, 2011 11:53 pm
Location: New York, NY

Re: HSNCT and its Problems

Post by naan/steak-holding toll »

Not part of the HSNCT audience here, but I'm extremely baffled as to why the 2 second policy is being continued for online events, especially given that many NAQT moderators are extremely aggressive about negging you. It's very likely that Zoom - a platform which eats up a lot of bandwidth - will cause lag multiple times throughout a tournament. Look, I'll play an online nationals if I have to, and in fact eagerly signed up for ICT this year, but this seems guaranteed to produce a miserable play experience with lots of people getting negged despite saying their answer in time simply because there was lag in the mod hearing it.

Actually I'm not baffled, since NAQT's members have consistently voted against changing the 2-second timing policy despite it being loathed by most of the player base, but still.
Will Alston
Dartmouth College '16
Columbia Business School '21
User avatar
cchiego
Yuna
Posts: 890
Joined: Fri Jun 06, 2008 7:14 pm
Contact:

Re: HSNCT and its Problems

Post by cchiego »

etotheipi wrote:Meanwhile (and this is all in the same packet, mind you), we see a commonlink on souls, which attempts to clue a Sam & Dave song, whatever that is (seems to be more "boomer trash," as NAQT trash has often been described), along with the video game Dark Souls and DuBois' The Souls of Black Folk, before ending with a clue from Henley's Invictus and a weak description. Leaving aside the weird, certainly-not-pyramidal difficulty of this tossup, I'm curious to know what writer even thought this was a good idea and what editor approved this absurd trash-literature-philosophy common link, and why it is taking up space from, say, more fine arts content (another policy decision that makes no sense, NAQT, why no writer tags?).
I've been guilty of overly dramatic complaining about NAQT and pop culture before, but this example isn't a good one. Sam and Dave are a hugely important group and "Soul Man" is a quite significant song both musically and socially. Plus, it wasn't used as a whole TU or bonus, but rather the lead-in clue for a tossup in a finals packet at HSNCT. That seems a wholly appropriate place for such a clue. Such interdisciplinary/mixed TUs are, I think, a good place to put these kinds of clues on significant but not necessarily "canonical" topics and it's good that NAQT encourages those. Whether or not there should be more of a clear pop culture canon and what counts as "boomer trash" vs. culturally significant arts is a worthwhile debate for another thread; my current view (shaped in part by listening to arguments against my old views) has moved towards encouraging more "significant" pop culture and expanding what counts as "art" to encompass more significant contributions from more "popular" forms of art, but I understand how the current setup and lack of clarity can be frustrating for players.

I do agree on the point of NAQT tossup and bonus difficulty consistently seeming to be more variable than in other sets, but it would be ideal to focus on statistical evidence for that as a comparison rather than just a couple of specific questions. Analyzing scoresheets for the numbers of 0/10/20/30 as well as dead TU% between similar sets of teams would be more helpful here.

More broadly, I have had good experiences with providing feedback to NAQT officials in charge of running national events. I would recommend that people directly contact NAQT people with various concerns, especially since it does indeed appear from their extremely extensive guide to online quizbowl that NAQT is taking all of the potential issues with online quizbowl seriously (including the need to pay staffers a decent wage for online quizbowl work, which I suspect is a large contributor to the costs at nationals).

It might be helpful in the future for more ways to have productive community dialogues back and forth with NAQT given its increasingly-dominant role in quizbowl. But the way that the initial arguments in this thread were framed as being inherently correct or more worthwhile since they are coming from the "elite players" and "top teams" is not a productive frame. A good part of the effectiveness of the arguments for pyramidal quizbowl against non-pyramidal quizbowl over the years has stemmed from how pyramidal quizbowl benefits all teams in many different ways, making quizbowl a better activity as a whole (as well as one that has more appeal to more schools and teams, which as Jon mentions is quite important for a company like NAQT to consider compared to a small group of volunteers). And given that there have been many worthwhile concerns in the past raised about "insiderness" and "insularity" within the quizbowl community at a number of levels, such elite-centered framing is likely to undermine the arguments being made.
Chris C.
Past: UGA/UCSD/Penn
Present: Solano County, CA
User avatar
Stained Diviner
Auron
Posts: 5085
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: HSNCT and its Problems

Post by Stained Diviner »

I am not speaking here for anybody or any organization except myself. I am not currently an employee of NAQT, though I was a question writer for them in the past and wrote some You Gotta Knows this year. I'll also state that I have not seen a single question from this year's HSNCT or NSC, and I do not have access to those sets.

I am going to push back here on a lot of Aadi's points, but I am going to do so respectfully because I respect him and his opinions. (I don't know him, but I read his post and respect him based on that.) A lot of people have been making similar points for the last fifteen years or so, and there is no reason to get upset because somebody is saying them here as opposed to saying them in person or on discord or wherever kids say things these days. My response is long because I am addressing many of the points Aadi made in his long post.

The cheating issue has already been addressed, and unfortunately it is a complex issue that will continue to be addressed before, during, and after this coming nationals season.

I can only address the cost issue a little bit because I don't know what factors went into it, but keep in mind that NAQT is paying its writers and editors to make a set with a lot of rounds and that probably will only be used once. Also, though tournaments are saving a lot of money by not using hotels and buying plane tickets for staffers, they are now paying staffers significantly more than usual because it is difficult to recruit a lot of staffers when you can't lure them with travel and the ability to see their friends from different places. It is especially difficult to recruit staffers when you are running an enormous tournament that requires an enormous number of staffers. Additionally, for most teams this tournament will end up being a lot cheaper than usual because the teams are not paying for travel and hotels for themselves. As far as schools covering the cost, that varies a lot from school to school, but there are actually a lot of school districts that have more money than usual for these types of things because they have spent a lot less on travel and activities this year than normal years. If some schools are not participating due to the cost, then that is unfortunate, and those schools should consider describing their financial situation to NAQT. I honestly have no idea whether NAQT will do anything in such cases, but the question can be asked.

As has already been pointed out, it is difficult to compare NAQT to PACE. NAQT has some full-time employees, so my guess is that the loss of revenue over the past year made things difficult for them. PACE does not, so it is much easier for PACE to spend less money when it takes in less money.

Hopefully NAQT will eventually guarantee more than eight games. Either way, the top teams will play more than eight games, and some of them will play considerably more. Some of this probably depends on how many teams and staffers NAQT gets, which is difficult to predict because nobody has ever run an online HSNCT before. That being said, if HSNCT guarantees 8 games this year instead of the usual 10 because this year is completely messed up in all sorts of ways, it's a problem worth noting but not a crisis.

As far as the timing rules are concerned, let's see what NAQT does. NAQT has already run one online national tournament, and they will run several more between now and HSNCT, and they probably will adjust rules that do not work. At any rate, while the timing rules are something to reasonably disagree with, I'm not sure how they got grouped with cheating, cost, and question quality, all three of which are fundamental long-term quizbowl issues.

As far as question quality is concerned, the reactions to HSNCT question sets are generally positive within the community. NAQT sells their questions rather than posting them, which puts you at a disadvantage when critiquing the questions that is not your fault. NAQT does that because selling questions makes money, which in turn allows them to pay people to work on quizbowl and control some of the costs for their questions and tournaments. The current sample HSNCT packet was used in the Championship Match, which is available online. It is possible that some of the questions in that packet were specifically held for the playoffs and were judged to be too difficult for the first day of HSNCT when all of the teams were playing.

The NAQT distribution and distributional variance are things some people don't like. I agree with some of your points--I myself would like to see more Fine Arts and less Pop Culture than NAQT gives us, but all we can do is express our opinion. Keep in mind that NAQT more than other quizbowl organizations has polled people at their tournaments on these types of matters, and the results of those polls are one reason that the distribution is what it is. Even at HSNCT, there are a lot of players who enjoy the pop culture questions. As far as the variance, it is fairly small, and playing with a clock means that there is going to be some variance anyways because many matches don't get to the last few questions. (NAQT has determined that the clock is a good way to keep an enormous tournament running efficiently and to encourage moderators and teams to keep things moving.) I would prefer to get rid of the variance myself, but it's generally a minor concern.

NAQT questions are short. The character limit is 425 characters. Almost all NAQT questions are exactly the same length. If they are three sentences, then the three sentences are on the long side. The short question length does make it more difficult to differentiate between teams and does lead to more buzzer races. That being said, NAQT knows that their questions are short, and they work hard to put lots of buzzpoints in their questions and cram in clues as best they can. The three sentence question you point to has "people who are murdered after voting to divert water from a wealthy rancher", "Handcarved Coffins", "Music for Chameleons", "Floyd Wells snitches on a former roommate", "investigator Alvin Dewey", "execution of Dick
Hickock and Perry Smith", "murder of the Clutter family", and "In Cold Blood", all of which are buzzable clues.

The issue of lag could be a problem. I don't know what the solution is other than to have in-person quizbowl as much as possible, but I agree that a one-second lag could be an issue.

As far as bonus difficulty variation is concerned, I agree with you that the first bonus you quote seems easier than the second, though I believe you overstate your case. The first question has a clear easy part, while the other two parts are difficult. The second question starts with a hard part before moving on to an easy part and a medium part that an elite team is going to convert. That being said, all three answers in the astronomy question are terms that are used often in astronomy and are described in a straightforward way, so a team that has somebody who is into astronomy is going to sweep it. Unfortunately, the astronomy bonus was never used, so we have no data on it. The music question was swept, but the national champion had to work to pull the first part. The first part is more difficult than the HFT question you cite, because the HFT question states, "David Tudor marked the beginning and end of movements by raising and lowering the lid of a piano, and it’s marked Tacet." Usually the last notes mark the end of movements, and the only reason something else had to be used was because this piece has no notes.

I want to add three points about NAQT bonus difficulty variation. The first is that NAQT uses conversion data from past tournaments more and better than anybody in quizbowl, and it is not even close. Some of their decisions seem counterintuitive, but they are often backed by data. The second is that after its tournaments NAQT makes a thread in which they discuss and give conversion results from many of their questions, and once again this is something that NAQT does much better than anybody else. Sometimes conversion data indicates that a particular question was bad, and NAQT admits it. Sometimes conversion data is surprising--we learn that high school quizbowlers are either more knowledgeable or less knowledgeable than we thought they were on some specific topic.

My third point on NAQT bonus difficulty variation is that NAQT bucks the canon more than many other question producers. They take some chances, hoping that students have learned something outside of quizbowl. The astronomy bonus is an example. Sometimes those chances pay off by teaching us something new and allowing students to get credit for knowledge gained outside quizbowl, and sometimes those chances result in bad questions that penalize students for not knowing something obscure. To be clear, anybody attending HSNCT should expect that as always there will be plenty of clues and answers that have come up in quizbowl before. You should also expect some stuff that hasn't, and it's fine to have an opinion one way or the other on that.

As far as the tossups you cite, keep in mind that these questions were used in one match, and in that match 21 out of 22 tossups were answered. Multiple inheritance is the only tossup that went dead. They didn't finish the packet, so Zadie Smith did not get asked. I have actually written a Zadie Smith tossup myself, which was asked in the Championship Match at the 2016 Scobol Solo and answered by Ali Saeed of Stevenson. My guess is that between Beavercreek and Uni High, somebody would have answered it, though it would have been a poor answer choice on the first day of HSNCT.

I asked a question on trigonal planar at this year's Scobol Solo. It was used in 32 rooms, and it was answered correctly in 29 of them. That's a high school solo tournament, albeit one that is designed so that only students with winning records were playing that round. As has been said before, do not equate search frequency with difficulty. (As a note, this year's Scobol Solo is not cleared, so do not give away content without my permission. I gave myself permission, so I'm allowed to do it.)

The soul question is the type of question that only NAQT writes that combines pop culture with academic content. I am against those questions myself because I am against pop culture questions generally, but this one doesn't seem particularly bad, though it could be criticized because it is so focused on titles of various things. In the actual match, Beavercreek got it because of some deep video game knowledge. The Sam & Dave song is a good song, but I'm so old that I sometimes get mistaken for a boomer, so my opinion isn't what matters. More seriously, if you are going to have pop culture, it probably is good to pull from a variety of eras, and this question seems to do that appropriately. Soul Man isn't some hit from the 60s that has been completely ignored and forgotten since then. It was a hit before I was born, but I listened to The Blues Brothers, and perhaps some of you kids watched Drake & Josh reruns.

The Tau Manifesto isn't really a pop culture clue. It's a geek culture clue that people into math and math education noticed. It's not the most academic content, but it's something that got people to pay attention to that particular number, which is as legitimate as many quizbowl clues are.

I appreciate the fact that some people are frustrated by certain aspects of NAQT. NAQT has a high profile, and part of having a high profile is that people have strong opinions. When elite teams play each other on IS questions, there are a ton of buzzer races because the combination of short questions with an intended audience outside the actual audience makes for buzzer races, but that is an IS issue that is reduced significantly at HSNCT. Some people have different visions than NAQT of what quizbowl ideally should be, and it is frustrating to put a lot of time and effort into quizbowl and have the activity be different than how you want it. There are many members of the community who want lots of deep clues and more of an academic and canon focus than NAQT provides them, and those community members are left with a choice of playing NAQT tournaments or not. That's a decision for individuals, teams, and groups of teams to make.

Personally, I am glad that a lot of teams, including a lot of elite teams, play HSNCT. I am glad that it is considered a legitimate national championship, and I hope that continues. When you look at the big picture, NAQT has done a great job of increasing the amount of quizbowl while remaining true to the goals of the activity. Of course, they are not above criticism, and it is better to have that criticism out in the open rather than hidden. If NAQT frustrates you to the point that you do not want to play their tournament, then don't play it. If you believe that participating in a boycott will lead to improvements in NAQT and/or quizbowl, then participate in a boycott. Personally, I believe that quizbowl has had more than its share of drama and infighting, and we should try to get along with people and organizations in quizbowl that we can get along with, and that includes NAQT. We have an organization that is capable of recruiting 352 teams to a tournament and running it well, and that is a great thing.
David Reinstein
Head Writer and Editor for Scobol Solo, Masonics, and IESA; TD for Scobol Solo and Reinstein Varsity; IHSSBCA Board Member; IHSSBCA Chair (2004-2014); PACE President (2016-2018)
User avatar
Skepticism and Animal Feed
Auron
Posts: 3238
Joined: Sat Oct 30, 2004 11:47 pm
Location: Arlington, VA

Re: HSNCT and its Problems

Post by Skepticism and Animal Feed »

jonpin wrote: Fri Mar 05, 2021 3:43 pm
Joshua Rutsky wrote: Fri Mar 05, 2021 3:21 pm Does anyone else get uncomfortable when we start having conversations about what "elite teams" are deciding should and should not be done as part of a national tournament, and when the threat that the elite teams will cease attending the event because it does not meet their standards is put out there? This is an ongoing issue in quizbowl, to be sure, but one that really should be addressed if a community is going to exist. Quizbowl is played by people at all levels of skill. Some teams are good, some teams are very good, and some teams are ungodly. Does being an "elite" team give you special status in terms of saying what should and should not be done in quizbowl nationally? Isn't that the same argument that breaks out once every few years about this board in general, and the "That's bad quizbowl and you shouldn't play it because you are encouraging bad quizbowl!" argument?

There's nothing wrong with critiquing an organization. There's nothing wrong with critiquing rule sets. When we start talking about what the elite demand, however, I think we are going in the wrong direction. Quizbowl isn't just for the elite. 280 teams don't come to Nats thinking they all have an equal shot at winning. They come because they want to play against other teams that love the game as much as they do. They come because they want to test themselves against strong competition. They come because it is a TREAT to take a big trip (not so much this year, obviously) to a major city, and a fun way to cap a year's worth of hard work. They come for reasons that are their own, and they don't all directly correspond with the needs of the top 10-20 teams at that event. They aren't less important for that reason.
I would not agree with this take. The reason modern quiz bowl exists in its current, not terrible, format is because the "elite" collegiate teams of the early 1990s looked at the product that College Bowl was putting out and said "this sucks, let's do better". Over the next 15 years, the reputation of CBI degraded to the point where it was no longer a legitimate national championship because enough of the really good collegiate teams didn't consider it one, and so stopped attending. At the high school level, the same thing happened to Questions Unlimited (though NAC didn't completely die out). Hundreds of teams travel to NAC to have a big ol' hurrah and play teams from across the nation, and almost nobody who posts on these boards considers that a national championship anymore. Heck, HSNCT itself no longer includes computational math tossups because a lot of strong teams thought they were antithetical to quiz bowl.
The point of a NATIONAL CHAMPIONSHIP should be, at least at the top of the list, to determine the national champion. If the teams that are reasonably described as contending for that title think a tournament has flaws that prevent it from doing so, that's a legitimate complaint.

This is not to say that the complaints registered in OP are accurate, are shared by a critical mass of elite teams, or are sufficiently damaging to render HSNCT a fake national championship. Merely to say that, a set of complaints that is accurate, is shared by a critical mass of elite teams, and is significant in scope would indeed damage a tournament's credentials as a national championship.
Indeed, you could have used the example that back in 2007 - 2008, a bunch of elite college players harshly criticized NAQT's collegiate product as outdated, and the result was that NAQT took their criticism to heart, hired a bunch of new editors, and significantly improved their product.
Bruce
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source
User avatar
The Favourite
Wakka
Posts: 118
Joined: Mon Aug 11, 2014 8:01 pm

Re: HSNCT and its Problems

Post by The Favourite »

Disclaimer: Not at an elite school, nor is my team attending HSNCT (although we are attending SSNCT). I don't completely agree or disagree with the points in the OP. A big problem that stood out to me immediately when looking at HSNCT stuff this morning was the fact that a school is sending 8 teams to a NATIONAL tournament capped at 128 teams, while other strong, possibly trophy contending A teams (including the 6th ranked team in the nation according to Groger Ranks!) are sitting on the waitlist that may or may not be admitted to the field.
Last edited by The Favourite on Wed Mar 10, 2021 3:33 pm, edited 1 time in total.
Tracey Hickman
Rock Creek 12
Redlands 14
Central Oklahoma 16
Coach, Murray State College (OK)
Former Coach, Coalgate Schools (OK)
Curriculum Director, Oklahoma Quizbowl Camps
alexdz
Rikku
Posts: 458
Joined: Wed Sep 03, 2008 7:29 pm
Location: Conshohocken, PA

Re: HSNCT and its Problems

Post by alexdz »

The Favourite wrote: Wed Mar 10, 2021 10:03 am ...the fact that a school is sending 8 teams to a NATIONAL tournament capped at 128 teams...
I want to build on this with some additional nuance. Of course, presumably this school was able to qualify 8 teams for the tournament, which is an awesome accomplishment. These students certainly earned a spot based on the published qualification rules, and in that sense alone, there would appear to be nothing "wrong" with their registering. However, perhaps there ought to be some limit, at least initially, on how many teams from a single school can enter the field. I might suggest 3 as the initial cap, with any other teams from the same school being placed on the waitlist. We've done something similar for our statewide events in Missouri -- limit the number of teams per school to 1 or 2 initially, but place C, D, etc. teams on the waitlist. We do this for the same reason Tracey I think is getting at here -- give more schools a chance to enter, which provides for diversity of competition, more geographic representation, and a chance to play against teams with different experiences, knowledge bases, and circumstances.
Alex Dzurick
====
Owner/Editor, SAGES Quizbowl Questions
Middle school teacher, Rohan Woods School
====
South Callaway '08 -- Mizzou '12 -- Illinois '17
SCMS coach '12-'13 -- EFIP coach '20-'21 -- RWS coach '22-present
User avatar
quizbowllee
Auron
Posts: 2180
Joined: Thu Feb 12, 2004 2:12 am
Location: Alabama

Re: HSNCT and its Problems

Post by quizbowllee »

Both the HSNCT and SSNCT pages very specifically say there is a two-team per school limit in place until a later date. Yet, both tournaments have schools that have registered more than two teams.

From the HSNCT Site:

Registration is limited to invited teams through Sunday, March 21. Beginning Monday, March 22 at midnight Central time, any high school may register, with a limit of two teams per school. Beginning Monday, April 4 at midnight Central time, the two-teams-per-school limit will be lifted and any high school may register any number of teams (subject to a cap on the total field size).
Lee Henry
AP Lit and APUSH Teacher
Quiz Bowl Coach
West Point High School
President-Elect/Past President- Alabama Scholastic Competition Association (ASCA)
matthewspatrick
Lulu
Posts: 76
Joined: Tue Jul 15, 2014 11:30 am
Location: Wilmington, DE

Re: HSNCT and its Problems

Post by matthewspatrick »

quizbowllee wrote: Thu Mar 11, 2021 2:59 pm Both the HSNCT and SSNCT pages very specifically say there is a two-team per school limit in place until a later date. Yet, both tournaments have schools that have registered more than two teams.

From the HSNCT Site:

Registration is limited to invited teams through Sunday, March 21. Beginning Monday, March 22 at midnight Central time, any high school may register, with a limit of two teams per school. Beginning Monday, April 4 at midnight Central time, the two-teams-per-school limit will be lifted and any high school may register any number of teams (subject to a cap on the total field size).
You're misreading it. (I had the same misreading at first.)

From the point of the announcement though 21 Mar, any invited team (i.e., a team that qualified through the usual top 15% with ties in an NAQT-linked invitational, or winning a state title) can register.

From 22 Mar - 3 Apr, any school may register up to two teams.

Then, from 4 Apr to the deadline, any school can register as many teams as they like.

Wayzata placed eight teams into the top ten in RAT-RACE. RAT-RACE had 64 teams, which meant the top ten finishers, with ties, got automatic bids for HSNCT.
Patrick Matthews
University of Pennsylvania 1989-94
NAQT Member Emeritus and co-founder
I do not speak for NAQT in any way, shape, or form.
User avatar
quizbowllee
Auron
Posts: 2180
Joined: Thu Feb 12, 2004 2:12 am
Location: Alabama

Re: HSNCT and its Problems

Post by quizbowllee »

Ah. I see that now. That was a little unclear. Thanks for the clarification.
Lee Henry
AP Lit and APUSH Teacher
Quiz Bowl Coach
West Point High School
President-Elect/Past President- Alabama Scholastic Competition Association (ASCA)
Scottietodd
Wakka
Posts: 122
Joined: Sun Nov 13, 2011 1:55 pm

Re: HSNCT and its Problems

Post by Scottietodd »

There are lots of excellent points here from all parties. From my perspective, I don't really "engage" too much when reading specific points/criticisms of 2021 because we are in the middle of a pandemic.... and it is my fervent hope that all of these "online" issues will not be as relevant soon. I do read and think about posts that are relevant to quiz bowl year in and year out and think about them.

Our approach at Glasgow this year has been to a) try to get better, b) try to have fun, c) and try not to get too bent out of shape when something does not go well due to being virtual because everyone is doing the best they can. There have certainly been times this year in which, if an outsider were to come into my classroom (virtually or otherwise,) they might criticize my approach to teaching Algebra and Calculus. To that I would only be able to respond.."I have been doing my best in a year that has been mentally challenging for me personally."

So, while for instance I am certainly a little antsy about some of the rules and procedures for SSNCT this weekend, we will try to respond to stressful situations with patience and grace.

Meanwhile, the main thing I want to do is to thank NAQT and PACE for putting on their championships this year. Being able to continue to have a goal and something to look forward to this year has helped with our mental state and given at least some sense of normalcy in a year that has obviously been very challenging.
Todd Garrison
Glasgow High School
Coach - 2012 to present
scottkim
Lulu
Posts: 70
Joined: Tue Feb 07, 2012 7:49 am

Re: HSNCT and its Problems

Post by scottkim »

Not necessarily related to HSNCT directly, but coaches and players better make sure their internet doesn't drop out in the middle of a tossup, especially on Sunday playoffs!

NAQT will not be flexible with their online rules (https://www.naqt.com/online/technical-problems.jsp) and will allow a phantom buzz from a player who has clearly disconnected from zoom and buzzin.live mid-tossup before they are reconnected to count as a neg!
Scott Kim
Quiz Bowl Coach
North Gwinnett MS (2017-Present)
Collins Hill HS (2006-2017)
User avatar
CPiGuy
Auron
Posts: 1070
Joined: Wed Nov 16, 2016 8:19 pm
Location: Ames, Iowa

Re: HSNCT and its Problems

Post by CPiGuy »

scottkim wrote: Sun May 02, 2021 2:14 pm Not necessarily related to HSNCT directly, but coaches and players better make sure their internet doesn't drop out in the middle of a tossup, especially on Sunday playoffs!

NAQT will not be flexible with their online rules (https://www.naqt.com/online/technical-problems.jsp) and will allow a phantom buzz from a player who has clearly disconnected from zoom and buzzin.live mid-tossup before they are reconnected to count as a neg!
Buzzin.live does not randomly "phantom buzz" players in; if this were a thing that happened it surely would have been documented more frequently (and would presumably not be used as widely as it is). It is unfortunate that disconnecting after buzzing in is ruled as a neg, but any alternative to this rule would open up far worse possibilities for unethical behavior. The possibility of technical difficulties is an unavoidable negative aspect of online quizbowl, but NAQT's rules for mitigating them are clear, reasonable, and in my opinion generally turn to the least bad resolution to bad situations.

(It's possible the player buzzed accidentally, too! But accidental buzzes happen at in-person tournaments too and they're not allowed to be withdrawn there either.)

Coach Kim, I have a great deal of respect for you as a coach, and I understand your frustration that your team was eliminated in a game where technical problems affected the outcome. Blaming NAQT for enforcing their rules and posting in a forum thread about a different tournament just isn't the move, though.
Conor Thompson (he/it)
Bangor High School '16
University of Michigan '20
Iowa State University '25
Tournament Format Database
alexdz
Rikku
Posts: 458
Joined: Wed Sep 03, 2008 7:29 pm
Location: Conshohocken, PA

Re: HSNCT and its Problems

Post by alexdz »

CPiGuy wrote: Sun May 02, 2021 2:57 pm
scottkim wrote: Sun May 02, 2021 2:14 pm Not necessarily related to HSNCT directly, but coaches and players better make sure their internet doesn't drop out in the middle of a tossup, especially on Sunday playoffs!

NAQT will not be flexible with their online rules (https://www.naqt.com/online/technical-problems.jsp) and will allow a phantom buzz from a player who has clearly disconnected from zoom and buzzin.live mid-tossup before they are reconnected to count as a neg!
It is unfortunate that disconnecting after buzzing in is ruled as a neg, but any alternative to this rule would open up far worse possibilities for unethical behavior.
Exactly this. Or else, I might buzz in, realize I don't actually know the answer, and simply disconnect myself to avoid giving my team a neg. Plus, given everything else a moderator or other room staffer is doing, it may be near impossible to catch momentary disconnections from either Zoom or buzzin.live. A player must have taken some action which resulted in a buzz before being disconnected, and as unfortunate as a disconnection is, any other resolution leaves open the possibility that I could buzz, disconnect myself from Zoom temporarily, take a few seconds to cheat, and return claiming a technical error.
Alex Dzurick
====
Owner/Editor, SAGES Quizbowl Questions
Middle school teacher, Rohan Woods School
====
South Callaway '08 -- Mizzou '12 -- Illinois '17
SCMS coach '12-'13 -- EFIP coach '20-'21 -- RWS coach '22-present
scottkim
Lulu
Posts: 70
Joined: Tue Feb 07, 2012 7:49 am

Re: HSNCT and its Problems

Post by scottkim »

CPiGuy wrote: Sun May 02, 2021 2:57 pm
scottkim wrote: Sun May 02, 2021 2:14 pm Not necessarily related to HSNCT directly, but coaches and players better make sure their internet doesn't drop out in the middle of a tossup, especially on Sunday playoffs!

NAQT will not be flexible with their online rules (https://www.naqt.com/online/technical-problems.jsp) and will allow a phantom buzz from a player who has clearly disconnected from zoom and buzzin.live mid-tossup before they are reconnected to count as a neg!
Buzzin.live does not randomly "phantom buzz" players in; if this were a thing that happened it surely would have been documented more frequently (and would presumably not be used as widely as it is). It is unfortunate that disconnecting after buzzing in is ruled as a neg, but any alternative to this rule would open up far worse possibilities for unethical behavior. The possibility of technical difficulties is an unavoidable negative aspect of online quizbowl, but NAQT's rules for mitigating them are clear, reasonable, and in my opinion generally turn to the least bad resolution to bad situations.

(It's possible the player buzzed accidentally, too! But accidental buzzes happen at in-person tournaments too and they're not allowed to be withdrawn there either.)

Coach Kim, I have a great deal of respect for you as a coach, and I understand your frustration that your team was eliminated in a game where technical problems affected the outcome. Blaming NAQT for enforcing their rules and posting in a forum thread about a different tournament just isn't the move, though.
The post was obviously written out of frustration, but I still think others ought to know that NAQT will enforce their rules, even in the case where a "common sense" outcome given the situation (for those of us who were in the game room and who know all the details) isn't the actual outcome.

The rule on NAQT's website mentions scenarios where a player is ruled incorrect if they are disconnected after buzzing in (completely understandable for the reasons stated above), but it's unclear if that same rule applies if a player is disconnected and then buzzes in while not in the zoom. The rule cited to me from the Technical Problems page, doesn't address situations like the one we experienced.

The moderator said he saw <player> (Disconnected) on his screen immediately after the buzz and acknowledged that it seemed accidental since they had to readmit the player into the zoom call. Upon returning, the player had no idea he had buzzed in.

Conor, I posted here since there were folks out there concerned with certain aspects of HSNCT. Since not all HS players/coaches will check the Middle School forum, I wanted to give those folks a heads up of the potential situation that could occur.
Scott Kim
Quiz Bowl Coach
North Gwinnett MS (2017-Present)
Collins Hill HS (2006-2017)
User avatar
Santa Claus
Rikku
Posts: 285
Joined: Fri Aug 23, 2013 10:58 pm

Re: HSNCT and its Problems

Post by Santa Claus »

Now that HSNCT has happened (and Lambert has attended and placed t-19), do you still stand by anything you said in this post?
Kevin Wang
Arcadia High School 2015
Amherst College 2019

2018 PACE NSC Champion
2019 PACE NSC Champion
User avatar
etotheipi
Lulu
Posts: 78
Joined: Thu Jun 13, 2019 9:13 pm

Re: HSNCT and its Problems

Post by etotheipi »

Santa Claus wrote: Mon May 31, 2021 11:25 pm Now that HSNCT has happened (and Lambert has attended and placed t-19), do you still stand by anything you said in this post?
Short answer: Of the criticisms I've made, yes.

Long answer:

First of all, I'd like to give a long-overdue apology for the inflammatory nature of this post, specifically alleging things like people "not even trying" to control difficulty (without proof). It's been a long time since this was a topic of conversation in the Discord, but the point was raised that the perceived low quality of NAQT questions could be due to the different process involved in writing them compared to most housewrites. I failed to consider this when I wrote this post, and thus unwarrantedly accused NAQT employees of malice, which I regret doing. I also used some rather stupid turns of phrase (most prominently "revolution", but also I think I should have chosen a word better than "elite", say just "playoff teams", which would have worked as well without any of the stigma), which I highly regret using. My goal was not (and is not) to sink NAQT as an organization, or to ensure housewrite supremacy, or anything like that. Instead, it was to draw attention to the many issues that I (and many other high schoolers) see in NAQT questions, and hope that NAQT improves by our standards. I think I failed in this regard in the original post.

Second, I'd like to respond to the accusations of hypocrisy that have been levelled (in much less offensive terms, of course) about me attending HSNCT. I do not think it is proper for me to abandon my team at a national competition due to my personal issues/grudges. Lambert's performance at HSNCT is important to a lot of people, not just me: the rest of the HSNCT team, the rest of Lambert quizbowl, both our past and present coaches, the people who we may be able to recruit to the club thanks to the relatively high placement, parents, Lambert's principal - who was kind enough to fund us to play.

On the other hand, there are many NAQT competitions that I can choose not to go to that do not have this kind of impact on my community, and I have made the choice not to go to them whenever possible. This year, I only played in a single regular-season NAQT tournament (to ensure HSNCT qualification), despite the availability of several. I did not play IPNCT. I will continue these trends for next year unless NAQT's question quality becomes better.

That being said, I think that this year's HSNCT was not particularly well-received, based on both my perception (having played both the 2019 HSNCT and this HSNCT, I do think that 2019 was a bit better) and the (admittedly anecdotal) opinions of others who played the set. As I played HSNCT, I noticed that it suffered from many of the question quality issues that I had pointed out in this post. Without getting into details (I was not involved in any live-streamed rounds to my knowledge, and I don't think the earlier rounds are clear), I observed significant swing in the difficulty of tossups/bonuses. It's not that other sets don't do this; it's that HSNCT has done this so much more than practically any housewrite I've played.

I will also comment that the timing rules were inconsistently enforced, at least in the rules I was in. We had one moderator who would call time exactly two seconds after each buzz, and others who gave significantly more leeway to the person who buzzed.

My thoughts about HSNCT's price have not changed, nor have my thoughts about NAQT not releasing their sets for free after every tournament. I really do wish that I emphasized both of these more in the original post. Specifically, the free release of sets that's so common to "housewrite culture" makes quizbowl a significantly more accessible game, reducing the cost of study/practice to practically zero. This has had a large effect, at least in my case - the fact that quizbowl questions were free, and thus that there was no initial commitment required to study quizbowl, played a big role in my beginning to actually study for quizbowl three years ago. In addition, I think that releasing sets helps dampen the negative reaction to those sets.

On the price issue, I'm not arguing that NAQT is conspiring to raise the price of nationals to hike its profits without regard for the teams that play its tournaments. Most of the responses to this issue seem to be in the vein of NAQT being unable to decrease the price any further; I'll trust the people who have advanced this view, as I am not privy to NAQT's finances. I only seek to point out that, whatever the reason, NAQT has charged the same amount for the 2021 HSNCT as the 2019 HSNCT (which was both in-person and guaranteed fewer rounds), an amount that I feel is stretching the ability of teams to pay. We were lucky in this regard, in that our principal agreed to use school funds to pay for our HSNCT berth. Regardless of whether NAQT needed to do this or not, the fact remains that the tournament was still too expensive.

Finally, I'm happy that NAQT (whether in response to my or others' concerns) instituted a cheating policy that at least I felt to be adequate. I do not think HSNCT was at all marred by either accused or actual cheating, a pretty big achievement for such a large tournament.
Arya Karthik (they)

Lambert HS, 2018-22
Georgia Tech, 2022-24
St. Catherine's College, Oxford, 2024-25

2022 PACE NSC; 2023 ACF Nationals
matthewspatrick
Lulu
Posts: 76
Joined: Tue Jul 15, 2014 11:30 am
Location: Wilmington, DE

Re: HSNCT and its Problems

Post by matthewspatrick »

etotheipi wrote: Tue Jun 01, 2021 12:09 pm I will also comment that the timing rules were inconsistently enforced, at least in the rules I was in. We had one moderator who would call time exactly two seconds after each buzz, and others who gave significantly more leeway to the person who buzzed.
This sort of inconsistency can be rather frustrating. Would you say that you experienced more, less, or about the same variance than in 2019?
etotheipi wrote: My thoughts about HSNCT's price have not changed, nor have my thoughts about NAQT not releasing their sets for free after every tournament. I really do wish that I emphasized both of these more in the original post. Specifically, the free release of sets that's so common to "housewrite culture" makes quizbowl a significantly more accessible game, reducing the cost of study/practice to practically zero. This has had a large effect, at least in my case - the fact that quizbowl questions were free, and thus that there was no initial commitment required to study quizbowl, played a big role in my beginning to actually study for quizbowl three years ago. In addition, I think that releasing sets helps dampen the negative reaction to those sets.
Everyone loves free stuff. The more question sets NAQT gives away for free, though, the less they can afford to pay the writers and editors to do the writing and editing. Reduce those fees enough, and some will decide that it's no longer worth their while.

Plus, the list prices on NAQT practice sets, while not free, are not exactly exorbitant: the HS sets range from $2.17/packet for the IS-A sets to $2.58/packet for the IS sets. The per packet rates on the old *SNCT sets looks to be a little lower, in the $1.95 range. Yes, it adds up for a year's worth of material, but these things cost money to produce. If you're saying they should be free, you're also saying that the time people spent creating them has no value.

(I have no idea how much writers/editors for housewrites typically net, but whatever it is, I suspect that it is not enough.)
etotheipi wrote: On the price issue, I'm not arguing that NAQT is conspiring to raise the price of nationals to hike its profits without regard for the teams that play its tournaments. Most of the responses to this issue seem to be in the vein of NAQT being unable to decrease the price any further; I'll trust the people who have advanced this view, as I am not privy to NAQT's finances. I only seek to point out that, whatever the reason, NAQT has charged the same amount for the 2021 HSNCT as the 2019 HSNCT (which was both in-person and guaranteed fewer rounds), an amount that I feel is stretching the ability of teams to pay. We were lucky in this regard, in that our principal agreed to use school funds to pay for our HSNCT berth. Regardless of whether NAQT needed to do this or not, the fact remains that the tournament was still too expensive.
Entry fees are a tough issue. It's true that for a lot of programs, funding this year was tighter and perhaps even non-existent. NAQT didn't set the entry fee where they did to screw those programs, though. I'm not privy to their finances, but I would guess that as any business would, they carefully weighed what they needed to be able to pull off a successful event, what it would take to make things worth their while, and what their customers were willing and able to pay, and charged what they thought was reasonable. Given that their were 224 teams willing to pay that price, I'd say that they did an at least passable job on that estimate.

The entry fee was essentially unchanged vs 2019, and there were two fewer games guaranteed, but bear in mind that teams also did not have to spend time or money traveling to the venue. Not all prospective HSNCT participants are as close to the site as Lambert HS. A team which would have to fly to Atlanta would definitely spend more than the entry fee on airfares and lodging. For many teams, perhaps most, the total cost of participating was less in 2021 than in 2019.

Guaranteeing eight games vs ten was a tough one. It would have been difficult logistically to add 2-3 more rounds on Saturday, and extending the prelims into Sunday would have meant a very late ending. There was also a risk, if there were additional prelim rounds on Sunday, that a fair number of teams with no shot at the playoffs might have simply bailed. Among the 141 non-playoff teams, only ~60 signed up for consolation play, so at least among the teams which did enter HSNCT, I'm not sure there was a ton of unmet demand for more gameplay. (It could well be that lots of teams which decided not to register would have played had there been an upfront ten game guarantee; I have no way of knowing that one way or another.)

Also, while NAQT didn't have to rent hotel meeting space or fly in staff, to a large extent those costs got replaced by other costs (e.g., Zoom fees, paying staff, etc.). No one was getting rich off of this.
etotheipi wrote: Finally, I'm happy that NAQT (whether in response to my or others' concerns) instituted a cheating policy that at least I felt to be adequate. I do not think HSNCT was at all marred by either accused or actual cheating, a pretty big achievement for such a large tournament.
Having spent all of ICT, HS IPNCT, SSNCT/MS IPNCT, MSNCT, and HSNCT in the control room, I can tell you that NAQT took anti-cheating measures very seriously. Things weren't perfect by any means, but the effort that went into this was considerable.
Patrick Matthews
University of Pennsylvania 1989-94
NAQT Member Emeritus and co-founder
I do not speak for NAQT in any way, shape, or form.
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 6112
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: HSNCT and its Problems

Post by Important Bird Area »

etotheipi wrote: Tue Jun 01, 2021 12:09 pm Without getting into details (I was not involved in any live-streamed rounds to my knowledge, and I don't think the earlier rounds are clear), I observed significant swing in the difficulty of tossups/bonuses.
The 2021 HSNCT set is actually clear (and I'll have a discussion thread later today).
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
Locked