Housewrites vs. NAQT

[slimg]

With housewritten tournaments becoming more and more abundant, I was wondering if they are truly better than buying questions from a company. It's obviously more lucrative for a team to do a housewrite, but can they provide the same or better experience than company questions? What are your thoughts?

[Matt Weiner]

If you do it properly, you will get a large benefit to your team in terms of learning the material covered in the questions and getting an inside understanding of how questions are constructed in general. It will also provide you with an experience in organizing a large team project that may help you in areas besides quizbowl. The best house-written sets are usually considered as good or better than the best vendor sets. Again, that's if you do it properly.

The worst house-written sets provide the hosting team with no real benefit besides saving the cost of buying questions, and are way, way worse than even the most egregious outliers in an NAQT set. (I can't really conceive of anything being worse than Questions Unlimited, so I won't say you could really undershoot whatever their "standards" are even if you tried, but we all should set the bar a little higher than that.)

If you want to be ready to write a tournament in-house, then you should make sure you have several good writers who are willing to redo less experienced teammates' questions to bring them up to par, make up inevitable shortfalls from less committed teammates flaking out entirely, and generally take on the burden of getting a good set done no matter what it takes. If you're not sure where you stand, you can always write a few questions and send them to an experienced collegiate editor for comment.

[slimg]

You bring up another problem with housewrites: assurance of quality for the other teams in attendance. Quizbowl teams are forced to make a huge commitment of time and funds in order to attend a tournament to which they have no assurances (ie Moravian or State College going to Yale, or, on a much larger scale, Shanghai American coming to Harvard).

[Stained Diviner]

If you want housewritten questions to be better than what vendors are selling, then you need the leaders on your team to agree on what the weaknesses of, say, an IS set are. If you have some dedicated students who want to provide a tournament that is more challenging than IS, or a more academic distribution, or something along those lines, then you have a chance of producing something better. Otherwise, there isn't much point--there is still a good chance that you will end up with something worse than IS, but there is at least some chance that you will end up with something better.

If you think IS and QU are comparable, then you probably shouldn't be writing questions. Nothing personal. It is possible to write questions worse than QU, though.

Additionally, that core group of students needs to be dedicated and needs to be willing to criticize each others' questions and put that criticism to good use--repeatedly editing the questions until they are good. It takes the right group of students, and it takes a lot of time. You also need to set some intermediate deadlines and, if they aren't met, then be ready to cancel the tournament or order some questions.

Attracting teams is a different issue. Generally, tournaments are small in their first year of existence. A lot of it comes down to convenience and publicity--pick a good date well in advance and contact as many teams as you can. If you want teams to travel long distances, give them a reason to believe that your tournament will be a very good one, which is a difficult argument to make and should involve the quality of the field and the questions.

[Sen. Estes Kefauver (D-TN)]

If you don't feel you have the capability to write an entire tournament on your own, there are some other options you can pursue. A movement in tournament writing is to have schools collaborate on tournament sets with distant schools and have both of them run tournaments on the set. This season, there are lots of these types of tournaments, just read through this thread. If you are interested in getting some writing experience but not writing an entire 10 or 12 round tournament, I would think you could get in touch with some of these writers and work out something where you write a few games worth of questions, send them in to the more experienced editors, and get to use to set.
Independently of whether you run a tournament on questions you write, I can not stress enough that writing 6 line or more clue dense tossups on topics that come up frequently in quizbowl is one of the best ways to improve your skill, as you internalize the information, and you can then save the questions for later uses - one thing worth exploring is freelancing packets for lower level college tournaments.

[Kyle]

You bring up another problem with housewrites: assurance of quality for the other teams in attendance. Quizbowl teams are forced to make a huge commitment of time and funds in order to attend a tournament to which they have no assurances (ie Moravian or State College going to Yale, or, on a much larger scale, Shanghai American coming to Harvard).

To be fair, it doesn't matter how good the questions are or how many other excellent teams are in attendance because when Shanghai decides to attend a regular season tournament in the United States, that is an irrational decision on their part. But top teams from closer, like Dorman or MLK or TJ, start agreeing to commit their resources to attend a tournament after hearing that other top teams will do the same. The quality of the field is not an unknown when teams start signing up, even if those teams coming have no control over the questions they will be asked.

[slimg]

The quality of the field is not an unknown when teams start signing up, even if those teams coming have no control over the questions they will be asked.

Just because the field of a housewritten tournament is strong, it doesn't mean that the tournamnet will be a good, fair one (ie Dorman A vs. Moravian -315 to 285 at Harvard - Sorry Kyle- with Moravian not even at full strengh.) Compare that to Dorman B vs. Moravian - 375 to 230- at the NAQT HSNCT. I'm sure Dorman A did not feel the contest at Harvard was fair. And there in lies the problem. How can a school, such as Dorman, evaluate a housewritten tournament before they come all the way from SC? At least when a school attends a NAQT tournament, they have assurances about the quality of the questions.

[Sen. Estes Kefauver (D-TN)]

What on earth?

[Kyle]

I guarantee you that no one on the Dorman team found Harvard's tournament "unfair" because they weren't winning by enough.

[Magister Ludi]

I would be shocked if Dorman thought that Harvard's questions were "unfair" in any way. As far as I've heard the HFT set was praised by top players as one of the better sets of the year (far superior to NAQT sets). If you have specific qualms about the quality of the questions please email me, Andy Watkins, or Kyle haddad-Fonda so we can improve for next year.

As to your real question. One can easily evaluate the quality of a house-written tournament by looking at its editors. If you see that Matt Weiner is the chief editor of a set you are assured a certain caliber for the questions. If you see that the tournament is being written by a first time writer then you must be more dubious. For evaluating first time tournaments I would suggest looking at whether the editors understand good quizbowl. If someone who understands the underlying principles of pyramidal question (such as Chris Carter in the upcoming No Name Tournament)is editing a set then you can at least be assured of a certain quality.

[slimg]

Just to clarify- I don't have any problems with the Harvard tournament. I just knew the most about yours from our teams attendance, so I used examples from your tournament .

[Kyle]

No, what you said was that our tournament was less fair than a tournament run on NAQT questions and cited as "evidence" the fact that Dorman didn't beat you by a very large margin in the one round you played against them and your assumption about the way other people feel. Please think for a moment about whether that evidence supports that allegation.

(Also, re: Guy, I'm not defending the quality of a set I edited, which is for others to judge; I'm only defending the fact that our tournament was fair.)

[Sir Thopas]

The quality of the field is not an unknown when teams start signing up, even if those teams coming have no control over the questions they will be asked.

Just because the field of a housewritten tournament is strong, it doesn't mean that the tournamnet will be a good, fair one (ie Dorman A vs. Moravian -315 to 285 at Harvard - Sorry Kyle- with Moravian not even at full strengh.) Compare that to Dorman B vs. Moravian - 375 to 230- at the NAQT HSNCT. I'm sure Dorman A did not feel the contest at Harvard was fair. And there in lies the problem. How can a school, such as Dorman, evaluate a housewritten tournament before they come all the way from SC? At least when a school attends a NAQT tournament, they have assurances about the quality of the questions.

Since all the people who have defended the Harvard set so far have been from Harvard—as one of the so-called elite team players(TM) that Ted's talking about, HFT was indeed awesome. Much, much better than the HSNCT set. There are a couple of possible reasons for the results you mentioned, some or all of which came into play. The stats no longer seem to be available, so I can't quite confirm some of my ideas entirely.
1) Both Dorman A and Dorman B have at least about a 150-point advantage against Moravian on paper. Based on what I've seen from the three teams, this is definitely a possibility.
2) Dorman is better at NAQT; Moravian is better at mACF. Dorman certainly has a distinct buzzer advantage thanks to The Dorman Buzz(TM), so that might have widened the margin of victory on NAQT questions a bit, but they also have really deep knowledge, so I wouldn't put too much into this.
3) Moravian might have just had a good game, dude. Or Dorman, a bad one. Or the subdistributions in the packet might have played more to your strengths. This happens, and doesn't necessarily reflect poorly on the questions. Perhaps Dorman just got caught up in a NEGSTORM or something similar.

In short, a surprising or closer than expected result doesn't make the packets terrible. It's when people get pissed at the packets, and when way more surprising results are consistently happening than everyone agrees should occur, that people start grumbling. Did you, personally, dislike the HFT set?

(EDIT: This post has been partly superseded by the previous two posts as I was writing this. Oh well.)

[Kyle]

Dorman indeed had five negs in that game.

(Actually, this is interesting but totally irrelevant both to this discussion and especially to a thread that has wandered far from its original topic: both Dorman and Moravian scored 75 points on tossups, but Dorman had a bonus conversion of 24 compared with Moravian's 18.67)

[Mechanical Beasts]

Just because the field of a housewritten tournament is strong, it doesn't mean that the tournamnet will be a good, fair one (ie Dorman A vs. Moravian -315 to 285 at Harvard - Sorry Kyle- with Moravian not even at full strengh.) Compare that to Dorman B vs. Moravian - 375 to 230- at the NAQT HSNCT. I'm sure Dorman A did not feel the contest at Harvard was fair. And there in lies the problem. How can a school, such as Dorman, evaluate a housewritten tournament before they come all the way from SC? At least when a school attends a NAQT tournament, they have assurances about the quality of the questions.

You're looking at a score. I'm sure whoever read that match or played in it could tell us more.

I'd point you to the finals of the 2005 HSNCT. Thomas Jefferson over Lakeside, 415-325. Should we excoriate that final packet (or the set, or the writers, or whatever) because Lakeside wasn't crushed? Lakeside was 9-1 in the prelims but scored slightly over half the points that TJ did. Looks like a bad set! (Humorously enough, this anecdote deserves a "sorry Kyle" too.)

But really, that's not the case. Listening to that match, TJ negged an incredible amount, especially in the first half. And looking to the sixth playoff round, we have TJ A 760 to Lakeside 30. Maybe that game, TJ played better and didn't have what, ten negs, like they did in the final?

Seriously, I think it's wrong to judge a tournament as not good or not fair because of the result of one game. If Moravian swept the field, beating Whitman and Dorman handily, then there would probably be problems; the latter two teams are decisively better than the first. Looking at the overall results--Whitman went 11-1, losing to Dorman once; Dorman went 10-1, losing to Whitman once. In the top bracket, it went Whitman, Dorman, State College, TJ A. Are those results that disquieting? I don't really think so.

I'd like to echo Ted's response, though, in that if you have legitimate complaints about the set, please email me your concerns and I will address each of them in full. I'd ask, however, that you refrain from judging our tournament based on the result of one match out of 161.

That said, any publicity for the THIRD HARVARD FALL TOURNAMENT ON 11/15/2008 is good publicity.

YOU ALL SHOULD ATTEND THIS YEAR. I will be sitting in a dunk tank and if you can fling a copy of the offending packet (packet one, I believe) at the button hard enough, I'll plummet into a lake of cyclohexane. Which really isn't too different from my day to day work in the Matyjaszewski lab, granted.

Just kidding. But seriously, for two years people have said that it's a great tournament, and this year's is going to be the best yet.

[Sir Thopas]

And Kyle (and others), therein lies your answer. Moravian's bonus conversion in that round was nearly FIVE points per bonus above their peak in the other four games. Seeing as Dorman's bonus conversion* was pretty consistent in that round, I have to conclude that Moravian just got really lucky with the bonuses they received that round. And Dorman's five negs couldn't have helped them, either.

*EDIT: And everyone else's.

[cvdwightw]

How a particular team does in a single game against a single opponent cannot be construed as "evidence" that a tournament was or was not of high quality. I mean, I've beaten a team by 100 points and lost to that team by 100 points in the same tournament, at least twice (once in high school on NAQT and then at ACF Nationals last April, and that's just off the top of my head), probably much more.

I'm not sure what this "assurance of quality" means. I think all of us have at some point gone to a tournament because it's supposed to be good (presumably we have been "assured" of its "quality") and then realized halfway through that it isn't so good. You can't actually give teams an "assurance of quality" unless you send them the questions and they agree that they are indeed quality questions. What NAQT and several well-regarded house-written tournaments have is a reputation for quality, which they may or may not actually deliver on for a given set in a given year. However, I don't think any new tournament comes in with a "reputation for quality", even if it's written by a bunch of writers who have those reputations - the reputation of the tournament itself has to be earned.

[Kyle]

Okay, back to the point of this thread — there are an increasing number of house-written tournaments or mirrors of house-written tournaments already announced, all around the country, that you can expect to be of high quality. These include tournaments at Brown, Illinois, Harvard, VCU, Minnesota, Minneapolis South, Georgetown Day School, Notre Dame, Hunter, Tulane, Raleigh Charter, San Diego, Thomas Jefferson, and many others I don't have the time to track down. There are also a lot of college tournaments or open tournaments whose tournament directors will consider letting elite high school teams play.

There are a lot of reasons why you should go to these tournaments as well as tournaments with questions provided by established vendors. The main reason is that the questions will be of very high quality (not every question, necessarily, but every tournament). Others include: you'll get better quality opponents, who won't travel for NAQT sets they can play at home; in certain cases, as advertised, you'll get harder questions; you'll be supporting a circuit that sustains itself by having teams write questions and run tournaments; you'll get a glimpse of what it will be like someday when you edit a tournament or at least write questions for one; you'll almost certainly get a distribution with less pop culture than NAQT questions offer; you'll have the opportunity to visit colleges you might want to attend someday (this is a great way to justify making your parents send you to HFT); you'll get better by playing in more tournaments, and there is a limit to the number of sets NAQT puts out each year; and you'll get practice at a format somewhat closer to PACE and get more chances to qualify for the NSC.

[cvdwightw]

Wait, now I'm confused. Is the point of this thread to discuss the benefits of:

A. Writing or mirroring a house-written tournament as opposed to running a tournament with questions from a national vendor (i.e. from an administrative end), or
B. Attending a house-written tournament as opposed to a tournament with questions from a national vendor (i.e. from a participatory end), or
C. Both A and B?

[dtaylor4]

Wait, now I'm confused. Is the point of this thread to discuss the benefits of:

A. Writing or mirroring a house-written tournament as opposed to running a tournament with questions from a national vendor (i.e. from an administrative end), or
B. Attending a house-written tournament as opposed to a tournament with questions from a national vendor (i.e. from a participatory end), or
C. Both A and B?

I think the initial intent was writing one's own vs. NAQT, but the writer did not know of other options, now it's developed into C.

[theattachment]

Note that in replying I only read to the post I quote below.

First off,

Since all the people who have defended the Harvard set so far have been from Harvard—as one of the so-called elite team players(TM) that Ted's talking about...

Screw you, Guy

Anyhow, housewrites are certainly a different animal than NAQT. I don't necessarily believe that there's as much of a difference in overall, per-tournament performance between a team on an mACF set and an NAQT set -- see the fact that Dorman performed about as well as they did at WoQ Saturday, HSNCT, and PACE. However, round to round there is a definite difference in packeting between house sets and NAQT sets. For a full set, NAQT will have a distribution similar (save for trash and CE) to an mACF set, but within packets you'll have five trash compared to four lit, a bunch of SCIENCE and the occasional terrible question. House sets usually are of a similar quality with per-packet distributions. For playability, a housewritten set is preferable to an NAQT set if and only if the quality of writing is the same.

Because of that last point, you have to take caution when undertaking a housewritten set. To be totally honest, most high school players should not yet be writing questions. Unless they're talented within subjects, the average player simply does not yet have the correct grasp of the canon to write good questions that are pyramidal; worse still, often you find high school students that can't write. To make a solid set, you need writers that know information and can write clues, as well as editors to make sure that the clues are ordered correctly. If this comes together, you get a set that probably is better than the average IS set. If not, you get something beyond sub-par.

As for which house sets to play, usually if a tournament has a track record of continually drawing really good teams (namely ones from far away), play it. Elite teams don't come back to play terrible questions. In addition, I would love to pimp the No Name Tournament, as Chris and Ian are excellent writers that will put together a set fitting their talent. Also, since an ex-teammate is apparently science editing it, Prison Bowl should be stellar this year.

[rchschem]

To be totally honest, most high school players should not yet be writing questions.

This is true, but it is also a very good reason to teach them how. As was mentioned earlier, writing is a great way to learn, both content, and how to "feel" questions that others have written. Editing is the key here. At Raleigh Charter our entire team writes questions, after we have had several writing and critiquing workshops.

[Sir Thopas]

Also, since an ex-teammate is apparently science editing it, Prison Bowl should be stellar this year.

Hmm, first I'm hearing of that. Good to know, though.

[Auks Ran Ova]

To be totally honest, most high school players should not yet be writing questions.

This is true, but it is also a very good reason to teach them how. As was mentioned earlier, writing is a great way to learn, both content, and how to "feel" questions that others have written. Editing is the key here. At Raleigh Charter our entire team writes questions, after we have had several writing and critiquing workshops.

I have a feeling that Colin meant that most high school players should not yet be writing tournaments, as most high schoolers are not yet experienced enough with the many many facets of question-writing to produce a good set. However, this is all the more reason they should be writing questions on their own, to get better at it.

[Sen. Estes Kefauver (D-TN)]

I will stand by my solution to the idea of improving your writing - if you don't have a house tournament to use your stuff in, submit the questions to other tournaments, either high school sets you won't be playing in, or novice college sets with good editors, and then solicit feedback from different people after the tournament. I personally think it would be a fantastic idea for everyone to take the summertime to just write 1 complete ACF style packet.

[NoahMinkCHS]

Has anyone ever tried a packet submission (optional) HS tournament? I'm imagining a college or HS team doing an essentially house-written tournament but offering significant discounts for writing a packet. You probably wouldn't get many takers, but I'd guess (in a few regions at least) at least 2-3 teams would give it a shot. And top-tier HS teams' writing is probably comparable to (or better than) most inexperienced college players, so as long as the set editor knew what they were doing, quality shouldn't suffer.

Potential pitfalls that I see:
--Discounts might not be that useful since HS teams are often school-funded with less limited budgets than college clubs. Another incentive of some kind might need to be worked out; maybe free lunch? Also, if you had a lot of teams doing it, a best packet award might encourage some to do it just for competitiveness's sake.
--Some coaches would end up writing their team's packet or at least doing significant work on it. This wouldn't hurt the tournament but would defeat the purpose of the exercise. I guess given that, the best defense would just be to remind the coaches why you're doing it and ask them to be honorable about it.
--Speaking of honor, it would have to be doubly and triply stressed to keep questions blind to B/C/D teams, etc., since HS teams have no experience with that sort of thing. (That said, if I were TDing, I would still try to make sure all of a school's teams -- as much as possible -- had byes in the same round.)
--The hosting team(s)/editor(s) would need to have a ton of extra questions available since HS teams, even less than college teams, would be likely to underestimate the time/dedication needed to write a good packet, and are also likely to flake out at the last minute due to that.

Anyway, has this ever been attempted? Would anyone be interested in trying it?

[cdcarter]

Has anyone ever tried a packet submission (optional) HS tournament? I'm imagining a college or HS team doing an essentially house-written tournament but offering significant discounts for writing a packet. You probably wouldn't get many takers, but I'd guess (in a few regions at least) at least 2-3 teams would give it a shot. And top-tier HS teams' writing is probably comparable to (or better than) most inexperienced college players, so as long as the set editor knew what they were doing, quality shouldn't suffer.
[...]
Anyway, has this ever been attempted? Would anyone be interested in trying it?

NNT is semi-unofficially offering discounts to teams who send me a packet. The plan we are going to use is just not to use that packet at all at that teams home site. So far we have one team who is probably sending us a packet.

[Mechanical Beasts]

--Speaking of honor, it would have to be doubly and triply stressed to keep questions blind to B/C/D teams, etc., since HS teams have no experience with that sort of thing. (That said, if I were TDing, I would still try to make sure all of a school's teams -- as much as possible -- had byes in the same round.)
--The hosting team(s)/editor(s) would need to have a ton of extra questions available since HS teams, even less than college teams, would be likely to underestimate the time/dedication needed to write a good packet, and are also likely to flake out at the last minute due to that.

A potential solution to both of these problems would set a restriction on the scale of the tournament but would be a great solution if that's not a problem: bracketing. Make teams submit half-packets or (if you're really lucky and can manage it) quarter packets. Granted, you'd have to give each fourth of the teams slightly different specifications, since (in ACF) one of the half packets would contain the SS and another the Geo, and the only thing that would be constant in the quarter packets would be the Big Three... but you see the possibilities.

[theattachment]

To be totally honest, most high school players should not yet be writing questions.

This is true, but it is also a very good reason to teach them how. As was mentioned earlier, writing is a great way to learn, both content, and how to "feel" questions that others have written. Editing is the key here. At Raleigh Charter our entire team writes questions, after we have had several writing and critiquing workshops.

I have a feeling that Colin meant that most high school players should not yet be writing tournaments, as most high schoolers are not yet experienced enough with the many many facets of question-writing to produce a good set. However, this is all the more reason they should be writing questions on their own, to get better at it.

To some extent, Rob is right about my assessment, but he misses something that isn't just a QB problem but a problem at large. Rob's suggestion for people to write questions to gain experience writing is very good for people whose only problem is with writing questions. It certainly helps players learn new knowledge about stuff that comes up, but equally important it helps players understand the game a lot more.

My comment, however, is directed at people that don't know how to write instead of those who don't know how to write questions. Cooperatively writing a house written set, even if it's a trash one, shows off the exact way to screw up a question by ambiguous wording, unreadable syntax, and terrible grammar. You see questions that could ask twenty different things because of pronoun usage. No matter how good the question is structured clue-wise, the fact remains that someone has to re-write it to make it not burn your eyes out. When you have to compile a set before a deadline, this becomes a major issue. Having to write 5/5 in a night because you're short turns in to having to write 15/15 to make decent questions readable and playable.

It doesn't seem that the college game has as many problems with this, possibly because people usually get breaks on writing questions as first-time players and during that honeymoon period they're forced to take some sort of freshman composition class. High schools just teach a half-way version of composition unless you're in an acutal composition class. Because of this, in a high school tournament it helps to have a style editor whose job it is to read over questions and ensure they're properly written.

[Sen. Estes Kefauver (D-TN)]

Every high schooler I've met who has the motivation to improve at quizbowl has enough of a grasp of the English language to not make your concerns a problem.

[Gautam]

To some extent, Rob is right about my assessment, but he misses something that isn't just a QB problem but a problem at large. Rob's suggestion for people to write questions to gain experience writing is very good for people whose only problem is with writing questions. It certainly helps players learn new knowledge about stuff that comes up, but equally important it helps players understand the game a lot more.

My comment, however, is directed at people that don't know how to write instead of those who don't know how to write questions. Cooperatively writing a house written set, even if it's a trash one, shows off the exact way to screw up a question by ambiguous wording, unreadable syntax, and terrible grammar. You see questions that could ask twenty different things because of pronoun usage. No matter how good the question is structured clue-wise, the fact remains that someone has to re-write it to make it not burn your eyes out. When you have to compile a set before a deadline, this becomes a major issue. Having to write 5/5 in a night because you're short turns in to having to write 15/15 to make decent questions readable and playable.

It doesn't seem that the college game has as many problems with this, possibly because people usually get breaks on writing questions as first-time players and during that honeymoon period they're forced to take some sort of freshman composition class. High schools just teach a half-way version of composition unless you're in an acutal composition class. Because of this, in a high school tournament it helps to have a style editor whose job it is to read over questions and ensure they're properly written.

huh?

So far, I've come across a fair number of packet submissions and unedited questions written by a variety of people (including high school students), but I have yet to see submissions rife with problems you are describing. I speak of both the college level and at the high school level.

Sure there are some problems with non-clues sneaking in to some questions, or pyramidalty issues, but major issues with the usage of english language are rare. What person, who is not particularly good at writing, will assume responsibility of writing something as specialized as writing tossups and bonuses?

Gautam

EDIT removed unnecessary quotes at the beginning

[STPickrell]

Another thing that a mid-level team might consider is getting an alumnus and/or other QB 'expert' to edit the house-written set. The edits have to be substantial -- indicating why you're moving clues around, if there's too many questions in subject area 'X', etc. To make this work, the matches must be in to the editor at least a month before the tournament -- otherwise you get hastily edited questions.

[ClemsonQB]

Perhaps Dorman just got caught up in a NEGSTORM or something similar.

Yes, that was me for the most part

[slimg]

Another thing that a mid-level team might consider is getting an alumnus and/or other QB 'expert' to edit the house-written set.

What constitutes an expert?

[Matt Weiner]

What constitutes an expert?

A good question. This is a good place to start: viewtopic.php?p=80410#p80410 . There are more good writers not listed, but that is the majority of them.