The Quizbowl Resource Center

Our state is bidding on a new vendor for our tournament questions. Assuming this is an appropriate forum, can you share your opinion of Avery Enterprises and Questions Gallore?
thanks!

People certainly can and will do that. However, be aware that you asked a question and will get a brutally honest answer.

Selecting either of those companies will result in a set that is nothing short of an insult to students who enjoy playing this game.
Bryce Avery has a track record of ignoring criticism, threatening lawsuits, and name-calling at people who are young enough to be his child, as well as producing questions that are riddled with factual errors and buzzer races.
Questions Galore, while not coming with the ethical baggage that Bryce Avery brings to the table (as far as we know), produces sets that are even worse in quality, have impossible calculations, multiple questions per round on things nobody in high school can reasonably be expected to know, a lack of giveaway clues that might actually help conversion, nothing but speed checks for the questions that are answerable, and a propensity for blatantly ignoring the distribution they are assigned.
If I were you, I would steer clear of either of these options.

It is the general consensus of many on this board, supported by varying degrees of hard evidence, that coaches of bad teams will generally attempt to get short, buzzer-race questions often rife with factual errors, because the deciding factor is then buzzer speed and not knowledge, and thus they have a chance to spring the upset on even the most knowledgeable teams. This can be incredibly frustrating to players who are either on good teams or are willing to put in the effort to become good.

Any company that cannot offer pyramidal questions which have difficult information at the beginning and taper down to an easily accessible giveaway should, plain and simple, not even be considered. The reason for this is that non-pyramidal questions must use the relative obscurity of answers to differentiate between teams of different knowledge levels, so more questions go dead in more matches. Thus, these questions are generally not actually as fun to play as questions where most teams will know the answer, even if players do have to wait for the giveaway clue to answer the question.

Based on what little I have seen from these companies, neither of them can make an honest claim to provide pyramidal questions. However, as I have not had direct interactions with either of these companies, I would check with third parties to verify whether or not their questions are pyramidal.

You can check my above post as one of those third parties - I've played 6 tournaments on Avery and 1 on Questions Galore (that was heavily edited because there was almost no literature in the original set) as well as seen many of their sets in practice and moderated at one tournament held on them. All of these providers are unacceptable and not worthy of the money that your state would be paying them.

Ditto to everything Charlie said. I've coached teams who have played at tournaments using these questions, and I wouldn't recommend either of them.

You might try writing letters (better yet, get coaches and students of strong or want-to-be strong teams to write letters) to the members of the Oklahoma board recommending a better vendor, with some examples and samples. In other words, be part of the solution.

I know many people on this board do not like NAQT A Level sets, but that would be a start and hundreds of times better.

Our conference uses Avery Questions and I cannot tell you how horrible they are. They are not pyramidal in the sense that the first clue is often the easiest clue. All literature seems to be pop lit, meaning you will hear more questions on Harry Potter than you will on the works of Tolstoy or Dickens. The math always seems to be really easy or really hard. We recently had the tossup "Multiply (1/10) times (2/3)." The best advice that I can give you is to not use them. I have not played Questions Galore enough to tell you my opinion, but I know that they are not well thought of. My advice would be to go with other companies, even if it costs you more.

I know the OSSAA wants questions to be two lines or shorter. Sadly, many of the pyramidal companies will not write questions that short, as (and I agree with them) those are simply too short to adequately distinguish between the knowledge levels of the two teams.

I would strenuously advise the OSSAA to consider letting questions at regions and states be 3-4 lines due to the higher talent levels of the teams involved.

If the format cannot be changed, here is what I would do:
1. Seek a question writer that will *attempt* to put 2-3 clues in a question. e.g. "Who created the character of A in book B and the character of C in book D?" as opposed to "Who wrote book D?" or even worse, questions with 1.5 lines of fluff "He wrote novels full of suspense and drama. Who wrote book D?" or a 90-degree turn "Someone else wrote book A. But who wrote book D?" or questions that ask "why?" or "how?" (Unless, of course, 10-minute protests or asking the teacher who's volunteered to help out are your cup of tea.)
2. Someone that at least attempts to follow a distribution, and doesn't let pet categories dominate within said distribution.
3. Someone that is factually correct, and includes alternate acceptable answers/answers that are close and shouldn't be prompted).

As for lightning rounds, you want someone that composes lightning rounds that are (1) answerable and (2) written so they can be completed in 60 seconds. Is there a reason the time cannot be extended to 90 or 120 seconds? This allows for multi-discipline lightning rounds which are, IMO, better (e.g. 'things that start with B' or 'things related to the month of April.')

Is what I advise the ideal? Of course not.

Is it an improvement? I believe it is. I believe (and others might disagree) that it is better to play quizbowl that is 20% of the ideal instead of quizbowl that is 10% of the ideal.

Good luck. Are there any other bidders?

BG MSL Champs wrote:We recently had the tossup "Multiply (1/10) times (2/3)."

That's Numberwang!

Unfortunately, I know that I played on Avery. As far as I remember (this was 2005-06 DVC conference), the questions lacked any semblance of pyramidality. I don't remember any skewed canon, but, then again, nearly EVERY tournament that year had 1/1 Harry Potter (or so it seemed).

As for QG, our school's F/S tournament we run uses these questions. I've agreed to edit them, so I'll report once the tournament is held just how bad the questions were. In the meantime, you can view some of last year's questions here.

All my opinion.

Just wondering: can you post the call for proposals? From my own standpoint, I would like to find out what Oklahoma is specifically looking for and values in the questions to pick the vendor that best fits its programmatic goals. How are the vendors going to be accountable for satisfying those criteria? I say this because as it has been pointed out, if the selection committee is being told by its membership to find writers who can produce a bunch of original one-sentence buzzer-race questions with 40% math calculation without justifying how that distribution meets the programmatic goals of Oklahoma (or other state) quiz bowl, then there are going to be problems. I presume each of those vendors sent in sample packets from previously used events, and a sampling of positive references?

Yes, squeaky wheels that many people are describing as "bad teams" are going to be more vocal than the "good teams" because in competitions like this, there will be more teams that do not win the state championship (or even advance) than there are teams who will win. I agree that everyone in education would like to see more parity in a competitive activity, but I also note that there are other ways to do this than lowering curricular standards. How amenable quiz bowl (state, nationally) is to pursuing these other avenues is another discussion.

But yes, the advice you receive here from many of the better teams and coaches is very valuable as you evaluate your proposals, and I would always want to know with any q-writing company appropriate probative questions on how they adjust appropriately to feedback.

The other major thing is question delivery. That is always an issue that I think many complain about (will you deliver questions when I ask you to... say two weeks before the tournament).

The last is appropriate proprietary issues: are you going to summarily edit questions from your contractor without the contractor's permission, then claim any problems with the questions are the fault of your contractor? I think that if this continues, I would expect to see q-writing companies have a stated warranty on their questions where users will be subject to an additional fee if any of the question sets are modified without prior permission. That's like buying a Mercedes, but then deciding to put in your own engine. How far should q-writing companies go to customize their question sets to conform to certain game formats should be explicit, and purchasers should be upfront with their intentions on why they are looking to work with said company. Furthermore, if you are going to take that much time to edit someone else's work, you might as well write your own.

Since I haven't purchased questions in ages for tournaments (from vendors), I'm just relying on what people have posted here. I don't think there is such acrimony with a lot of the companies that are being used most frequently, but I think that the quality of our competitions can be much improved if appropriate communication of expectations and subsequent/frequent progress and follow-through is done. To that end, I'm not sure that most of our cottage q-writing companies have reached that level of customer service (nor should I expect them to) so I think anything that can be done to strengthen them through constructive feedback would be great.

Shawn's post is one example of articulated question expectations that I guess I'd like to see each vendor state philosophically and each purchaser articulate. I think Illinois and Kentucky do this clearly, and whether people like that game format or not, that's up to each state association/host to decide.

For the IHSA (Illinois) State Series, the questions are written and edited by a fairly large group of former coaches and players, some of whom have no connection to Illinois. This approach, which came about after years of being frustrated by a particular vendor who is not one of the two you mentioned, has its pros and cons. The cons are that it ends up being a lot of work, since willing writers are sometimes difficult to find, writers sometimes back out of commitments, and it is pretty much impossible to get college students and former coaches to agree on what constitutes good questions. The pros are that we can put somebody in charge who makes sure that the questions meet our constraints and that deadlines are met. Additionally, we have gotten many very good writers who would not have been interested in writing the entire State Series but have been willing to contribute a fair number of very good questions.

The questions we end up with are certainly imperfect, though I believe that they are better than what is available from commercial vendors willing to write our State Series in our format.

I am sure that BA and QG submitted plenty of sample questions to you, and they both have a long history of delivering lots of questions on time to lots of tournaments. At some point, this decision will be made by the people in charge looking at the samples and deciding what they like better. However, much of the advice given above is very good advice. In Quizbowl, Issue #1 is Question Quality. A well-run Quizbowl organization with bad questions is like a well-run sports franchise with bad athletes. Take a look at different questions that are out there, including ones found here, here, and here, and ask yourself which questions do the best job of promoting learning. Because Quizbowl is largely about learning, those probably are the best questions.

Good vendors:
-NAQT. While shorter and flawed, the questions are still pyramidal. IS sets are varsity sets, A sets are junior varsity sets. A sets are terrible, and IS sets have serious problems, but they are a major improvement over the vendors you mentioned. I belive the NAQT website has a sample packet.
HSPAQ-probably the best vendor whose sets I've played. Good questions, good distribution. There's a link to a sample packet in the discussion forum.
Ageis-I've never played their sets, but they have a good reputation, so you should probably check them out.

Question: how much computational math does Oklahoma want? This will greatly influence who will be willing to write.

Well, if Oklahoma coaches wil revolt at the suggestion that mathcomp should be kept to a minimum or even eliminated, I think NAQT would allow hosts to add mathcomp.

Anti-Climacus wrote:Well, if Oklahoma coaches wil revolt at the suggestion that mathcomp should be kept to a minimum or even eliminated, I think NAQT would allow hosts to add mathcomp.

NAQT IS and IS-A sets have a pretty good amount of mathcomp. Also, I believe that NAQT creates special sets for customers with specific requests. I'm sure NAQT would be willing to up their mathcomp to get the Oklahoma contract. Hell, if Oklahoma wanted, they could have NAQT speedcheck sets, which would be, umm, bad for everyone.

la2pgh wrote:

Anti-Climacus wrote:Well, if Oklahoma coaches wil revolt at the suggestion that mathcomp should be kept to a minimum or even eliminated, I think NAQT would allow hosts to add mathcomp.

NAQT IS and IS-A sets have a pretty good amount of mathcomp. Also, I believe that NAQT creates special sets for customers with specific requests. I'm sure NAQT would be willing to up their mathcomp to get the Oklahoma contract. Hell, if Oklahoma wanted, they could have NAQT speedcheck sets, which would be, umm, bad for everyone.

Not being from NAQT, I can't officially comment on this, but I know NAQT has some special policies about doing state series sets that probably don't align with the Oklahoma bid.

I was warned that my question would elicit some frank answers and I guess I got them.

Without violating the ethical obligations of my role as a member of the OSSAA Advisory Board, let me just say that my question was based on the fact that the aforementioned vendors made bids and were unknown to me.

I appreciate the very specific and pointed comments. We can probably close this forum because the bidding process is closed and most of the suggestions are irrelevant. Is there a thread here that discusses vendors in general? That might help serve the general purpose of what I was trying to do here.

Thanks for your help.

You might want to look through these links.

I understand that many of the comments above probably are too late to be relevant to this year, but I think many of them should be relevant if you are going to take a long-term view of what you want Oklahoma Quizbowl to be like in five or ten years.

Good luck then with your selection process.

I suppose what we can do on another thread or a wiki spreadsheet is to list the major competitions and list the question sources used for those events. This would include a list of all the state championship competitions run by various organizations. But it would be interesting to do that type of comparison, and it would be a great place for people to ask for referrals for the next rounds of bids.

Who's got some free time this holiday break?

I am interested to hear how the selection pans out. The problem with QB in Oklahoma is that sets have to be challenging enough to encourage school like Booker T, Jenks, and Edmond while still being accessible to hillbilly schools who also play (of which there are surprisingly many).

lasercats wrote:I am interested to hear how the selection pans out. The problem with QB in Oklahoma is that sets have to be challenging enough to encourage school like Booker T, Jenks, and Edmond while still being accessible to hillbilly schools who also play (of which there are surprisingly many).

I'm...pretty sure that's a problem with QB everywhere.

I think it's a problem because, well, for the most part, this is an issue that quiz bowl needs to confront at most regular events, high school and at least novice-level college. Questions should be written in a good, accessible, pyramidal, challenging way to appeal to both the novice and the veteran alike. There certainly have been good examples of this happening.

(I am not arguing for any sort of "dumbing-down" of quiz bowl.)

That is perhaps the hardest challenge to confront because it is extremely difficult to appeal to teams that meet every other day for a couple of hours to those school teams that just met each other on the van/bus trip to the competition. There's no way we can have everyone run the same 5K if some of the competitors have never put on running shoes before. There need to be enough events where the lowliest team feels interested in competing again and getting better without compromising the enjoyment and develop other teams for national-caliber play.