Page 1 of 1

Rethinking Middle School/Novice Difficulty

Posted: Sat Sep 21, 2019 7:59 pm
by cchiego
Quizbowl should be in every school in the country or, at the very least, as many schools as those that currently have debate, Model UN, drama, or chess clubs (which is a pretty high percentage!). But it's not going to get there if the easiest questions that are out there are a very high cliff to climb for students at a large proportion of schools.

A good place to start is at what is widely considered the "easiest" difficulty in quizbowl, Middle School difficulty. This is, ostensibly, where a new player or sponsor looking to start a new program should start (and where they'll likely be directed to if they get contacted by the quizbowl community). But what will they see?

Look through some of the current Middle School question sets. Some of them are unbelievably difficult. One recent set has tossups on Madame Butterfly, Krishna, the Czech Republic, Oedipus Rex (!!), the Italian language with Dante as the giveaway, and the hajj. And that's all in the first packet. The current Sample NAQT MS packet has TUs on Orion, Switzerland, Edward Snowden, Tennessee Williams (!!), William the Conqueror, Cain, Byzantines, and Aaron Copland (!!).

While yes, there are a handful of middle schoolers nationwide who can blow through those questions (and which people in the quizbowl community will point to as a need to raise the difficulty of middle school questions), the vast majority of middle schoolers are not like that. I know because I've talked with middle school teachers at a wide variety of schools and tried to get middle school teams started at a number of schools across the country. The number of "easy" questions going dead when I read is visibly discouraging (a recent experience with this and several conversations with Middle School teachers is what prompted this post).

These students are often gifted students at solid-to-excellent schools as well. But if quizbowl wants to be truly universal then it can't just appeal to those students but rather a broader set of students from all schools. Should these students not also have pyramidal questions on which they can reasonably expect to buzz and answer too? Telling people that the current set of MS questions are considered the "easiest" in quizbowl is likely going to just get them to not respond, much less start a quizbowl team. While HS teams can start by reading MS questions to help ease into quizbowl (although even then this is often challenging), MS teams don't really have anywhere to go for even a set of just "here is how you buzz in and play quizbowl" questions.

It doesn't have to be this way. Here are some ideas:

1. One actual totally-new-to-quizbowl tournament a year that is as accessible as possible.
Yes, this may mean changing up the vaunted canon by cutting a lot of literature and arts, adding a lot of general knowledge, adding more basic math theory questions, etc. It may also mean making the questions shorter to 2-3 lines. But the goal is simply to get students on the buzzers thinking like a quizbowl player and motivating them to want to learn more. One place to look here is to go back to the curriculum and see what students are actually supposed to be learning at that grade, then building from there. While local curricula do vary widely, they are good places to look for what students might be learning and the kinds of concepts that can come up that students at least would have encountered before.

2. Clear Study Guides for New-to-Quizbowl Players
SCOP has done incredible work in putting together study guides appropriate for new players. (Unfortunately NAQT You Gotta Knows have veered off into very difficult material; I understand the need to appeal to a broad audience of HS and College players, but it makes it harder to recommend those You Gotta Knows to new players who get dragged down with lots of stuff). This, along with the well-written questions, make SCOP probably the most accessible question set out there and easily the first thing I recommend to new teams and players. More of these kinds of guides are needed, as are guides to "what does it actually mean to study for quizbowl." Some people can, of course, figure it out by coming online and asking the quizbowl community (I've tried to offer a basic guide here that's more focused at the HS level). The vast majority will not do that, not because they're stupid, but because figuring out how to learn the things that come up a lot in quizbowl is challenging, complicated, and somewhat quirky compared to how you learn for school.

3. Holding down overall Middle School difficulty.
This has to be one of the overall priorities as well. Be judicious in selecting the most accessible answerlines possible. Figure out a way to incorporate more general knowledge, easier giveaways, etc. as much as possible. And be very cautious of creep from the higher difficulties. I have fewer qualms with national championships like MSNCT using harder questions, but I do think that the regular season sets should be kept as accessible as possible. The top MS teams can compete on HS sets in MS-only divisions or even at some HS tournaments (in limited circumstances; you probably also don't want a new HS team's first experience with quizbowl to be losing a middle school team).

As I've written before, there are, of course, many other issues involved in getting new schools and teams and players into quizbowl, particularly "novice" events that aren't always novice and other issues with managing tournament fields that also exist at the MS level. But one place to start is by rethinking the easiest difficulty in quizbowl as something a bit more accessible to all.

Re: Rethinking Middle School/Novice Difficulty

Posted: Sat Sep 21, 2019 8:25 pm
by alexdz
This is a goal I've been trying to achieve somewhat with the SAGES sets I've written. The whole company was inspired by a need for good quality, pyramidal questions for *elementary school* players, and they have been successfully used for middle school tournaments targeting largely newer players or those not connected to the national scene. I've always consciously incorporated as much general knowledge as the set seems to be able to handle while still feeling like it has a reasonable distribution, as well as shying away from the more difficult answerlines. That doesn't mean I haven't let some slip through, but I have tried to keep the difficulty toned down to things that a reasonably intelligent, curious middle school student would know about (i.e., the typical gifted middle school kid at a typical US middle school that Chris mentioned).

Middle school (and lower) difficulty is a huge population of people's exposure to quizbowl, regardless of the nuances of the format, and I think we as a community can do a much better job of providing resources for those teams. I'm happy to be able to participate in this effort with what SAGES is doing, but with the power of this community joining in and welcoming these folks, we can achieve a lot!

Re: Rethinking Middle School/Novice Difficulty

Posted: Sun Sep 22, 2019 9:48 am
by Stained Diviner
Thanks to Chris for a great post. I think his first suggestion is a great one. (His other suggestions are good too, but he provides a link to show that his 2nd suggestion has already been done, and his 3rd suggestion is one of those things that has been said before and that needs to be repeated over and over.)

I don't know whether focusing on what students learn in class is the answer, because I don't know how practical that is. Middle school students spend a lot of time on math and English, and the stronger ones learn a foreign language, so a large part of their day is spent on material that does not translate well to tons of great questions. I'm not saying that the suggestions is horrible--just that it's value is limited. There is a fair amount of geography, current events, and pop culture that can be asked about that is not in the curriculum.

As to his first suggestion, it reminds me of some of the problems high school quizbowl was having 10-15 years ago. There was a desire to bring in new teams, and there was also a desire to keep things interesting for the good teams (which were growing in both quantity and quality). The solution was to produce some high school novice sets and tell the top 100-200 high school teams that they would not enjoy playing those sets. Another part of the solution was to allow for the fact that the truly elite teams can crush regular difficulty. That solution has worked out well. Middle school quizbowl may be at the same point--NAQT already produces different level questions for MS, MSNCT qualifying rounds, and MSNCT playoff rounds because the elite MS teams are crushing both MS and MSNCT difficulty.

I also think that we need a different distribution and a different canon for introductory MS quizbowl. Literature should probably be around 2/2, Fine Arts should probably be around 1/1, and all of thought/social science/philosophy should probably combine for around 1/1. Some of the literature and fine arts should be stuff that doesn't get emphasized much at the high school level--juvenile lit and name that instrument/key signature. This might mean that rounds should be shorter, or it might mean that we're OK with somebody producing a set that is only 5-6 rounds (which would mean that the numbers early in this paragraph could be increased). It may also mean that sets are more focused on the U.S. in categories such as literature, geography, and history, and more focused on Christianity in the religion category.

Re: Rethinking Middle School/Novice Difficulty

Posted: Sun Sep 22, 2019 1:46 pm
by Cheynem
Obviously, to some extent, quizbowl can always get easier, and I think having some very basic set for MSers or novices that just basically gives them an idea of what pyramidal quizbowl is would be a good idea.

I do question Chris' assertion that the sample NAQT packet is outrageously hard for MSers. Some of the questions he singled out as unreasonably difficult were a tossup on the constellation Orion, a tossup on Switzerland's geography (i.e., the Alps, the Matterhorn, Zurich), and Cain from the Bible. I don't understand why those are difficult for MSers, and none of them are from the "literature" and "arts" canon.

Re: Rethinking Middle School/Novice Difficulty

Posted: Sun Sep 22, 2019 2:40 pm
by cchiego
Cheynem wrote: Sun Sep 22, 2019 1:46 pm I do question Chris' assertion that the sample NAQT packet is outrageously hard for MSers. Some of the questions he singled out as unreasonably difficult were a tossup on the constellation Orion, a tossup on Switzerland's geography (i.e., the Alps, the Matterhorn, Zurich), and Cain from the Bible. I don't understand why those are difficult for MSers, and none of them are from the "literature" and "arts" canon.
I recently read sample quizbowl questions to probably ~300 middle schoolers. Questions (ones I re-wrote and shortened to make them even more accessible) on Sherlock Holmes, energy, Germany (with Berlin as the giveaway), and the Flood (from the Bible) had something like 5-30% conversion rates. Questions on Rosa Parks, Hercules, and Mars (the planet) had more around 70-80% conversion rates (we switched to using the latter three much more often after it became apparent the former ones had issues). This suggests to me that there are some things derived from the current "middle school" difficulty better suited to truly introductory middle school-level quizbowl questions than others.

I think those that I highlighted in the NAQT packet would be pretty challenging for most middle schoolers. My own experience as a world geography teacher suggests that Switzerland would not be well-converted just off the names of those cities (maybe a giveaway that talks about watches, mentions chocolate, and stresses the mountains/middle of Europe part rather than just "Alpine" would be better-converted). More anecdotally, it seems Biblical knowledge is getting lower and lower (even among religiously affiliated schools!) and "Cain" would be a challenge ("Adam" or "Eve" might be better converted).

I'd love to see more systematic data on this, but I'd also be cautious about interpreting conversion data from quizbowl tournaments. There's already considerable self-selection into which middle schools have active quizbowl teams and attend tournaments. Note, for instance, the stats from the recent Bay Area Novice tournament--the Middle School team PPBs are higher than the Novice HS team PPBs on the same IS-A set. I hope nobody sees that and decides that middle school difficulty is thus too low! The experiences of the few middle schools--and right now there definitely seem to be relatively fewer middle schoolers playing quizbowl nationwide compared to high schools--who have active quizbowl teams are not representative of the overall national population.

I would love to hear more from middle school teachers at a range of schools/locations about their experiences. But the ones I've talked to have noted the difficulty of even the easiest MS-level packets. Teachers at some middle schools that I've tried to get to start quizbowl teams at (again, at good-to-great schools even) have flat-out told me the questions are too hard for their students. I don't know how accurate this necessarily is in practice, but it seems a barrier to getting more teams started that could be overcome with at least one purposefully introductory question set to use as a demonstration/sample set.
Deviant Insider wrote: Sun Sep 22, 2019 9:48 amThanks to Chris for a great post. I think his first suggestion is a great one. (His other suggestions are good too, but he provides a link to show that his 2nd suggestion has already been done, and his 3rd suggestion is one of those things that has been said before and that needs to be repeated over and over.)
I do think there that there could be more study guides though, especially for those subjects and specific types of questions (i.e. "Literature" vs. English) that don't come up much in the curricula. While a list of frequently-asked-about composers and some of their works is a good general starting point (and again, it's fantastic that SCOP went through the efforts to write it and a huge improvement over not having a study sheet like this at all), a guide that went into a bit more detail on each of these composers (not to mention each instrument, various keys, etc.) and explained a bit more about classical music could help. A wonderfully ambitious idea might be a short online course targeted at quizbowl knowledge that provides a brief overview of classical music and opera along with sound clips (but see "wonderfully ambitious").

This doesn't mean that there needs to be a complete study guide, but rather some general introductory guide to orient students to these different subject areas. While students can, of course, google things, use wikipedia, and look up old questions on quizdb, more user-friendly tools that help bridge the divide from lists to understanding--especially in categories that may be more unfamiliar for many students and coaches--could be really helpful. This is again where I'd note that You Gotta Knows can be quite useful with the opportunity explain a bit more about a topic or answerline. I would, however, favorably compare, say, an older You Gotta Know to a more modern You Gotta Know in terms of friendliness to what newer players might encounter and in avoiding overwhelming new players with too many details.

Re: Rethinking Middle School/Novice Difficulty

Posted: Mon Sep 23, 2019 1:50 pm
by Stained Diviner
cchiego wrote: Sun Sep 22, 2019 2:40 pm I recently read sample quizbowl questions to probably ~300 middle schoolers. Questions (ones I re-wrote and shortened to make them even more accessible) on Sherlock Holmes, energy, Germany (with Berlin as the giveaway), and the Flood (from the Bible) had something like 5-30% conversion rates. Questions on Rosa Parks, Hercules, and Mars (the planet) had more around 70-80% conversion rates (we switched to using the latter three much more often after it became apparent the former ones had issues). This suggests to me that there are some things derived from the current "middle school" difficulty better suited to truly introductory middle school-level quizbowl questions than others.
Just so I can understand what happened--
When you say 5-30%, do you mean 5-30% of individuals or 5-30% of times you read those questions to a group of 10 or however many students?

Also, was there any explanation of what pyramidality is and the importance of sticking with a question when you don't know the first part(s) of it, and was there any encouragement for students to make their best guess even if they were not certain? I ask because a lot of people new to quizbowl do not understand those two things.

Re: Rethinking Middle School/Novice Difficulty

Posted: Mon Sep 23, 2019 2:17 pm
by cchiego
Deviant Insider wrote: Mon Sep 23, 2019 1:50 pm Just so I can understand what happened--
When you say 5-30%, do you mean 5-30% of individuals or 5-30% of times you read those questions to a group of 10 or however many students?

Also, was there any explanation of what pyramidality is and the importance of sticking with a question when you don't know the first part(s) of it, and was there any encouragement for students to make their best guess even if they were not certain? I ask because a lot of people new to quizbowl do not understand those two things.
Sure, happy to add on though to be clear this was not conducted under ideal experimental conditions: it was a booth at a large citywide high school open house fair with 15,000+ attendees. It was, however, an excellent opportunity to appeal to a wider audience than quizbowl usually gets. Thus, the goal was "get lots of students a chance to buzz-in and then talk about starting/joining a quizbowl team." There was indeed encouragement to guess and occasional additions of "it starts with X letter." There wasn't much time to explain pyramidality (trying to explain quizbowl in a 20-second pitch is a challenge in itself since not everyone knows what Jeopardy! is either) other than "buzz when you think you know it and listen carefully." Most of the time the group was 1-3 students (with their parents nearby and occasionally joining in), but the sizes varied. Out of those low-conversion questions, I'd estimate that Sherlock Holmes got the best conversion (~30% of the time answered by anyone in a group) with the rest much lower (these others were also used less as time went on and it became clear they were getting lower conversion rates). That said, I'm sure too the people who chose to come over to the booth were also self-selecting as well so this wasn't exactly a random sample of all students either. One particular sequence of events repeated itself 7-8 times: a parent read the sign, got very excited, rushed to find their child, and then steered their child over to the buzzers while saying some variant of "this is the activity for them."

One more thing: to be clear, I'm not advocating turning quizbowl into some kind of AUK-ful trivia game. I am, however, convinced that the easiest difficulty in quizbowl can still be easier (somewhat along the lines of what Alex D. describes above in SAGES) and that a student's first experience with quizbowl matters. Are students going to be inspired or intimidated by not knowing the answer to multiple questions in their first few encounters with quizbowl? Are prospective sponsors going to want to sponsor an activity that their students struggle with immediately (esp. if there isn't a powerful statewide org/association coordinating and sponsoring it)? If quizbowl really wants to expand to encompass more schools and students, I think this would be helpful to consider.

Re: Rethinking Middle School/Novice Difficulty

Posted: Mon Sep 23, 2019 2:47 pm
by Cheynem
When I was in HS, we would occasionally get students who wanted to try out who were very intelligent, very strong students (i.e., did well in classes, smart--I am not saying the other team members weren't, of course). In many cases, they found that quizbowl wasn't for them. In other instances, kids who perhaps weren't the best students were delighted to find an activity that rewarded them for stuff that school wasn't--the books they read on their own time or the history/geography/music they were interested in.

Now, of course, this was like 50 years ago, so the quizbowl we were playing was probably not very good and way too hard. It's gotten better and easier and could certainly stand to continue to get easier, especially for introductory levels. I do think we are faced with a mild dilemma though--can quizbowl retain that "an activity for the intellectually curious student" (which has nothing to do with G.P.A. or I.Q. and only mild connections with class status, in my opinion) feel but also avoid being an "elite" activity with a high barrier to entry?

One thing that could be done, as Chris points out, is make the "easiest" difficulty in quizbowl using a slightly different distribution, one that appeals more to GK or classroom-oriented topics. I don't have a problem with this, as the important point about the "easiest" difficulty is seeing if newcomers like, well, quizbowl in general--do they enjoy trying to match answers to clues and recalling things? If they don't like it at really easy difficulty level, it's pointless to keep trying at harder difficulty. There are also sound writing strategies, such as choosing the easiest possible answerline in most cases, and avoiding writing from the "canon" (or what's come up).

I do wonder what the potential end result of all of this is: if we create a super easy novice set of packets, and kids play it, where do they go from there? Will they continue the theoretical cycle of moving on to harder packets and getting frustrated again? Or will they become more emboldened and confident based on the early success and do better?

I still maintain that much of this could be resolved more through coaching (which is far easier said than done, as people who have the time or ability to coach high school or college quizbowl are rare). We would not expect anyone in activities like baseball or chess to immediately waltz in and start hitting .300, routinely winning chess matches, or whatever. Good coaches and preconceived expectations help dampen the idea that someone quits immediately. In quizbowl, the expectations aren't as preconceived--I do not doubt, having seen this myself, that someone unable to answer 15 out of 20 questions in a packet, might want to quit immediately. I do not know how to resolve this aside from instilling to coaches and players that quizbowl is much like any other sport or activity--you improve, sometimes very rapidly, as you start playing and practicing more.

Sorry, this was rambling, but I find the topic very interesting and, as a full time quizbowl writer, I like to throw out thoughts.

Re: Rethinking Middle School/Novice Difficulty

Posted: Mon Sep 23, 2019 4:00 pm
by cchiego
Mike's post is quite thoughtful. I agree that there's always the worry that people will like super-easy questions and never want to move beyond that. I'm well-aware that there's a history of people/teams who should know better playing far too easy questions. That always needs to be taken into consideration, though in general I'd rather use things like eligibility restrictions to address that rather than setting high minimum question difficulties.

I also want to address something else: I am not advocating for 2-3 new 10-12 packet sets of these kinds of very accessible questions per year (the MS answerspace is already very compressed). To be more specific, I'm advocating for: 1) adding a single sample "set" of 4-5 packets (maybe produced once per year) that could be easily used as an introduction to quizbowl in every school, 2) helping players and coaches understand how to study for quizbowl, especially subjects/types of questions that don't always come up in normal curricula (I fully agree with the point in Mike's post about how better coaching can help here; but, you also have to induce teachers or community members to want to run a team and coach in the first place too in many regions), and 3) holding down middle school regular difficulty level so that it comes down on the side of "more accessible" as much as possible and keeping in mind the many, many, many schools that don't already have quizbowl teams, much less those who have incredibly self-motivated students who can quickly master the MS canon.

Re: Rethinking Middle School/Novice Difficulty

Posted: Tue Sep 24, 2019 9:27 am
by the return of AHAN
I've entered a twilight zone with respect to middle school coaching. The advent of Discord has created a subculture of players who crave more and more difficult questions, to the point where a few are questioning my use of PSACA for the Challenge Division of my tourney... because it's "too easy."
The Flynn Effect is real.

Re: Rethinking Middle School/Novice Difficulty

Posted: Tue Sep 24, 2019 6:35 pm
by Kevin
For the past few years I've edited a MS set for Texas Quiz Bowl Camp. The set has been aimed at the youngest and most inexperienced players at the camp. I think there are a number of different challenges in producing sets that are accessible to new MS players (and to new HS players as well).

1) The number of potential answer lines is quite small. Take literature, for example. Without getting into stuff that could be considered children's literature, you can't really ask about that many works or authors. You can tossup Shakespeare, and maybe a few of his plays, but nowhere near as many as you could in a high school tournament. A lot of things that a high schooler comes across in the normal HS curriculum simply don't come up in MS.

2) Quiz bowl writers are usually good players, and they often have been good players since middle school. Their expectation of what a middle schooler should know may be drastically different from what middle schoolers actually know. I've had numerous questions submitted that are on answer lines that are way too hard for a typical middle school player.

3) A typical HS distribution is tough to fill out. There's not that much you can ask about in literature, fine arts, physics, or chemistry. We've ended up relying a lot on children's lit, language arts, general knowledge, common link, and trash.
cchiego wrote: Mon Sep 23, 2019 4:00 pm To be more specific, I'm advocating for: 1) adding a single sample "set" of 4-5 packets (maybe produced once per year) that could be easily used as an introduction to quizbowl in every school,
I think this would be helpful. I also think it would benefit immensely from input from a number of middle school teachers (whether or not they are already involved with quiz bowl).

Re: Rethinking Middle School/Novice Difficulty

Posted: Tue Sep 24, 2019 7:54 pm
by i never see pigeons in wheeling
I've long been an advocate for msqb vastly expanding its distribution of general knowledge. I'm not a fan of collegiate quiz bowl having more than a little trivia, but it's perfectly apropos for middle school (and hs!) qb to ask more directly about elements of their culture and surroundings they're likely to have actually encountered--the alternative to that is the expectations gap Chris identified. I've been told by NAQT writers that writing on the sorts of answerlines used by J! Kids Week or Trivial Pursuit would be tricky to execute pyramidally, and while that may be the case, (1) ms quiz bowl questions are short enough so that pulling those off is actually viable, and (2) it may be worth devoting a much greater proportion of bonus space than tossup space to such questions. Pointing to the trash distro would be missing the point, as that is a form of cultural engagement distinct from what I'm talking about--it's a distribution that is tilted towards people who engage in TV/movies/video games/pop radio, which is fine to reward. Why not more questions cluing game/sports rules, instrument answerlines that don't entirely cite classical music pieces, furniture, gardening, appliances (of course make it actually good and not like that infamous blender TU from NAC), wood/metalworking, fashion, outdoor activities, etc. amongst the range of hobbies that middle schoolers engage in? There is some element of these things in existing sets and particularly in the extant general knowledge distribution, but with some imagination, there could be more pyramidal questions like these, and middle school sets could reach out to more teams with a wider range of life experiences. An example of something that's very good that lots of kids learn about/can answer is animal science.

Re: Rethinking Middle School/Novice Difficulty

Posted: Wed Sep 25, 2019 7:52 pm
by tiwonge
Even if there is an easier MS distribution (something I've kind of toyed around with, too), I don't think this precludes using it as an intro to harder questions. Even if the questions are short, as long as they're pyramidal, you can still use early clues or hard bonus parts to introduce harder answers (especially fine arts?) that might come up later. And I think that even with an easier tournament, it's still OK to have 1-2 slightly harder/more regular questions per packet.

Re: Rethinking Middle School/Novice Difficulty

Posted: Thu Sep 26, 2019 5:07 pm
by Cheynem
What would people think about the creation of a "starter set" that would be:

-basically 3-5 packets at most, perhaps only 1-2
-be specifically targeted to teach new players how to play and to see if they like quizbowl
-would feature extremely easy, short questions

Such a set, in my opinion, would only need to be produced every few years--certainly not every year. There is literally no point in people who have played such a set before to ever play another one (unless you're so inexperienced or there's such a gap that you will probably not remember any answerlines anyway). The benefit of not producing the set every year is obviously lowering the workload of quizbowl writers and also avoiding the risk of difficulty creep that comes as people churn out a set on an annual basis ("Oh, last year's set had The Great Gatsby...I don't want to do that again.").

I'm kind of imagining this set to be the quizbowl equivalent of those intro games that appear in beginners manuals for Dungeons and Dragons type games. Not only do you get the packets, but you would get copious amounts of suggestions and notes for coaches explaining, for example, "What to do if your team has trouble answering many questions at first" or "How to move on from these packets to harder material."

Re: Rethinking Middle School/Novice Difficulty

Posted: Fri Oct 25, 2019 7:26 pm
by Bloomington
So what would it say if your team has trouble answering many questions at first?

My coach read an MS NAQT packet on Thursday's practice (we are planning to go to the Jr. Wildcat Tournament) are Varsity team, (9 people now) played a solid match against each other (475-270), after that, we brought in new sixth graders. I think they played on an old RAMS set, and I am pretty sure about 5-7 tossups went dead. Fortunately, our sixth graders seem pretty decent at USH and Trash but I am not sure they are ready for regular MS difficulty.

Re: Rethinking Middle School/Novice Difficulty

Posted: Fri Oct 25, 2019 9:34 pm
by Stained Diviner
Bloomington wrote: Fri Oct 25, 2019 7:26 pm So what would it say if your team has trouble answering many questions at first?

My coach read an MS NAQT packet on Thursday's practice (we are planning to go to the Jr. Wildcat Tournament) are Varsity team, (9 people now) played a solid match against each other (475-270), after that, we brought in new sixth graders. I think they played on an old RAMS set, and I am pretty sure about 5-7 tossups went dead. Fortunately, our sixth graders seem pretty decent at USH and Trash but I am not sure they are ready for regular MS difficulty.
I wouldn't worry about it. I am looking at the first round of RAMS, and I see tossup answers such as Madame Butterfly, Krishna, and Oedipus Rex. A tossup on sulfur probably would have much better conversion among 8th graders than 6th graders. It's perfectly fine for a new group of 6th graders to miss some of those questions. If they practice regularly and some of them put some effort towards learning some things, then I see no reason why that couldn't be a good team by the end of the year and possibly a very good team in a year or two.

Re: Rethinking Middle School/Novice Difficulty

Posted: Sat Oct 26, 2019 11:27 am
by Irreligion in Bangladesh
Deviant Insider wrote: Fri Oct 25, 2019 9:34 pm
Bloomington wrote: Fri Oct 25, 2019 7:26 pm So what would it say if your team has trouble answering many questions at first?

My coach read an MS NAQT packet on Thursday's practice (we are planning to go to the Jr. Wildcat Tournament) are Varsity team, (9 people now) played a solid match against each other (475-270), after that, we brought in new sixth graders. I think they played on an old RAMS set, and I am pretty sure about 5-7 tossups went dead. Fortunately, our sixth graders seem pretty decent at USH and Trash but I am not sure they are ready for regular MS difficulty.
I wouldn't worry about it. I am looking at the first round of RAMS, and I see tossup answers such as Madame Butterfly, Krishna, and Oedipus Rex. A tossup on sulfur probably would have much better conversion among 8th graders than 6th graders. It's perfectly fine for a new group of 6th graders to miss some of those questions. If they practice regularly and some of them put some effort towards learning some things, then I see no reason why that couldn't be a good team by the end of the year and possibly a very good team in a year or two.
Agreed completely. Commend them for getting the questions they do get right, and emphasize what Reinstein says - with study and practice, they can go from rocking half the packet to rocking the whole packet.

Re: Rethinking Middle School/Novice Difficulty

Posted: Sat Oct 26, 2019 11:31 am
by Cheynem
5-7 tossups isn't too bad, really, especially if this is the first time playing such packets. I think, like Reinstein mentioned above, perhaps try and figure out why something went dead--did they not know? Were they reluctant to guess? Did they just forget something they knew? A lot of times, especially among new players, a dead tossup doesn't inherently mean lack of knowledge.

Re: Rethinking Middle School/Novice Difficulty

Posted: Tue Nov 12, 2019 8:50 pm
by rpfeynmanjr
I think that the key thing is that the kids who play on these middle school quizbowl teams are not your average middle schooler.

User reminded to add signature. -mgmt

Re: Rethinking Middle School/Novice Difficulty

Posted: Wed Mar 25, 2020 2:30 pm
by Stained Diviner
Cheynem wrote: Thu Sep 26, 2019 5:07 pm What would people think about the creation of a "starter set" that would be:

-basically 3-5 packets at most, perhaps only 1-2
-be specifically targeted to teach new players how to play and to see if they like quizbowl
-would feature extremely easy, short questions

Such a set, in my opinion, would only need to be produced every few years--certainly not every year. There is literally no point in people who have played such a set before to ever play another one (unless you're so inexperienced or there's such a gap that you will probably not remember any answerlines anyway). The benefit of not producing the set every year is obviously lowering the workload of quizbowl writers and also avoiding the risk of difficulty creep that comes as people churn out a set on an annual basis ("Oh, last year's set had The Great Gatsby...I don't want to do that again.").

I'm kind of imagining this set to be the quizbowl equivalent of those intro games that appear in beginners manuals for Dungeons and Dragons type games. Not only do you get the packets, but you would get copious amounts of suggestions and notes for coaches explaining, for example, "What to do if your team has trouble answering many questions at first" or "How to move on from these packets to harder material."
Some work is going to go into this project this weekend. Please critique the following distribution:
Literature 1/1 (include children's and young adult fiction)
US History 1/1
Non-US History 1/1
Biology 1/1
Physics & Chem 1/1
Math 1/1
Other Science 1/1
Music 1/1 (focus on instruments and scales)
Other Fine Arts 1/1 (focus on famous buildings)
Religion 1/1
Myth 1/1
US Geography 2/2
Non-US Geography 2/2
Current Events/Government 1/1 (focus on stuff that will still be true in a couple of years)
Interdisciplinary 1/1 (if you can be careful about overlaps; do this before or after everything else)
Pop Music 1/1 (focus on the last two years and stuff that is family friendly)
Movies and TV 1/1 (focus on the last two years and stuff that is family friendly)
Other Pop Culture 1/1 (focus on the last two years and stuff that is family friendly)

Re: Rethinking Middle School/Novice Difficulty

Posted: Wed Mar 25, 2020 3:26 pm
by alexdz
As someone who writes MS questions a lot, this overall feels like a pretty accessible distribution and one i'd feel good about as an introduction to the game.

If anything, I feel like there might be room to expand basic literature just a little bit more. Most myth questions will feel literature-y, so that's more or less 2/2 "lit" guaranteed per packet. But still, it would be nice to see a guaranteed 3/3 of lit+myth. My best suggestion that doesn't involve re-distributing topics is to require at least one of the religion questions in each packet to focus on a "story" from religion (i.e., Biblical characters) rather than a practice/holiday; and require one of the interdisciplinary questions to include literature.

Re: Rethinking Middle School/Novice Difficulty

Posted: Wed Mar 25, 2020 3:32 pm
by Cheynem
I would probably do something like 1/1 Literature and 1/1 "Popular Culture Literature," with the acknowledgement that the 1/1 Lit can possibly include some popular culture clues. I think there's 1/1 literature to do a novice set on, assuming this isn't too huge a set.

Re: Rethinking Middle School/Novice Difficulty

Posted: Wed Mar 25, 2020 3:32 pm
by ryanrosenberg
alexdz wrote: Wed Mar 25, 2020 3:26 pm As someone who writes MS questions a lot, this overall feels like a pretty accessible distribution and one i'd feel good about as an introduction to the game.

If anything, I feel like there might be room to expand basic literature just a little bit more. Most myth questions will feel literature-y, so that's more or less 2/2 "lit" guaranteed per packet. But still, it would be nice to see a guaranteed 3/3 of lit+myth. My best suggestion that doesn't involve re-distributing topics is to require at least one of the religion questions in each packet to focus on a "story" from religion (i.e., Biblical characters) rather than a practice/holiday; and require one of the interdisciplinary questions to include literature.
In a similar vein, I'd like to see some part of the geo have a historical component (maybe 1/0 or 0/1).

Re: Rethinking Middle School/Novice Difficulty

Posted: Wed Mar 25, 2020 4:03 pm
by i never see pigeons in wheeling
Stained Diviner wrote: Wed Mar 25, 2020 2:30 pm
Cheynem wrote: Thu Sep 26, 2019 5:07 pm What would people think about the creation of a "starter set" that would be:

-basically 3-5 packets at most, perhaps only 1-2
-be specifically targeted to teach new players how to play and to see if they like quizbowl
-would feature extremely easy, short questions

Such a set, in my opinion, would only need to be produced every few years--certainly not every year. There is literally no point in people who have played such a set before to ever play another one (unless you're so inexperienced or there's such a gap that you will probably not remember any answerlines anyway). The benefit of not producing the set every year is obviously lowering the workload of quizbowl writers and also avoiding the risk of difficulty creep that comes as people churn out a set on an annual basis ("Oh, last year's set had The Great Gatsby...I don't want to do that again.").

I'm kind of imagining this set to be the quizbowl equivalent of those intro games that appear in beginners manuals for Dungeons and Dragons type games. Not only do you get the packets, but you would get copious amounts of suggestions and notes for coaches explaining, for example, "What to do if your team has trouble answering many questions at first" or "How to move on from these packets to harder material."
Some work is going to go into this project this weekend. Please critique the following distribution:
Literature 1/1 (include children's and young adult fiction)
US History 1/1
Non-US History 1/1
Biology 1/1
Physics & Chem 1/1
Math 1/1
Other Science 1/1
Music 1/1 (focus on instruments and scales)
Other Fine Arts 1/1 (focus on famous buildings)
Religion 1/1
Myth 1/1
US Geography 2/2
Non-US Geography 2/2
Current Events/Government 1/1 (focus on stuff that will still be true in a couple of years)
Interdisciplinary 1/1 (if you can be careful about overlaps; do this before or after everything else)
Pop Music 1/1 (focus on the last two years and stuff that is family friendly)
Movies and TV 1/1 (focus on the last two years and stuff that is family friendly)
Other Pop Culture 1/1 (focus on the last two years and stuff that is family friendly)
I find the lack of a general knowledge distribution startling. For a set like this, (family-friendly) GK should be 3/3 or 4/4. Pop culture and geography can be reduced somewhat to accommodate it.

Re: Rethinking Middle School/Novice Difficulty

Posted: Wed Mar 25, 2020 6:04 pm
by tiwonge
Stained Diviner wrote: Wed Mar 25, 2020 2:30 pm
Some work is going to go into this project this weekend. Please critique the following distribution:
Literature 1/1 (include children's and young adult fiction)
US History 1/1
Non-US History 1/1
Biology 1/1
Physics & Chem 1/1
Math 1/1
Other Science 1/1
Music 1/1 (focus on instruments and scales)
Other Fine Arts 1/1 (focus on famous buildings)
Religion 1/1
Myth 1/1
US Geography 2/2
Non-US Geography 2/2
Current Events/Government 1/1 (focus on stuff that will still be true in a couple of years)
Interdisciplinary 1/1 (if you can be careful about overlaps; do this before or after everything else)
Pop Music 1/1 (focus on the last two years and stuff that is family friendly)
Movies and TV 1/1 (focus on the last two years and stuff that is family friendly)
Other Pop Culture 1/1 (focus on the last two years and stuff that is family friendly)
If I collect these into categories, I get:
Lit 1/1
History 2/2
Science 4/4
Arts 2/2
Religion/myth 2/2
Geography 4/4
Current events 1/1
Interdisciplinary 1/1
Pop culture 3/3

Looking at this (and even with Alex's observation that a lot of the religion/myth might come from stories or books), Lit seems to be getting a short shrift. I think that there's room for both a YA/pop lit and a more "serious" lit of something that might be read in class. (Something like Animal Farm, To Kill a Mockingbird, Gift of the Magi, all of which I remember reading in class in middle school. Romeo and Juliet would also be appropriate. Maybe Of Mice and Men.)

It also feels like a lot of science compared to other academic subjects, but science might be an easy thing to write about because they lend themselves to pyramidality? And, I feel, maybe a bit much geography and pop culture? I think that art, especially if the visual art focuses on famous buildings, could incorporate some of the geography.

One of the things I'd like to do--and I can't do this right now because libraries are closed down--is to try to tie some of the questions, especially the academic questions, to curricula and what might be found in textbooks. (I guess I can still check online, but I was also thinking of going to the education section of the library and just pulling out middle school textbooks.)

Thinking of the middle school curriculum, I wonder if there's also room for something like questions about grammar? (Asking about pronouns, or verbs, or something?) Maybe not 1 per packet, but possibly 1 or 2 over the course of the set. What else is covered in the middle school curriculum that isn't emphasized as much in high school and college? Home Ec/finances? (This might also fall under general knowledge.) I think it's worth thinking about what middle school students might study, and what middle school teachers might teach, and try to align a set to what they're learning and teaching.

Re: Rethinking Middle School/Novice Difficulty

Posted: Wed Mar 25, 2020 6:07 pm
by tiwonge
The other thing I'm thinking right now, is that if I come up with 4/4 (the median of the number of packets suggested) answerlines for each tossup, are there any categories where I'm scraping to get appropriate questions, or any categories where I feel like I'm leaving out a lot of good questions?

Re: Rethinking Middle School/Novice Difficulty

Posted: Wed Mar 25, 2020 6:23 pm
by Stained Diviner
Animal Farm, To Kill a Mockingbird, Gift of the Magi
I'm trying to avoid answers like these. It's true that some 6th graders can get those, but that's not who this set is for. Those answers will go dead too often if they are asked to novice middle schoolers. I wouldn't even toss up Orwell.

Re: Rethinking Middle School/Novice Difficulty

Posted: Wed Mar 25, 2020 8:11 pm
by tiwonge
Stained Diviner wrote: Wed Mar 25, 2020 6:23 pm
Animal Farm, To Kill a Mockingbird, Gift of the Magi
I'm trying to avoid answers like these. It's true that some 6th graders can get those, but that's not who this set is for. Those answers will go dead too often if they are asked to novice middle schoolers. I wouldn't even toss up Orwell.
I still think that 1/1 Lit is pretty low.

Re: Rethinking Middle School/Novice Difficulty

Posted: Wed Mar 25, 2020 8:17 pm
by i never see pigeons in wheeling
tiwonge wrote: Wed Mar 25, 2020 8:11 pm
Stained Diviner wrote: Wed Mar 25, 2020 6:23 pm
Animal Farm, To Kill a Mockingbird, Gift of the Magi
I'm trying to avoid answers like these. It's true that some 6th graders can get those, but that's not who this set is for. Those answers will go dead too often if they are asked to novice middle schoolers. I wouldn't even toss up Orwell.
I still think that 1/1 Lit is pretty low.
Remember that this is a novice set meant to initiate brand new middle schoolers (including kids who are shy or have other hang-ups about buzzing) into getting comfortable with qb-style buzzing. I think that sacrificing distributional purity is worthwhile for the purposes of absolutely maximizing end conversion. There is plenty of anecdata that ms lit conversion is really low, especially for newer players.

Speaking of which, this is why we need a lot of GK, including stuff they may learn inside and outside of school. Grammar and foreign language can fall under this category, but so can the subjects I mentioned in my earlier post.

Re: Rethinking Middle School/Novice Difficulty

Posted: Wed Mar 25, 2020 8:52 pm
by tiwonge
i never see pigeons in wheeling wrote: Wed Mar 25, 2020 8:17 pm
tiwonge wrote: Wed Mar 25, 2020 8:11 pm
Stained Diviner wrote: Wed Mar 25, 2020 6:23 pm
Animal Farm, To Kill a Mockingbird, Gift of the Magi
I'm trying to avoid answers like these. It's true that some 6th graders can get those, but that's not who this set is for. Those answers will go dead too often if they are asked to novice middle schoolers. I wouldn't even toss up Orwell.
I still think that 1/1 Lit is pretty low.
Remember that this is a novice set meant to initiate brand new middle schoolers (including kids who are shy or have other hang-ups about buzzing) into getting comfortable with qb-style buzzing. I think that sacrificing distributional purity is worthwhile for the purposes of absolutely maximizing end conversion. There is plenty of anecdata that ms lit conversion is really low, especially for newer players.

Speaking of which, this is why we need a lot of GK, including stuff they may learn inside and outside of school. Grammar and foreign language can fall under this category, but so can the subjects I mentioned in my earlier post.
I was just thinking that a lot of kids like reading, so having questions about books might be good.

On the other hand, that is so broad that it might be hard to find things (other than popular literature) that a good chunk of people might have read.

Re: Rethinking Middle School/Novice Difficulty

Posted: Wed Mar 25, 2020 9:34 pm
by alexdz
I've often used lists of things like Newberry and Caldecott award winners, ALA "best of" lists, etc., to get a handle on what "kid lit" books are popular and widely known. I do believe there is room for more than 1/1, but I'd rather see something like 2/2, as there seems to be a consensus that 3/3 might be pushing it for the goals of this set.

Also, regarding "general knowledge" -- this definitely is important and can be used very well in themed bonuses, particularly. I remember one of the earliest bonuses I wrote for an elementary set involved question on apples in various contexts. I've also written questions where the answer is a number or a letter, which allow you to ask about random general knowledge things. That being said, some things can be categorized in one of these categories even if the question is what we'd call "GK"... for example, I've put grammar questions into lit distributions and included cooking/food questions in science or geography. If we use this distribution as a rough guide and plug general knowledge in where it fits most relatedly, then I'm 100% cool with sacrificing "purity" for the sake of playability.

Re: Rethinking Middle School/Novice Difficulty

Posted: Wed Mar 25, 2020 11:03 pm
by i never see pigeons in wheeling
alexdz wrote: Wed Mar 25, 2020 9:34 pm Also, regarding "general knowledge" -- this definitely is important and can be used very well in themed bonuses, particularly. I remember one of the earliest bonuses I wrote for an elementary set involved question on apples in various contexts. I've also written questions where the answer is a number or a letter, which allow you to ask about random general knowledge things. That being said, some things can be categorized in one of these categories even if the question is what we'd call "GK"... for example, I've put grammar questions into lit distributions and included cooking/food questions in science or geography. If we use this distribution as a rough guide and plug general knowledge in where it fits most relatedly, then I'm 100% cool with sacrificing "purity" for the sake of playability.
There's general knowledge that doesn't fit in neatly anywhere--if you wanted to ask a question about knitting or fishing (to take two hobbies middle schoolers might know a little about), it doesn't really fit anywhere in the current distro. I don't think there's a particular need to have that much pop culture or geography in particular.

Also, I do see the point about having relatively few packets, but if this many person-hours are being devoted to it, perhaps there could be an entire set made out of it? I could imagine that early year hs novice tournaments could run a separate ms *novice* division and use this set instead of the usual MS set or SCOP MS. And, maybe, for the first time, we could have a sample portion of a set run purely as an outreach/team-building tool (like the first 3 packets or so being read at individual elementary/middle schools) and then say "if you want more of that, you can continue playing against other kids from other schools new to the game at [x] tournament."

Re: Rethinking Middle School/Novice Difficulty

Posted: Thu Mar 26, 2020 7:16 am
by Stained Diviner
When thinking about this set, keep in mind that there are already middle school appropriate sets out there. They are easy to find. This set is supposed to be significantly easier than those. The reason this set exists is because a lot of people don't know much about major writers, composers, artists, etc. If you ask average students about almost any Newberry and Caldecott award winners, they have not heard of them. Therefore, that stuff should not go in this set, except for maybe bonus hard parts. This set is especially being written for people who don't know the names of many writers, composers, and artists, because such people are common. It's just a few rounds, so if they like these questions they will soon get to questions with a typical distribution. It's fine to work lit into some of the other questions.

It's fine to carve out some space for general knowledge, but there is already a Miscellaneous Pop Culture, Interdisciplinary, and Other Science that allow for pretty much anything you want to ask about.

The reason this set is short is so that the writer can focus on using the easiest answer lines available in a category rather than 30th easiest answer lines in a category.

Re: Rethinking Middle School/Novice Difficulty

Posted: Thu Mar 26, 2020 7:00 pm
by alexdz
i never see pigeons in wheeling wrote: Wed Mar 25, 2020 11:03 pm knitting or fishing
Knitting could be "other visual art" and fishing could be sports, wherever that might fit. But the point is that nearly anything GK you could think of would be able to fit in any of these categories, particularly if you widen "literature" to mean more like "language arts."

I didn't mean to suggest that any Newbery/Caldecott Honor book is fair game, just that it's one way to narrow down books that would most likely be found in school libraries that kids might have read. The list for me has always been a scope-narrowing device, and then I look into some of the titles and look for other ways to know if a book is popular (has it been turned into a film or TV series, is it by an author who has written other highly-regarded books, etc.). It's my virtual way of trying to figure out which books are going to be on that big display right in the front of the library that a generally intellectually-curious kid might pick up if they wandered into their school's library.

Re: Rethinking Middle School/Novice Difficulty

Posted: Wed Aug 19, 2020 1:58 pm
by tiwonge
By the way, if you haven't heard, a small group of high school students and I have been writing a 5 packet set along these lines that should be released soon. We playtested it with some non quiz bowl middle school students (although, generally, they were smart kids), and for the most part, there was high conversion of the tossups and most bonuses. I hope to release this on Monday, if the people who are reviewing it now don't find many problems with the questions. (If you'd like to look it over before I release it, let me know.)

Re: Rethinking Middle School/Novice Difficulty

Posted: Sun Aug 23, 2020 11:21 pm
by tiwonge
Mike and Tejas have looked over the set. Mike says that he thinks it meets PACE standards, and Tejas gave us a bunch of corrections (thank you!).

I think I want to print out copies of the packets tomorrow just to look over a printed version and see what it looks like. Other than that, I think it's done.