Writing science questions: sources and topics

[mattreece]

While editing the science for ACF Nationals, Susan and I noticed some fairly widespread problems with questions that were submitted, so we want to offer some suggestions. An earlier thread (viewtopic.php?f=9&t=8822&p=159369#p158855) covered some similar issues, and might also be worth reading. And of course you should read Jerry's "How to Write Questions" (http://www.acf-quizbowl.com/documents/howtowrite.php) on the ACF website. I'll lead off with some comments, and Susan will add more.

What sources to use

The best sources to use when writing science questions, especially if you're not an expert on the topic you're writing about, are widely-used textbooks. This guarantees that you're writing on a well-known topic and that the clues you choose are things that knowledgeable people can reasonably be expected to know about. The more you know about a topic, the more free you should feel to venture to more specialized sources, but good general textbooks are always a good place to start. Which isn't to say that a library book chosen at random is a good place; try looking at syllabi for courses to get a sense of what books people are most likely to be familiar with.

Wikipedia on some topics is very good, on others less so; it can be a good place to do some reading and get a quick sense of what might be important about a topic. But writing an entire question from Wikipedia (as some people pretty clearly did) is a bad idea. One issue that arose from this, which we managed to catch, were nearly-identical tossups written for the NAQT ICT and for ACF Nationals by different people. I can't tell you definitively that both writers were using Wikipedia, but the selection and ordering of clues was identical, and the lead-in was a fairly recent development that featured prominently in the beginning of the Wikipedia article.

Using primary literature is dangerous, although it can work well if you're familiar with the subject you're writing about or willing to spend some time reading. The trouble is that the scientific literature is vast, and even experts in a field usually can't read more than small fraction of the new papers in that field. If you grab one of those papers at random and use it for your lead-in, the result will be a lead-in that no quizbowler will know, and perhaps only a handful of people in the entire world will recognize. This isn't an exaggeration -- some of the submitted questions were guilty of it. Just as textbooks are good guides to what is well-known, a well-written review article on a subject can be a good place to get a sense of what is important and what isn't, and might be a better source of clues than a random journal article. If you're just Googling a topic and clicking at random, you'll end up with nonsense. Google Scholar displays moderately accurate lists of how many times a paper has been cited, which can be a rough proxy for how interesting it is -- and, at minimum, whether anyone has ever heard of it. Individual fields might have their own more useful databases of papers (in my field, particle physics, the SLAC SPIRES database is best, although often sluggish).

To give some examples, without naming the guilty parties or quoting the questions: one submitted physics tossup began with a clue taken from a 1996 paper that has never been cited (according to searches on Google Scholar and the SLAC SPIRES database), by a little-known author with a marginal publication record. Another began with two references to recent literature: a 2009 paper with no citations according to Google Scholar (and one of the author's names spelled incorrectly in the question!) and a 2007 paper that has been cited once. In the latter case, the clue arose from a misreading of the abstract and was not correct. A common feature of questions with lead-ins from recent literature was phrasing copied almost directly from the abstract of the paper; these questions showed no evidence that the writer understood what they were writing about, and often showed evidence that they didn't. If you don't understand what you're writing, and you can't give an explanation of why the result is significant or relevant, it's probably a sign that you need to find a different clue.

There's nothing inherently wrong with using recent literature as a clue; for instance, I wrote a tossup mentioning the Kerr black hole/CFT correspondence, a 2008 result of Strominger and collaborators that has been cited 96 times (and it's relevant for several different subfields of physics, so it's fairly broadly known). The threshold for looking at a citation count and deciding if a result is well-known varies by field, so it's hard to declare a sharp cutoff. But in general, if you find something with zero or only a handful of citations, it is probably not well-known even to experts, much less to the quizbowl community. Such a clue won't reward expertise, it will just confuse even the experts.

Distribution

Many, many teams completely failed to follow the expected distribution. Some submitted packets didn't include any chemistry at all. Others included oddly slanted subdistributions, like a heavy emphasis on math or CS in all of the "other science" questions. The ACF distribution is:

* Biology 1/1
* Chemistry 1/1
* Physics 1/1
* Math or computer science: 1 question
* Astronomy, earth science/geology, or other science not covered above: 1 question
* Any science: 1/1 (you can write a math question here if you wrote computer science above, or earth science here if you wrote astronomy above, but don't write a second question on any of the "minor" fields. You can also just write more biology, chemistry, or physics here.)

Note that, e.g., 1/1 math or 1/1 CS is never acceptable, and that it's perfectly OK to use the "any science" distribution to write more bio or physics or chem.

I'll end this here and let Susan add her comments, though I might have more to say later.

[Susan]

To add to what Matt's said:

-Matt's covered the Wikipedia issues pretty well, I think, but I want to stress again how one really must avoid pulling all one's clues from Wikipedia when writing a tossup (this results in tossups that are boring at best and at worst end up accidental copies of other tossups where people have done the same thing). For example, if you decide you want to write a tossup on promoters, the thing you should NOT do is go to http://en.wikipedia.org/wiki/Promoters and write a tossup consisting of a clue on Rubinstein-Taybi syndrome followed by a clue on beta-thalassemia followed by an easy-middle clue followed by your giveaway. I saw quite a few tossups that were written along these lines.

-using primary literature--the spectrum of problems I observed in people using primary literature for tossups was somewhat different than what Matt observed. I saw a fair amount of clues drawn from primary literature that were just not specific at all (for example, if you're going to claim that a particular assay or technique can be used to study a protein, you need to google around and make sure it's really specifically used only for that protein; usually googling "[particular assay]" will give you a decent sense of this) or that were specific in a trivial way ("Obscure Scientist used super-common technique to study this organ...").

-ways to ensure you're using primary literature well: As Matt said, citation indices can be helpful for ensuring that you're not trying to mine a paper that no one would ever have had any reason to read. Some of the primary literature that I cited in my questions included a paper that coined the term for a major scientific argument (the Judson and Normark review on ancient asexual scandals in the asexuality tossup), a set of experiments (Gurdon's work with exogeneous mRNA expression in Xenopus oocytes), and some more recent work by a scholar whose work is defining a current scientific debate (Harmit Malik's work on centromeres, which is pretty inescapable if you work on anything that ever touches on centromeres). The Judson and Normark review has ~230 citations (though it's actually undercited--the term "ancient asexual scandals" is more widespread than the citation index indicates) and the work referenced in the Gurdon and Malik clues gave rise to multiple highly-cited papers.

So far as I can tell, you can't get Google Scholar to rank a list of search hits by number of citations. What you can do, as Matt suggests, is focus on looking for reviews, which a) sum up the most important recent literature and hopefully do a bit of separating the wheat from the chaff, and b) present information in a way that's easier for a non-expert to understand. Both Google Scholar and PubMed allow you to limit your search to review articles only; I'd suggest further limiting it to reviews/perspectives published in top journals like Nature (and by extension Nature Reviews...), Cell, Science, Annual Reviews in..., etc.

-textbooks: Again, as Matt says, widely-used textbooks are one of the most useful resources around. PubMed Bookshelf has a lot of useful ones for biology freely available online (good general ones: Annual Reviews Collection, The Cell: a Molecular Approach, Developmental Biology, GeneReviews, Janeway's Immunobiology, Alberts' Molecular Biology of the Cell, etc. etc. etc.; there's also a bunch of more specific ones, particularly in cancer and neuro, and then a bunch of useless ones about like molecular probes and stuff). OMIM (Online Mendelian Inheritance in Man) is a useful index of phenotypes associated with human genes; SGD, Flybase, and Wormbook serve similar functions for S. cerevisiae, D. melanogaster, and C. elegans, respectively.

-distribution: remember to distribute questions well within your categories; just as you wouldn't send in 5/5 British literature, you ought not to make both of your bio questions plus one of your other science questions on immunology (or particle physics, or whatever). A fair amount of our cutting of questions was motivated by this sort of distributional concern. I don't think you ever need more than 1/0 or 0/1 on any scientific subtopic.

[Mechanical Beasts]

These are both great posts, even though I'm certain my questions were guilty of many of the flaws Matt outlines, particularly. (I'm much better at preaching than practicing the week after writing HI; I get tired.) I would like to say that there are ways to use journal articles--at least in chemistry--that
a) don't require that the player has read the article, thereby making "this result has a lot of citations" important but not crucial, and
b) communicates the essential result of the article in a chemically important way.

If I didn't have two tournaments to edit at the moment, I would give an example with actual papers and everything. But let me think of a functionality near and dear to my heart: epoxides. The clue "[obscure scientist not especially famous for working on epoxides] reacted these compounds with amines and a MacMillan catalyst." That clue sucks, since MacMillan catalysts are just organic catalysts to make reactions go asymmetrically--I'm not particularly sure that there's a MacMillan catalyst that works with epoxide-opening reactions, but it wouldn't surprise me much if there were one. So ideally, you'd find a better clue, but even this clue is improved tremendously if you replace "reacted" with "opened." That makes this clue terribly specific; to a chemist, only rings can be opened and it sure helps if there's a big electronegativity difference for reactions to happen in general: this clue pretty strongly points to epoxides now. And the degree to which the clue is good is now, basically, the degree to which someone would care to find out about the chemistry this clue describes. There aren't any really great openings of epoxides with amines, I happen to know, so this might be pretty interesting to someone.

That might have been too obtuse, so here's a gloss: the main result of a chemistry paper is usually presented with a whole lot of assumed context and is otherwise not specific enough just to copy some words you see therein. Presenting a common fact about the subject of the paper (epoxides can be "opened") within the context of that paper does a better job of communicating new and interesting facts while rendering the clue still buzzable. (Of course, when you're ready to take it to the next level, just select the most important papers in epoxide chemistry and give clues about the innovative way the author used an addition funnel and... well, yeah, don't do that.)

[Susan]

I'm replying to Andy's post on my way out the door, so forgive me if I've horribly misread something.

If you do come up with the sort of clue that Andy's describing--something from the primary literature where the paper's pretty obscure but the clue itself will be helpful to people who know about the topic--I wouldn't bother mentioning the researcher's names; you get the same information across in a slightly more compact way by saying "these compounds can be opened with amines and a Macmillan catalyst" (or some rearrangement thereof), and you don't waste time saying a clue (ObscureScientist's name) that no one has heard of and no one can buzz on. (Plus, in the specific case you describe, you avoid having someone with deep knowledge hear [ObscureScientist] and buzzing with his or her more famous non-epoxides work.)

[Mechanical Beasts]

Quite true. It's generally a pretty fast procedure to note what ObscureScientist is best known for, to whatever extent you can say he's known for anything at all. Generally people who aren't terribly well-known just have one or two areas in which they or the postdocs they supervise are doing research (though more often than not the common tie won't be "it's all about [functional group]!"). Just see what a supermajority of his papers seem to discuss. If he's an inorganic chemist, he's probably obsessed with a certain transition metal and its uses, or with a certain area of thought (small molecule activation--and which small molecule?, metal-ligand multiple bonds, C-H activation); if he's an organic chemist, he might be looking at total synthesis, or he might be interested in a set of related reactants, or... Some cursory reading will help you not throw people off.

[cvdwightw]

I'm not by any means the world's greatest science writer/editor, but here are a few more tips on writing for science:

Science Current Events
I'm interested to see what people think about including "Science Current Events." By this I mean a lead-in discussing an article that appeared recently (like, a couple of issues before you submit your question) in a high-impact journal that a lot of science-type people read (e.g. Science, Nature) or that's extremely important in a certain field (e.g. Cell, Brain). Personally, I like a few of these clues per tournament - obviously not one per packet or anything near that frequency. I think they're a good way to keep the science canon fresh; in the first instance (lots of various science-type people read it) it's a good way to reward basic curiosity about what's going on in science, and in the second it's a way to reward people with knowledge of and interest in a certain discipline without going overboard difficulty-wise.

Effective Web Searches
A quick way to filter out a lot of junk is to search .edu sites only. You'll still get a few Wikipedia clones and, depending on the obscurity of the clue/answer, a bunch of papers that may or may not be important, but you should also get plenty of class notes/syllabi/online Q&A. At the very least you should be able to judge what level (high school, lower division, upper division, graduate) the topic regularly appears at. One of the great (or not so great, if you're a non-scientist) things about science is that the canon is largely molded by forces outside the game.

However, you need to be careful even when searching course notes and stuff to make sure that whatever term you're using is actually widespread. In the course of my science writing I've accidentally victimized people multiple times by referring to something by a less-common name or using a clue that apparently one professor in America likes to talk about. Ultimately it's the job of the editors to make sure that doesn't happen.

For God's Sake, Use Actual Science Clues!
This problem is terribly prevalent in chemistry and present to some degree in other sciences. No one is going to fault you for trying to write a science question using real science clues and failing miserably; a competent editor is (typically) going to take a clue that doesn't make sense and rewrite it so that it does. Infinitely worse is the science question that makes no attempt to use clues that actual scientists would know and care about; this wastes everyone's time since the editor has to cut your question and completely rewrite/replace it. The canonical example of this is the lazily-written element question.

Use Packet Archives Judiciously
I might take a lot of flak for writing this, but there is a pretty standardized set of middle and late clues for any given regular-or-lower-difficulty science answer and you really can write a half-decent second half of a science question just by mining the packet archives for clues that regularly show up. However, writing the first half of a science question straight out of the packet archives is a serious no-no.

Don't Be Afraid to Explore the Shadow Canon
When most people think about science lead-ins, they think about pushing so deep into knowledge of a topic that even people who work on that topic have no idea what's going on in the lead-in. This is bad! An alternate way to expand the canon is to look in what I'm terming the "shadow canon." The "shadow canon" consists of everything that is canonical in a certain field but not canonical in quizbowl. Finding an interesting clue from an under-explored area of study (engineering disciplines come chiefly to mind) helps alleviate the problem of "I do research on X, and I'd never heard of lead-in Y."

Science Clues That Are Almost Always Non-Unique

• Symptoms of diseases (focus instead on the mechanism of the disease, specific techniques used to diagnose and classify the disease, etc.)
• Mechanisms of reactions
• General uses of elements/compounds
• Most clues in a biology tossup to which the answer is a class of molecules (these are extremely tricky to write well; it's much better to write a question on, e.g., niacin than it is to write on B vitamins)
• The dreaded "associated with" link
• Name-dropping scientists without mentioning their relevance to the answer

I'm sure there are many more that I haven't immediately thought of

[Huang]

What are some must-have science textbooks for Quizbowl?

This is what I currently have:
Biology by Campbell
Principles of Biochemistry by Lehninger
Fundamentals of Physics by Halliday
Chemistry: The Central Science by Brown
Organic Chemistry by McMurry (Is the Wade book better?)

For cost reasons, I like having just one textbook for each major science topic. So I guess I'm really just asking what you guys consider the best science textbook is for each major topic.

[Gautam]

What are some must-have science textbooks for Quizbowl?

Please browse through: http://hsquizbowl.org/forums/viewtopic.php?f=9&t=8822 and http://hsquizbowl.org/forums/viewtopic. ... ks#p137233

[Mechanical Beasts]

[*]Mechanisms of reactions

To emphasize this one: if you think you are going to write a tossup on some reaction of enolate chemistry,
1) Stop and do something else;
2) Don't use mechanism clues if you really just have to write that tossup, because the very first mechanism clue you use (unless you toss up something ridiculous like the Baylis-Hillman reaction) will point players directly to "an enolate reaction" and the very last clue will, if you are very, very good with wording, get them a little more focused than that. But just a little.

Same, though to a lesser extent, with pericyclics: all sigmatropic rearrangements are basically the same.

[Sima Guang Hater]

To emphasize this one: if you think you are going to write a tossup on some reaction of enolate chemistry,
1) Stop and do something else;

That's good advice

(unless you toss up something ridiculous like the Baylis-Hillman reaction)

I was going to do this...

[Mechanical Beasts]

My lips are sealed.

(Seriously, though, that reaction is something you learn about in (some) second-semester organic classes, whenever you learn about enolate chem, but it's never been a Quizbowl Thing.*)

* light and non-binding guarantee totally invalid in Canada where Diels-Alder is a stranger.