Friday, January 27, 2012

Rejected by Science !!

I just got a rejection for a manuscript sent to the journal Science. That's strike two for me with Science (I sent them my paper on agricultural terraces, back in the early 1990s; it ended up in JFA). I have something else up my sleeve for Science; maybe the third time will work. Because so many papers are submitted to Science, they have a bulk system for evaluating them. Each manuscript gets a quick once-over by one of a small number of editors, who say "no" to most papers. If they say "yes," then the paper gets sent out for peer review. This means that rejections come fast - it took them just a few days to reject my paper. I have to admire that efficiency.

This experience got me thinking about how the Science review process affects the kinds of archaeology papers published in the journal. If you pay attention to the journal, you will know that they tend to favor high-tech methods, archaeometry, fancy quantitative methods, and reports about "the earliest" this or that. While I can only recall one or two papers in Science that I thought were incompetent (a much better record than most archaeology journals, some of which are full of incompetent articles), their selection of archaeology papers is definitely biased in a certain direction. I think one way of expressing this might be that Science publishes archaeology articles that will appeal on methodological grounds to non-archaeological scientists. My guess is that papers that are more synthetic or less methods-heavy don't make it through the initial review (which is done by non-archaeological scientists).

This suggestion links up with the issue of what does "science" mean in archaeology. Not in some big ontological sense, but in practical terms. What kinds of archaeology can be called scientific, and what kinds of archaeology are recognized by other scientists (such as editors at the journal Science) as being scientific in nature? "Scientific method" in archaeology has two meanings. (1) On the one hand science means research done following a scientific epistemology (empirically testable, logically coherent, done with a critical spirit, etc.), whether it employs high-tech methods or not. (2) Scientific methods in archaeology also means the use of non-archaeological scientific techniques: archaeometry and the like. Now ideally, these two meanings of "science" go together, but often they do not. Much research that is epistemologically scientific does not use jazzy methods (as in the paper that was recently rejected by Science). And scientific methods (sense #2) are often used in non-scientific research (sense #1).

What do I mean by that last observation? Consider two examples. First, there are post-processual archaeologists who explicitly reject a scientific epistemology for archaeology, yet they embrace archaeometric methods. This is science of definition #2, done in opposition to science of definition #1. Would this kind of research get by the editors of the journal Science? Good question. Second, there is research that would claim to follow a scientific epistemology, but is too sloppy to be considered good science. Many archaeometric sourcing studies fit here. The archaeologists picks a bunch of artifacts of type X, and subjects them to technical provenience analyses. But if those artifacts were not selected with a rigorous sampling scheme, then this is simply not a good scientific research design. The results cannot be generalized beyond the sample that was analyzed (although archaeologists who are sloppy in picking their samples tend to also be sloppy in overgeneralizing their results). Now this kind of work can easily get past the editors and reviewers of journals, which always puzzles me and bugs the heck out of me. I have pissed off a number of authors and editors over the years with my complaints about the publication of such papers.

So, what's a scientific archaeologist (definition #1, whether or not using methods from definition #2) to do? I guess try another journal. For the sake of the discipline, one can only hope that these powerful editors at Science are not too often fooled by science #2 that does not conform to science #1.


Anonymous said...

This post just got assigned to my arch theory class for Monday. Thanks!

Jason Ur said...

Your perception of what sorts of studies get accepted by Science- earliest, biggest, choose your superlative- is spot-on.

Michael E. Smith said...

@Jason- I hadn't thought much about this previously, but now I wonder if it might be useful if someone would gather systematic data on the kinds of archaeology papers that appear in Science. If in fact the leading scientific journal publishes a biased selection of archaeology papers, this would be of interest to the profession, and perhaps could serve as leverage with the journal.

Any energetic students (or others) out there who want to take this on???

Anonymous said...

Yes, Science does feature a "biased" selection of archaeological research; selection is just that--biased (i.e., directed). I have followed the archaeology papers in Science for some time and I predict that the results of a decent bibliometric analysis will reveal the weight of topics falling under (1) palaeolithic stuff [time period-wise]; (2) "firsts" (esp. technological and agronomic origins) [topics-wise]; (3) high-tech [methodology-wise].

The enumerated converge into just about every archaeology paper I can recall from Science (e.g., earliest evidence for starch grain processing given by some hopelessly sketchy but newly-released on the wannabe-a-scientist scholar market statistical and spectroscopic/dna analyses.

My apologies if this comment sounds bitter; I quite enjoy these articles--and the supplementary material can be quite fun to review. Supplementary-material preparation well done seems to be an undersung art.

Anonymous said...

well, if you're recruiting a private army, you should at least target laa first (as per your previous post).

Michael E. Smith said...

The paper Science didn't want has now been accepted by PNAS, "Papers Not Acceptable to Science". Oops, that should be, "Proceedings of the National Academy of Sciences." But now it is under an embargo and I can't talk about it till just before its published. The paper describes top-secret biomedical research! No, that's not it. Hmmmm..... I am getting punchy after a day of writing with a view of the Dragoon Mountains.

Anonymous said...

"I think one way of expressing this might be that Science publishes archaeology articles that will appeal on methodological grounds to non-archaeological scientists."

This is true of *all* the social science stuff that they publish. Can you think of a single paper from, say, sociology or psychology that doesn't mimic the positivist predict / test / control methodological paradigm? Their editors lack even a basic respect for pluralism in method (not to mention diversity in background).

(The PNAS paper was very interesting, btw).

Michael E. Smith said...

I've updated my discussion of the two types of archaeological science here: