Friday, January 13, 2017

Carl Sagan's Toolkit for Skeptical Thinking (or call it Smith's epistemology)

I just read a nice blog post by A.P. Van Arsdale, "Size, Science, and Scientific Truth on bias in scientific thinking. I differ from Van Arsdale somewhat in my view that science is not about "Truth," but about reducing error. As Professor Indiana Jones once said,  "Archeology is the search for fact, not truth. If it's truth you're interested in, Dr. Tyree's Philosophy class is right down the hall.'   Archaeology, like all science, is about facts and patterns and explanations, not about truth. But I do understand that many people use the word "truth" informally to refer to facts, patterns, and explanations.

In his post Van Arsdale lists nine principles from Carl Sagan that comprise a "Toolkit for sceptical thinking." These are from Sagan's book, "The Demon-Haunted World: Science as a Candle in the Dark." These are great precepts, and they neatly describe my own epistemology.

  1. Wherever possible there must be independent confirmation of the “facts.”  
  2. Encourage substantive debate on the evidence by knowledgeable proponents of all points of view.
  3. Arguments from authority carry little weight – “authorities” have made mistakes in the past. They will do so again in the future. Perhaps a better way to say it is that in science there are no authorities; at most, there are experts.  
  4. Spin more than one hypothesis. If there’s something to be explained, think of all the different ways in which it could be explained. Then think of tests by which you might systematically disprove each of the alternatives. What survives, the hypothesis that resists disproof in this Darwinian selection among “multiple working hypotheses,” has a much better chance of being the right answer than if you had simply run with the first idea that caught your fancy.”
  5. Try not to get overly attached to a hypothesis just because it’s yours. It’s only a way station in the pursuit of knowledge. Ask yourself why you like the idea. Compare it fairly with the alternatives. See if you can find reasons for rejecting it. If you don’t, others will.
  6. Quantify.  If whatever it is you’re explaining has some measure, some numerical quantity attached to it, you’ll be much better able to discriminate among competing hypotheses. What is vague and qualitative is open to many explanations. Of course there are truths to be sought in the many qualitative issues we are obliged to confront, but finding them is more challenging.
  7. If there is a chain of argument, every link in the chain must work–not just most of them.
  8. Occam’s Razor. This convenient rule-of-thumb urges us when faced with two hypotheses that explain the data equally well to choose the simpler.
  9. Always ask whether the hypothesis proposed can be, at least in principle, falsified. Propositions that are untestable, unfalsifiable are not worth much. Consider the grand idea that our Universe and everything in it is just an elementary particle – an electron, say – in a much bigger Cosmos. But if we can never acquire information from outside our Universe, is not the idea incapable of disproof? 
If you have read any of my ranting and raving in this blog about science, scholarship, and the deleterious effects that postmodernism, postprocessualism, and social archaeology have had on the advancement of archaeology, these points are no surprise. For more formal statements of some of my epistemology, see Smith (2015; n.d.). Or see many of my prior posts, especially my series on science in archaeology, starting with "Science, Social Science, and Archaeology: Where do we Stand?"

Smith, Michael E.
2015 How can Archaeologists Make Better Arguments? The SAA Archaeological Record 15 (4):                 18-23.

n.d. Social Science and Archaeological Inquiry. Antiquity  (in press).

And if you want to see Carl Sagan in some wild and wonderful videos, check out the mash-ups by Melody Sheep. I especially like this one.  Whoop  Whoop......


Friday, December 16, 2016

Why I Find Foucault Useless



People periodically try to convince me that I should pay attention to the work of French philosopher Michel Foucault, because it will enrich archaeological interpretations of the past. Maybe. But here is why I remain dubious, after reading various books and articles by and about Foucault.

Highly abstract theories and concepts—such as Foucault’s governmentality, power and discipline (things archaeologists have mentioned)—describe the operation of the world on a very general level. This kind of approach, termed “grand theory” by C. Wright Mills, is “so general that its practitioners cannot logically get down to observation. They never, as grand theorists, get down from the higher generalities to problems in their historical and structural contexts” (Mills 1959:33). This is pretty basic stuff in social science epistemology: grand theory is so abstract that it cannot explain individual cases or variation among cases (Abbott 2004:218; Ellen 2010; Mjøset 2001; Smith 2011). My post. "What is wrong with abstract social theory" has links to other posts and resources.  Foucaultian power and governmentality simply exist—presumably for all societies—so how can they explain change and variation?

In Abend’s (2008) classification of types of theory in sociology, Foucault’s concepts are examples of theory type 3 (a statement about the meaning of social phenomena, an interpretation, a reading, or a way of making sense) or type 5 (a weltanschauung, an overall perspective from which one sees and interprets the world). What this means is that Foucault’s concepts are of limited utility in explaining specific social phenomena, and their empirical adequacy cannot be tested. In the words of Kevin Fisher (2009:440), for archaeoloigsts, abstract theory like this “does not offer the tools needed to analyze the material remains on the ground.” I discuss this notion further in Smith (2015, n.d.). See my previous post on Abend's scheme of theory.

Pierre Bourdieu recognized this problem with Foucault’s work. As reported by Callewaert (2006:92), Bourdieu complained that “the philosophical method was used [by Foucault] for answering questions that are basically empirical sociological questions.” Foucault’s methods were faulty from the perspectives of both historiography and social science methodology (Garland 1987). In fact, his method of social analysis has been called “politically engaged journalism” (Vallois 2015). If one is interested in abstract, philosophical notions about the human condition, then the work of Foucault may be full of insights. But if one is interested in a social-scientific explanation of the dynamics of past cities and human societies, one has to look elsewhere for concepts and models.


In the social sciences, theory that is more grounded and testable is termed “middle-range theory” (Hedström and Udéhn 2009; Merton 1968:39-72; Sampson 2010). In Abend’s (2008) scheme, this corresponds to theory type 1 (a general proposition about the relationship between two variables) and type 2 (an explanation of a particular social phenomenon). In this approach explanation consists of identifying the causal mechanisms responsible for observed changes: “to explain a fact is to exhibit the mechanism that makes the system tick” (Bunge 2004:182). This is part of a basic scientific approach to archaeological knowledge. See my post, "Why is it important to strive for a more scientific archaeology?"

So, if you don’t care about explanation and causality, or about being able to tell when your interpretation is wrong, then the work of Michel Foucault may be fine for you. But for me, I cannot find anything interesting or useful in his work. 

(And, of course, I am really bugged by Foucault using the term "archaeology" to refer to the past history of any old thing. It is insulting that Google searches for archaeology plus something else turn up Foucault instead of turning up archaeology!)


Abbott, Andrew
2004 Methods of Discovery: Heuristics for the Social Sciences. Norton, New York.

Abend, Gabriel
2008 The Meaning of "Theory". Sociological Theory 26: 173-199.

Bunge, Mario
2004 How Does It Work?: The Search for Explanatory Mechanisms. Philosophy of the Social Sciences 34 (2): 182-210.

Callewaert, Staf
2006 Bourdieu, Critic of Foucault: The Case of Empirical Social Science Against Double-Game-Philosophy. Theory, Culture and Society 23 (6): 73-98.

Ellen, Roy
2010 Theories in Anthropology and "Anthropological Theory". Journal of the Royal Anthropological Institute 16: 387-404.

Fisher, Kevin D.
2009 Placing Social Interaction: An Integrative Approach to Analyzing Past Built Environments. Journal of Anthropological Archaeology 28: 439-457.

Garland, David
1987 Foucault's Discipline and Punish: An Explosition and Critique. Law and Social Inquiry 11 (4): 847-880.

Hedström, Peter and Lars Udéhn
2009 Analytical Sociology and Theories of the Middle Range. In The Oxford Handbook of Analytical Sociology, edited by Peter Hedström and Peter Bearman, pp. 25-49. Oxford University Press, New York.

Merton, Robert K.
1968 Social Theory and Social Structure. 3rd ed. Free Press, New York.

Mills, C. Wright
1959 The Sociological Imagination. Oxford University Press, New York.

Mjøset, Lars
2001 Theory: Conceptions in the Social Sciences. In International Encyclopedia of the Social and Behavioral Sciences, edited by Neil J. Smelser and Paul B. Baltes, pp. 15641-15647. Elsevier, New York.

Sampson, Robert J.
2010 Eliding the Theory/Research and Basic/Applied Divides: Implications of Merton's 'Middle Range'. In Robert K. Merton: Sociology of Science and Sociology as Science, edited by Craig Calhoun, pp. 63-78. Columbia University Press, New York.

Smith, Michael E.
2011 Empirical Urban Theory for Archaeologists. Journal of Archaeological Method and Theory 18: 167-192.

2015 How can Archaeologists Make Better Arguments? The SAA Archaeological Record 15 (4): 18-23.

n.d.   Archaeology and Social Science Inquiry. Antiquity  (accepted for publication).

Vallois, Nicolas
2015 Michel Foucault and the History of Economic Thought. Œconomia: History, Methodology, Philosophy 5 (4): 461-490.



Sunday, November 20, 2016

Am I the most literary archaeologist of all time?

How many archaeologists can say that they have participated in a joint project with the likes of Gore Vidal, F. Scott Fitzgerald, Barbara Kingsolver, and Charles Frazier? The list also includes Annie Dillard, Larrie McMurtry, and Jane Smiley. Well, I have published an essay in a volume together with these and other literary (and historical) luminaries. I guess that makes me a very literary archaeologist! What was I doing together with all these famous novelists? Unfortunately, it was not hobnobbing with them  at a literary cocktail party in Manhattan (nor at a Gatsby party on Long Island, for that matter).

I was invited to contribute an essay to a book edited by historian Mark C. Carnes called Novel History: Historians and Novelists Confront Amerca's Past (and Each Other). Carnes had a bunch of historians write essays about specific historical novels, and then had the novelists write replies. The idea was to stimulate thought and discussion about history, fiction, and the past, but without slipping into nit-picky historical details. My contribution was the exotic case in the volume: Gary Jennings sprawling novel of Aztec adventure, sex, and violence, titled simply Aztec.

When Carnes first asked me to do this, I told him I needed to read the book first! I had started the novel as a graduate student, but had to put it down to avoid confusion. Jennings had immersed himself in the primary sources on Aztec society and history, and he really knew the details. Then, as a novelist, he elaborated where necessary. I found myself getting confused. Where did I read about people avoiding priests because they were worried they might be picked to be sacrificed? Was that in Sahagun, or was it an invention of Gary Jennings? So I dropped the novel, until Mark Carnes's request led me to pick it up again.

I loved the book. It was mostly accurate and full of adventure. The main character was a merchant who could travel in both elite and commoner social contexts. Jennings created practices that were contrary to fact only in key situations necessary for the novel. Thus he portrayed the Aztec writing system as more complete than it actually was, so that he could have people writing messages to one another, a practice that advanced the story in key places. Of course Jennings got a number of picky minor things wrong. But on the other hand, he actually predicted a finding that archaeologists had not yet dared to formulate until well after the novel was published!

Jennings has a merchant carrying obsidian and other goods back and forth across the fortified boundary that separated the Aztec and Tarascan empires. The written sources on the Aztecs, however, claim that this was an impenetrable border that nothing crossed. Aztec archaeologists, being traditionally under the spell of the written record (don't get me started....), had not even considered the possibility of Aztec-Tarascan trade. But if you think about it for more than a couple of seconds, it it clear that one trait EVERY fortified and defended border has in common, is that people and goods move back and forth illegally (I could make a crack here about a proposed wall along the US-Mexican border...). So it was not hard for Jennings to have his characters involved in contraband and smuggling. But only after the novel was published did we get incontrovertible evidence of an active trade across the Aztec-Tarascan border. Obsidian sourcing studies now show a two-way exchange of obsidian across the border, and lead isotope studies of bronze artifacts I excavated, by Dorothy Hosler, show a west-to-east trade).

It was fun writing my essay, and I was looking forward to seeing Gary Jennings's reply. My main beef with him was that he did not include the typical section where he lists his main sources and perhaps thanks some experts. But this is a minor point. Unfortunately, Jennings died before he could reply to my essay. I was really bummed out! So Carnes published, instead, the reply he got to his initial invitation to Jennings to participate in the volume. It is sort of cranky, railing against academics in general who get picky about historical novels. "It may sound to you, Mark, as if I'm already compiling my indignant response to whatever historian may eventually do the critical review of Aztec." He counseled the editor to find "non-ivory-tower historians" for the books in the volume.

I would like to think I would be considered a non-ivory-tower historian (or archaeologist). This was one of my most enjoyable essays. And by keeping company (of sorts) with Gore Vidal, F. Scott Fitzgerald, and the rest, I think I can be considered one of the most literary archaeologists of all time!

Carnes, Mark C. (editor)
2001    Novel History: Historians and Novelists Confront America'sPast (and Each Other). Simon and Schuster, New York.

Jennings, Gary
1980    Aztec. Avon Books, New York.

Smith, Michael E.
2001    The Aztec World of Gary Jennings. In Novel History: Historians and Novelists Confront America's Past (and Each Other), edited by Mark C. Carnes, pp. 95-105. Simon and Schuster, New York.


Tuesday, November 8, 2016

Problems with authors who publish books but not articles

Do you ever get annoyed when reading book-length studies and the author feels justified in ignoring the scholarly literature on the topic? I have found this to be the case with a number of authors. If they would publish in journals, they would be forced to cite other studies on the topic and contextualize their work within the scholarly literature. But because they are publishing a book (and the editors/press don't seem to care), they feel free to write what they like, and other studies of the topic be damned. I think this practice is harmful to scholarship.

Here is a portion of a book review I published a number of years ago. I've anonymized it, since my goal here is not to dump on Dr. X. But it does express my frustrations with this particular book, something I have seen in other book-authors who do not publish journal articles:


I am in agreement with X’s overall goals and approach. This type of revisionist history, in which political explanations are applied to phenomena previously interpreted in particularistic and ideological terms, is welcome. Nevertheless, I am uncomfortable with many of X’ specific arguments, largely because I am unable to assess their strengths and weaknesses. In part this is owing to his style of scholarship. X identifies an important and unresolved issue, summarizes what the primary historical sources say, discusses the pros and cons of alternative interpretations of the data, and then states his preference. His exposition sounds logical and convincing but because he does not cite the relevant secondary literature, one would never know that a given topic is the subject of considerable published scholarship and debate among specialists, many of whom draw on data and methods not presented by X.  Scholars Y and Z, for example, have made fundamental contributions to the topics covered by X, but he does not cite the relevant publications  This failure does not make X’s arguments wrong, but the reader is prevented from evaluating them within the context of contemporary scholarship.

 I am also disappointed by X’s treatment of archaeological data. He presents incorrect dates (which support his interpretations) for several key buildings, including the New Fire temple on Mount Huixachtecatl and the twin-temple pyramids of Tenayuca and Teopanzolco. Contrary to X’s assertions, these latter temples are dated quite firmly to the Early Aztec period (several centuries before the Aztec empire) and thus cannot possibly have had the imperial significance attributed to them by his model. X’s book is an intriguing study with a fresh theoretical approach and many promising interpretations of Aztec history, time and calendars. However, to be assessed properly, X’s interpretations must be debated within the community of scholars working on these issues so that the strength of his arguments can be evaluated. 

Give me a series of journal articles any day. Or, if you write a book, please be scholarly and complete about it, even if it is not subject to peer review.

Saturday, October 29, 2016

Anonymity ain't what it used to be

(NOTE: I just found that this post was sitting around, not submitted. So here it is)

Sometimes scholarly anonymity is not upheld by journals, even if they promise this. A while ago I reviewed a manuscript for Journal A (I am being anonymous here to protect the guilty). The paper in question had a lot of problems and I did not recommend publication. Recently I saw that the paper (presumably a heavily revised version) had been published in Journal B. Then a colleague emailed me with some remarks on the paper, and made an offhand remark that I must have seen the paper and provided help or comments to the authors. But I don't know the authors and had never communicated with them! I checked the paper, and I was indeed thanked in the acknowledgements for my comments. I went back to my original review and confirmed that I had indeed submitted it anonymously (sometimes I sign my reviews, and sometimes not; when I DO sign a review, I make it very clear that I do not want to remain anonymous).

I contacted the editors of Journal A, who did not know what had happened. I contacted the lead author, who sent along a copy of my review. It was the review I had submitted to the journal, but an extra line had been added at the top: "Reviewer: Michael Smith." This was supposed to be a single-blind review system (that is, the names of the authors are known to the reviewers, but the names of the reviewers are not supposed to be revealed to the authors). Hmmmmmmmm. I was pretty angry at the journal, and slightly miffed at the authors. I think the authors should have contacted me, acknowledged that they saw my review, and ask if they minded if they cited me in the acknowledgements. That might have led to a more lengthy correspondence, and perhaps a more beneficial use of my advice to them. But this is a minor point. Another consideration is that the authors (presumably) used my review to improve the paper, so I am happy to have contributed in that sense.



I started off my career, back in pre-email days, writing reviews on university letterhead, a BIG mistake! One time a publisher asked me to review a couple of chapters from a new textbook (by a colleague I knew pretty well) and promised me anonymity. I thought the chapters left a lot to be desired and said so. The next time I talked to the author, they thanked me for my "frank" review. "How did you know it was me?" I asked. Well, it seems the publisher had covered up my signature but not the university letterhead! I was teaching at Loyola University of Chicago, and I was the only archaeologist on the faculty. It was pretty clear to the author that I must have written the review. That was when I switched to writing reviews on a separate sheet with no identifying marks. That seemed to work, until now.

So, what is the lesson here? Although I often feel that anonymity should be avoided in the interests of collegiality and collaboration, it does have its uses. If a journal claims to use blind peer reviewing, then they should adhere to their standards and not compromise them. And if you are writing a review and really want to remain anonymous, consider the reputation of the journal and its editor. You can always contact the editor to mention your concern about anonymity; that might reduce the chances of slippage as in my case today.

For more information about anonymity in peer review (in relation to double-blind reviewing), see my older post on this. Both of those older posts have some bibliographic citations on types of peer review and their implications.

From: PhD Comics

Thursday, October 20, 2016

Should you consider a 3-article dissertation?

The three-article format for dissertations is becoming more common in archaeology. In my proposal-writing class we cover some professional issues, and students were interested in discussing the 3-article dissertation. I am not an expert here, but I did supervise creation of a policy and guidelines for the three-article thesis in my unit, and I am sitting on committees where students have chosen this option. I found a nice blog post (from 2014) on The Thesis Whisperer, "Thesis by publications? You're joking, right?" (guest post by earth scientist David Alexander). The (many) comments on that post contain many good insights and ideas, and I'll quote a bunch of them below.


First, here is a brief summary of our requirements (in SHESC) for a three-article dissertation:
  • The dissertation will consists of an introductory chapter, three article-chapters, a conclusion, and a single bibliography.
  • The paper must be considered publishable by the committee. This includes evaluation of the appropriateness of the target journal.
  • If the student has gathered a significant amount of data that will not go into an article, then the data should be presented in an appendix to the dissertation 
  •  

 Advantages

  • Divides the thesis into manageable sections
  • Less of a big push at the end to get the thesis done
  • Professional critique and feedback at an early stage
  • Articles improve one’s CV, and gives you a head start on publishing from the thesis.

 Here are some positive quotes from the comments on the Thesis Whisperer blog post"


For mine, the greatest advantage is that I now don’t have to sit down and write publications associated with my thesis.
It would be fairly hard for examiners to say that your research is crap when it has already been peer reviewed. 
What this discussion raises for me, or reminds me of, is that too often PhD students spend so much time on this one research project that they end up having narrowed their vision of the field that they are in. In addition, their actual amount and breadth of experience with research methodology is quite brief, because the research methods utilised and analysis of data/results is limited in scope.   I would argue that a PhD by publication can not only be rigorous, due to the peer review process, whether internal or external, but it also likely demands that the student demonstrate a variety of research skills across a number of research studies.   The PhD by publication is much closer to the real life work of a career academic, where quantity of publications is a ‘fact of life.'

Disadvantages:



  • The format may not fit all dissertation topics
  • Can be looked down on by humanities-oriented scholars or disciplines
  • Can be looked down on by older, more traditional scholars
  • Can be a big delay if all of the articles must be PUBLISHED before the Ph.D. is granted.
  • The dissertation is less useful as a doorstop.

Here are some negative quotes from comments on the Thesis Whisperer post:"


I could imagine that a paper could possibly be produced from the lit review or maybe the discussion chapter from the end, but I can’t imagine how a chapter about theory, methodology or findings could function as a stand-alone paper??
I was strongly advised against it as the monograph plus 1 or 2 article is still the expected norm within English departments. I asked the same question, PhD by publication or PhD as monograph, at a conference a few months later where three scholars in my field (from UK, Australia and USA) gave a seminar for post grads on the job market, and was virtually scoffed at for even suggesting that a PhD by publication was a possibility in English.
I was incredibly surprised at the range of (often strongly voiced) opinions that academics and university administrators, as well as PhD students, have on the issue of PhD by publication. I’ve seen people having angry, loud arguments (particularly in the social sciences) about whether a PhD publication is a positive development for the academy.    I think the reason for the divergent opinions is that this issue goes to the very heart of what we think scholarship should be about. The monograph model suggests that good scholarship should be based on an ability to produce an in-depth, book length analysis of a given issue, whereas the publication model tends to correlate more closely to existing research evaluation frameworks, which value large numbers of papers more highly than other forms of research output (such as books).

So, here are some things to consider in making a decision about whether the three-article dissertation is right for you:

  • Check your university regulations. Some British institutions (based on the comments on the Thesis Whisperer post) require the three articles to be completely published before the thesis is accepted (obviously a potentially dangerous source of delay). Some institutions promote this form of dissertation, others try to restrict it. Check your options.
  • Is your archaeology more science- or humanities-oriented. The 3-article dissertation is definitely a development in the sciences, and it is more widely accepted in scientific disciplines. Humanities-oriented archaeologists, perhaps Classicists, who have spent their whole career on one type of building in one time period may be less likely to approve of the 3-article format. If you have such people on your committee, or if you intend to apply for jobs in programs consisting of such scholars, you may want to go with the traditional dissertation format.

Theses nailed to the wall in Uppsala
I wonder what they think about 3-article dissertations in Sweden, where they still nail their theses to the wall.