Decision, Risk, and Values: The Philosophy of Churchman and Ackoff

A couple of months ago, I suggested a possible conflict of interest between STS and the history of science.  Effectively, the aspirations of STS to contemporary relevance is at least partially dependent on potential contributions arising from new research results.  For these results to have impetus, conclusions should be novel.  Historians of science usually see their own opportunities in confirming STS results by mining examples from history, which, as illustrative examples, are treated as effectively “lost” to the present.

However, novelty can be augmented by conveniently forgetting the history of the ideas underlying the conclusions on offer.  By mining deep history for ideas that are, in some sense, to be considered “lost” (or by seeking evidence that the ideas have never existed at all), historians can inadvertently create an “anti-history” of the subsequent history of those ideas.  A better opportunity, I would argue, is to be found in placing the claims of STS and philosophical peers within their historical traditions.  Historians could keep track of who else is currently espousing these ideas based upon much fuller accounts of their history extending to the present.

Unfortunately, historians’ bookkeeping methodologies are woefully inadequate to this task.  But it is still possible to fill in pieces of the history where the opportunity arises.  This particular post is prompted by a recent post at The Bubble Chamber, which posits a recent move in the philosophy of science, which takes efficacy as a key criterion of knowledge.  However, my own historical work on the figures of philosophers West Churchman and Russell Ackoff (who just died last year) suggests that the tradition is neither new nor lost — perhaps just misplaced by philosophers (though I trust philosophers can clarify this point).  Neither was obscure: Churchman was actually editor of Philosophy of Science from 1948 to 1958.  However, both turned from philosophy of science to operations research before ultimately winding up in the eclectic realm of “systems thinking”.

To make this point in a timely fashion I am going to break informal blog policy to present a short extract from my forthcoming book, presently titled In Pursuit of Rationality: Science and the Rise of Policy Analysis.  This is very much part of my professional output, so if you wish to use any of the material, please cite either the book or my 2007 dissertation (ProQuest link) in which the material also appears.  Also, since this is a manuscript in progress, please feel free to offer suggestions in the comment space.  Although I don’t think he has published anything on the topic, also note that Alan Richardson, a philosopher of science at the University of British Columbia, has done some work on this specific topic as well.  Finally, the images of Churchman and Ackoff are not being used in the book.  I encourage you to click on them to go to the original pages on which they appear.

–Will

It’s hard to recall how and why I moved my intellectual dwelling some half century ago from epistemology to management.  The two questions, “What’s wrong with logical positivism’s theory of knowledge?” and “How many 15½-33 men’s shirts should be kept in a retail store’s shelves?” do seem a bit different, don’t they?

C. WEST CHURCHMAN, 1994[1]

Virtually all support of the early professionalization of operations research was based on the desire to replicate the successes of wartime OR in a peacetime context.  The most important exception was the establishment of an OR research and education program at the Case Institute of Technology in Cleveland by two transplants from the philosophy department at the University of Pennsylvania, West Churchman and Russell Ackoff.  Although Churchman had been impressed by Abraham Wald’s work on sequential analysis while working at the Frankford Arsenal during the war, neither Churchman nor Ackoff had had any contact with the wartime OR groups.  Instead, they were attracted to OR as a means of putting into practice the “experimentalist” ideas of Churchman’s mentor, Edgar Singer, Jr., a philosopher of science from the pragmatist tradition who had been a student of William James.[2]

Singer, Churchman, and Ackoff all eschewed the idea embraced by logical positivists that the idea of science could be defined in terms of a cumulative and constantly expanding body of knowledge founded upon basic empirical truths.  Instead, they preferred to define science around its methodology and its ability to achieve goals.  From their perspective, science was something that occurred when a goal was explicitly stated, various means of achieving the goal were hypothesized, and these means were explored by means of experimentation in order to find the best possible option.  They understood “scientific” method to differ from non-scientific acts of problem solving only insofar as science actively sought out a “best” or most efficient solution, rather than suffice with a less-than-optimal solution.  They defined sufficing with a less-than-optimal solution as “lag” and the movement from a better to a worse solution for the sake of change as “anti-lag”.[3] Thus, the conduct of “science” required a record of stated problems and a record of attempted solutions.  To practice science meant to adhere to the best attempted solution; to do otherwise was to lag behind science.

Statistical inference was at the heart of Churchman and Ackoff’s vision of science.  In certain simple scenarios one hypothesis out of many might prove obviously superior, but advances in statistical methodology promised to revolutionize scientific method by sharpening humans’ ability to judge between competing hypotheses, even if the impossibility of arriving at absolute certainty persisted.  To demonstrate the point Churchman offered a simple example at the beginning of his 1948 book, Theory of Experimental Inference.  He pointed out that the industrial problem of deciding whether a new and improved material is stronger than an existing material is ultimately resolved by performing a series of tests yielding results that might or might not appear to offer a resolution to the question.  A simple side-by-side comparison of test results might show the new material to be consistently stronger than the older one, but a large variance in the strength measurements of the new material might hint at some inconclusiveness in the test.  Fortunately, he observed, advanced statistical methodology offered ways of distinguishing between the sets of data in a rigorous way by determining just how likely it was that the new material was actually stronger.[4]

One of the points that statistical analysis—and especially the wartime development of sequential analysis—made apparent was that the act of deciding whether or not any given inference was true ultimately became a value judgment about just how certain the experimenter wanted to be that the statement was, in fact, true.  For instance, because the only way to determine absolutely that the new material was in fact stronger was to conduct an infinite series of tests, experimenters were forced to define their tolerance of the risk that their conclusion was in error.  Defining such tolerances is always to an extent arbitrary, but Churchman and Ackoff hastened to point out that the definition of tolerance was also a function of values.  One had to ask oneself just how important it was that the correct conclusion had been reached.  Because these decisions could have a clear ethical dimension—for instance, in determining the lethal dosage of a chemical—declarations of correctness and ethics could not be separated.  Defining how confident one can be in determining a lethal dosage, and, likewise, whether it is better to overestimate or underestimate that value, becomes a measure of the value one places on preventing fatalities.[5]

In positing the interconnectedness of knowledge and values, Churchman and Ackoff acknowledged a certain debt to the relativist resolution of the conflict between rationalism and empiricism, which held that truth was contingent upon one’s willingness to believe it.  However, they were intent on not falling into the trap of simple skepticism that relativism implied.  By building on pragmatist foundations, their experimentalism admitted that it had no guaranteed basis for knowledge.  They did not see this admission as a weakness: by acknowledging the arbitrary assumptions and values that sustained visions of truth, one could confront these assumptions and values head-on.[6] In fact, this acknowledgement presented philosophers with an opportunity to apply scientific method all at once to problems in the natural sciences, the social sciences, policymaking, and even ethics.  Because scientific inquiry simply involved the testing of competing solutions to problems, it followed that issues such as the just allocation of resources were as amenable to research as the properties of the natural world.  In fact, Churchman and Ackoff were especially exercised by the possibility of including the problems of an ethical society within their vision of empirical science.  To them, a scientific ethics rejected the notion that ethical rules followed from an understanding of the natural world.[7] Instead, experimental science could be used to study the values that people held and the means they used to come to agreements that they judged to be fair.[8]

In the 1950s, Churchman and Ackoff would become early adherents to decision theory.[9] Yet, even prior to their acceptance of the conceptual language of decision theory, they emphasized the need to find valid means of observing values and their manifestation in decisions, which involved establishing a firm logical relationship between the design of experiments and the hypotheses to be evaluated.  If this most basic problem were not solved, the entire activity of experimentation would be severely weakened and rendered stagnant.  Some hypotheses, of course, simply could not be tested, because limitations in prior knowledge prevented the design of an experiment capable of correlating the result with the hypothesis.  To decide which hypotheses could be reliably tested and how, it would be necessary to situate the hypothesis and the experiment within the entire system of experimental knowledge assembled to that point.[10]

The clear problem preventing the design of valid scientific experiments was the need to correlate all prior knowledge into a single provisional system.  With a rapidly expanding and diversifying scientific enterprise in the middle of the 20^th century, Churchman and Ackoff identified the fracturing of knowledge through specialization as the most pressing problem facing experimental science.  To confront the problem, they advocated a “unification” of scientific work distinct from the unification of knowledge still being advocated by some logical positivists in the postwar years.[11] Their view of unification stressed the interrelation of disparate viewpoints in the art of experimental design, pointing out, for instance that biology, psychology, and sociology could all be related to the act of observation, deploying as a case in point Friedrich Bessel’s famous development of the “personal equation” in precise astronomical observation.[12] By Churchman and Ackoff’s reckoning, because science was a method rather than a body of knowledge, no science was more fundamental than any other science.  Rather, each science leaned on the others in the search for more effective knowledge.  The question was how these sciences could continue to do so in the face of the mounting complexity of information.

Churchman and Ackoff looked to an institutional solution, proposing the establishment of a series of “Institutes of Experimental Method”.  These institutes, divided into four sections, would train methodologists—specialists in the issues of scientific methodology—who would serve as consultants to scientists.  Members of a “general methodology” section would keep track of scientific knowledge so as to criticize and improve scientists’ methods of experimentation and the criteria used to evaluate the results of those experiments.  A mathematical statistics section would develop the insights of statistical theory and ensure that statistical theories were properly employed.  Experts in a separate sampling techniques section would scrutinize and help formulate the “presuppositions” behind experimental samples.  Finally, in recognition of the fact that scientific work does not always remember why it has taken the paths that it has, a history of science section would investigate the ways scientific investigations of the past influenced the ways current scientific inquiries were framed and would help determine what aspects of past scientific work could be revived most fruitfully in the present.[13]