Auxiliary Hypotheses

Podcasts: BSPS Annual Conference

 

If you didn't make it to this year's BSPS annual conference in Oxford, we've teamed up with Philosophy Streaming to record the Presidential Address and the plenary discussions for your listening pleasure!

Lakatos Award Lectures

Endowed by the Latsis Foundation, the Lakatos Award is given to an outstanding contribution to the philosophy of science. Winners are presented with a medal and given the chance to deliver a lecture based on the winning work. To celebrate the 2015 and the 2016 award winners—Thomas Pradeu and Brian Epstein, respectively—they each delivered a lecture at the LSE last week. Introduced by Hasok Chang, Pradeu's lecture is entitled 'Why Philosophy in Science? Re-Visiting Immunology and Biological Individuality' and Epstein's is 'Rebuilding the Foundations of the Social Sciences'.

You can listen to the lectures here.

 

 

The Limits of the Numerical | Stephen John

 

There are many good reasons to want social policy to be based, where possible, on numerical evidence and indicators. If the data clearly shows that placing babies on their back reduces the risk of cot death, this information should guide the advice which midwives give to new parents. On the other hand, not everything that matters can be measured, and not everything that can be measured matters. The care a midwife offers may be better or worse in ways that cannot be captured by statistical indicators. Furthermore, even when we are measuring something that matters, numbers require interpretation and explanation before they can be used to guide action. It is important to know if neo-natal mortality rates are rising or falling, but the proper interpretation of this data may require subtle analysis. To make matters worse, many actors aren't interested in proper interpretation, but in using the numbers to achieve some other end; as a stick with which to beat the midwifery profession, say.

Anna Alexandrova and I are co-PIs on a project funded by the ISRF and based at Cambridge trying to think through such issues around the ‘Limits of the Numerical’ in the context of healthcare policy. As the sketch above suggests, there are lots of different senses in which the ‘numerical’ might be (or should be) limited: it may be impossible to measure some things accurately; it may be possible, but politically, morally, or socially inadvisable to measure others; some measures may be fine in some contexts but there may be limits to their use in others. Therefore, our team includes expertise not only in philosophy of science, but in political philosophy (Gabriele Badano) and anthropology (Trenholme Junghans). We are also lucky to have sister projects looking at the uses and abuses of numerical indicators in two other domains of social life: climate change policy (based at Chicago), and higher education policy (UCSB).

One reason to explore these topics is their obvious practical relevance; another is more theoretical. The use of numerical indicators is often praised as a way of ensuring that policy is ‘objective’. However, there are at least two senses of ‘objectivity’ at play in such claims: we might think that using numerical indicators, as opposed to human judgement, means policy is more likely to be based on an understanding of the world as it really is. Alternatively, we might think that using numerical indicators is more likely to ensure that policy is not swayed by idiosyncratic interests and biases. These two concerns can come apart: rolling a die to make a treatment decision may ensure that the decision is not swayed by a doctor's interests, but it does not increase our chances of identifying what is, in fact, the ‘best’ treatment. On the other hand, even if relying on trained judgement is more likely to get us to the truth, such reliance may seem to leave us at the mercy of a physician’s whims. Philosophers of science spend a lot of time worrying about whether measurement tools are objective in the sense of mirroring nature; many political debates, however, are more concerned with ensuring that measures are objective in the sense of being fair or impartial. Showing a policy-maker that her shiny new evidence hierarchy is epistemologically flawed may not speak to the reasons she values that tool: that it cannot be ‘gamed’ by big pharma.

 

To make these general comments a bit more concrete, consider a case study that has fascinated our team: the work of the National Institute of Health and Clinical Excellence (NICE) in the UK. Very roughly, NICE’s role is to advise NHS Trusts as to which drugs to buy. As part of this process, NICE (in)famously calculates the amount of health benefit (measured in the metric of Quality Adjusted Life Years, QALYs) that can be expected to result from purchasing a drug. In turn, NICE typically recommends that drugs should not be purchased when they cost more than £30,000/QALY. (Strictly, for the purists, per incremental QALY—but leave that to one side). This ‘threshold’ is, of course, incredibly controversial, because it means that NICE often recommends against buying drugs that would, undoubtedly, benefit some patients, but only at great cost. Where, then, does the number come from?

 

The official justification appeals to the fact that the NHS only has a limited budget, and, as such, every decision to purchase drugs has some opportunity cost; in rough terms, if you are spending more than £30,000 to get one QALY, the money could be spent somewhere else in the system to get more benefit. Of course, this form of reasoning raises deep and important questions in moral and political philosophy. Note, however, that these questions only seem interesting if £30,000/QALY does reflect the ‘true’ opportunity cost. Does it? It seems not. Rather, recent research from health economists implies that the ‘true’ threshold should be much lower—around £13,000/QALY. To put it another way, NICE is green-lighting very many treatments that, given its purported aims, it should not be.

 

This seems to be a scandal! After all, regardless of the ethical questions around health resource allocation, it seems that if (part of) NICE’s job is to allocate resources efficiently, they should do it properly. However, the official response to the studies has been rather surprising. The chief executive of NICE replied by pointing out that reducing the threshold would have a detrimental effect on the UK’s pharmaceutical industry. There is something fascinating about this response. It makes no sense at all if we think that the function of numbers in public life is to try to measure some fact about the world (in this case, the ‘true’ opportunity cost). Consider, however, the other role that numbers play: they provide a kind of stability, allowing different actors—the pharmaceutical industry, patient advocacy groups, and so on—to plan their strategies and policies. Changing the number would be like changing the rules of football halfway through the game. Would it be unfair to do that? This may seem like an odd question to ask, but it’s not clear that we get very far in thinking about numbers and objectivity in policy without understanding that fairness—or at least, the impression of fairness—is, often, a key concern. Oddly, even if the £30,000/QALY threshold is unmoored from reality, it can play this second role, much as the rules of football can be arbitrary but enable fair competition.

 

It would be nice if all good things came together, but they don't. Our research into particular tools in health policy opens up, then, a far larger question for philosophers of science: which forms of objectivity matter?

 

University of Cambridge

Getting Tough on Armchair Models | Anna Alexandrova & Robert Northcott

Models and modelling practices in science were once ignored in philosophy of science; however, in the past fifty years they have been anything but. From Mary Hesse’s pioneering work in the 1960s, to the writing of Ron Giere, Uskali Maki, Nancy Cartwright, Mary Morgan, and Margaret Morrison in the 80s and 90s, to today’s contributions from Michael Weisberg, Mauricio Suarez, Wendy Parker, and too many others to mention, scientific models are now studied left and right. This work is no longer quirky or marginal, and it spans many scientific fields. There are detailed and intricate accounts of what models are, of the variety of different models, and of the epistemic and social roles played by models. But we would like to suggest that in one respect, more should be done.

One of the most useful distinctions to have come out of this work is that between theoretical and phenomenological models, or targeted and targetless models to use the language of Cartwright and Weisberg respectively. For our purposes here, the two distinctions amount to the same thing: phenomenological or targeted models are evaluated on their ability to represent or predict certain features of the phenomenon in question. They are widespread in engineering, applied econometrics, meteorology, and other fields where accurate forecasting matters. Theoretical or targetless models, by contrast, explicitly eschew directly empirical criteria. They are for the most part minimal representations of a single causal factor or mechanism, and their goal is to study the relation between variables in the absence of complications introduced by the real world. Cellular automata models of complexity, Ising models of phase transitions, and rational choice and game theory models in economics and theoretical biology are all examples of targetless modelling. Provocatively, let us call it armchair modelling. It is these models that we need to get tougher on.

By and large, philosophers’ contribution has been to specify the various roles these models play: they isolate causal factors, construct credible worlds, structure research agendas, explore possible explanations, provide understanding, and finally they are playgrounds for creativity. We urge that philosophers should be more than just interpreters of these armchair models; they should also be evaluators of them. But evaluation of what sort?

E. O. Wilson has written:

[In biology] factors in a real-life phenomenon are often misunderstood or never noticed in the first place. The annals of theoretical biology are clogged with mathematical models that either can be safely ignored or, when tested, fail. Possibly no more than 10% have any lasting value. Only those linked solidly to knowledge of real living systems have much chance of being used.

Notice the terms in which he couches his critique: of course, some armchair models have eventually led to genuine explanatory successes while others have gathered dust in the drawers. Wilson is not dismissing the success cases, but he is implying that there are not enough of them.

In our view, this is the key question: has too much been invested in armchair models? Take a particular scientific problem; take a range of methodologies that scientists could be using to make progress on it; take the human, intellectual, and economic resources available to them; and ask ‘what is the rational way of apportioning these resources across the methodologies?’. Just as a community can plan more or less rationally which problems their scientists should tackle—that’s the ideal of a well-ordered science that Philip Kitcher formulates—so can this community make rational decisions about which methodologies to pursue. A methodology has to be an efficient one, given the task and the available resources. Let’s call that ‘efficiency analysis’.

Like E. O. Wilson on biology, we suspect that economics, to take one example, is not practicing well-ordered science. Intricate and sophisticated armchair models are central to the education of students, to publishing in top journals, to getting hired by top departments, and in general are the main currency of prestige in the discipline. This has been true for roughly 50 years. In economics at least, armchair science has had plenty of opportunity to prove itself. Has it? That’s a big empirical question. From a number of case studies by philosophers and historians of economics, including some of our own, our initial verdict is ‘not so much’. Armchair models can do a lot in economics, but not nearly enough to justify the level of investment they receive.

Philosophers of science are particularly well qualified to carry out efficiency analysis because they have the right distance and level of knowledge to question an ‘armchairist’ status quo. But to carry it out properly, we need to move on from our own status quo. Many papers have roughly the following structure: ‘scientists in a discipline X cannot conduct controlled experiments, so they build armchair models. I present a possible justification of this practice’. But there are many other methods available besides building armchair models: process tracing, case studies, observational studies, natural and quasi-experiments, small-N causal inference, and more.

We are all for naturalism and the so-called practice turn in philosophy of science. Engaging with the details of science is crucial. But this should not mean that we just look at what scientists do and then use our ingenuity to rationalize it. Sometimes naturalism calls for being more critical and asking what scientists could be doing instead.

Anna Alexandrova
King’s College
and
Department of History and Philosophy of Science
University of Cambridge

Robert Northcott
Department of Philosophy
Birkbeck
University of London

The Mysterious Methodology of Rational Choice Theory | Christopher Clarke

 

In the last few decades, economists have puzzled over the curious phenomenon of so-called ambiguity-averse preferences. You are indifferent between (A) receiving a cash prize if a coin lands heads, and (B) receiving the prize if a coin lands tails. You are also indifferent between (A*) receiving the prize if the Nikkei stock index goes up and (B*) receiving the prize if it goes down; for you are totally ignorant about the Japanese stock market. But you prefer (A) to (A*), and you prefer (B) to (B*). Thus, intuitively, you prefer gambling on the more familiar toss of a coin than on the less familiar stock market.

 

Now, your indifference between (A) and (B) suggests that you think that the coin landing heads is just as likely as tails; standard rational choice theory says it has probability ½. Similarly, your indifference between (A*) and (B*) suggests that you think that the Nikkei going up is just as likely as its going down; it also has probability ½. So standard rational choice theory says that (A), (B), (A*), and (B*) really amount to the same thing, namely, receiving the cash prize with probability ½. Therefore the standard theory is inconsistent with your preference for (A) over (A*), and (B) over (B*). And to accommodate such ambiguity-averse preferences, many economists have proposed variations to standard rational choice theory. But how are economists to go about modifying the standard theory, the theory of individual decision-making that constitutes a bedrock assumption of most economic models?

 

Introductory courses in economics teach students that there is a sharp distinction between two branches of economics. On the one hand there is ‘positive economics’ whose job is to describe and explain economic phenomena such as prices, unemployment, and consumer demand. On the other hand there is ‘welfare economics’ whose job is to tell national governments how they should intervene in the economy. Students are thus taught to keep the descriptive and the evaluative projects within economics sharply distinct. Given economists' invocation to keep descriptive and evaluative projects distinct, one might expect economists to distinguish two theories of individual decision-making. On the one hand, there might be a theory of what decisions—as a matter of actual fact—individual agent's will make in various contexts. On the other hand, there might be a theory of what decisions an individual has reason to make in these contexts; or of what is required of the agent in order to count as rational.

 

But this expectation will be confounded. For when it comes to rational choice theory, economists tend to mix the descriptive project with the evaluative one. The really curious thing, I find, is that economists conflate the question of whether a new proposal gives an accurate description of actual agents' choices, from whether the proposal delineates the requirements of rationality. In particular, those economists who propose theories that can accommodate ambiguity-averse preferences (for descriptive purposes) think it very important to defend ambiguity-averse preferences as rational. They wouldn't be satisfied to claim that ambiguity-averse preferences are widespread, but irrational. (See the contributions to the 2009 special issue of Economics and Philosophy, for instance.)

 

This practice raises some foundational questions. Does this practice signal a shift in the philosophical assumptions of economists, a jettisoning of the sharp distinction between description and evaluation, between positive and normative economics? Or is something deeper going on in the special case of rational choice theory? Do these economists think there is a deep conceptual tie between theories that describe decision-making and those that evaluate it as rational or irrational?

 

University of Cambridge