Posted on

What is the point… of social simulation modelling?

Previously, I mentioned a thread on SIMSOC initiated by Scott Moss. He asked ‘Does anyone know of a correct, real-time, [agent] model-based, policy-impact forecast?. Following on to the responses to that question, earlier this week he started a new thread entitled ‘What’s the Point?:

“We already know that economic recessions and recoveries have probably never been forecast correctly — at least no counter-examples have been offered. Similarly, no financial market crashes or recoveries or significant shifts in market shares have ever, as far as we know, been forecast correctly in real time.

I believe that social simulation modelling is useful for reasons I have been exploring in publications for a number of years. But I also recognise that my beliefs are not widely held.

So I would be interested to know why other modellers think that modelling is useful or, if not useful, why they do it.”

After reading others’ responses I decided to reply with my own view:

“For me prediction of the future is only one facet of modelling (whether agent-based or any other kind) and not necessarily the primary use, especially with regards policy modelling. This view stems party from the philosophical difficulties outlined by Oreskes et al. (1994), amongst others. I agree with Mike that the field is still in the early stages of development, but I’m less confident about ever being able to precisely predict future systems states in the open systems of the ‘real world’. As Pablo suggested, if we are to predict the future the inherent uncertainties will be best highlighted and accounted for by ensuring predictions are tied to a probability.”

I also highlighted the reasons offered by Epstein and outlined a couple of other reasons I think ABM are useful.

There was a brief response to mine then and then another, more assertive, response that (I think) highlights a common confusion of the different uses of prediction in modelling:

“If models of economic policy are fundamentally unable to at some point predict the effects of policy — that is, to in some measure predict the future — then, to be blunt, what good are they? If they are unable to be predictive then they have no empirical, practical, or theoretical value. What’s left? I ask that in all seriousness.

Referring to Epstein’s article, if a model is not sufficiently grounded to show predictive power (a necessary condition of scientific results), then how can it be said to have any explanatory power? Without prediction as a stringent filter, any amount of explanation from a model becomes equivalent to a “just so” story, at worst giving old suppositions the unearned weight of observation, and at best hitting unknowably close to the mark by accident. To put that differently, if I have a model that provides a neat and tidy explanation of some social phenomena, and yet that model does not successfully replicate (and thus predict) real-world results to any degree, then we have no way of knowing if it is more accurate as an explanation than “the stars made it happen” or any other pseudo-scientific explanation. Explanations abound; we have never been short of them. Those that can be cross-checked in a predictive fashion against hard reality are those that have enduring value.

…

But the difficulty of creating even probabalistically predictive models, and the relative infancy of our knowledge of models and how they correspond to real-world phenomena, should not lead us into denying the need for prediction, nor into self-justification in the face of these difficulties. Rather than a scholarly “the dog ate my homework,” let’s acknowledge where we are, and maintain our standards of what modeling needs to do to be effective and valuable in any practical or theoretical way. Lowering the bar (we can “train practitioners” and “discipline policy dialogue” even if we have no way of showing that any one model is better than another) does not help the cause of agent-based modeling in the long run.

I felt this required a response – it seemed to me that difference between logical prediction and temporal prediction was being missed:

“In my earlier post I wrote: “I’m less confident about ever being able to precisely predict future systems states in the open systems of the ‘real world'”. I was careful about how I worded this [more careful than ensuring correct formatting of the post it seems – my original post is below in a more human-readable format] and maybe some clarification in the light of Mike’s comments would be useful. Here goes…

Precisely predicting the future state of an ‘open’ system at a particular instance in time does not imply we have explained or understand it (due to the philosophical issues of affirming the consequent, equifinality, underdetermination, etc.). To be really useful for explanation and to have enduring value model predictions of any system need to be cross-checked against hard reality *many times*, and in the case of societies probably also in many places (and should ideally be produced by models that are consistent with other theories). Producing multiple accurate predictions will be particularly tricky for things like the global economy for which only have one example (but of course will be easier where experimental replication more ogistically feasible).

My point is two-fold:
1) a single, precise prediction of a future does not really mean much with regard our understanding of an open system,
2) multiple precise predictions are more useful but will be more difficult to come by.

This doesn’t necessarily mean that we will never be able to consistently predict the future of open systems (in Scott’s sense of correctly forecasting of the timing and direction of change of specified indicators). I just think it’s a ways off yet, that there will always be uncertainty, and that we need to deal with this uncertainty explicitly via probabilistic output from model ensembles and other methods.Rather than lowering standards, a heuristic use of models demands we think more closely about *how* we model and what information we provide to policy makers (isn’t that the point of modelling policy outcomes in the end?).

Let’s be clear, the heuristic use of models does not allow us to ignore the real world – it still requires us to compare our model output with empirical data. And as Mike rightly pointed out, many of Epstein’s reasons to model – other than to predict – require such comparisons. However, the scientific modelling process of iteratively comparing model output with empirical data and then updating our models is a heuristic one – it does not require that precise prediction at specific point in the future is the goal before all others.

Lowering any level of standards will not help modelling – but I would argue that understanding and acknowledging the limits of using modelling in different situations in the short-term will actually help to improve standards in the long run. To develop this understanding we need to push models and modelling to their limits to find our what works, what we can do and what we can’t – that includes iteratively testing the temporal predictions of models. Iteratively testing models, understanding the philosophical issues of attempting to model social systems, exploring the use of models and modelling qualitatively (as a discussant, and a communication tool, etc.) should help modellers improve the information, the recommendations, and the working relationships they have with policy-makers.

In the long run I’d argue that both modellers and policy-makers will benefit from a pragmatic and pluralistic approach to modelling – one that acknowledges there are multiple approaches and uses of models and modelling to address societal (and environmental) questions and problems, and that [possibly self evidently] in different situations different approaches will be warranted. Predicting the future should not be the only goal of modelling social (or environmental) systems and hopefully this thread will continue to throw up alternative ideas for how we can use models and the process of modelling.”

Note that I didn’t explicitly point out the difference between the two different uses of prediction (that Oreskes and other have previously highlighted). It took Dan Olner a couple of posts later to explicitly describe the difference:

“We need some better words to describe model purpose. I would distinguish two –

a. Forecasting (not prediction) – As Mike Sellers notes, future prediction is usually “inherently probabalistic” – we need to know whether our models can do any better than chance, and how that success tails off as time passes. Often when we talk about “prediction” this is what we mean – prediction of a more-or-less uncertain future. I can’t think of a better word than forecasting.

b. Ontological prediction (OK, that’s two words!) – a term from Gregor Betz, Prediction Or Prophecy (2006). He gives the example of the prediction of Neptune’s existence from Newton’s laws – Uranus’ orbit implied that another body must exist. Betz’s point is that an ontological prediction is “timeless” – the phenomenon was always there. Einstein’s predictions about light bending near the sun is another: something that always happened, we just didn’t think to look for it. (And doubtless Eddington wouldn’t have considered *how* to look, without the theory.)

In this sense forecasting (my temporal prediction) is distinctly temporal (or spatial) and demands some statement about when (or where) an event or phenomena will occur. In contrast, ontological prediction (my logical prediction) is independent of time and/or space and is often used in closed system experiments searching for ‘universal’ laws. I wrote more about this in a series of blog posts I wrote a while back on the validation of models of open systems.

This discussion is ongoing on SIMSOC and Scott Moss has recently posted again suggesting a summary of the early responses:

“I think a perhaps extreme summary of the common element in the responses to my initial question (what is the point?, 9/6/09) is this:

**The point of modelling is to achieve precision as distinct from accuracy.**

That is, a model is a more or less complicated formal function relating a set of inputs clearly to a set of outputs. The formal inputs and outputs should relate unambiguously to the semantics of policy discussions or descriptions of observed social states and/or processes.

This precision has a number of virtues including the reasons for modelling listed by Josh Epstein. The reasons offered by Epstein and expressed separately by Lynne Hamill in her response to my question include the bounding and informing of policy discussions.

I find it interesting that most of my respondents do not consider accuracy to be an issue (though several believe that some empirically justified frequency or even probability distributions can be produced by models). And Epstein explicitly avoids using the term validation in the sense of confirmation that a model in some sense accurately describes its target phenomena.

So the upshot of all this is that models provide a kind of socially relevant precision. I think it is implicit in all of the responses (and the Epstein note) that, because of the precision, other people should care about the implications of our respective models. This leads to my follow-on questions:

Is precision a good enough reason for anyone to take seriously anyone else’s model? If it is not a good enough reason, then what is?

And so arises the debate about the importance of accuracy over precision (but the original ‘What is the point’ thread continues also). In hindsight, I think it may have been more appropriate for me to use the word accurate than precise in my postings. All this debate may seem to be just semantics and navel-gazing to many people, but as I argued in my second post, understanding the underlying philosophical basis of modelling and representing reality (however we might measure or perceive it) gives us a better chance of improving models and modelling in the long run…

Posted on

Millington et al. 2009 – A landscape fire-succession model

As promised, I’m hollering:

Millington, J.D.A., Wainwright, J., Perry, G.L.W., Romero-Calcerrada, R. and Malamud, B.D. (2009) Modelling Mediterranean landscape succession-disturbance dynamics: A landscape fire-succession model Environmental Modelling and Software 24 1196-1208

http://dx.doi.org/10.1016/j.envsoft.2009.03.013

Email me if you would like a reprint: jamesdamillington at gmail.com

Posted on

End of the Line

Coinciding with World Oceans Day, today was the UK premier of End of the Line the “world’s first major documentary about the devastating effect of overfishing“. It already seems to be making an impact with one of the UK’s leading food retailers, Pret a Manger, announcing that it will be switching to a more sustainable species of tuna. It’s good to see commercial organisations moving toward more sustainable food sources but as usual we need to think about what we can do as individuals too. Checkout the guide below to see which seafood and sushi are currently better or worse options for sustaining another of our important natural resources.

http://widgets.clearspring.com/o/497088da2c5e5ebd/4a2530b53603f423/4a0874e363907df0/f43fa7a9

Posted on

nighthawks

Chicago Skyline
I love the energy of big cities. Sometimes there’s so much it spills over into the early hours of the morning. On a trip to Chicago last weekend we saw Hopper‘s evocative depiction of that straggling energy – when you just don’t want to follow everyone else home to call it a night – in his famous Nighthawks.

Edward Hopper's Nighthawks
I know the feeling, it reminds me of visits to Don Quixote. Not the Don Quixote – Cafe Don Quixote on the Kingsway in London.

Cafe Don Quixote, London
Sometimes the 4am conversation at Cafe Don Quixote was reminiscent of the Spanish Knight; other times it was as lonely as Hopper’s nocturnal scene. But there was always people-watching those other stragglers waiting at the bus stop opposite the cafe, and the warm tea always provided just enough energy to survive the night bus home.

[Nighthawks photo by Mollie E Tubbs]

Posted on

Developing Sustainable Lifestyles

It can be hard not to abandon hope for a sustainable future when you read about our rapidly growing global population and the hopes of those in the developing world (growing the fastest) to lead more ‘western’ lifestyles. For ‘western’, read ‘consumptive’. Last year Jared Diamond came up with new numbers to make us feel even more hopeless; economically more developed countries are consuming resources and producing waste 32 times faster than less developed countries. That means, Diamond estimates, if everyone on earth were to eat as much meat, drive their cars as far and use electricity as prodigiously as Europeans, Americans and Japanese currently do it would be as if the human population had suddenly ballooned to 72 billion.

In an editorial in the latest issue of Conservation Biology, R. Edward Grumbine and Jianchu Xu use Diamond’s example when discussing the rise of China as a global economic power and consumer and the potential implications for conservation, the environment and the climate debate:

“China’s rapid economic rise has not helped conservation much. The country faces severe environmental challenges as the largest human population in history builds highways, factories, and housing to fully join the modern industrial world. The PRC [People’s Republic of China], however, remains relatively poor. Per capita income in 2007 was a mere one-fifth of the U.S. average; a typical American teenager has more discretionary income than the total annual salary of the average Chinese citizen.

Despite the importance of biodiversity issues, we want to draw attention to less-discussed environmental concerns that involve China at regional and global scales and which will likely transform life for all of us over the rest of the 21st century.”

Focusing on their discussion about issues related to climate change, Grumbine and Xu point out;

“Even if the European Union and the United States magically reduced their greenhouse gas emissions to zero while you are reading this sentence, China’s current pace by itself may keep global emissions rising through 2020.

China should not be blamed for the world’s runaway greenhouse gas emissions; the United States never even ratified the Kyoto Protocol. And we emphasize that China’s development dream is not a vision exclusive to the PRC. Beyond the Middle Kingdom, there are at least 1.2 billion people desiring cars, a decent house attached to a sewer system, potable water, and a fair measure of education and health care.”

The consequences of Chinese, and other poorer nations, realising their hopes of economic development?

“China and the rest of the less-developed world are driving wealthy countries toward a global reckoning with the fossil-fuel-powered, high-consumption, industrial way of life.

… The Tyndall Centre for Climate Change Research in the United Kingdom has estimated that some 23% of China’s total emissions result from net exports to the developed world. The Earth’s atmosphere bears a message: we are all in this together. China and climate change have collapsed us and them into we.”

Grumbine and Xu reckon China is poised to assume a leadership role in solving our international environmental problems despite, or maybe as a consequence of, its rapidly growing population and ecological footprint. The US government also seems to now recognise that we’re all in this together. In February, US Secretary of State Hillary Clinton set out to discuss these issues during her visit to China, and it appears her path may have been previously beaten (behind closed doors) during the preceding administration. In vowing to “restore science to its rightful place” President Obama named Nobel Prize laureate Steven Chu as his Energy Secretary. However, it seems that despite wanting to put science first, domestic political opposition to emissions cuts and to changes in the US energy mix are hindering these moves. Chu said recently to the BBC;

“As someone very concerned about climate I want to be as aggressive as possible but I also want to get started. And if we say we want something much more aggressive on the early timescales that would draw considerable opposition and that would delay the process for several years. … But if I am going to say we need to do much, much better I am afraid the US won’t get started.”

However, Chu went on to discuss his aims for a “massive programme of efficiency for commercial buildings”, vastly improved cost-effectiveness of solar energy, and an interconnected wind power grid. The Obama climate change bill is making progress, but the slow movement on energy policy because of domestic resistance to change has potential global consequences. If the economically more developed countries of the world cannot show that their populations are willing and able to change their lifestyles to be less consumptive, negotiations with developing countries will be hindered.

Pressure from lower levels of government will help push things along. Last week 178 Michigan scientists (including myself) signed a letter to the Michigan Congressional delegation calling for actions to achieve strong and effective federal climate change solutions policies. And scientists can (and need) to do more than just write letters and do their basic (physical) research in their laboratories and at their computers. Reiterating his commitment to science in an address to the National Academy of Science, President Obama asked scientists and academics to engage in society to inspire and enable people “to be makers of things, not just consumers of things”.

A paper by David Pimentel and colleagues, entitled Energy efficiency and conservation for individual Americans, provides some solid numbers and ideas about how we as individual citizens in the economically more developed world can modify our residential energy use, reduce the impact of personal transport, and make informed decisions about what we eat. I’ve listed some of their more interesting suggestions for a sustainable lifestyle below. These are rational and effective ways we can change our lifestyles to live more sustainably and show that we are willing to share the responsibility of mitigating the human impact on the global environment. If we don’t want to be left with mere hope for a sustainable future, we need to show how others in the world can realise their hopes of development whilst conserving energy, water and our other natural resources.

Residential Energy Conservation

  • Improve and upgrade windows – 25% of residential heating and cooling energy is lost directly through single pane windows

  • Plant trees – deciduous on south to shade the house in the summer and allow full-sun in the winter, evergreen trees to the north can act as a wind-break

  • Use the microwave – it’s the most efficient way to steam, boil, and bake vegetables

  • Power-down your computer when it’s not in use – “computers should be turned off if the unit will be left for 2 hours or more and if left for 30 min the machine should be set in standby mode”

Pimentel and colleagues suggest that implementing these, and other, measures around the home would save around 5,600 kWh/year, resulting in savings of about $390/year on home energy costs.

Personal Transport

  • Drive slower – “A reduction in speed from 104 kmph (65 mph) to 86 kmph (55 mph) will reduce fuel consumption 19% (UrbanPlanet). For a 104 km trip, only an additional 11 min would be required if one traveled at 86 kmph. This extra 11 min would repay the person nearly $1.86 in fuel saving, or repay the person $10/h.”

  • Inflate your car tires properly – this will decrease the fuel consumption by up to 3%

  • Get rid of that junk in your trunk – “each 45 kg (100 pounds) of additional load in the car will reduce fuel mileage about 1%”

  • Ride your bike – bicycling uses 25 kcal/km (34 kcal/mile)compared with 938 kcal/km (1,510 kcal/mile) for a mid-sized car

In summary: “[c]urrently, the average American uses about 1,900 l (500 gallons) of fuel/year in personal transport in contrast to the average person in the United Kingdom who consumes 1,700 l (450 gallons) (Renner 2003). If Americans implement the suggestions listed above [and others I haven’t listed] over a 10-year period, it would be possible to reduce fuel oil consumption between 10% and 20% from the current 20 quads of vehicle fuel [approximately 600 billion l or about 16 billion gallons of fuel] consumed in the U.S.”

Food system
The authors highlight several ways in which farmers and policy-makers can aggressively pursue sustainable agricultural practices. They are less precise about what individuals’ can do but offer some general ideas:

  • Eat local products – reduces transport energy costs [and find out where you should buy your wine from here]

  • Eat less (especially less meat) – read more about meat and the environment here

  • “Select aluminum and steel packaging over glass or plastic, for energy conservation. For the same reasons, however, plastic and especially recyclable plastic should be selected instead of glass and/or paper.”

Pimentel et al. summarise: “[w]ell-directed, serious conservation strategies influenced by individuals with supportive state and federal leadership and policies will have an enormous positive impact on transitioning to a sustainable energy future for the United States.”

Posted on

Apture

I’ve just discovered Apture – it looks like a pretty cool tool for integrating media into websites and blogs like this one. I’ve just installed it and will be experimenting to see how well it works. When you see an icon like this or this , the link it accompanies should open some related content in an interactive Apture window (which you can reposition or enlarge as you please). Here’s an example:

The Guinness Premiership 2009 try of the season was scored at the Memorial Ground in Bristol by David Lemi.

Posted on

US-IALE 2009: GLP Agent-Based Modelling Symposium

The second symposium I spent time in at US-IALE 2009, other than the CHANS workshop, was the Global Land Project Symposium on Agent-Based Modeling of Land Use Effects on Ecosystem Processes and Services. My notes for this symposium aren’t quite as extensive as for the CHANS workshop (and I had leave the discussion part-way through to give another presentation) but below I outline the main questions and issues raised and addressed by the symposium (drawing largely on Gary Polhill’s summary presentation).

The presentations highlighted the broad applicability of agent-based models (ABMs) across many places, landscapes and cultures using a diverse range of methodologies an populations. Locations and subjects of projects ranged from potential impacts of land use planning on the carbon balance in Michigan and rangeland management in the Greater Yellowstone Ecosystem, through impacts of land use change on wildfire regimes in Spain and water quality management in Australia, to conflicts between livestock and reforestation efforts in Thailand and the resilience of pastoral communities to drought Kenya. It was suggested that this diversity is a testament to the flexibility and power of the agent-based modelling approach. Methodologies used and explored by the projects in the symposium included:

  • model coupling

  • laboratory experiments (with humans and computers)

  • approaches to decision-making representation

  • scenario analysis

  • visualisation of model output and function

  • approaches to validation

  • companion modelling

Applied questions that were raised by these projects included:

  • how do we get from interviews to agent-behaviours?

  • how well do our models work? (and how do we assess that?)

  • how sensitive is land use change to planning policies?

  • how (why) do we engage with stakeholders?

In our discussion following the presentation it was interesting to have some social scientists join in the discussion that was dominated by computer scientists and modellers. Most interestingly was the viewpoint of a social scientist (a political scientist I believe) who suggested that one reason social scientists may be skeptical of the power of ABMs is that social science inherently understands that ‘some agents are more important than others’ and that this is not often well reflected (or at least analysed) in recent agent-based modelling.

Possibly the most important question raised in discussion was ‘what are we [as agent-based modellers] taking back to science more generally?’ There were plenty of examples in the projects about issues that have wider scientific applicability; scale issues, the intricacies of (very) large scale simulation with millions of agents, the integration of social and ecological complexity, forest transition theory, edge effects in models, and the presence of provenance (path-dependencies) in model dynamics. Agent-based modellers clearly deal with many interesting problems encountered and investigated in other areas of science, but whether we are doing a good job at communicating our experiences of these issues to the wider scientific community is certainly something open to debate (and was in the symposium).

A related question, recently raised on the SIMSOC listserv (but not in the GLP sumposium) is ‘what are ABMs taking back to policy-making and policy-makers’? Specifically, Scott Moss asked the question; ‘Does anyone know of a correct, real-time, [agent] model-based, policy-impact forecast? His reasoning behind this question is as follows:

“In relation to policy, it is common for social scientists (including but not exclusively economists) to use some a priori reasoning (frequently driven by a theory) to propose specific policies or to evaluate the benefits of alternative policies. In either case, the presumption must be that the benefits or relative benefits of the specified policies can be forecast. I am not aware of any successful tests of this presumption and none of my colleagues at the meeting of UK agent-based modelling experts could point me to a successful test in the sense of a well documented correct forecast of any policy benefit.

The importance of the question: If there is no history or, more weakly, no systematic history of successful forecasts of policy impacts, then is the standard approach to theory-driven policy advice defensible? If so, on what grounds? If not, then is an alternative approach to policy analysis and an alternative role for policy modelling indicated?”

The two most interesting replies were from Alan Penn and Mike Batty. Penn suggested [my links added]:

“… the best description I have heard of ‘policy’ in the sense you are using was by Peter Allen who described it “at best policy is a perturbation on the fitness landscape“. Making predictions of the outcome of any policy intervention therefore requires a detailed understanding of the shape of the mophogenetic landscape. Most often a perturbation will just nudge the system up a wall of the valley it is in, only for it to return back into the same valley and no significant lasting effect will be seen. On occasion a perturbation will nudge the trajectory over a pass into a neighbouring valley and some kind of change will result, but unless you have a proper understanding of the shape of this landscape you wont necessarily be able to say in advance what the new trajectory will be.

What this way of thinking about things implies is that what we need to understand is the shape of the fitness landscape. With that understanding we would be able to say how much of a nudge is needed (say the size of a tax incentive) to get over a pass. We would also know what direction the neighbouring ‘valleys’ might take the system,
and this would allow predictions of the kind you want.”

Batty:

“I was at the meeting where Scott raised this issue. Alan Wilson said that his company GMAP was built on developing spatial interaction models for predicting short term shifts in retailing activity which routinely produced predictions that were close to the mark. There are no better examples than the large retail units that routinely – every week – run their models to make predictions in retail markets and reportedly they produce good predictions. These are outfits like Tesco, Asda, M[orrisons] and S[ainsbury’s] and so on. I cant give you chapter and verse of where these predictions have been verified and documented because I am an academic and dont have access to this sort of material. The kinds of models that I am referring to are essentially land use transport models which began in the 1960s and are still widely used today. Those people reading this post who arent familiar with these models because they are not agent based models can get a quick view by looking at my early book which is downloadable …

I think that the problem with this debate is that it is focussed on academia and academics don’t traditionally revisit their models to see if longer term predictions work out. In fact for the reasons Alan [Penn] says one would probably not expect them to work out as we cant know the future. However there is loads of evidence about how well some models such as the ones I have referred to can fit existing data – ie in terms of their calibration. My book and lots of other work with these models shows that can predict the baseline rather well. In fact too well and the problem has been that although they predict the baseline well, they can often be quite deficient at predicting short term change well and often this arises from their cross sectional static nature and a million other problems that have been raised over the last 30 or more years.”

In response to Batty, Moss wrote:

“It is by no means unusual for model-based forecasts to be sufficiently accurate that the error is less than the value of the variable and perhaps much less. What systematically does not happen (and I know of no counterexample at all) is correct forecasting of volatile episodes such as big shifts in market shares in retail sales, macroeconomic recessions or recoveries, the onset of bear or bull phases in financial markets.

Policy initiatives are usually intended to change something from what has gone on before. Democratic governments — executive and legislative branches — typically investigate the reasons for choosing one policy rather than another or, at least, justify a proposed policy before implementation. Sometimes these justifications are based on forecasts of impacts derived from models. Certainly this is happening now in relation to the current recession. So the question is not whether there are ever correct forecasts. Certainly on the minimal criteria I suggested, there are many. The question is strictly about forecasts of policy impacts which, I conjecture, are rather like other major shifts in social trend and stability.

I believe this particular question is important because I don’t understand the point of policy modelling if we cannot usefully inform policy formation. If the usefulness we claim is that we can evaluate policy impacts and, in point of fact, we systematically (or always) produce incorrect forecasts of the direction and/or timing of intended changes, then it seems hard to argue that this is a useful exercise.”

But is focussing on the accuracy of forecasts of the future the only, or indeed best, way of using models to inform policy? In recent times some policy-makers (e.g. Tony Blair and New Labour) have come to see science (and it’s tools of modelling and predictions) as some kind of a ‘policy saviour’, leading to what is known as evidence-based policy-making. In this framework, science sits upstream of policy-making providing evidence about the real state of the world that then trickles down to steer policy discourse. This may be fine when the science is solving puzzles, but there are many instances (climate change for instance) where science has not solved the problem and rather has merely demonstrated more clearly our ignorance and uncertainty about the material state of the world.

Thus, when (scientific) models are developed to represent ‘open’ systems, as most real world systems are (e.g. the atmosphere, the global economy), I would argue that model forecasts or predictions are not the best way to inform policy formation. I have discussed such a perspective previously. Models and modelling are useful for understanding the world and making decisions, but they do not provide this utility by making accurate predictions about it. I argue that modelling is useful because it forces us to make explicit our implicitly held ‘mental models’ providing others with the opportunity to scrutinise the logic and coherence of that model and discuss its implications. Modelling helps us to think about potential alternative futures, what factors are likely to be most important in determining future events, how these factors and events are (inter)related, and what the current state of the world implies for the likelihood of different future states.

Science, generally, is about finding out about how the material world is. Policy, generally, is about deciding and making how the world how it ought to be. In many instances science can only provide an incomplete picture of how the world is, and even when it is confident about the material state of the world, there is only some much it can provide to an argument about how we think the world should be (which is what policy-making is all about). Emphasising the use of scientific models and modelling as a discussant, not a predictor, may be the best way to inform policy formulation.

In a paper submitted with one of my PhD advisors we discuss this sort of thing with reference to ‘participatory science’. The GLP ABM symposium is planning on publishing a special issue of Land Use Science containing papers from the meeting – in the manuscript I will submit I plan to follow-up in more detail on some of these participatory and ‘model as discussant’ issues with reference to my own agent-based modelling.

Posted on

Peter Orszag and Economic Models

A while ago I heard this interview with Peter Orszag, Director of the US Office of Management and Budget and one of President Obama’s key economic advisors. Interestingly to me, given what I’ve written previously about quantitative models of human social and economic activity, Orszag is interested in Behavioural Economics and is somewhat skeptical about the power of mathematical models:

“Too many academic fields have tried to apply pure mathematical models to activities that involve human beings. And whenever that happens — whether it’s in economics or health care or medical science — whenever human beings are involved, an approach that is attracted by that purity will lead you astray”

That’s not to say he’s not going to use some form of model forecasting to do his job. When Jon Stewart highlights (in his own amusingly honest way) the wide range of economic model projections out there for the US deficit, Orszag points out that he needs at least some semblance of a platform from which to anchor his management of the US economy. But it’s reassuring for me to know that in managing the future this guy won’t be seduced by quantitative predictions of it.

http://media.mtvnservices.com/mgid:cms:item:comedycentral.com:222776

Posted on

US-IALE 2009: CHANS Workshop

Coupled Human and Natural Systems (CHANS) research is all about relationships – that seemed to be one of the main conclusions of the Challenges and Opportunities in Research on Complexity of Coupled Human and Natural Systems workshop at the US-IALE meeting in Snowbird, UT. The processes of identifying relationships between system elements and fostering them between researchers are key to realizing successful CHANS research. The workshop followed-up on a symposium in which principle investigators from several NSF-funded CNH projects presented their work, and was an opportunity to ask questions that went unasked during that symposium. The workshop was also the kick-off event for the CHANS-Net website.

In my notes below I have not identified individual workshop participants, both because I may have mis-interpreted their actual opinions or thoughts, but also because in some cases I can’t identify from my notes who said what. The workshop started with a panel discussion (the panel composed of the symposium speakers) followed by break-out groups to continue the discussion.

The first question from the audience asked how the panel approaches the dichotomy between abstract and contextualised research. Just as many dichotomies are false, it seems this one is also not always appropriate. For example, one response was that just because we can explain some characteristic about a specific place does not mean we didn’t use any theory whilst arriving at that explanation. Talking to local people can generate interesting, if contextualised, questions and one panel member highlighted the usefulness of ‘stakeholder steering groups’ (composed of local decision-makers and actors) to identify diverse opinions and direct research in ways that may not have happened otherwise. Another suggestion was that communication tools (such as role-playing, hypothetical scenarios, model output, etc.) are useful as a starting point for discussion, even if the theory underlying those tools is not discussed. To summarise the responses to this question I’ll paraphrase one of the panel members; ‘it was Louis Pasteur that said the question is not about whether the science is abstract or applied, but whether it is good science or bad science’.

A subsequent question along similar lines touched on the interplay of theory and practice; “what happens when your research proposal does not match ‘messy reality’? How do you explain why you ended up doing what you did do [to the people that accepted your proposal]?”. No original research goes entirely to plan – as some famous scientist once said; ‘if we knew what we were doing, it wouldn’t be called research’. In reality, there is always ‘wriggle-room’ in resolving this issue – if you start with a broad question it is easier to stay with a research theme even if the details get modified. Similarly, it is useful to make sure your research question is more important than the place where you will address that question. One panel member described how a research project they worked on needed to change the country in which is was situated. By focusing on the general research question they were able to negotiate this seemingly insurmountable problem. Other respondents from the panel got into more ‘messy’ details about the execution of such research. For example, in a project that involved both social and physical scientists there was initially confusion about how the two different types of scientists perceived and undertook measurements. A useful suggestion was to read your colleagues synthesis/review papers from other disciplines or backgrounds. Through commitment and patience in working together, an objective should be to identify a common language between researchers that can then push the research goals forward. Again, the importance of relationships was stressed.

An issue that came up both in response to this theory versus practice question, and frequently throughout the workshop, was the importance of good project management. One panel member suggested that an individual needs to be designated with the task of keeping the project on timeline, and that this person may need to take tough decisions (e.g. to drop researchers from the project) if deadlines or standards are not met. Finally, changing research can be a healthy thing – there will be frequent opportunities to extend research in new directions because new questions will arise as understanding develops. We shouldn’t be afraid to pursue those new directions.

One participant wanted to talk about fields that remain under-represented in CHANS current projects. They asked; “what about landscape architects and other ‘professional’ individuals?” A variety of missing experts and knowledge were suggested: the built environment, technology, environmental psychology, historians, political scientists, and communications experts (cartographers, public relations consultants, etc.) amongst others. The need for greater engagement and strengthening of relationships with political scientists seemed to be particularly important to several participants: under what conditions does a policy succeed or fail? How do we achieve good governance of the systems being studied? The US EPA (for example) are making decisions all the time – how are CHANS researchers engaging and influencing them?

Another workshop participant suggested that the presentations in the symposium had highlighted several different ways to conceive the relationship of humans with their environment, from ‘invaders’ to ‘managers’ to ‘components’. “How do we cross the boundaries between these different conceptualizations?” The first respondent suggested that researchers tend to pick a perspective (on the relationship between humans and their environment) and stick with it throughout their research – a better approach might be to consider different perspectives within the same project. However, the discussion quickly moved on to address the entire concept of ‘coupled’ human-natural systems. Several panel respondents voiced concerns about the coupling metaphor – one suggested that (human-natural) systems are not coupled, rather there is just one system. Another highlighted how the US perspective [remember this was the US-IALE meeting] on the human-nature relationship is rather unique – Europeans arrived with ideas of wilderness, protection and exploitation which differ from those in other places. Many of our ideas about how humans are related to their environment, one panel member suggested, likely stem from the Judeo-Christian philosophy which states that man was given dominion over nature. During the development of that philosophy humans got separated [in their minds?] from ecosystems and a difference soon emerged between a perspective in which humans rightly dominate nature versus one in which humans are viewed as being part of nature [which might be more consistent with Eastern religions such as Taoism or Buddhism].

To conclude the panel discussion someone asked; “what direction does CHANS research need to go in?” I thought the most interesting response was that CHANS research should be about easing transitions between different environmental conditions, and not trying to stop those transitions. The speaker suggested that CHANS research needs to focus on the sustainability of communities in the face of environmental transitions, adopting a perspective closely aligned with the view that humans are a part of nature rather than a controller of nature. A second respondent (possibly a geographer) identified the problem of scale. Whilst pretty much every presentation in the symposium contained a ‘spider diagram’ depicting a system as arrows linking boxes of elements, scale didn’t figure much. Yet, the respondent argued, all the systems presented were to some degree scale-dependent (but note there are cases where scale-invariant behaviour is manifest [.pdf]).

The workshop then broke up into groups to discuss some the issues outlined above in more detail. Correspondingly, there was plenty of feedback when the groups re-convened. Put in the most simple terms, our group decided that there are four things that characterize CHANS research:

  1. It is hard (e.g. issues of coupling systems, scaling, policy work, management, interdisciplinarity, and many more)
  2. It’s all about relationships (both in the systems of study and between the researchers studying those systems)
  3. Face-to-face interaction is key (between researchers themselves, and between researchers and other stakeholders – policy makers, managers and importantly the people in the systems and places being studied)
  4. It takes time (because of all of the above)

This last point was emphasized in several places; it takes time to generate links between disciplines. And it can be frustrating. For CHANS research to be successful, one of the key steps is to identify individuals that are willing to make the same leap across a disciplinary divide that you want to. CHANS researchers aren’t alone in having these kinds of discussions right now, and there are lessons to be learned from many different groups investigating the web of human-environment relationships. That’s where the workshop ended in Utah, but no doubt the discussion about relationships will continue – possibly in forums like that offered by CHANS-Net.

Posted on

Cedar Swamps and Deer

Right now I should be back in East Lansing after a week of fieldwork in our Michigan Upper Peninsula (the UP) study area. We’ve been in the UP this last week to finish up on our mesic conifer planting and white-tailed deer density fieldwork that I’ve written about previously. However, an incident with a deer has delayed us (see the bottom of this post) so I’m doing some data entry and writing in Marquette while our Jeep is repaired.


In previous posts about the fieldwork we’ve done in the UP, I have included photos from forest stands containing deciduous hardwood species such as Sugar Maple or American Beech. Generally, it’s understood that white-tailed deer browse juveniles trees in hardwood stands during the daytime in the winter, but shelter overnight in nearby lowland conifer stands. One of the aspects of our project is to identify some quantitative relationships for this behaviour, and so we’ve often had take measurements in the cedar swamps adjacent to northern hardwood stands.


As you can see from the picture above, the density of cedar swamps can make tree measurements a bit tricky. A standard measure of forest stand density (or stocking) is ‘stand basal area’ – a measure of the area occupied by tree stems (i.e. trunks) in a given area. The northern hardwood stands in our study area can have a stand basal area of anywhere between 60 and 100 square feet per acre. Cedar swamps are much more densely populated, with stand basal area values of 280 to 350 square feet per acre. An example of the transition between these stand types is shown in the picture below (click for a larger image).


The high density of the cedar swamps combined with continual cover provided by the evergreen canopy (generally) make winter snow depths lower and winter air temperatures higher compared with the deciduous hardwood stands. The soggy conditions underfoot make surveying cedar swamps even trickier – one has to hop from tree-root island to tree-root island over puddles whilst trying not to impale oneself on the lower branches. Even with care given enough time you’re guaranteed scratches and wet boots.


We’ve completed our fieldwork for now and are just waiting for our Jeep to be fixed after we hit a deer on our last day of work. With so many deer in the area and the high number of miles we drive around our study area, it was only a matter time before we hit one. We were on a major highway and the deer came out of nowhere. We’ve often spooked deer driving on tracks through the forest – it seems to me that when they’re startled they just bolt in whatever direction they happen to be facing at the time. Even if that means running across the road in front of your vehicle. As you can see below, it left quite a dent in the radiator. But Megan did a good job of keeping us on the road and thankfully the only casualty was the deer.