chickerinoDOTcom

tumble-log » The danger of models

Back to chickerinoDOTcom Written on 06-Dec-2008 by chickerino

lenny smith.jpg Physicist Lenny Smith is interviewed in the New Scientist about the 'danger' of climate change prediction models. He says that whilst the models are certainly indicative of the affect of human activity upon climate change, it is dangerous to place too much value on their actual results.

He also points out that "national research centres are charged with both advancing the science and selling their results commercially". This amplifies the issue of over-valuing the results of the models because when an actual fiscal value is applied to it, there is a danger of a) over-selling - the scientist acts like a sales man and the incentive is tied to a need to have a strong belief that your data is correct, and b) over-valuing of the data by the purchaser - why would you purchase data from the experts if you don't trust that they are correct? This unavoidable human bias undermines the models themselves because they should be *entirely* objective.

Another more simple problem with the models is that they are often too ambitious. Next year a largely government funded body - the UK Climate Impacts Programme intends to run models which will allow the goverment and commercial organisations to 'predict' weather patterns by the hour well beyond the year 2060! We are currently at the stage where we can barely predict the weather accurately for the next week, let alone the next year or 50. I'm not the greatest fan of conventional probability (IMHO it's an artificial man-made concept that was invented to help us understand the world but that's another story) but I can use it to explain how when compounded, the accuracy of a model can be vastly reduced. Imagine that our clever weather prediction model could predict the temperature of the earth in a year's time with a ±1% margin of error. Might not make difference in the short term, but compounded over 50 years there is a ±40% margin of error. That's 1 minus 99% (or 0.99) to the power 50.

Anyway, my point here is not really to argue that weather prediction models are inaccurate or over-valued. I know nothing of this and my reasoning would be simplistic - I'd rather leave this to the experts such as Lenny Smith. I am more concerned about the use of models in a more broad sense. A model by definition is a mathematical or data representation of a scenario in the real world. It is also vastly simplified - of course it has to be since you can't actually model the real world - that in itself is a paradox. Because models are always simplifications, they often fail to take into account various small details and always fail to take into account the details that we don't yet know.

As a conclusion, I am not saying that models are useless. Far from it. I just think it's worth be cautious about relying on them too much. The investment banks relied on massively complex financial models that no one thought could fail - and we know what happened there! - And why did it happen? Because their models like many others weren't able to match the true complexity of the real world.

Send to a friend

Tags: weather,chance,financial,models,legend,banks,credit crunch,scientist,lenny smith,probability,scepticism

Comments

written on 08-Dec-2008

russell.volckmann [http://www.webjam.com/brandintegrity] says:

I agree that models in general need to be put into perspective—and credence in modeled forecasts needs to be balanced with the understanding that modeled results are only as good as the validity and completeness of the constants and variables. In other words, the modeler may leave out parts of a true, accurate, and complete equation, and this could be for any number of reasons: the unknown; false assumptions; ulterior motives; greed; human error; incompetence; ignorance; more...

Models can be relied on most often as possibilities, sometimes as probabilities, and rarely (if ever) as absolutes.
written on 09-Dec-2008

patencia [http://www.webjam.com/nice_new_rattle] says:

Apologies in advance for the frivolous comment.... But I just cannot be objective and not be biased by the look of this guy in the photo. I just can't take very seriously what he says with that look of illuminated profet.
BAD, I know, but I can't help it
written on 09-Dec-2008

patencia [http://webjam.com/nice_new_rattle] says:

proPHet, sorry...
written on 09-Dec-2008

chickerino says:

Unavoidable human bias. It's what makes life interesting!
Jesus certainly wasn't a PHd type though. That's for sure!
written on 09-Dec-2008

svirsk says:

Is it just something as prejudices, without them it would be impossible to get anything done, because every tiny thought needs reconsidering, though you should be aware that you are actually using a lot of prejudices in daily life, so not take things too seriously.

My favourite model is the concept of being 'english' (or Dutch for that matter) what does it mean? when does it end, for how long does the citizen model work, and when does it start to become dangerous.

good: people do something for the greater good, for their country etc people even fight wars to defend this idea, it keeps the country together

bad: people act violent when they come across people who are believers of an other model (Germans for example Whilst its only a model.

So, I guess models are good, because they make our world workable, but we shouldn't take them too serious kinda 80/20 or 90/10 take it 80% serious but know that its a 20% joke?
written on 01-Jan-2009

paulsari says:

Interesting post because of the problematisation of an over-reliance of models. Like you say, models simplify things and, as has variously been expressed neatly above as well, they set certain assumptions. Assumptions in themselves are OK, I think, but frequently those operating the models are unaware of what models presume which conditions are to be set aside. That makes (new) application(s) a big problem. (In other words, assumptions are related to the issue of how we arrive at we take as knowledge and should comprehend when that which was previously a minor nuance becomes a significant factor)______________ As an aside, I was under the understanding that a) yes, the models were too simplistic to ascribe a rating to the mortgages that have now proved to be exceedingly unrealistic / over-optimistic. That issue was confounded by two things; on the one hand that mortgages lent to those with reputable credit ratings were bundled with those to people with dire personal situations and, thus, it became easier to ambiguate the value of the packages of loans (usually in threes) and, on the other hand, that these bundles were sold from lending agency to lending agency very quickly (previously unprecedented, I believe) and so by the time the commercial banks were required to rate, the picture was rather blurred - they lost the Ueberblick b) quite simply, the commercial banks' statistical sample was far too small. ________________ Sjors (and indeed any and everyone else), should you have not already heard of it, you may be interested in the book Imagined Communities.