Podcast - how some researchers reduce trust in research - The Oxford Review - OR Briefings

Podcast – how some researchers reduce trust in research

Organisational Success Podcast

Source manipulation

Podcast links



Why do some people do this? How some researchers reduce trust in research. Hi, I’m David Wilkinson, the editor of the Oxford Review. I came across this last week and it’s both blatant and appalling because it really reduces people’s trust in research. Now you may have come across this concept here, it’s called the learning pyramid. Basically what it’s saying is it came out of the thing called the National Training Laboratories in the late 1980s. Basically what it’s saying at the top of the pyramid, the smallest part, is saying that students retain about 5% of everything that’s coming out of a lecture, about 10% from reading, about 20% from audio visual or video. They retain about 30% from watching a demonstration. They retain about 50% from group discussions, about 70% from practicing something, and 90% right at the bottom of the pyramid, they retain about 90% from teaching others.

The learning pyramid


I’ll say, you may have come across something like this before. It’s been around for a very, very long time. It doesn’t mean that it’s true by the way, which we’ll come onto in a minute. So last week I’m reading a research paper and this is it. Developing Management Effectiveness: The Nexus Between Teaching and Coaching. It was published in the International Journal of Management Education just last month.

The study

Now the paper, which I was surprised about actually references the learning pyramid. It uses a reference here called [Latrud 00:01:36] and it says adapted from Latrod. Now, the reason I was surprised about this was that as far as I’m aware, this isn’t true and it’s not true for a number of reasons. One is there are so many variables involved in learning. Things like depending on your expertise, your prior knowledge, whether you’ve already got a framework of thinking about the subject, whether you’re a good critical thinker or not. Reading can be every bit as effective as teaching somebody else.


In fact, teaching somebody else has actually been shown not to be as effective as some other things again, depending on the circumstances. So I’m intrigued now as to why somebody would be using this and also this reference of being adapted from Latrud. So let’s just go on and see what the paper’s saying about this. It says the complimentary nexus between classroom-based management education and coaching is however, and this is an important part, most strongly reinforced by studies conducted by the National Training Laboratory or NTL in Bethel, Maine US in the late 1980s.


So most strongly reinforced by studies, which is suggesting that there are some studies. The NTL studies empirically reveal the notion of a learning pyramid. That suggests that there are some experiments. They’ve done experimentation and this is right based on those experiments. In that learning is retained at graduated stages of learning interventions. So just like the pyramid’s saying that students tend to learn or retain less from attending a lecture or reading than they do from teaching this stuff. They’re using as a reference here of Latrod, that’s a person’s name based on a paper that was published in 2012.


So just thinking about that, just remember what they’ve said, the NTL studies empirically reveal the notion of a learning pyramid. So when I go to the bottom of the paper and look at the reference, this is what I find.

The reference in the paper
The reference in the paper

Latrud, A Rebuttal of NTL’s Institutes Learning Pyramid. Now I’m confused because that doesn’t sound like just from the title, like empirically revealed the notion of a learning pyramid, that it’s evidence of it. It sounds like the opposite. So anyway, let’s go and have a look at the paper and see, because titles could be anything.

Letrud's title

So I’ve got the paper out and this is what it says. The article discusses the learning pyramid propagated by NTL. It presents and compliments the historical methodological critique. What that means is lots of other people have said this is not true, right? There’s lots of reasons for it. Against the learning pyramid and calls for the NTL Institute to retract their model.


Now, for researchers to actually put that on a paper means that there’s very good evidence for saying that. It’s a very unusual thing to do. Bearing in mind what they’ve said is the NTL studies in particular reveal the notion that this is saying the opposite. They’ve used this as the reference for backing up the assertion that the learning pyramid is true when blatantly it isn’t. So let’s just have a quick look at the conclusions of Latrud’s paper.

Letruds conclusion
Letruds conclusion

The NTL’s Learning Institute lacks empirical evidence, the exact opposite of what that paper’s saying. Any attempt to perform empirical tests of the model will encounter methodological problems. As I said, there’s too many variables. Despite its inability to stand up to close examination, the model is still corroborated by the NTL Institute and continued distribution of this model cannot be justified. In order to reduce further dissemination, the NTL Institute ought to retract it.


That can’t get much less ambiguous than that. That’s pretty unambiguous statement. Yet the paper that we looked at is saying the studies, Latrod’s saying the exact opposite of what they’re saying Latrod is saying. You couldn’t get any more blatant. There’s no mistaking what Latrod’s saying. There’s no ambiguity in any of this. What I’m confused about is this. Why do people do this? I think there may be a couple of reasons and my thoughts are, and these are the kinds of practices that are see in the review when we’re reviewing papers because we probably get rid of around about 40% because of either the evidence is lacking or it’s poor studies or something. That’s on a good day. On a bad day, it can be up to 80% of the research that we read. I think there are a number of reasons why this occurs.


I think the first reason is that people kind of make up a theory based on their intuition. What they try to do is they try to dress it up as research or retrofit evidence from other places. I think that a lot of it is just poor quality research. So you get people doing things like they’ll publish a study based on 10 people in my office, which is hardly representative, right? So we get a lot of that. Or the evidence is just inconvenient. Therefore, what we’re going to do is we’re going to manipulate it and present it as something that it isn’t. That’s blatant and lying basically. Or just sheer laziness. There’s quite a lot of laziness goes on in, well, just about every aspect of work. Now, obviously the basis of this is if you’re reading research, check the sources because they may not be saying what you think they’re saying. What I’m interested in is what do you think about this?


What I’d like you to do is just put what you think and feel about this kind of practice. Why do you think people are actually doing it? Why are researchers doing this? It’s nuts and it’s pretty blatant and obvious. Anyway, I’m David Wilkinson, editor of the Oxford Review. If you want to be the most well-informed person in the room, go to Oxford-review.com and you can sign up for free research briefings and a whole load more. Okay. To become the most well-informed person in the room about people and organizations, including topics like leadership, management, organizational development, organizational change, human resource, human resource management, learning, organizational learning, learning and development, coaching, and work psychology and decision-making, go to Oxford-review.com.

Be impressively well informed

Get the very latest research intelligence briefings, video research briefings, infographics and more sent direct to you as they are published

Be the most impressively well-informed and up-to-date person around...

Powered by ConvertKit
Like what you see? Help us spread the word

David Wilkinson

David Wilkinson is the Editor-in-Chief of the Oxford Review. He is also acknowledged to be one of the world's leading experts in dealing with ambiguity and uncertainty and developing emotional resilience. David teaches and conducts research at a number of universities including the University of Oxford, Medical Sciences Division, Cardiff University, Oxford Brookes University School of Business and many more. He has worked with many organisations as a consultant and executive coach including Schroders, where he coaches and runs their leadership and management programmes, Royal Mail, Aimia, Hyundai, The RAF, The Pentagon, the governments of the UK, US, Saudi, Oman and the Yemen for example. In 2010 he developed the world's first and only model and programme for developing emotional resilience across entire populations and organisations which has since become known as the Fear to Flow model which is the subject of his next book. In 2012 he drove a 1973 VW across six countries in Southern Africa whilst collecting money for charity and conducting on the ground charity work including developing emotional literature in children and orphans in Africa and a number of other activities. He is the author of The Ambiguity Advanatage: What great leaders are great at, published by Palgrave Macmillian. See more: About: About David Wikipedia: David's Wikipedia Page