Why scientific myths not only endure in public but also some scientists minds, infecting some research with misleading counter-factual pseudo-research.
Interview with Kare Letrud – Scientific myths
Asst. Professor Dr. Kare Letrud talks to David about a common such research myth, the learning pyramid and how such scientific myths become to believed even by other scientists.
University Profile Page: https://eng.inn.no/about-inn-university/employees/kare-letrud
Research Gate profile: https://www.researchgate.net/profile/Kare_Letrud
This video about scientific myths invading research and why the learning pyramid is not supported by the evidence. https://oxford-review.com/really-bad-research-what-do-you-think/
Get free research briefings, infographics and more
Transcript – how scientific myths and fake research even gets into research papers
Welcome back. I’m David Wilkinson, the editor of the Oxford Review. And today we’ve got Kare Letrud, who’s done some really interesting papers around scientific myths, and how they’ve managed to grow and invade scientific thinking and papers. So welcome, Kare. Do you just want to start off by telling us a little bit about who you are, what you do, and a bit of your journey and background, that’s led you to this place, in terms of the research?
Yeah, I’m an assistant professor at Inland Norway University of Applied Sciences, the campus Lillehammer. Here, I mainly teach a course called Exum Philosophicum, which is a 10 ECCS introductory course in the philosophy of science and critical thinking. It’s a compulsory part of several bachelor degrees. And I quite recently, a couple of months ago, I defended my thesis in, at the University of Bergen, in the philosophy of science. This, this doctorate is not strictly a PhD, it’s called a Doctor of Philos, and it’s the independent study without the supervisor, and I’m not part of any PhD program. So I, in [inaudible 00:02:55] I’ve referred it to as a DIY PhD. Right?
It’s a busy day and the research is good. I’ve read it.
So, and yeah, so this thesis was about scientific myths and particularly the learning pyramid, which is a very central, very, my main example of, scientific myth, scientific myth those discussions.
Yes. Yeah. And congratulations on your, your defense of the thesis. As I say, I’ve read the thesis. It’s very good. Very interesting. So you know, you’ve, over the last few years, since 2012, from what I can gather, is you’ve published a series of papers around, scientific myths, and how they’ve pervaded some scientific thinking, but other people’s thinking as well. And the first one that I picked up on, and we’ll come back to this a little bit later, was the paper that’s entitled A Rebuttal of NTL Institute’s Learning Pyramid, but also the ones that followed on from there, the Affirmative citation bias in scientific myth debunking: A three-in-one case study, excavating the origins of learning pyramid myths. And then obviously, your thesis that you were talking about. So do you just want to give us an overview of what led to this stream of research?
Yeah, it’s, the project goes way back really, maybe 12 years.
It started when the local chief librarian, Sigbjørn Hernes, and I were presented with this learning premium model. And we found, it’s strange, there were many weird things about it. Like, it seems like you remember maybe 10%, 20% if you attended a lecture, but if you watch the same lecture on the video tape, you apparently remembered 40, 50% of the same material.
And that’s kind of, it didn’t seem quite consistent. And the numbers were too even, it was kind of strange, but seems, seemed to be, accepted. No, they show that way at first week. Okay. Maybe it’s this, maybe it’s so. And it corresponded to how we thought about teaching, and we were frustrated doing lectures. How much do you actually remember from this?
And so, okay. Maybe this is so, but we started to investigate. And when we, this was about 2008, yeah, 2008, and we started to investigate this learning pyramid, and we didn’t find any research about it. So, and we try to, what we did find was that several used them, but, and some criticize it. And we made a paper in, we published a paper in 2009 in the Vision Jet publication, where we just did a review about what do we know about this learning premise, and how often are they, or how common are they, in the new vision of literature on the education?
So that was just, this is what we know, this is how common it is. And Sigbjørn and I, we had a good, very good club collaboration, and we proceeded to work with this issue. And we increased the range of it. So we started looking at the international diffusion of this model.
So, and that was in 2016, we published a paper in general curriculum studies, we reported 418 peer reviewed journals, publishing some version of their impairment.
And now consider it that there is babies, 70 million papers published by now. 418 it doesn’t sound that impressive, really. It is too many, but not dramatically, I would say. But what we considered maybe more important is that also several fields, specifics, encyclopedia articles, featured some version of the learning pyramid. And when they, when knowledge gained this level in publication, when they come into the encyclopedias, they are like, this has got consensus. This is what we know about these things.
And this paper took some time. It took several years to publish, or to write and to get it published. Because this learning pyramid is, it is just basically a blanket term. It’s an umbrella term for a lot of different models, and they don’t…some of them don’t really look like each other. So we had to establish the history and genealogy, genealogy of the [inaudible 00:06:37] business, to be able to say, this is the version of the permit, this is a version of that, and that that is not.
So this was a very complex time consuming piece of work, and a lot of time went into that. And following that, we worked on, about, we’ve carried this kind of genealogy, its historical perspective, a bit further, and doing this archeology, we don’t go way back to…also, looking for the origin of this is learning pyramid, also was a quest to find what research was there available, that could support this model.
So we dug our way back, and back, and back, into the model. It seems to change consistent, consequently, and we ended up in the 1850s.
That’s where we found the first identifiable version of the learning pyramid. And from what I can gather, that is not the first one, or sometimes maybe 18, late 1850s, they were talking about this as common knowledge, as, this is what we used to say, this is common, this is common expression. It is…yeah. So it is probably decades older than 1815.
But from there, we weren’t able to get hold of any more digitalized, digitized papers. So we had to just put, stop there. But for us, it was adequate, because we could now say that there is no scientific basis for the learning pyramid, because simply, psychology, psychologists, started studying retention years later. Decades later.
So…if literally and pretty much, it was somehow based on research, we have to, we will have to rewrite entire history of lots of psychology, basically. So we were content that, okay, there is no research for this.
So that was the first finding. That we can establish that this was basically a folk psychology, hearsay, or what do you call…there’s an expression-
David (09:15):[crosstalk 00:09:14].
And saying. Yeah. And, that’s the one finding. And the other finding is the quite paradoxical, discovered. This model is actually 160 years old, and we have found it in academic publishing, or inside the academic publishing, outside, for all these years. And usually science or research is able to remove false or unsubstantiated claims. But of course we, this would take some time. But at least 160 years, that seems a bit much.
So I’m, from primarily the 2016 and 2018 papers I wrote, I use these two papers as a case to try to develop some conceptual theoretical work on top of this, and to use the learning permit case, as a case of a scientific myth, and try to develop the kind of concepts and, scientific new concept, and some explanation as to how scientific myths persist, how they, how this diffused, how they persist debunking, et cetera. So that was, that ended to be, ended up being my thesis basically.
Yeah. Yeah. We’ll come back to that in a second.
Let me…the story of how I came across you actually, is paradoxical of all of this. I was actually [inaudible 00:00:10:59], my Masters in, or Doctorate, or, in Education. So, do a lot of work around learning, obviously, and the whole thing around expert and novice learners and how they learn differently, and various other bits and pieces like that.
So I, actually happened to be reading a study. I don’t know whether you can hear that. It’s rain. It’s amazing! We’ve got a lot of thunder and lightning.
So I was reading a paper that was published this year called Developing Management Effectiveness. I don’t know why academics use this kind of language, the nexus between teaching and coaching. Right?
So in there, they reference your work. Right? And they referenced the paper, the Rebuttal of NTL Institute’s Learning Pyramid paper that you published, some time ago. Right? But they use that paper as evidence to support the learning pyramid, saying that you have scientifically proved the pyramid. And I was like, what? This is exactly the opposite of what this guy’s saying. I met you at that point. And I think, how can they actually publish this? And and I’m still a bit stunned about it, like they would actually do that. I don’t know what your…have you seen that paper?
I was aware, there’s, it is not often cited, that this paper is not often cited. And by the time I checked it, I assumed that it would be citing it incorrectly. Yeah, it is. It’s definitely not uncommon.
It’s amazing! So, can you just, what I’m interested in really is, what thoughts you’re coming up with is, is how come these myths, not only pervade scientific thinking, but actually get to a stage where people are using it in the completely opposite direction. So they’re using other people’s research in a completely opposite direction and, than it was actually intended for. I’m, I don’t understand how that can happen.
I think the most [inaudible 00:13:24] explanations is that they haven’t read the paper. That is my guess. It is quite clear on how I feel about the learning pyramid. So I don’t think it’s ambiguous in any way-
David (13:37):[crosstalk 00:00:13:38].
But, so that’s the first part of explanation, perhaps they haven’t read it. And most interesting, more interesting, is perhaps asking, so why didn’t they read it? And there’s several possible explanations for that. And one thing is that the learning pyramids, and several of the myths that I have looked at, are kind of intuitive. They kind of sound like they got, they’re truthy. They kind of agree with our experience, our informed experience. We can easily think of several examples of how this is correct.
Like the idea of the learning pyramid, that you learn more efficiently by teaching others.
Of course, when I stand in teaching life, for instance, The Cave Metaphor of Plato, and suddenly I have a realization that, okay, this is probably what he meant, as I stand there, I get this epiphany. And when I look to, look back at, how I have worked with Plato, how I have studied Plato, and how I taught him, this epiphany stands out more clear to me in my memory than all the tedious work, [inaudible 00:14:55] this kind, this, or the English is bad, Norwegian or English translations, or trying to get grips with the Greek, or-
Yeah, all the language you’ve been to-
Yeah. Or all the lectured I’ve followed myself, all these, archived, they’re way back, they aren’t memorable. But this epiphany…so it easily lends itself to this narrative that, okay this is how it works.
So, and this is one thing. And also the learning pyramid is very malleable. It’s, it has, it is plastic. It is, kind of, we can shape it any way you like. So if your preference is doing digital teaching, if you’re into MOOCs, if you’re into using iPads, there’s always room to put in these things somewhere high up the hierarchy in the learning from it.
If you think that practicing, if you think that discussion groups are essential, this also can be stressed in the model they, I remember we joked about.
I don’t remember. I don’t, I’m not sure how to say this in English, but in Sunday Schools in olden days, they had this kind of flannel graph, it’s fabrics, it’s figures that you put on a flat surface.
To tell biblical stories. I mean, you joke, okay, these people are using this model to argue for the efficiency of specific, method, technologies. We could even find an example flannel graph, and if I actually did that, they had found, they put the flannel graph, into the version of the learning pyramid.
So, whatever idea you want to present, you can use the learning pyramid to do that.
So it’s kind of a confirmation bias, and the learning pyramid’s ambiguous enough, to allow for that to occur.
It is, it is. And several of, yeah. It kind of corresponds also to ideas about activating students. Of course, these are healthy ideas, as, but they are quite diffuse, quite vague, and they speak to all these ideas about how we ought to teach in some way.
And okay, so that’s, those kind of explanations go without the particularities of the learning pyramid, and similar myths. And there’s also other reasons like, you don’t have time, you’re stressed, you need to publish things. Okay. And also, there can be minor breaches like, I think I have read this paper, I do believe I have read this paper, but I actually didn’t. And my [inaudible 00:17:55] base has 3,000 papers, and it’s very easily, to imagine that I have read it, and I believe, I know what’s in that paper, so I don’t have to read it again, but actually I’ve just put it in my note, and I haven’t actually read it. So, that is very easily done, actually.
Yes, it is. Yeah. Fallibility of memory, definitely-
So, you used to make, you make about hundreds of claims in, we should, probably, several papers. You make hundreds of claims, and make hundreds of references, hundreds of citations, and okay, errors are bound to happen. And when these small areas accumulate, you can get this, this phenomenal, these scientific myths.
Yes. And it’s largely because the information is coming at us from lots of different places.
And, if we don’t engage, and it’s what you’re talking about and what you’re teaching, if you don’t engage in critical thinking and start going, hang on a minute, so a video of the lecture is more impactful than the lecture, and it has greater retention. How can that be? When you start to think like that, then it starts to unravel.
Yeah. Definitely. And that’s based on a whole series of other knowledge constructs and things like that.
Yeah, that’s really interesting. One of the problems with all of this is that, it can erode people’s faith in science, when people start to realize that these things actually aren’t based on anything. And I did a thing on LinkedIn yesterday about these neuromyths that pervade a lot of people’s thinking, around having spherical dollar dominance, this idea of a right and left brain, and-
Being more logical and more creative, it’s just not true. But it’s spawned a whole industry, just about, and it catches people’s imagination, I think.
So are there any other findings from your research, about why these things end up being disseminated so widely?
Well, I have, and there’s a paper in my thesis that is not yet published on it, I need to work on it a bit further. But I identify minor epistemic breaches, minor epistemic flaws, like they sometimes should read critical critics. They refer to a critic, but they sometimes don’t engage with it. They can’t just…but this idea is controversial, and they leave it there. They don’t engage with the debate itself.
So they just go through with the motions, like ah, here’s the learning pyramid, it says so-and-so, it is controversial, but it seems to accord with how we think, how we…and it’s, and also, it is very commonly used. So they kind of just take it up, lift it up, shelf it and put it aside as, as an issue. And there’s several from, several problems that are citations. Yeah. This basically is, I tried to make a list of this small, small breaches of acceptable research practices.
Mm-hmm (affirmative). And they all act as kind of cognitive nudges. And, and that again, I think what you’re talking about, about the lack of engagement, is a lack of critical thinking. We, it’s…and it’s that, because it takes effort to critically think-
And it’s quite easy, also to consider, okay, this model is widely published. So it is authoritative. Who am I to question these, all these researchers, there are hundreds researchers publishing [inaudible 00:21:51]?
How can they be wrong?
So, maybe it’s also a lack of confidence to say, hang on a minute!
Yeah. So I’ve got a voice in this. Yes. And I think a lack of confidence, particularly with people, if you see professors, and esteemed academics publishing, it can be quite intimidating, particularly if you’re just starting out a PhD student, or somebody who’s not even an academic and they read it, it’s got to be true.
And that, I think that gets going, that, there was an interesting, and I think this also has something to do with this, that there was a paper that I was reading late last year, that was looking at the lag between research being published, and it actually getting into practice. So, the original paper I was looking at, was actually looking at the lagging clinical practice in medicine. And they say on average, on average, it takes about 17 years for things in research to end up in practice in hospitals, medicine, things like that.
And there was another one published about MBAs, showing that it takes about 13 years, from publication to ideas from the publication and ending up on MBA programs. So there’s this lagging. And what’s interesting is what happens in that lag, how all of these, how, what is it that creates the conditions where those pieces of research are published? Cause there’s an enormous amount of publication going on.
I estimate roughly about 105,000 papers published every month, new papers every month. Which is, yeah I know, it’s incredible, and what’s happening in that time period, and what’s creating the conditions, where individuals become conscious of a stream of thinking, a stream of research, in order to then bring it into practice.
And I think that’s part of the whole idea of little nudges, little, just little things, that are accumulating over time. And I think that’s one of the reasons why we get this huge lag, because some of the research actually takes quite a mind shift. You’ve really got to go, hang on a minute! I’ve now got to reconsider everything here, rather than it just fitting it in, in a stream of thinking. And I think that’s, that could be part of the story as well, which is interesting.
Kare (24:30):[crosstalk 00:24:31].
Yeah, go on.
There’s probably some also institutional hinders to this because they have to find the way into textbooks. They have to find a way into curricula. They have to buy into it.
Yeah. So, and by the time the students get to their workplaces, there are cultures, established cultures, that they will have to fight. So I guess I’m not surprised that 17 years is, is the time.
As described, yeah.
Definitely. Yeah. Yeah, and I’m not either. What I’m particularly interested in is what goes on in people’s heads over that period of time, to make it acceptable, or to make it important, or whatever happens to be.
So, if you were to pull out a takeaway or a couple of takeaways for practitioners from this, who will be listening to this, so people in organizations and things like that, about scientific research, what would they be?
I guess, be aware of simplistic answers to complex questions, and sweeping generalizations of the human behavior. Because if you find an idea intuitive, needs and comes very easily to mind, chances are, they have also come very easily to the mind of the researchers. So if, maybe that is, that’s the kind of a general rule of thumb I can give, because the myths I have been looking at have been looked at, all this very general vague descriptions of human behavior, without consideration of content, context, of social context, or with time of a person, personalities, culture, or anything. This kind of a one-size-fits-all, into the clear idea about how people work, how they function, how we learn, how we behave, how we respond.
So yeah, these kind of ideas.
Yes, I get it-
I guess, that would be the most relevant, for practitioners at least.
Yeah. I think that’s really good advice. Definitely. Definitely.
Okay. Thank you very much, Kare. I really appreciate your time, and the work that you’re doing. I think it’s really important work.
Is there some way that if people want to follow you, follow your research, or contact you, that they can?
Yes. They can contact me via email and-
On the website, yeah.
Yeah. And I’m not, I don’t have a huge presence in, on the internet, but I have a ResearchGate page. So several, some of my papers are available there, and if they contact me via ResearchGate, I can send them copies, or via, by email, of course.
Yeah, we’ll put your link to the ResearchGate profile for you.
That’s brilliant. Thank you very much, Kare. I really appreciate your time.
Thank you for having me.
Really nice making contact with you at last! Right?
Take care, and good luck for, well, for the new term-
Thank you for that-
The teaching, and with all of the issues that we’re facing with coronavirus. And hopefully we’ll talk again some other time.
Yeah. Thank you.
You take care.
Be impressively well informed
Get the very latest research intelligence briefings, video research briefings, infographics and more sent direct to you as they are published
Be the most impressively well-informed and up-to-date person around...