Why you shouldn’t just believe research published in research journals

Why you shouldn’t just believe research papers just because it is published in an academic research journal. Take a new study citing ‘evidence’ for the learning pyramid for example…

How does this statement sound?

“Teaching a subject is better, in terms of retention or memory than just reading about the subject.”

Sounds plausible right? You may even have heard this and believe it.

Surely the learning pyramid is right isn’t it?

How does the learning pyramid look to you?

It’s common sense, right?

Only it isn’t. There is no evidence that this is correct, indeed there is good evidence that it isn’t that simple and that reading can lead to every bit as much retention as teaching it does.

A new study with ‘evidence’ to support the learning pyramid

Last week I came across a paper published in the International Journal of Management Education which looked interesting.

One of the diagrams within the paper was this, which talks about supposed previous research about average student retention rates increasing dependent on the learning activity the student was engaging in.

‘Quoting’ research

The paper quotes a study by Letrud (2012) by stating “The complementary nexus between classroom-based management education and coaching is, however, most strongly reinforced by studies conducted by the National Training Laboratories (NTL) in Bethel, Maine, US in the late 1980’s. The NTL studies empirically revealed the notion of a learning pyramid, in that learning is retained at graduated stages of learning interventions (Letrud, 2012)”

In essence, the study is saying that Letrud confirms a study conducted by the NTL in the late 1980s.

Really?

I was intrigued as this was not my understanding of the current state of research around average student retention rates due to activity. In fact my understanding was that there was little evidence to support the “learning pyramid”.

Just a glimpse at the title of the paper in the references suggested that something was not quite right…

What Letrud really said

On obtaining KÅRE Letrud’s paper clearly titled ‘A rebuttal of NTL Institute learning pyramid‘, not as the paper suggests an “empirically revealed” notion of the learning pyramid. Infact you can see this in the original papers references.

As the title of the paper would lead you to expect, Letrud is saying the exact opposite to what this paper is claiming: the learning pyramid has no valid evidential support and should be withdrawn. Letrud’s conclusion is pretty clear and leaves it in pretty unambiguous language what her conclusions are about the Learning Pyramid:

“Conclusion

NTL Institute’s learning pyramid lacks empirical evidence, and any attempt to perform empirical tests of the model will encounter major methodical problems. Despite its inability to stand up to close examination, the model is still corroborated by NTL Institute. The continued distribution of this model cannot be justified, and in order to reduce further dissemination, NTL Institute ought to retract it.
Presentations of other versions of the learning pyramid also need to be confronted with correspondingly critical questions in order to clarify the scientific statuses of these models. If these cannot be adequately satisfied, one should abstain from using them all together.”

How can this have happened?

I think that’s pretty clear. This is not support for the learning pyramid, which, together with other popular theories such as learning styles, hemispheric dominance and Dale’s experience cone are understood in evidence-based education circles as being either:

• Without evidence – no research was actually done and the theory was made up (usually based on intuition),
• That the ‘research’ conducted is of such poor quality (based on a small sample survey of people in the same class or workplace for example ) or
• There is good evidence that the theory in question is not true but this is an inconvenient truth so we will act as if it is and ignore the evidence or we are just too lazy to actually read the papers we are quoting or do the basic research.

Common belief of myths

A recent 2018 study found that approximately 74% of learning design professionals believe these spurious and use them as a basis of their learning materials design.

Other recent studies have found that more than 85% of pre-and in-service teachers in the UK and Netherlands believed in a range of learning maths including learning styles and hemispheric dominance as well as the NTL’s Learning pyramid.

It is difficult to see how papers with such blatant misinformation can get through the editorial and or peer review process but they do.

The upshot of all of this is quite simply to check your sources. Any researchers worth their salt will double check any statements and sources quoted even in studies that have been published in respectable peer-reviewed journals.

Even non-researchers can perform the simplest of Internet searches on a search engine like Google, Bing etc. or even Google scholar which should rapidly ascertain the evidence for claims for such theories.

What do you think?

My question is, when this kind of blind acceptance of myths in what is supposed to be ‘research’ surfaces what are your thoughts and feelings?

References

Reid, A., Cook, J., Viedge, C., & Scheepers, C. B. (2020). Developing management effectiveness: The nexus between teaching and coaching. The International Journal of Management Education, 18(1), 100334.

Letrud, K. (2012). A Rebuttal Of NTL Instituteʼs Learning Pyramid. Education, 133(1). https://eric.ed.gov/?id=EJ996977

Sloan-Lynch, J. K. (2018). Designing with learning myths in mind. https://psyarxiv.com/na8u2/

Tardif, E., Doudin, P. A., & Meylan, N. (2015). Neuromyths among teachers and student teachers. ​Mind, Brain, and Education ​, ​9​(1), 50-59. https://doi.org/10.1111/mbe.12070