In 2015, for instance, the first round of attempts by the “Reproducibility Project” to redo 100 prominent studies got the same results as the original for only one-third. That doesn’t mean what the original researchers reported (that, for example, students learn more effectively if they’re taught in the “learning style” that matches theirs) didn’t really happen. It could simply be that what was true for the participants isn’t true of many, or even most, other people. The replication crisis made me look back over my columns for Mindful to see if I’ve misled you, however inadvertently. So far, I’ve been lucky (and I emphasize lucky: I don’t claim any superior ability to sniff out problematic findings): I was glad to see that I warned against believing the wilder claims about mirror neurons (my June 2014 column), about biophilia (August 2015), and about sex differences in the brain (February 2016). But I wouldn’t be surprised if some of the results I described in neuroeconomics (April 2015) and generosity (August 2016) don’t hold up as well. Scores of claims that have gotten extensive media coverage, and even made their way into textbooks, are questionable. I’ve chosen ones that offer some general lessons for consumers of psychology research:

  1. Those learning styles: Although the majority of studies disprove the popular idea that students learn better if the pedagogic technique matches their supposed style, the myth persists. That may be because when people try to learn something according to what they believe to be their learning style, they feel they have learned the material better—but haven’t, found a 2016 study in the British Journal of Psychology led by psychologist Roger Van Horn of Central Michigan University. (Yes, I know every time I cite research I could be on thin ice. I try to include only findings with support from multiple, independent studies.) But the most effective pedagogic technique varies according to the type of material, not the student. Nobel-winning social psychologist and economist Daniel Kahneman asked, “The question I have is: If your effect is so fragile that it can only be reproduced [under strictly controlled conditions], then why do you think it can be reproduced by schoolteachers?” 2) The power stance: Stand with your feet apart and your hands on your hips, or sit with your legs on a desk. Such a “power pose,” researchers reported in 2010 in Psychological Science, made their 42 volunteers feel bolder, elevated their testosterone levels, decreased their levels of the stress hormone cortisol, and increased their tolerance for risk, as shown by their willingness to make risky bets. The TED talk version is that the power stance can change your life. Alas, when other scientists redid the study in 2015, with five times the participants, they found no such effect. And although the original scientists protested that 33 other studies found a power-pose effect, an objective analysis of those 33 found something quite different: The statistics in those 33 are such that they can equally support the conclusion that the power stance has no effect, and hint that researchers deep-sixed power stance studies that did not find an effect. They called the evidence “too weak to advocate for people to engage in power posing to better their lives.” Lesson: If a claim is based on results from only a few dozen people, take it with a grain of salt, and keep the shaker nearby until a larger study replicates it. 3) Smiling makes you happy: This one has been around since at least 1988, when a study reported that holding a pen between the teeth to force a smile (try it) caused people to find cartoons funnier than when they held a pen between their lips. Unfortunately, when 17 independent labs ran the make-me-smile test with just under 2,000 volunteers, they found no effect of mouth position on how funny people found cartoons. This doesn’t mean no one feels happier if something forces him to smile; maybe if you force yourself to smile, without the annoying pencil, you feel a little happier. But the replication failure does mean the effect, if any, is too weak to appear reliably in large numbers of people. Lesson: If a psychological effect that is taken as applying to humans as a species applies only to some of us in some circumstances, it’s not a legitimate human universal like confirmation bias and loss aversion. 4) Finite willpower: This is considered “one of the most influential psychological theories of modern times,” as the British Psychological Society put it. The idea is that if you draw on your limited store of willpower to, say, resist the dessert cart at lunch, you have less to use when you walk past a store advertising exactly the shoes you’ve long admired. Dozens of studies have found such an effect, which is also called “ego depletion,” so it would seem to be robust. Yet 23 labs studying nearly 2,000 participants found that “draining” self-control in one task had “close to zero” effect on people’s capacity for self-control in a subsequent task. A separate analysis of 116 studies, in Journal of Experimental Psychology, similarly came away unimpressed. Lesson: If there’s an effect at all it’s small, it doesn’t apply to everyone, and could even be opposite the one usually claimed. That is, exerting self-control in one situation made some people better at it in the next one. 5) The Lady Macbeth Effect, in which people exposed to, or made to engage in, unethical behavior are driven to wash their hands or otherwise clean themselves, as researchers reported in 2006 in Science. Strictly speaking, the claim was based on a lab study in which people copied, by hand, an account of sabotaging someone and then found products like soap and toothpaste more desirable than if they had copied a story about helping someone. Later studies found that people felt guilty after washing. Again, when other scientists redid the original study they found no such effect. Maybe some people do have a Lady Macbeth thing going on, while others didn’t. The more important lesson here is the need to be cautious in extrapolating an artificial lab setup (copying a story, not actually engaging in unethical behavior; rating soap and toothpaste, not actually scouring yourself) to real life. 6) Big Brother watching: A poster of watchful eyes caused people, on the honor system, to chip in more for coffee than when the walls were bare. This 48-person 2006 study made headlines and influenced public policy, with some British police departments putting up posters of staring eyes in an effort to keep people honest. But in 2011, a redo with 138 people failed to find a pro-social effect in people being “watched” by the eyes of a poster. 7) Wear red to attract a mate: Several studies have reported that men rate women wearing red as sexier and more attractive than women wearing other colors, something that scientists have spun into a “Just So Story” about how our primate ancestors advertised their sexual availability. But in a 2016 paper in Evolutionary Psychology scientists described three experiments with 800 young men (vs. two dozen in the original study) finding no such effect. Lesson: Even if there is a weak red effect, it’s a relatively unimportant influence on how we judge potential partners—certainly long-term ones, but even one-night stands. It’s easy to become cynical about psychology, or at least the exciting results that the media pick up. The general point is not that the original, dubious claims are wrong. They might be—heck, they probably are—true for some people. Some of us likely do feel bolder in a power stance. Maybe believing that it can transform your life in a good way produces changes for the good that bring that about. For if there is one psychological effect that has stood the test of time, and countless replications, it is the placebo effect: that believing in the power of something can make it so. At least for some people, a little or a lot, in some circumstances some of the time.

Research: Not Myths

While many findings in psych studies have turned out to be mythical, a number of cognitive biases—mental shortcuts we use to make quick decisions—have been amply demonstrated. With confirmation bias, we seek data to support what we already believe. Loss aversion points to putting more effort into avoiding losses than making gains. They’re discussed in a popular new book by Michael Lewis: The Undoing Project, which is about two Israeli psychologists whose research on bias broke new ground.