ICPS 2015

Symposium

Testing Psychological Theories: False Positives, False Negatives, and Other Issues

Saturday, 14 March 2015, 17:00 - 18:20
Ontvangkamer

Subject Area: Methodology

Chair: Jelte M. Wicherts
Tilburg University, The Netherlands
Chair: Robbie C.M. van Aert
Tilburg University, The Netherlands

This symposium concerns problems in the use of Null Hypothesis Significance Testing (NHST) that may affect the psychological literature, namely: errors in the reporting of statistical results, false negatives, incorrect intuitions about power, and the effects of questionable research practices on meta-analytic results.

The Prevalence of Reporting Errors in Psychology: An Automated Approach
Michèle B. Nuijten
Tilburg University, The Netherlands
We evaluate reporting errors in over 250,000 p-values from 30,717 articles from journals in most sub disciplines of psychology, using the new automated procedure “statcheck”. Half of the articles contain at least one erroneous p-value. We discuss differences between journals and trends over time.

Co-Author: Chris H.J. Hartgerink, Tilburg University

Co-Author: Marcel A.L.M. van Assen, Tilburg University

Co-Author: Sacha Epskamp, University of Amsterdam

Co-Author: Jelte M. Wicherts, Tilburg University


Too Good to be False: Nonsignificant Results Revisited
Chris H.J. Hartgerink
Tilburg University, The Netherlands
The failure to detect genuine effects (false negative findings) may impede scientific progress. We obtain evidence for false negatives in 66.7% of 10,425 papers reporting nonsignificant results in the psychological literature (1985-2013). These results highlight the importance of considering statistical power and false negatives in psychological science.

Co-Author: Marcel A.L.M. van Assen, Tilburg University

Co-Author: Michèle B. Nuijten, Tilburg University

Co-Author: Jelte M. Wicherts


Flawed Intuitions About Power in Psychological Research
Marjan Bakker
University of Amsterdam, The Netherlands
We surveyed 291 psychological researchers concerning their power intuitions and found large discrepancies between the preferred amount of power and the power calculated based on respondent’s typical sample size, effect size, and alpha level. Furthermore, we found differences between responses from a researcher’s and a reviewer’s perspective.


Harmful effects of questionable research practices and publication bias on results of meta-analysis
Robbie C.M. van Aert
Tilburg University, The Netherlands
Two substantial problems in psychological research are questionable research practices (QRPs) and publication bias. We studied the impact of QRPs and publication bias on the performance of techniques for assessing publication bias. These techniques are affected by QRPs meaning that they can hardly distinguish between publication bias and QRPs.

Co-Author: Esther Maassen, Tilburg University

Co-Author: Marcel A.L.M. van Assen, Tilburg University


Email Bookmark and Share