Addressing the challenge of p-value and sample size when the significance is borderline: the test of random duplication of participants as a new approach

Abstract

The issue of borderline p-value seems to divide health scientists into two schools of thought. One school of thought argues that when the p-value is greater than or equal to the statistical significance cut-off level of 0.05, it should not be considered statistically significant and the null hypothesis should be accepted no matter how close the p-value is to the 0.05. The other school of thought believes that by doing so one might be committing a Type 2 error and possibly missing valuable information. In this paper, we discuss an approach to address this issue and suggest the test of random duplication of participants as a way to interpret study outcomes when the statistical significance is borderline. This discussion shows the irrefutability of the concept of borderline statistical significance, however, it is important that one demonstrates whether a borderline statistical significance is truly borderline or not. Since the absence of statistical significance is not necessarily evidence of absence of effect, one needs to double check if a borderline statistical significance is indeed borderline or not. The p-value should not be looked at as a rule of thumb for accepting or rejecting the null hypothesis but rather as a guide for further action or analysis that leads to correct conclusions.