Discussion

Note

This section will be updated with each new release of the report.

Current version: Studies 2016 - 2020

Summary of Results

The meta-analytic assessment of all micro-PK experiments conducted at the LMU Micro-PK lab currently encompasses 14 studies with a total of 23 experimental conditions and 13 control conditions, including data from 55668 participants and reveals several key insights. Although individual studies included in this report often tested specific hypotheses, the meta-analysis focused on a general micro-PK effect to ensure comparability. Descriptions of the hypotheses for each study can be found in the respective experiments section.

The overall effect size for micro-PK was found to be non-significant (0.017, p = 0.155) across experimental conditions, with moderate heterogeneity (45.34%). Similarly, control conditions showed no significant effect (-0.005, p = 0.646). These results suggest that the experimental manipulations may not be robust enough to produce a consistently meaningful effect. The moderate heterogeneity in both analyses indicates variability due to study design or implementation differences, though the lack of significant heterogeneity in control conditions suggests that variability in experimental conditions is more likely due to differences in interventions.

Change of Evidence Measures

Since Psi-effects may be influenced by the accumulation of evidence over time, new measures have been devised to analyze how this evidence changes. By comparing these results to simulations, empirical p-values can be calculated. A meta-analytic assessment of these change-of-evidence measures was conducted using the inverse chi-square invchisq-function to combine p-values across studies. Three characteristics were analyzed: Maximum Bayes Factor (BF), BF Energy, and the amplitude sum of the BF curve’s Fast Fourier Transform (FFT).

For all experimental conditions, the combined chi-square statistics were calculated with 35571 degrees of freedom, yielding the following results: 36155.12 for the Maximum BF (p = 0.015), 36096.14 for the BF Energy (p = 0.025) and 36174.83 for the FFT amplitude sum (p = 0.012).

The analysis of change of evidence measures demonstrated statistically significant results for experimental conditions, suggesting that observed evidence changes are unlikely due to random variation. This could be the result of a volatile micro-PK effect. The tests are robust to using methods that do or do not account for varying sample sizes, thus addressing potential issues of overweighting studies that show a noticeable BF early on and whose data collection is extended.

In contrast, none of the three characteristics demonstrated statistically significant results for the control conditions. However, these significant results in experimental conditions do not necessarily translate into a substantial overall effect, as no significant effect was found in the aggregate meta-analysis of micro-PK interventions. They do, however, indicate that the observed changes in evidence are not likely due to random variation.

Addressing Publication Bias

A potential concern in meta-analyses is publication bias, where significant results are more likely to be published compared to non-significant findings. In our case, the comprehensive inclusion of both published and unpublished studies from our research group effectively mitigates this concern. By incorporating all relevant studies, our analysis is less susceptible to selective reporting, enhancing the reliability of our findings.

Implications of Findings

The significant change of evidence measures in experimental conditions suggest dynamic factors influencing evidence accumulation. According to the Model of Pragmatic Information, novel, autonomous effects may decrease in detectability over time. This model may explain the observed volatility and highlights the potential of change of evidence measures for understanding temporal dynamics in micro-PK effects.

Despite these insights, the non-significant overall effect size indicates that micro-PK interventions, as tested, may not consistently produce a robust effect. Future research should focus on refining experimental designs, replicating studies, and exploring the underlying mechanisms of micro-PK effects to establish more consistent evidence.

Conclusion

While our meta-analysis at this point in time did not demonstrate a statistically significant effect of micro-PK interventions, the significant change of evidence measures in experimental conditions indicate that the phenomena observed are not purely due to chance. Future research should focus on refining experimental designs, conducting replication studies, and maintaining transparent reporting practices to further explore and validate micro-PK effects. Addressing these aspects will help in understanding the underlying mechanisms and improving the robustness of evidence in this field.

We would like to additionally encourage the scientific community to follow this example of transparent reporting and data sharing to enhance the reproducibility and reliability of research findings in the field of parapsychology and related disciplines and to counteract publication bias and selective reporting practices.