Discussion

Note

This section will be updated with each new release of the report.

Current version: Studies 2016 - 2025 (v2)

Summary of Results

The meta-analytic assessment of all micro-PK experiments conducted at the LMU Micro-PK lab currently encompasses 27 studies with a total of 34 experimental conditions and 19 control conditions, including data from 45787 individual participants and reveals several key insights. Although individual studies included in this report often tested specific hypotheses, the meta-analysis focused on a general micro-PK effect to ensure comparability. Descriptions of the hypotheses for each study can be found in the respective experiments section.

The overall effect size for micro-PK was found to be marginally significant (0.016, p = 0.072) across experimental conditions, with moderate heterogeneity (36.71%). Control conditions showed no significant effect (-0.008, p = 0.308) with low heterogeneity (23.1%). These results suggest that while the experimental manipulations may produce a weak effect, it is not robust enough to reach conventional significance levels. The moderate heterogeneity in experimental conditions compared to the low heterogeneity in control conditions indicates that variability is more likely due to differences in interventions rather than methodological factors.

Change of Evidence Measures

Since Psi-effects may be influenced by the accumulation of evidence over time, new measures have been devised to analyze how this evidence changes. By comparing these results to simulations, empirical p-values can be calculated. A meta-analytic assessment of these change-of-evidence measures was conducted using the inverse chi-square invchisq-function to combine p-values across studies. Three characteristics were analyzed: Maximum Bayes Factor (BF), BF Energy, and the amplitude sum of the BF curve’s Fast Fourier Transform (FFT).

For all experimental conditions, the combined chi-square statistics were calculated with 41493 degrees of freedom, yielding the following results: 41336.33 for the Maximum BF (p = 0.706), 42142.89 for the BF Energy (p = 0.012) and 42011.57 for the FFT amplitude sum (p = 0.036).

The analysis of change of evidence measures revealed mixed results for experimental conditions. While the Maximum BF showed no significant deviation from chance expectations (p = 0.706), both BF Energy (p = 0.012) and FFT amplitude sum (p = 0.036) demonstrated statistically significant results. This pattern suggests that while the peak evidence accumulation may not differ substantially from simulated data, the overall energy and frequency characteristics of evidence changes show systematic deviations from random variation. The assignment of \(p_{MaxBF} = 1\) to studies in the direction of H0 (MaxBF = 1) might be responsible for the difference and suggests that the MaxBF criterion could be limited.

Nonetheless, the significant findings for BF Energy and FFT amplitude sum indicate the presence of a subtle but detectable pattern in how evidence accumulates over time in experimental conditions, potentially reflecting a volatile micro-PK effect that manifests not as consistent directional changes but as systematic alterations in the dynamics of evidence accumulation. The tests are robust to using methods that do or do not account for varying sample sizes, thus addressing potential issues of overweighting studies that show a noticeable BF early on and whose data collection is extended.

In contrast, none of the three characteristics demonstrated statistically significant results for the control conditions. However, these significant results for two of the three measures in experimental conditions do not necessarily translate into a substantial overall effect, as only a marginally significant effect was found in the aggregate meta-analysis of micro-PK interventions. They do, however, indicate that the observed changes in evidence accumulation patterns are not likely due to random variation alone, suggesting that experimental manipulations may influence the temporal dynamics of evidence accumulation even when they do not produce detectable overall effects.

Addressing Publication Bias

A potential concern in meta-analyses is publication bias, where significant results are more likely to be published compared to non-significant findings. In our case, the comprehensive inclusion of both published and unpublished studies from our research group effectively mitigates this concern. By incorporating all relevant studies, our analysis is less susceptible to selective reporting, enhancing the reliability of our findings.

Implications of Findings

The significant change of evidence measures for BF Energy and FFT amplitude sum in experimental conditions suggest dynamic factors influencing evidence accumulation patterns. This goes hand in hand with theories about decline effects in (parapsychologcial) science. E.g., the Model of Pragmatic Information suggests that novel, autonomous effects may decrease in detectability over time. It may explain the observed volatility and highlights the potential of change of evidence measures for understanding temporal dynamics in micro-PK effects.

Weak and volatile effects are often observed in parapsychological research, exemplified by failed direct replications, a phenomenon known as the decline effect. This pattern is not unique to Psi research but has been documented in other scientific areas as well. The change of evidence measures presented here might offer a fruitful approach to identify decline patterns within individual studies and differentiate between false positives combined with regression to the mean versus genuine effect-plus-decline patterns.

The marginally significant overall effect size (p = 0.072) suggests that micro-PK interventions may produce a weak but detectable effect, though not at conventional significance levels. This finding, combined with the significant change of evidence measures, indicates that while micro-PK effects may be present, they are subtle and may manifest primarily through alterations in evidence accumulation dynamics rather than consistent directional effects. Future research should focus on refining experimental designs, replicating studies, and exploring the underlying mechanisms of micro-PK effects to establish more robust evidence.

Conclusion

While our meta-analysis demonstrated only a marginally significant effect of micro-PK interventions (p = 0.072), the significant change of evidence measures for BF Energy and FFT amplitude sum in experimental conditions indicate that the phenomena observed are not purely due to chance. These findings suggest that micro-PK effects, while subtle, may influence the temporal dynamics of evidence accumulation in ways that traditional effect size measures might not fully capture.

Future research should focus on refining experimental designs, conducting replication studies, and maintaining transparent reporting practices to further explore and validate micro-PK effects. Particular attention should be given to understanding the mechanisms underlying the observed changes in evidence accumulation patterns. Addressing these aspects will help in understanding the underlying mechanisms and improving the robustness of evidence in this field.

We would like to additionally encourage the scientific community to follow this example of transparent reporting and data sharing to enhance the reproducibility and reliability of research findings in the field of parapsychology and related disciplines and to counteract publication bias and selective reporting practices.