Discussion

Note

This section will be updated with each new release of the report.

Current version: Studies 2016 - 2025 (v3)

Summary of Results

The meta-analytic assessment of all micro-PK experiments conducted at the LMU Micro-PK lab currently encompasses 27 studies with a total of 34 experimental conditions and 19 control conditions, including data from 45787 individual participants and reveals several key insights. Although individual studies included in this report often tested specific hypotheses, the meta-analysis focused on a general micro-PK effect to ensure comparability. Descriptions of the hypotheses for each study can be found in the respective experiments section.

The overall effect size for micro-PK was found to be marginally significant (0.016, p = 0.072) across experimental conditions, with moderate heterogeneity (36.71%). Control conditions showed no significant effect (-0.008, p = 0.308) with low heterogeneity (23.1%). These results suggest that while the experimental manipulations may produce a weak effect, it is not robust enough to reach conventional significance levels. The moderate heterogeneity in experimental conditions compared to the low heterogeneity in control conditions indicates that variability is more likely due to differences in interventions rather than methodological factors.

Change of Evidence Measures

Since Psi-effects may be influenced by the accumulation of evidence over time, change of evidence measures compare how evidence develops during data collection with Monte Carlo simulations of the same studies under the null hypothesis. Three characteristics of each Bayes factor trajectory are assessed (its maximum, its energy and the amplitude sum of its Fourier transform) and combined per condition into an aggregate CoE p-value, the harmonic mean of the three. The aggregate p-values were combined across conditions with the inverse chi-square method, which weights conditions by sample size, and with Fisher’s method, which weights them equally. The first two versions of this report tested the three measures separately, and the aggregate replaces these three strongly correlated tests with one.

In experimental conditions, 7 of 33 conditions (21%) showed an aggregate CoE p-value below .05, compared with the 5% expected by chance. Combined, the aggregate p-values deviated from the null distribution in the weighted analysis (χ²(41493) = 42238.1, p = .005) as well as in the unweighted analysis (χ²(66) = 119.6, p < .001). Control conditions showed no such deviation. Here 1 of 19 conditions fell below .05, and neither the weighted (p = .888) nor the unweighted analysis (p = .552) differed from chance.

The direct comparison of the two condition types is the critical test of the volatile effect hypothesis, because artefacts of the analysis would appear alike in both condition types, which pass through identical procedures. Aggregate CoE p-values were smaller in experimental than in control conditions (Mann-Whitney U, W = 170, p = .003; permutation test on the difference in Fisher chi-square statistics with 10,000 permutations, p = .007). The combination methods treat all conditions as independent, although some reuse the same participants, such as the two time points of the Monks study and the three median splits of the Psychoscanner sample. When only one condition per participant sample is kept, the experimental conditions still deviate from chance (weighted p = .010, unweighted p < .001).

Among the individual measures, which the CoE meta-analysis reports as an exploratory breakdown, the weighted deviation of the experimental conditions is carried by BF Energy (p = .011) and FFT amplitude sum (p = .042), while the Maximum BF shows none (p = .724). The assignment of \(p_{MaxBF} = 1\) to conditions whose evidence never favoured H1 (MaxBF = 1) may limit that measure.

Evidence dynamics thus deviate systematically from chance in experimental but not in control conditions, while the overall effect size is only marginally significant. This pattern is consistent with a volatile micro-PK effect that shows less as a stable directional shift than as altered accumulation of evidence over time. The results hold for methods that do and do not account for sample size, which addresses the concern of overweighting studies whose data collection was extended after an early noticeable BF.

Addressing Publication Bias

A potential concern in meta-analyses is publication bias, where significant results are more likely to be published compared to non-significant findings. In our case, the comprehensive inclusion of both published and unpublished studies from our research group effectively mitigates this concern. By incorporating all relevant studies, our analysis is less susceptible to selective reporting, enhancing the reliability of our findings.

Implications of Findings

The aggregate change of evidence measure, which deviates from chance in experimental but not in control conditions, suggests dynamic factors influencing evidence accumulation patterns. This goes hand in hand with theories about decline effects in (parapsychologcial) science. E.g., the Model of Pragmatic Information suggests that novel, autonomous effects may decrease in detectability over time. It may explain the observed volatility and highlights the potential of change of evidence measures for understanding temporal dynamics in micro-PK effects.

Weak and volatile effects are often observed in parapsychological research, exemplified by failed direct replications, a phenomenon known as the decline effect. This pattern is not unique to Psi research but has been documented in other scientific areas as well. The change of evidence measures presented here might offer a fruitful approach to identify decline patterns within individual studies and differentiate between false positives combined with regression to the mean versus genuine effect-plus-decline patterns.

The marginally significant overall effect size (p = 0.072) suggests that micro-PK interventions may produce a weak but detectable effect, though not at conventional significance levels. This finding, combined with the significant change of evidence measures, indicates that while micro-PK effects may be present, they are subtle and may manifest primarily through alterations in evidence accumulation dynamics rather than consistent directional effects. Future research should focus on refining experimental designs, replicating studies, and exploring the underlying mechanisms of micro-PK effects to establish more robust evidence.

Conclusion

While our meta-analysis demonstrated only a marginally significant effect of micro-PK interventions (p = .072), the aggregate change of evidence measure deviated from chance in experimental conditions and differed from control conditions, indicating that the phenomena observed are not purely due to chance. These findings suggest that micro-PK effects, while subtle, may influence the temporal dynamics of evidence accumulation in ways that traditional effect size measures might not fully capture.

Future research should focus on refining experimental designs, conducting replication studies, and maintaining transparent reporting practices to further explore and validate micro-PK effects. Particular attention should be given to understanding the mechanisms underlying the observed changes in evidence accumulation patterns. Addressing these aspects will help in understanding the underlying mechanisms and improving the robustness of evidence in this field.

We would like to additionally encourage the scientific community to follow this example of transparent reporting and data sharing to enhance the reproducibility and reliability of research findings in the field of parapsychology and related disciplines and to counteract publication bias and selective reporting practices.