Meta-Analysis

Last update: 04 May 2020

Frequentist Meta-Analysis

In this analysis all Micro-Pk studies of the LMU Micro-Pk Lab are included that hypothesize the presence of an effect. In studies comparing experimental to control conditions, only the experimental conditions are included. The effect size for “Smokers 1” is inverted, since a two-tailed hypothesis was formulated and the resulting effect was negative. All other studies/conditions have directional hypotheses.

st <- readRDS("data/overview.rds")

# Select studies / conditions
data <- subset(st, st$Experimental == TRUE)

knitr::kable(data)
Study Experimental N Trials M SD Hits (%) t p ES Var BF Direction Year Labstudy
1 Loving Kind TRUE 171 100 49.725146 4.930148 49.72515 -0.7290206 0.7665039 -0.0557496 0.0058570 0.3034712 greater 2016 TRUE
2 Prayer TRUE 431 100 49.638051 4.805892 49.63805 -1.5635507 0.9406706 -0.0753136 0.0023268 0.1448423 greater 2017 FALSE
3 Monks T1 Exp TRUE 23 400 202.347826 6.859689 50.58696 1.6414415 0.0574628 0.3422642 0.0460249 1.8155253 greater 2016 TRUE
5 Monks T2 Exp TRUE 23 400 201.347826 8.020478 50.33696 0.8059304 0.2144544 0.1680481 0.0440922 1.0456588 greater 2016 TRUE
8 Meditation T2 TRUE 48 100 50.229167 4.539377 50.22917 0.3497646 0.3640389 0.0504842 0.0208599 0.7452738 greater 2016 FALSE
9 Sound preference TRUE 70 40 20.314286 2.931799 50.78571 0.8968906 0.1864480 0.1071989 0.0143678 1.2000567 greater 2017 TRUE
10 All-will-be-good TRUE 38 100 49.710526 5.098531 49.71053 -0.3499901 0.6358347 -0.0567759 0.0263582 0.4729100 greater 2016 TRUE
11 One-armed Bandit TRUE 40 200 101.125000 6.181807 50.56250 1.1509781 0.1283746 0.1819856 0.0254140 1.2718689 greater 2017 TRUE
12 Coin Toss TRUE 40 200 100.525000 7.709427 50.26250 0.4306924 0.3345315 0.0680985 0.0250580 0.8018915 greater 2017 TRUE
13 Incongruence Exp TRUE 236 10 5.182203 1.577994 51.82203 1.7738101 0.0386948 0.1154652 0.0042655 2.2417309 greater 2017 FALSE
15 Smokers 1 Exp TRUE 122 400 196.704918 9.874157 50.82377 -3.6859216 0.0003423 -0.3337077 0.0086531 66.0635985 different 2015 TRUE
17 Smokers 2 Exp TRUE 175 400 200.291429 10.375185 49.92714 0.3715825 0.6446721 0.0280890 0.0057165 0.0902946 less 2016 TRUE
19 Smokers 3 TRUE 203 400 199.137931 10.111786 50.21552 -1.2146808 0.1129529 -0.0852539 0.0049440 0.3958797 less 2017 TRUE
20 Psyscanner Style 1 Exp TRUE 1400 30 15.180000 2.678728 50.60000 2.5142470 0.0060201 0.0671961 0.0007159 10.4147205 greater 2018 FALSE
22 Psyscanner Style 2 Exp TRUE 1308 30 15.058104 2.707860 50.19368 0.7760390 0.2189332 0.0214575 0.0007647 0.4976420 greater 2018 FALSE
24 Psyscanner Style 3 Exp TRUE 1462 30 15.048564 2.735118 50.16188 0.6789043 0.2486530 0.0177556 0.0006841 0.4146918 greater 2018 FALSE
26 Relaxation TRUE 12571 100 50.017739 5.061680 50.01774 0.3929391 0.3471856 0.0035046 0.0000795 0.0993029 greater 2016 FALSE
27 Priming 1 Exp TRUE 4092 20 10.102395 2.265882 50.51197 2.8907394 0.0019318 0.0451899 0.0002446 19.8004286 greater 2018 FALSE
29 Priming 2 Exp TRUE 2063 20 9.961222 2.232693 49.80611 -0.7888809 0.7848638 -0.0173685 0.0004848 0.0884220 greater 2018 FALSE
31 Priming 3 Exp TRUE 6099 20 9.964092 2.314666 49.82046 -1.2115083 0.8871262 -0.0155130 0.0001640 0.0364067 greater 2019 FALSE
33 Priming 4 Exp TRUE 4060 20 9.995567 2.189693 49.97783 -0.1290108 0.5513223 -0.0020247 0.0002463 0.0840137 greater 2020 FALSE
35 Erotic Images 1 TRUE 241 200 100.510373 7.315573 50.25519 1.0830494 0.1399367 0.0697653 0.0041595 0.9734614 greater 2017 FALSE
36 Erotic Images 2 TRUE 678 50 25.128319 3.524935 50.25664 0.9478799 0.3435288 0.0364031 0.0014759 0.6359922 greater 2017 FALSE

Forest Plot

The forest plot visually represents the effect sizes and confidence intervals for each study.

#Frequentist m-a with metafor 
library(metafor)

# Specify parameters of Exp Studies
yi=data$ES
vi=data$Var
studies=data$Study
# invert ES of Smokers 1, since the hypothesis was two-sided and the results showed a negative effect (smokers are avoiding smoking-related images)
yi[which(studies == "Smokers 1 Exp")] <- -yi[which(studies == "Smokers 1 Exp")]

# Recommended method = REML: Langan et al. 2018 https://onlinelibrary.wiley.com/doi/10.1002/jrsm.1316
# knha = Hartung method to control ES non-normality Rubio-Aparicio et al. 2018

res <- rma(
  yi,
  vi, 
  method="REML",
  #mods = ~ data$Labstudy,
  knha=TRUE,
  slab=paste(studies)
  )

# forest plot
forest(res, 
       xlab="ES", mlab="RE", psize=1,
       ilab = data$N, ilab.lab = "N",
       header=T, shade=T)

Summary Statistics

These are the results of the meta-analysis. The model results reflect the overall estimated effect size, its confidence interval and whether it differs significantly from 0.

The Tau² represents the between-study variance, and the I² the heterogeneity. A significant test for Heterogeneity indicates a genuine variability of true effects across studies.

summary.rma(res)

Random-Effects Model (k = 23; tau^2 estimator: REML)

  logLik  deviance       AIC       BIC      AICc   
 23.2993  -46.5986  -42.5986  -40.4165  -41.9671   

tau^2 (estimated amount of total heterogeneity): 0.0006 (SE = 0.0005)
tau (square root of estimated tau^2 value):      0.0252
I^2 (total heterogeneity / total variability):   45.34%
H^2 (total variability / sampling variability):  1.83

Test for Heterogeneity:
Q(df = 22) = 43.4629, p-val = 0.0041

Model Results:

estimate      se    tval  df    pval    ci.lb   ci.ub    
  0.0170  0.0115  1.4729  22  0.1549  -0.0069  0.0409    

---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Funnel Plot

# funnel plot (publication bias)
funnel(res)

In this analysis all Micro-Pk studies of the LMU Micro-Pk Lab are included that hypothesize the absence of an effect. In studies comparing experimental to control conditions, only the control conditions are included.

# Select studies / conditions
data <- subset(st, st$Experimental == FALSE)

knitr::kable(data)
Study Experimental N Trials M SD Hits (%) t p ES Var BF Direction Year Labstudy
4 Monks T1 Con FALSE 29 400 198.344828 9.791672 49.58621 -0.9103017 0.8147832 -0.1690388 0.0349754 0.4860153 greater 2016 TRUE
6 Monks T2 Con FALSE 29 400 199.793103 9.648758 49.94828 -0.1154731 0.5455529 -0.0214428 0.0344907 0.6493717 greater 2016 TRUE
7 Meditation T1 FALSE 40 100 49.275000 5.764558 49.27500 -0.7954300 0.7844109 -0.1257685 0.0251977 0.4651069 greater 2016 FALSE
14 Incongruence Con FALSE 271 10 5.044280 1.538962 50.44280 0.4736623 0.6361235 0.0287729 0.0036916 0.4177974 different 2017 FALSE
16 Smokers 1 Con FALSE 132 400 200.530303 9.682312 49.86742 0.6292627 0.5302729 0.0547703 0.0075871 0.1630540 different 2015 TRUE
18 Smokers 2 Con FALSE 220 400 198.981818 10.341636 50.25455 -1.4603179 0.1456358 -0.0984546 0.0045675 0.2963604 different 2016 TRUE
21 Psyscanner Style 1 Con FALSE 1003 30 15.061815 2.691409 50.20605 0.7273804 0.4671629 0.0229674 0.0009973 0.4604948 different 2018 FALSE
23 Psyscanner Style 2 Con FALSE 1095 30 15.172603 2.740845 50.57534 2.0838687 0.0374037 0.0629743 0.0009151 3.2747936 different 2018 FALSE
25 Psyscanner Style 3 Con FALSE 941 30 14.882040 2.637678 49.60680 -1.3718490 0.1704378 -0.0447210 0.0010638 0.3519599 different 2018 FALSE
28 Priming 1 Con FALSE 4092 20 10.025660 2.260601 50.12830 0.7261019 0.2339089 0.0113509 0.0002444 0.2331401 greater 2018 FALSE
30 Priming 2 Con FALSE 2063 20 10.015511 2.234278 50.07756 0.3153284 0.3762721 0.0069425 0.0004847 0.2230455 greater 2018 FALSE
32 Priming 3 Con FALSE 6099 20 9.965076 2.362713 49.82538 -1.1543547 0.8758000 -0.0147812 0.0001640 0.0375426 greater 2019 FALSE
34 Priming 4 Con FALSE 4060 20 9.904926 2.255174 49.52463 -2.6862364 0.9963722 -0.0421581 0.0002465 0.0272478 greater 2020 FALSE

Forest Plot

The forest plot visually represents the effect sizes and confidence intervals for each study.

#Frequentist m-a with metafor 
library(metafor)

# Specify parameters of Exp Studies
yi=data$ES
vi=data$Var
studies=data$Study

# Recommended method = REML: Langan et al. 2018 https://onlinelibrary.wiley.com/doi/10.1002/jrsm.1316
# knha = Hartung method to control ES non-normality Rubio-Aparicio et al. 2018

res <- rma(
  yi,
  vi, 
  method="REML",
  #mods = ~ data$Labstudy,
  knha=TRUE,
  slab=paste(studies)
  )

# forest plot
forest(res, 
       xlab="ES", mlab="RE", psize=1,
       ilab = data$N, ilab.lab = "N",
       header=T, shade=T)

Summary Statistics

These are the results of the meta-analysis. The model results reflect the overall estimated effect size, its confidence interval and whether it differs significantly from 0.

The Tau² represents the between-study variance, and the I² the heterogeneity. A significant test for Heterogeneity indicates a genuine variability of true effects across studies.

summary.rma(res)

Random-Effects Model (k = 13; tau^2 estimator: REML)

  logLik  deviance       AIC       BIC      AICc   
 17.2904  -34.5808  -30.5808  -29.6110  -29.2474   

tau^2 (estimated amount of total heterogeneity): 0.0006 (SE = 0.0006)
tau (square root of estimated tau^2 value):      0.0235
I^2 (total heterogeneity / total variability):   42.66%
H^2 (total variability / sampling variability):  1.74

Test for Heterogeneity:
Q(df = 12) = 18.7120, p-val = 0.0957

Model Results:

estimate      se     tval  df    pval    ci.lb   ci.ub    
 -0.0051  0.0109  -0.4704  12  0.6465  -0.0290  0.0187    

---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Funnel Plot

# funnel plot (publication bias)
funnel(res)