Cohen’s d and effect size in R

Why a p-value isn’t enough, and how to compute and interpret Cohen’s d effect size in R — the standardized measure of how big a difference between two groups really is.

A p-value tells you whether an effect is there; an effect size tells you how big it is. With a large enough sample even a meaningless difference becomes “significant,” which is why every result should report an effect size too. For two means, that’s usually Cohen’s d. (Planning a study? Effect size is the key input to our sample size calculator.)

What Cohen’s d is

Cohen’s d is the difference between two group means expressed in pooled standard deviation units — so it’s comparable across studies and scales. Rough conventions:

• d ≈ 0.2 — small
• d ≈ 0.5 — medium
• d ≈ 0.8 — large

Compute it in R

Base R doesn’t have a built-in, but it’s a three-line function. We’ll use the sleep dataset (extra hours of sleep under two drugs):

cohen_d <- function(a, b) {
  n1 <- length(a); n2 <- length(b)
  s_pooled <- sqrt(((n1 - 1) * var(a) + (n2 - 1) * var(b)) / (n1 + n2 - 2))
  (mean(a) - mean(b)) / s_pooled
}

g1 <- sleep$extra[sleep$group == 1]
g2 <- sleep$extra[sleep$group == 2]

cohen_d(g2, g1)

[1] 0.8321811

A d of this size points to a substantial difference between the two drugs — regardless of whether the p-value happens to clear 0.05 at this sample size.

p-value vs effect size

t.test(g2, g1, paired = TRUE)$p.value  # significance

[1] 0.00283289

cohen_d(g2, g1)                        # magnitude

[1] 0.8321811

These answer different questions. Report both: significance says “unlikely to be noise,” effect size says “and here’s how much it matters.” A tiny d with a tiny p-value (from a huge sample) is statistically significant but often practically irrelevant.

Reporting

State the effect size with a confidence interval, and interpret it in the units your audience cares about — a “medium” d means little to a clinician until you translate it back into real outcomes.

---

Want effects reported and interpreted properly, not just p-values? That’s how we work.