Insensitivity to sample size
The tendency to under-expect variation in small samples
Alright, conversion optimizer, I’m looking at you on this one. You’re not immune to these cognitive biases. This bias is a big bias in split testing.Be careful of drawing actionable information from split tests that are drawn from small sample sizes.