Testing Terms
Testing terms cover experimentation and how to run reliable tests. Each definition explains the concept and how to interpret results.
8 terms
Bayesian TestingAn A/B testing approach that calculates the probability one variant is better than another, updating beliefs as data accumulatesTestingConfidence IntervalA range of values that likely contains the true effect of a test, showing the uncertainty around measured resultsTestingControl vs VariantThe two versions compared in an A/B test: the original (control) and the modified version (variant) being testedTestingExperiment PowerThe probability that a test will detect a real effect when one exists, typically set at 80% for A/B testsTestingHypothesis PrioritizationA systematic approach to ranking experiment ideas based on potential impact, confidence, and effort requiredTestingP-ValueThe probability that observed test results occurred by random chance, used to determine statistical significanceTestingSample Ratio Mismatch (SRM)A data quality issue where the actual traffic split between test variants differs significantly from the intended ratioTestingTest Duration CalculatorA tool that estimates how long an A/B test needs to run to reach statistical significanceTesting
