We investigate whether online A/B experimenters "p-hack" by stopping their experiments based on the p-value of the treatment effect. Our data contains 2,101 commercial experiments in which experimenters can track the magnitude and significance level of the effect every day of the experiment. We use a regression discontinuity design to detect p-hacking, i.e., the causal effect of reaching a particular p-value on stopping behavior.
Nearly every business leader (99% according to an HBR study) recognizes the need to build a data driven organization. However, so many struggle to use data for a variety of reasons, a big one of them being that data exists in all sorts of places. This is especially the case with startups, who use a number of out of the box tools that don’t connect to each other, and that only offer so much granularity and flexibility. The solution? A data warehouse.
The best articles on how to use analytics to grow your business. Analytics are the collection of both quantitative and qualitative data that are used to measure the performance of a business, product, marketing campaign or other initiative. Learn how growth marketers use analytics to find new opportunities for growth.
Join over 70,000 growth pros from companies like Uber, Pinterest & Twitter