Business Policy Experiments Using Fractional Factorial Designs
At DoorDash, we constantly strive to improve our experimentation processes by addressing four key dimensions, including velocity to increase how many experiments we can conduct, toil to minimize our launch and analysis efforts, rigor to ensure a sound experimental design and robustly efficient analyses, and efficiency to reduce costs associated with our experimentation efforts.
Experimentation isn’t just a cornerstone for innovation and sound decision-making; it’s often referred to as the gold standard for problem-solving, thanks in part to its roots in the scientific method.
Metrics are vital for measuring success in any data-driven company, but ensuring that these metrics are consistently and accurately measured across the organization can be challenging.
Running thousands of experiments effectively means carefully balancing our speed with the necessary controls to maintain trust in experimental outputs–but figuring out that balance is never easy.
For complex systems such as the DoorDash assignment system, simulating the impact of algorithmic changes is often faster and less costly than experimenting on features live in production.
DoorDash delivers millions of orders every day with the help of DeepRed, the system at the center of our last-mile logistics platform.
DoorDash’s new experimentation platform, built on a combination of SQL, Kubernetes, and Python, allows for quick iteration of data-driven feature improvements.
DoorDash’s principles and processes for democratizing Machine Learning
Six months ago I joined DoorDash as their first Head of Data Science and Machine Learning.
Within the dispatch team of DoorDash, we are making decisions and iterations every day ranging from business strategies, products, machine learning algorithms, to optimizations.
At DoorDash, we believe in learning from our marketplace of Consumers, Dashers, and Merchants and thus rely heavily on experimentation to make the data-driven product and business decisions.
