Restaurants on busy thoroughfares can use many elements to catch a customer’s eye, but online ordering experiences mostly rely on the menu to generate sales. When dining in it's not hard to imagine seeing the storefront and patio of a Mediterranean restaurant and being prompted to look at the menu in the window. The wide selection of mezzes, spreads, and breads on the menu serves as an attraction for where to get lunch.
In this way, in-person menus are just one part of the visual and experiential elements of the storefront, which helps attract customers. When creating an online experience on DoorDash, the different physical storefront elements, such as the one pictured for Aba Chicago, in Figure 1, are absent. This leaves the online menu as the main way to attract customers.
Since the menu is the main online touchpoint, an unattractive or poorly organized menu can have a huge negative impact on a merchant’s online conversion rate, regardless of the quality of the food. If a merchant does not design its menu correctly, customers won’t be as attracted to their online offerings and will not buy as often. In order to succeed online, merchants need to utilize a set of menu-building best practices to attract new customers.
In order to discover the characteristics that make for successful online menus, we utilized machine learning (ML) to analyze thousands of existing menus on DoorDash’s platform. We were able to translate characteristics of menus into a series of hypotheses for A/B experiments. We saw a hefty improvement in menu performance from experiments involving header photos and more information about the restaurant. We intend to conduct further experiments around adding targeted photos, increasing description coverage of key items, and making changes to item options.
Aligning merchant and customer menu needs
The first step in menu design is figuring out what the merchant needs to show and align it with customers’ expectations.
For merchants, ensuring their menu fairly and accurately represents their business online is usually the highest priority. This can be a challenge for merchants as the in-person and delivery dining experiences vary greatly. Furthermore, designing this online experience requires addressing several questions:
- Should merchants have the same on-premise and online menu to present the same experience across all channels, or should there be a different online experience?
- Consumers are able to make modifications in restaurants by asking the staff for help. To what extent should modifications be possible on the online menu?
- What are the food options a merchant should provide and how complex should they be? Should the menu provide elements such as pictures, detailed item descriptions, and standard substitutions, or will that amount of detail make browsing difficult.
These are all questions that determine an online menu’s complexity and how much a merchant is able to show off their items and recreate what it feels like to dine in.
Merchants need to make sure their own requirements align with customers, who want online menus to be a delightful experience that helps them make ordering decisions. While staff at restaurants can help sell the menu by crafting a story around a menu item or giving live recommendations and advice, this will not be the case online. For example, customers might not be familiar with muhammaras, shown in Figure 2, and require much more information from the online menu, including photos and descriptions, to make the leap and order it.
Customers will need help when parsing an online menu to make their decision, but assistance will have to be given in new ways that work online. This assistance might take the form of a simplified menu that prevents decision-fatigue by cutting out unpopular items, adding more detailed descriptions and labels to explain the remaining menu items, and clearly categorizing items to make the menu easier to navigate.
Defining the elements of an online menu
Before we jump into how we analyzed menus, let’s first define their key characteristics. Reconciling the key considerations of merchants and customers yields three key elements for online menu design:
- Structure
- Customizability
- Aesthetics
Structure: With structure, a merchant is considering the complexity of their menu. Do they want the menu to be short and simple? Or would they prefer an incredibly long and detailed menu?
Customization: Without staff explaining the menu or guiding decisions, the customizability of individual items and the addition of different options becomes a key consideration.
Aesthetics: Last but not least, what are the different visual elements that can be added to an online menu to make it as attractive as possible and a good brand representation?
With these key elements in mind, we can discover the top individual factors that lead to designing a great online menu.
Using ML to figure out the most influential menu features
It can be difficult to collect data on menus to discover their most influential features. Most menu data is quite unstructured and based on optical character recognition (OCR). While OCR is reliable for structured text within documents, it is a challenge to represent text on a menu. Menus are often not structured linearly and one would have to contextualize different menu categories with item names and descriptions for accurate representation.
Luckily, DoorDash has over a hundred thousand restaurants with menu data formatted in a way that is understandable and analyzable. We are able to include all of these menus and their corresponding conversion rates in a model that predicts the top factors for well-performing menus.
Defining the feature set for each layer of the menu design
We define a successful menu as one with a high conversion rate during a determined period of time. This model design is quite generalizable and reusable, as it's easy to swap out the target variable with another top line metric to get another angle on menu performance.
In building the set of features for the model, we looked at each layer of a menu, from the high-level menu appearance to detailed modifiers for each item. For each layer, we brainstormed features relating to the key elements mentioned above: structure, customizability, and aesthetics.
At the highest level, we considered the overall menu appearance. How many items and categories does the menu contain? Are the menu items well photographed? How do the different top-level designs of menus, as shown in Figure 3, below, determine customers’ first impressions?
We then move down to the category level. What types of categories are there? Are there special categories, such as “Nonna’s specials” if it is an Italian restaurant? Were the categories mostly representative of the menu’s appetizers, mains, sides, or desserts? As shown in Figure 4, below, some merchants use special categories to create meals with existing items while others use these categories to house off-menu or limited-time items.
In the next level down we consider items. What types of items are there? Do they have descriptions that help explain the item? How do price points vary across items? Adding descriptions, as Duck Duck Goat does in Figure 5, below, helps customers understand the ingredients in a dish and set expectations.
Finally, at the lowest level of detail, we have item options. How customizable are these items and what are the types of options and modifiers available? While some merchants provide a running list of options for each item, such as Pho888’s menu in Figure 6, below, some merchants prefer to keep the option list short to limit the online menu and simplify preparations.
Implementing our model
To find out which menu features were the most influential in menu conversion, we use the features mentioned above as inputs to regression models predicting menu conversion. We built our initial regression models with linear regression models and base tree models to achieve a baseline error. While the results were interpretable, the error rate was quite high. Furthermore, many of the features seemed to be correlated, which led to the issue of collinearity.
To further improve the model, we calculated VIF values to prune overly correlated variables. LightGBM and XGBoost proved the most accurate, so we chose them as the final models.
Despite the improved model performance, it was difficult to determine how changes in each feature impacted the target variable directionally. This issue of explainability has always been a pain point for black box models that perform better.
To solve this black box issue of not knowing the impact of each variable, we used Shapley (SHAP) values, a game theoretical approach towards model explainability. SHAP values represent the marginal contribution of each feature to the target variable and are calculated by computing the average marginal contribution to the prediction across all permutations before and after withholding that feature.
The resulting model summary plots are effective in demonstrating relative feature importance and directional impact on the feature variable.
The factors most correlated with a successful menu
Examining the resulting SHAP plots of the final XGBoost model, the top success factor was the number of photos on the menu. This is particularly important for the top items in a menu as photo coverage for such top items determines the overall appearance of the menu.
Other top factors and recommendations for merchants include higher customizability for their items. Customers enjoy some optionality within top items and the ability to customize an item provides a degree of familiarity associated with dining in. It’s easy to imagine a customer being turned off if they cannot ensure their desired dish is mild, medium, or spicy.
Menus with a healthy mix of appetizers and sides also converted better. This provides customers with more choices to complete their meal and can lead to higher cart values for merchants as well.
Specific cuisines have varying ideal menu characteristics
While the top factors that led to successful online menus are not surprising, rerunning the models across different merchant categories led to interesting variance in top menu factors. This variance in the successful menus is a case where “averages lie”, when the average does not represent a normal distribution, and suggests that not all menus should be built the same way to be successful. By better understanding customers' expectations about certain types of restaurants, merchants are better able to build and customize their menu into a truthful representation of their brand and food that these customers will understand.
The clearest example of customer expectations at work was with Chinese menus. Unlike most other menus, Chinese menus that were long and had many items tended to perform better. When customers dine in at Chinese restaurants, a long and complicated menu (sometimes with regional specialties) indicates authenticity, as the restaurant is capable of making all these different dishes. As an example, see the menu depicted in Figure 8, below. Customer expectations about an authentic Chinese menu leads to these complicated menus having greater conversion rates.
On the opposite end of that spectrum, menus for wings and pizza places tend to be shorter, well photographed, and have a myriad of options for customization. Customers usually visit these merchants with one or two items in mind and are just looking for customizations like extra sauces for wings or additional toppings for pizzas.
These specific recommendations and differences across successful menus are also not limited across cuisines. There are specific nuances as well within a single cuisine. For example, the menu of a Chipotle restaurant would be very different from the menu of a local food truck. While menus of large brands are relatively well known and convert well as customers know what they’re getting, the local spots have to put more effort into pictures and descriptions to help customers take the leap of faith and order from them.
What’s next in menu optimization
Given this prioritized list of features from our ML models, we are continuing to leverage the learnings and translate them into hypotheses around improving menu conversion and sales. Upcoming experimentation includes adding different types of photos across item types and the menu page and presenting items with a richer level of detail in their descriptions.
While it is one thing to let merchants know what factors lead to the highest conversion, we’re also working to make menu updates as intuitive and valuable for merchants on our platform. As such, we are continuing to experiment with new products to help merchants build ideal menus and better delight and intrigue their customers.
On the merchant enablement end, the learnings of which menu factors to prioritize can be collected into a much more detailed style guide for merchants of particular cuisine types. Tools such as the DoorDash Merchant Portal, our one stop shop for merchants to manage their performance on DoorDash, and the accompanying menu editor tool can be improved to make it easier to customize for extra options and give customers more context around the items they are ordering.
On the customer features side of the equation, this could mean new and improved menu designs to help highlight top items and categories, and allow for merchant stories to highlight their value proposition. The improved menu interface can help bridge the gap between the digital storefront and the traditional physical experience.
With these additions to our products and merchant education, we hope that merchants will be better equipped to control how they present their brand and digital storefront to their customers. Doing so will not just help improve their online sales but better reconcile and complement their physical and digital brands.
Acknowledgements
Thank you to Karthik Tangirala, Ariel Posner, Imani Joy, Govind Lingam, Selina Her for talking through product, design, and research insights on menus and Ezra Berger, Jessica Lachs, Alok Gupta, Kevin Teng, Jane Gladshteyn, Megan Han, Mihir Sathe for helping proofread and edit the article!
