Skip to content

Blog


Tackling technical challenges to build a global logistics platform

November 17, 2021

|

Navid Zolghadr

There are a variety of novel challenges involved when a growing tech company wants to expand quickly and efficiently into new markets. This was very much true at DoorDash, where we spent years updating our platform to ensure a successful 2015 launch in Canada, followed by our expansion into Australia in 2019, and Japan in 2021. To reduce the time between launches, we needed to go beyond addressing each country's specific challenges and build a platform that could make market launches more automated and faster to execute. 

Here we look at some of the specific challenges we are facing and describe our general approach to building a platform that can streamline international launches. In future posts, we will review the specific solutions we are building to solve these problems. 

Global expansion challenges and opportunities 

Despite the challenges of moving into new markets, tech companies can capture myriad opportunities through expanding internationally. In addition to enjoying diverse markets, companies can take advantage of local labor markets to bring more top talent to the team. Primarily, of course, companies growing abroad are seeking to grow the available pool of consumers. Generally speaking, companies cater to a specific customer base, which may be in limited supply in their home country. Expanding to new markets offers opportunities to attract new customers and increase revenue. 

Expanding internationally also makes it easier to attract and retain new, local talent. Because there are only so many experienced engineers in any given location, moving to a new market creates an opportunity to compete in that local hiring market and grow the team. An added advantage is that local engineers are more attuned to their home country’s unique needs.  

The challenges of building an international platform 

Building an international platform that will allow our team to quickly launch in a new country comes with a variety of challenges that we had to address. 

Challenge: Localizing a suite of products consistently on a global platform 

A central piece of the globalization challenge is localizing and translating all the products that are used on DoorDash’s global platform so that anyone can use them. Users in each market will prefer app interfaces that have a localized format, language, and numbering system. This will become more and more important as DoorDash expands into non-English speaking countries. To be successful in a market, products need to present data to all users in a format with which they are comfortable. Each data type that needs to be localized might bring its own set of challenges. One of our of previous posts elaborated on different types of data included in a localization pipeline, including: 

  • Text: A localization pipeline requires translating all strings in the apps and making them visible to users.
  • Addresses: If the app is dealing with a physical address, it is important to note that different locales write addresses in different formats. Some regions start addresses with a country and then work their way down to the unit number, while other regions require the opposite order.
  • Correct measurement units: Different regions show measures like the weight of an item in a variety of units; the app must use what’s most common for that region.
  • Naming customs: The ordering of first and given name might be different in various locales. Additionally, pronouns may be used in different contexts based on the audience and how formal the language is, such as honorifics in Japanese.
  • Numbers: Each language has its own way of writing numbers with respect to decimal points and digit group separators.
  • Currency: presents two challenges, converting prices and displaying them in local currency and dealing with the different annotations in the database as a period and comma mean different things for different currencies and can be hard for the backend to determine the number of significant units in every price.   
  • Date and time: Different regions have different conventions for showing the date. For example, showing countries where the convention is that the day is the first part of a date the month would cause confusion. Our app needs to be able to display the date differently depending on the location the app is being used. 

In addition to ensuring that all of these attributes are implemented, it is important that localization efforts are standardized. Without a standard, centralized localization system, internal teams will come up with their own solutions, which won’t scale and cannot be maintained for the long term.

Another typical challenge in accomplishing these translations is the binary size of the client-side apps, which grows linearly with the number of supported locales. The growing binary size will bloat the app, making it difficult to download and use, even though each individual user might only use one locale within the app.

Solution: Build a centralized localization service and libraries to provide a consistent experience

To address the localization of all DoorDash products, we are building a service to serve all translation needs across all the apps and binaries as well as a set of static libraries that would cover localization of a subset of other data types. This will standardize across all services, simplifying system maintenance. 

A key question for a good localization pipeline is what solution we would adopt for each of the data formats described in the previous section. There are two approaches we could take:

  • Make services that other services will call to get the localized version of the content. Although this solution will add latency and reliance on the localization service, it will ensure all localization lives in one place.
  • Make static libraries that services and binaries can integrate and call in their code flows. This solution offers the lowest overhead, but the least control because it would be up to the individual services and binaries to make sure they integrate with the libraries.

To take advantage of both approaches, we are adopting a hybrid solution that differs based on the data format.

The translation service we are building can serve the translated strings dynamically when requested during the runtime from our other microservices. This would help teams serve completely new strings from their databases without needing to rebuild and deploy their services. The service will also serve translations for the client-side apps that wish to compile all the translations in the app bundle. 

This localization service will be completely transparent to developers and improve developer velocity because they can call the translation service rather than wait for translations to happen in the background. In addition to the translation service, we have libraries across all languages and frameworks used within DoorDash to support other data localization formats, such as date/time, currency, and number formatting. We also leverage the platform APIs as much as possible within those libraries. These tools allow teams to opt into this common tooling across the company with minimum overhead while still giving them some autonomy to choose the language and framework they want to work with.

Challenge: adapting our platform to respect local laws, regulations, and customs

For a product to be ready to launch in a new market, it needs to respect local laws, regulations, and customs. This process creates a number of action items for businesses like DoorDash because we have a lot of different touchpoints that are all built in platforms designed to cater to all our customers. If there is a major law or regulation to address, we either need to create an entirely new experience for that specific locality or alter our existing platform so that it can address the local laws in their respective locales. 

Given how  business processes have many layers of complexity it would be time-consuming and require duplicate efforts to sift through these processes to rewrite operational flows. Instead, we needed a way to copy the flows that still work in a new market while being able to easily swap in new ones that adhere to local laws.  

Solution: Using a configuration store pattern for alternative flows 

The most practical way to deal with different configurations is to build an initial step that decides which flow to use based on the region or country code of the app user. This way, the app can sort out which experiences to show, negating the need to build separate applications, which would be harder to support. Each experience can be tailored to a particular market. This lets us build separate flows to satisfy the unique requirements of each location. When each user opens the app, their experience can be directed to an app flow that conforms to local regulations. 

Achieving this goal required building a set of reusable components that capture the different parts of the flow instead of creating a less-flexible monolithic application. These components are plug-and-play, allowing us to choose which to include in the pipeline via an external JSON config. Taking advantage of the reusable components, we can easily customize the flow for each market so the app automatically shows the correct model for that market.

Figure 1. Instead of having three different Dasher user journeys — one each for the US, Canada, and Japan — we built one Dasher onboarding platform with three different user flows based on the user’s location and its specific laws, regulations, and offerings. This means one app can cover all our locations and easily be reused when we have to onboard a new process.  

Challenge: How to integrate with local partners in each market

Integrating with local partners also is key to a successful launch. Local partnerships contribute to the company's success in a region by integrating with a product for more seamless usage and faster adoption. Of course, different businesses have varying integration points and APIs because each was designed independently. When there are no accepted open source standards or working groups standardizing API rules and formats in a particular domain, integration becomes a scalability problem; each new partner requires developing a new integration from scratch.

As a company’s number of partners grows, so does the breadth of its APIs. Without the right strategy in place, the effort required to onboard new partners and integrate with their APIs will grow linearly with the number of partners. This can be a cumbersome challenge that would not scale for any company that seeks to capture the long tail of small partners across the globe.

One example of the effort involved has been our integration with local banks to integrate their payment system in DashPass. Because we didn’t have these APIs and templated integration points, we had to invest a good deal of resources to deliver that product.

Solution: Build a library for all third-party APIs

The best way to tackle third-party integrations is to provide clear and understandable APIs for third-party developers to allow anyone to use our standards, rather than cater to every partner’s unique system. As we launch in different countries, we integrate with local partners in a variety of spaces such as local grocery stores and banks. To be able to generalize our system, we need to formalize the APIs of different products to allow any third party to integrate with them. By formalizing our APIs, we can solve the problem of integrating third parties through publishing our patterns and providing integration guides to third party developers.

A good set of APIs needs to be consistent across the board to minimize the integration effort. These were our guiding best practices: 

  • Consistency across the API surface: If there is a common field expected in different endpoints, all of them will expect them in the same way. For example, passing the developer key, which is the most common field in all the APIs, needs to be consistent. Differing APIs would make it harder for other parties to integrate with them. 
  • Adaptable APIs: Creating APIs so that callers can adapt them with queries allows them to receive the desired response in the right format such as  xml or json.
  • Backward compatibility and future evolution: Even as the API evolves and adds more functionality, it’s important that these changes do not break any existing integrations.

A consistent set of APIs allows us to capture the long tail of small partners that would be impossible to accommodate on a case-by-case basis.

To optimize this process of generalizing our API patterns, we are making big investments in Drive and DashPass to make it easy for banks and other retailers to integrate with them. This will allow us to expand our reach much faster and enable third parties to easily take advantage of our APIs.

Challenge: Keeping services running across all time zones

As a product launches in new countries and time zones, it will create market-specific issues that need to be addressed around the clock, without an undue burden on the supporting engineering teams. There is usually a user traffic pattern that changes based on the day of the week or time of day within each market. For example, the number of food delivery orders is higher during the day than at night. As such, different issues may have varying impacts based on where they occur.

The primary concern is getting the fastest response on important issues and outages while minimizing the number of people involved to avoid redundant noise. There might be employees in the affected time zone who can look into the issue during their working hours. That might give us the fastest response time but might not give us the most relevant expertise related to the outage. On the other hand, always alerting the owner team might get us optimal expertise but can cost precious time if the engineer called on  must wake up in the middle of the night and get in a good state of mind to manage the issue.

Solution: Ensure system health with an effective on-call strategy

A multilayered on-call system would be able to give us the best of both worlds in terms of optimizing expertise and efficiency. As DoorDash moves into new regions, it will expand its engineering footprint in each country. Over the course of building collaboration between these offices and establishing a good knowledge transfer path, we will assign each region’s engineers for that area’s on-call support. For example, we will assign the APAC engineers for the APAC on-calls and EMEA engineers for EMEA on-calls.

Figure 2: Visualization of the regions we use to organize our global teams.

This strategy ensures that each issue is directed to regional experts. The other positive note for this strategy is that, because of the day/night user traffic pattern, when traffic is high in a region the designated on-call engineer who is in the same time zone is less likely to be asleep, generating faster response times. If further assistance is needed from other teams, the regional on-call can escalate the issue to the domain team if necessary. In that case, the domain teams will only need to be involved if the issue was not resolvable immediately by the regional on-call or for the follow-up long-term fixes that are not as urgent as the original issue or outage.

Conclusion 

As more companies move overseas and enterprises continue to embrace remote work, these challenges will be felt even more intensely across the industry. Developing best practices helps all companies grow faster and more efficiently.

We have built a dedicated organization to provide support to other DoorDash teams in solving some of these challenges. This organization is heavily involved in launching DoorDash in new countries and allows DoorDash to innovate horizontally while core teams innovate vertically on other products. If you are interested in being apart of this engineering effort consider joining our international platform team by checking out open opportunities here.

Related Jobs

Location
San Francisco, CA; Sunnyvale, CA; Seattle, WA
Department
Engineering
Location
San Francisco, CA; Sunnyvale, CA
Department
Engineering
Location
San Francisco, CA; Sunnyvale, CA; Los Angeles, CA; Seattle, WA; New York, NY
Department
Engineering
Location
Sao Paulo, Brazil
Department
Engineering
Location
San Francisco, CA; Sunnyvale, CA; Los Angeles, CA; Seattle, WA
Department
Engineering