Routing in Google Maps is one of the most useful features that people use all the time. Figuring out the best way to get from point A to point B involves considering a bunch of factors like how long it’ll take to get there, tolls, how direct the route is, the condition of the roads, and user preferences. People’s preferences for getting around can be learned by looking at their actual travel patterns. This is where inverse reinforcement learning (IRL) comes in. IRL uses a process called Markov decision process (MDP) to analyze the road network and the routes people take. The goal is to understand what people value the most when choosing their routes. But up until now, it’s been hard to make IRL work on a large scale like the whole world. One of the problems is that IRL algorithms usually have to solve another problem every time they get updated. So even just trying to find the best way to go between two places in the whole world seems impossible. When you use IRL for routing, you have to think about all the possible routes between each starting point and ending point. This means breaking the problem down into smaller chunks is tough. But we’ve found a way to overcome these challenges. In a new paper, we show how we were able to scale up IRL in Google Maps. We used some old algorithms and added some new things like compression and parallelization. We also came up with a new IRL algorithm called Receding Horizon Inverse Planning (RHIP). RHIP lets us control how well the algorithm performs. The end result is a better route match rate in Google Maps. The RHIP policy is able to give a more accurate route and it’s faster too. We’re especially excited about how this can help people who use sustainable modes of transportation. Journey time isn’t the only thing that matters to them. We’ve tested RHIP in live experiments and it’s been a big success. It’s been able to improve routes for people all over the world. We’ve been able to train reward models on really huge datasets and made the most progress in IRL that anyone has made so far. You can read all about it in our paper. We want to thank all the people who worked on this project with us. It’s been a real team effort. We also want to thank the people who gave us their opinions and suggestions.