Big Data at UPS. Interview with Jack Levis.
“Data is always a bottleneck.”–Jack Levis.
I have interviewed Jack Levis, Senior Director, Industrial Engineering at UPS. Jack shared with us some of the main lessons learned from the development of the “On Road Integrated Optimization and Navigation system” (ORION).
RVZ
Q1. Can you give us a background on UPS and some of the challenges that UPS faces?
Jack Levis: The e-Commerce revolution brings with it some interesting challenges. First, from a cost-to-serve standpoint, residential deliveries are less dense in terms of distance between stops and pieces per stop. Second, residential customers want personalization in their delivery experience, which adds cost and other challenges. UPS has met these challenges by utilizing operations technology and advanced analytics.
Q2. UPS won the 2016 Franz Edelman prize for its On Road Integrated Optimization and Navigation system (ORION). You lead the four-year long development of ORION. ORION completed deployment in 2016. How did you manage to reduce 100 million miles driven annually and save UPS $300 to $400 million each year?
Jack Levis: UPS has a long history of innovation and constant improvement. The ORION story actually began in the late 1990s when UPS started building our Package Flow Technologies (PFT) data infrastructure. PFT created predictive models, a “virtual network”, and a suite of planning and execution tools. This was the foundation for ORION. PFT deployed in 2003 and reduced 85 million miles driven per year.
ORION was built on this robust foundation. Using the discipline of operations research, ORION created a proprietary route optimization brain. Research into ORION began in 2003, and the first model was field tested in 2008 in Gettysburg, Pennsylvania. The result was that ORION could find ways to serve all customers in a route while at the same time reducing cost.
It does so by systematically analyzing more than 200,000 different ways a route can be run and then presenting the optimal route to a driver. It does so in seconds.
The ORION savings of 100 million miles and $300 million to $400 million annually is in addition to the savings from PFT.
Q3. What data infrastructure and data analytics tools did you use to implement ORION?
Jack Levis: As mentioned above, ORION sits on top of our proprietary PFT technology. The analytics tools are also built in house by UPS’s operations research team. The marriage of operations research, IT and business processes is part of the ORION success story.
Q4. What were the main challenges and pitfalls you encountered in the project?
Jack Levis: The first challenge was to build the ORION optimization engine (brain) that could not only meet service while reducing cost but do so while thinking like a driver. This meant that ORION needed to balance consistency and optimality. It made no sense to throw things up in the air just to save a penny. To do so, UPS had to reevaluate business rules, methods, procedures, etc. In essence, UPS redesigned the delivery process.
The second challenge was ensuring the data fed to ORION was “pristine”, and maps were a major challenge in this regard.
Off-the-shelf maps were not accurate enough. UPS patented a process for utilizing our “big data” infrastructure to help make maps accurate enough for ORION.
For instance, if a speed limit changes or a bridge is out, this edit must be made and updated information sent to ORION. From the time the map is edited, an optimized route can be in a driver’s hands in 30 seconds.
The third and largest challenge was change management. By definition, optimization tools like ORION will require people to change behavior. UPS tested training procedures, new metrics, analysis tools, etc. to ensure the change would take place.
Ultimately, the deployment team grew to 700 people. This team impacted tens of thousands of front-line personnel. The deployment team and the front line are the true heroes of the story.
Q5. What are the three most important lessons learned during this project?
Jack Levis: Never assume you know the answers. The first four years were spent truly understanding the delivery problem. There were many guidelines but few rules. ORION had to turn these guidelines into acceptable algorithms. The team did so by working with drivers and the front line until ORION started thinking like a driver.
Data is always a bottleneck. Initially, the team thought maps that could be purchased would be accurate enough. When the optimizations gave bad answers, the team looked at the algorithm. As it turned out, the problem was the map data accuracy and not the algorithm. The algorithm could not be truly tested until the data inputs were “pristine.”
Don’t forget deployment and change management. Do not think that “If you build it, they will come.” ORION required extensive change management and front-line support. It is important to have support from the top and show that the results are achievable. Without understanding the importance of change management, new programs run the risk of becoming a “flavor of the month.”
Q6. Can you recommend three best practices so that other projects can have a smoother path?
Jack Levis: Focus on decisions. Put effort into areas where a better decision will make an impact.
Focus on deployment and data from the beginning. The ORION deployment strategy is now the standard for all operations projects at UPS.
Ensure the right data infrastructure is in place with proper data.
Utilize appropriate associations and networking for help like the Institute of Operations Research and the Management Sciences (INFORM).
Q7. What were the key elements that made this project a success?
Jack Levis: Support from senior management to allow the team to continue the research even when failures occurred.
Proving that benefits could be achieved. The teams ran 11 different tests in multiple operations to test things like benefit achievement, training, metrics, best practices, etc.
No site could be deployed unless an entrance criteria was met. No site could say deployment was completed unless exit criteria was met.
There was constant evaluation of metrics and issues.
ORION was built inside a delivery process. Operations do not know they are using such advanced mathematics. They are just doing their job.
Q8. In general, how do you evaluate if the insight you obtain from data analytics is “correct” or “good” or “relevant” to the problem domain?
Jack Levis: The key is to start with decisions. See below (Q9).
Q9. What are the typical mistakes done when analyzing data for a large scale data project? How can they be avoided in practice?
Jack Levis: We do NOT focus on technology. We focus on decisions. Big data is a how, not a what. We care less about Big Data than we care about big insight and big impact.
By focusing on decision that need to be made, priorities become clearer. Ensuring decision-makers have the right information to make decisions and then measuring the impact of better decisions helps with the process. It helps to ensure the proper data is in place to make an impact.
If an impact can be made from a simple tool, that is a good thing.
Q10. What are the main barriers to applying machine learning at scale?
Jack Levis: The largest barrier is not focusing on decisions.
Q11. What is your next project?
Jack Levis: UPS will build out ORION.
ORION will begin making suggestions to drivers throughout the day. ORION will also provide navigation to drivers.
We are also working on ORION making “dispatch” decisions. ORION will begin deciding which driver should serve customers. In essence, at some point it will automate the pickup/delivery process.
Simultaneously, UPS will provide ORION-like functionality in other areas of the business.
There will be a PFT/ORION for Transportation; city-to-city movements.
There will be a PFT/ORION for inside the building operations; sorting, loading, moving of vehicles, etc.
Qx Anything else you wish to add?
Jack Levis: The advances mentioned above along with automated facilities will begin to automate and optimize the network.
Today, ORION optimizes a single driver. Tomorrow, ORION will begin to optimize an entire delivery area.
UPS has a bold vision to optimize an entire network from end to end.
————————
Jack Levis, Senior Director, Industrial Engineering, UPS
“Through the marriage of technology, information, and analytics UPS reduces cost and improves services. These advanced technologies streamline our business processes and ultimately benefit our customers.”
Jack Levis, Senior Director of Industrial Engineering, drives the development of operational technology solutions. These solutions require advanced analytics to reengineer current processes to streamline the business and maximize productivity.
Jack has been the business owner and process designer for UPS’s Package Flow Technology suite of systems, which includes its award-winning delivery optimization, ORION (On Road Integrated Optimization and Navigation). These tools have been a breakthrough for UPS, resulting in a reduction of 185 million miles driven each year.
ORION has completed deployment and is providing significant operational benefits to UPS and its customers. UPS estimates that ORION is reducing costs by $300 million to $400 million per year.
Jack earned his Bachelor of Arts degree in psychology, from California State University Northridge. He also holds a Master’s Certificate in Project Management from George Washington University.
Jack is a fellow of the Institute for Operations Research and Management Sciences (INFORMS), receiving their prestigious Kimball Medal and the President’s Award.
Jack holds advisory council positions for multiple universities and associations, including the U.S. Census Bureau Scientific Advisory Committee.
————————-
Resources
–UPS Wins 2016 INFORMS Franz Edelman Award For Changing The Future Of Package Delivery
–UPS On-Road Integrated Optimization and Navigation (ORION)
–UPS ORION: The Driver Helper (Infographic)
Related Posts
– How Algorithms can untangle Human Questions. Interview with Brian Christian
– On Digital labor: Technology, Challenges and Opportunities. Interview with Michael Henry
– Big Data Management at American Express. Interview with Sastry Durvasula and Kevin Murray.
Follow us on Twitter: @odbmsorg
##
