Creating Autonomous Vehicle Systems

This is the first technical overview of autonomous vehicles where we share our practical experiences of creating autonomous vehicle systems. Autonomous vehicle systems are complex, consisting of three major subsystems: algorithms for localization, perception, and planning and control; client systems, such as the robotics operating system and hardware platform; and the cloud platform, which includes data storage, simulation, high-definition (HD) mapping, and deep learning model training. The algorithm subsystem extracts meaningful information from sensor raw data to understand its environment and make decisions about its actions. The client subsystem integrates these algorithms to meet real-time and reliability requirements. The cloud platform provides offline computing and storage capabilities for autonomous vehicles. Using the cloud platform, we are able to test new algorithms and update the HD map – plus, train better recognition, tracking, and decision models.

This book consists of 9 chapters: chapter 1 provides an overview of autonomous vehicle systems; chapter 2 focuses on localization technologies; chapter 3 discusses traditional techniques used for perception; chapter 4 discusses deep learning based techniques for perception; chapter 5 introduces the planning and control sub-system, especially prediction and routing technologies; chapter 6 focuses on motion planning and feedback control of the planning and control subsystem; chapter 7 introduces reinforcement learning-based planning and control; chapter 8 delves into the details of client systems design; and chapter 9 provides the details of cloud platforms for autonomous driving.

This book should be useful to students, researchers and practitioners alike. Whether you are an undergraduate or a graduate student interested in autonomous driving, you will find herein a comprehensive overview of the whole autonomous vehicle technology stack. If you are an autonomous driving practitioner, the many practical techniques introduced in this book will be of interest to you. Researchers will also find plenty of references for an effective, deeper exploration of the various technologies.

Table of Contents

Introduction to Autonomous Driving
Autonomous Vehicle Localization
Perception in Autonomous Driving
Deep Learning in Autonomous Driving Perception
Prediction and Routing
Decision, Planning, and Control
Reinforcement Learning-based Planning and Control
Client Systems for Autonomous Driving
Cloud Platform for Autonomous Driving

About the Author(s)

Shaoshan Liu, PerceptIn
Dr. Shaoshan Liu is chairman and co-founder of PerceptIn. He attended UC Irvine for his undergraduate and graduate studies and obtained a Ph.D. in Computer Engineering in 2010. His research focuses on Computer Architecture, Big Data Platforms, Deep Learning Infrastructure, and Robotics. He has over eight years of industry experience: before founding PerceptIn, he was with Baidu USA, where he led the Autonomous Driving Systems team. Before joining Baidu USA, he worked on Big Data platforms at LinkedIn, Operating Systems kernel at Microsoft, Reconfigurable Computing at Microsoft Research, GPU Computing at INRIA (France), Runtime Systems at Intel Research, and Hardware at Broadcom.

Liyun Li, Baidu US
Dr. Liyun Li is currently a software architect at Baidu’s Silicon Valley research center in Sunnyvale, CA. As one of the early members in Baidu’s Autonomous Driving team, he has been leading and driving the efforts of developing autonomous driving technologies including smart behavioral decision, motion planning, and vehicle control for Baidu’s autonomous vehicle. Before joining Baidu, he worked as a senior software engineer at LinkedIn, now a Microsoft subsidiary. He obtained his Ph.D. in Computer Science from New York University, with a research focus on applied machine learning.

Jie Tang, South China University of Technology
Dr. Jie Tang is currently an associate professor in the School of Computer Science and Engineering of South China University of Technology, Guangzhou, China. Before joining SCUT, Dr. Tang was a post-doctoral researcher at the University of California, Riverside and Clarkson University from December 2013 to August 2015. She received the B.E. from the University of Defense Technology in 2006, and the Ph.D. degree from the Beijing Institute of Technology in 2012, both in Computer Science. From 2009-2011, she was a visiting researcher at the PArallel Systems and Computer Architecture Lab at the University of California, Irvine, USA.

Shuang Wu, YiTu
Dr. Shuang Wu is currently a scientist at Yitu Inc. Previously senior research scientist at Baidu’s AI lab in Sunnyvale, CA, senior architect at Baidu USDC. He earned his Ph.D. in Physics from University of Southern California, and was a postdoctoral researcher at UCLA. He has conducted research in computer and biological vision, applied machine learning in industry for computational advertisement, and speech recognition. He has published in conferences such as NIPS, ICML etc.

Jean-Luc Gaudiot, University of California Irvine
Dr. Jean-Luc Gaudiot received the Diplome d’Ingenieur from ESIEE, Paris, France in 1976 and M.S. and Ph.D. degrees in Computer Science from UCLA in 1977 and 1982, respectively. He is currently a professor in the Electrical Engineering and Computer Science Department at UC, Irvine. Prior to joining UCI in 2002, he was Professor of Electrical Engineering at the University of Southern California since 1982. His research interests include multithreaded architectures, fault-tolerant multiprocessors, and implementation of reconfigurable architectures. He has published over 250 journal and conference papers. His research has been sponsored by NSF, DoE, and DARPA, as well as a number of industrial companies. He has served the community in various positions and was just elected to the presidency of the IEEE Computer Society for 2017.