Amr Suleiman, Member, Zhengdong Zhang, Student Member, Luca Carlone, Sertac Karaman, Member, and Vivienne Sze
doi: 10.1109/JSSC.2018.2886342
Abstract:
This paper presents Navion, an energy-efficient accelerator for visual-inertial odometry (VIO) that enables autonomous navigation of miniaturized robots (e.g., nano drones), and virtual reality (VR)/augmented reality (AR) on portable devices. The chip uses inertial measurements and mono/stereo images to estimate the drone’s trajectory and a 3‑D map of the environment. This estimate is obtained by running a state-of-theart VIO algorithm based on non-linear factor graph optimization, which requires large irregularly structured memories and heterogeneous computation flow. To reduce the energy consumption and footprint, the entire VIO system is fully integrated on-chip to eliminate costly off-chip processing and storage. This paper uses compression and exploits both structured and unstructured sparsity to reduce on-chip memory size by 4.1×. Parallelism is used under tight area constraints to increase throughput by 43%. The chip is fabricated in 65-nm CMOS and can process 752 × 480 stereo images from EuRoC data set in real time at 20 frames per second (fps) consuming only an average power of 2 mW. At its peak performance, Navion can process stereo images at up to 171 fps and inertial measurements at up to 52 kHz, while consuming an average of 24 mW. The chip is configurable to maximize accuracy, throughput, and energy-efficiency tradeoffs and to adapt to different environments. To the best of our knowledge, this is the first fully integrated VIO system in an application-specified integrated circuit (ASIC).