A Holistic View of Perception in Intelligent Vehicles

• Ghassan AlRegib, and Mohit Prabhushankar. Tutorial on ‘A Holistic View of Perception in Intelligent Vehicles’. IEEE Intelligent Vehicle Symposium (IV 2023), Anchorage, AK, USA, June 4, 2023.
• pdf: https://bpb-us-w2.wpmucdn.com/sites.gatech.edu/dist/4/3061/files/2023/07/IV2023_Tutorial_Perception_WatermarkCitation-1-compressed.pdf

---

• Perception and Autonomy
  1. Perception in Autonomous Vehicles includes serious technical challenges.
  2. For example, it is hard to differentiate the white side of the trailer in the brightly lit sky.
  3. For example, it is hard to recognize the smal objects while traveling in a fast speed.
  4. CURE-TSD
  5. CURE-TSR
  6. Choosing the appropriate sensor is also important

Sensors in AVs should cover all possible conditions however challengable.

Choose the sensor based on its own characteristics.

More companies are using only mono camera and radar in AVs now.

The perception requires numerous application steps (i.e., Detection, Localization..), but it can be unified as using Deep learning.

Deep Learning could promise the holistic solution to a number of above challenges.

 

---

• Deep Learning for Perception

Evolution of CNN architectures

• LeNet5 (1988)
  • Reduced number of learnable parameters, learned from raw pixels automatically.
  • stacking convolutional, activation, and pooling payers, ends with fully connected layers.
  • Good results on small datasets

• AlexNet (2011)
  • Implement ReLU to solve the vanishing gradient problem.
  • Apply dropout regularization to fully connected layer to control complexity.
  • Deeper, Wider than LeNet.
  • Data Augmentation
  • Deep CNN that runs on GPU hardware.

• ResNet (2015)
  • Introduced residual learning (shortcut connections with identity mapping)
  • 20 times deeper than AlexNet, 8 times deeper than VGG
  • less computational complexity without compromising generalization power.

• YOLO (2016)
  • Single-stage object detection: feature extraction, detection, and classification performed in one go.
  • Object Detection as a regression problem.

• PointNet (2017)
  • Address the challenge while utilizing LiDAR data.
  • Use the symmetric max pooling function for permutation invariance.
  • Use T-Net to get the transformation matrix in order for point data in a canonical space.

• Late Fusion (2019)
  • Use YOLO framework to extract features from camera and LiDAR

• Journal: "Exploiting the Distortion-Semantic Interaction in Fisheye Data"
  • https://seulgi-kim.tistory.com/entry/%EB%85%BC%EB%AC%B8-Exploiting-the-Distortion-Semantic-Interaction-in-Fisheye-Data

 

---

• Deep Learning at Inference
  1. Introduction
  2. Journal: "Backpropagated Gradient Representations for Anomaly Detection"
  3. Journal: "Probing the Purview of Neural Networks via Gradient Analysis"
     • https://seulgi-kim.tistory.com/entry/%EB%85%BC%EB%AC%B8-Probing-the-Purview-of-Neural-Networks-via-Gradient-Analysis
  4. Journal: "Introspective Learning: A Two-Stage Approach for Inference in Neural Networks"
  5. Journal: "Detecting and Classifying Anomalies in Artificial Intelligence Systems"
• Key Takeaways and Future Directions