Normal LiDAR in the market runs at 10hz, which is sufficient for state-of-the-art autonomous road vehicle, but not enough for autonomous racing vehicle that runs at 180mph. In order to make a "faster LiDAR", inspired by the fact that camera(30+hz) and LiDAR(10hz) holds different operating frequency, we propose a method that uses both camera and LiDAR history information to predict future LiDAR frames.
By replicating the state-of-the-art sensor fusion detection model ["PointPainting"](https://arxiv.org/pdf/1911.10150.pdf), we further use this tool to test/evaluate our 3D point cloud predictive model and end-to-end extrinsic sensor calibration model.
We deployed the state-of-the-art LiDAR perception model ["Point-Voxel CNN"](http://papers.nips.cc/paper/8382-point-voxel-cnn-for-efficient-3d-deep-learning.pdf) on MIT Driverless's full scale autonomous racing vehicle. The deployment includes converting model task from segmentation to classification, ROS integration and full scale vehicle testing. Find full story in the video provided here.
As my first project at [MIT Driverless](https://driverless.mit.edu/), my task was meant to find the visual explaination of the CNN-based object detection model that perception team is using, the [YOLOv3](https://arxiv.org/pdf/1804.02767.pdf). After reviewing the results, we concluded that the network has its most attention on the bottom part of the object(traffic cone), and in some cases the margin between cone and ground.
In order to make custom high-speed cameras that can deal with small patches of motion blur, I proposed a custom convolutional model that can detect motion blurred patches within images, achieved two sigma accuracy.