Depth Estimation using Mono Camera

Approach
1. Hardware Generalization 

a. The method's accessibility is limited due to the high cost of the Intel Realsense 435i hardware. I focused on testing the method using more affordable hardware options.

b. Most widely available hardware that comes with a camera and an IMU sensor is a mobile device. The algorithm required perfect synchronization of camera frames and  IMU sensor. 

d. Using android phone APIs, time synchronised camera feed and synced imu data was generated. 

c. Algorithm utilizes pickles file with different time calculation method. So, Python scripts were developed to preprocess the data for the algorithm. 

d. With new scripts we were able to get distance values using $100 phone camera. We also tested with different android camera phones and validate the performance of the method with ground truth measurements.

e. Original method only calculates the distance of objects within a fixed patch. I implemented image detection pipeline to calculate distances for objects within the entire frame.

f. I implemented an object detection pipeline using YOLO V5, a lightweight object detection model. Create an update patch function that resets the patch's dimensions and location based on the detected object's position in the frame.

g. The original method uses a static gain state observer (Luenberger Observer).  I utilized a dynamic Kalman gain for handling noisy measurements and output correction. 

Challenges faced:

1. • Calibration of the camera data and synchronization with IMU data.

   • Obtaining the required files: camera intrinsic data and a patch template.

   • Ensuring synchronized timestamps between camera frames and IMU data.

1. • Implementing the object detection pipeline using YOLO V5.

   • Updating the algorithm to handle multiple objects and their respective distances.

   • Implementing the Kalman filter and integrating it into the existing algorithm.

   • Comparing the performance of the Kalman filter with the Luenberger Observer and ground truth.

Project 3: Dynamic State Observer

1. Problem statement:

   • The original method uses a static gain state observer (Luenberger Observer).

   • Explore an alternative observer (Kalman filter) for better accuracy and versatility.

2. Approach:

   • Replace the Luenberger Observer with a Kalman filter.

   • Utilize a dynamic Kalman gain for handling noisy measurements and output correction.

3. Challenges faced:

   • Implementing the Kalman filter and integrating it into the existing algorithm.

   • Comparing the performance of the Kalman filter with the Luenberger Observer and ground truth.

4. Conclusion:

5. Conclusion:

   • Successfully implemented the object detection pipeline.

   • Modified the code to calculate distances for objects within the entire frame.

   • Plan to extend the approach to support distance calculation for multiple objects simultaneously.

   • Successfully used the Kalman filter to estimate τ and ϕ constraints.

   • Compared the results against available ground truth.

   • Identified the need for further fine-tuning of the Kalman filter for improved estimation.

   • Minimal increase in computational time observed with the Kalman filter.