Lucas Kanade Tracker

In this project, we implemented the Lucas Kanade Tracker to track various objects(cars, humans) in different lighting conditions. The challenge in this project is to implement a robust tracker capable of handling the variations in brightness, extremely rapid frames, and also, greater shift in relative position of the tracker objects.

Let us consider the Car dataset, which includes about 659 images of the car. These images though in grayscale for appearance, are converted to gray scale in the code as OpenCV will still try to read their channels. The challenging aspect of this dataset is the change in brightness of the frames when the car passes under the bridge.Initially, the tracker failed under the low brightness areas. However, the tracker performance improved on reducing the kernel size. Also, in order to compensate for the lateral movement of the car, we have used 2 templates for tracking. This ensures robust tracking of the car under various lighting conditions. The matrix multiplcation involved in this algorithm is computationally intensive. Therefore, in order to speed up the algorithm, we have flattened each matrix and then multiplied them. Also, in order to improve the warping, we have used a combination of the INTERCUBIC and WARPINVERSEMAP. The results of the algorithm are shown below:

Another dataset given was the Dragonbaby. The original video of the dragon baby is here. The original video is creative, and worth a watch actually! The object to be tracked was the baby's face. This was one of the toughest datasets, as the baby's face is lost in some frames, blurred in others. Take a look at our implementation on this dataset.

The repository for thus project is hosted on GitHub at : https://github.com/shubham1925/lucas-kanade-template-tracker