Track and field races usually have fully automatic timing (FAT) systems which consist of hardware and software that are used to accurately record the times that each runner finished at. Most systems on the market are expensive because they use specialized hardware. Our goal in this project is to create an FAT system that can operate using video from a cellphone. The output of a normal FAT system is an image where the horizontal axis represents time and each column of pixels is taken directly across the finish line. In addition to producing these sort of images, we would like our system to be able to identify the hip number on each finisher so that a list of finishers and their times can be produced automatically.
Our pipeline can be divided into two sections, runner identification, and hip number localization and classification.
Our inputs were stationary videos of a finish line with runners occasionally passing through the frame.
To start, a background frame (usually the first) was subtracted from each frame, so that the modified frames were mostly black except for the runners.
Each frame was then convolved with an hx1 filter so that the a wx1 image was left.
These wx1 images were stacked for each frame, and RANSAC was used to fit through all the pixels above a certain threshold.
Each line represented a single runner, and the intersection of these lines with the finish line gave us the finish time of the runners.
Our system did fairly well at identifying the point at which each runner crossed the finish line, although there were usually some false positives. The automatic hip number reading was not quite as accurate. The first network did a decent job of picking the actual hip number out of the window it was given, but the MNIST trained classifier did not predict the correct digit with great accuracy. It would have been better if we had trained on a dataset of actual hip numbers, rather than hand draw digits, but unfortunately we did not have access to a dataset with many unique hip numbers. Overall, we are able to detect a finish with about 80% accuracy and report the hip numbers with about 30-40% accuracy.