A open, freely accessible runner image search platform using racing bib number.
Webapp link: Runner Tag Platform
This Streamlit app is designed to be a user-friendly and accessible platform for runners. Its primary function is to enable users to search for and view their race photos using bib number search.
In a running race, photographers may capture hundreds of thousands of images. These photos are a valuable memento for the participants, capturing their reviving moments and achievement. However, finding one's own photos amidst such a vast collection can be like looking for a needle in a haystack.
As a marathon runner myself, I understand this challenge firsthand. The desire to easily locate and view my own race photos led me to dream of creating a solution - a web application specifically designed to simplify this process for runners. This service uses advanced AI technology to tag photos with the bib numbers of the runners featured in them.
Earlier before, I joined a half-marathon race in Hong Kong and soon realized that it was painful to search and find my photos one by one (out of 21229 photos) on the official photo website. Therefore, I decided to build on this platform for tagging these race photos. I hope this service not only enhance race experience of runners, but also add value to event organizers by increasing their after-race engagement with runners.
To get started with our platform, simply enter the bib number you're interested in into the search bar, and the application will retrieve all photos where that bib number (partially) appears.
demo-video.mp4
This platform utilizes two main computer vision techniques:
-
Object Detection: This technique is used to locate the bounding box of the bib number in the image. It helps to identify and isolate the area of interest in the image.
A custom YOLOv8 model (YOLOv8l) is built for object detection. YOLOv8 is a new state-of-the-art (SOTA) computer vision model built by Ultralytics. This open-source SOTA model is particularly strong in detecting objects in images with high speed and accuracy.
The model was trained on a combination of self-annotated images and multiple public datasets related to bib-number class available on Roboflow Universe. These public datasets include:
- Thomas Lamalle's Bib Detection
- RBNR's Bib Detector
- Sputtipa's BIP
- Bibnumber's Bibnumber
- Python Vertiefung's Python Vertiefung
- HCMUS's Bib Detection Big Data
- H1 QTGU0's Bib Number
To help improve the ability of our model to generalize and thus perform more effectively on unseen images, image augmentation techniques were applied to expand data size for training. As a result, a total of 8703 images were used to train the model.
Here is an example of our custom-trained model running on a image using Roboflow Inference:
Feel free to check out here if you also want to get a try on using the custom-trained model API! -
Optical Character Recognition (OCR): Once bib number area is isolated from each image using YOLOv8, OCR was used to detect and interpret the characters in the cropped bib number image. After comparing results from different open-source OCR models (e.g. EasyOCR, PaddleOCR, Tesseract OCR, etc), PaddleOCR turned out to be the best "out-of-the-box" tool with the highest accuracy for bib number recognition in our use case. This allows us to tag hundreds of thousands of images in reliable and efficient way.
The main combo of this platform were Streamlit and Google Sheets (for easy collaboration and updates by non-technical users):
● Step 1: Set up a Google Sheet with your data
● Step 2: Use Streamlit to read the data from the Google Sheet
● Step 3: Build the user interface and search functionality (e.g. textbox, pagination using buttons) using Streamlit
p.s. This post motivated me to get my hands dirty to build my app by myself!
To summarize, here’s how to build this platform to tag bib numbers in a few steps:
1. Annotate custom data and create datasets for training (using Roboflow)
2. Train a custom object detection model (YOLOv8) to detect bib number
3. Crop the bib number object(s) from original image
4. Use OCR toolkits (PaddleOCR) to recognize the number in each cropped image
5. Wrap the above steps into a python function and run model inference on cloud (In my case, I'm using GCP Vertex AI instance)
6. Export and save results in a Google Spreadsheet
7. Build a web service on Streamlit Community Cloud ((the free hosting and deployment service) for end-users to respond with the result
Having said that OCR method is applied for bib number recognition, cropped bib number image quality is considered to be a key constraint to affect recognition quality:
-
Lighting, shooting angle and folding of running bib
-
Horizontal alignment of bib number
-
Similar-looking characters like '0' vs. 'O', 'R' vs. '8' and 'M' vs. 'A1'.
In light of these cases, some rules or heuristics have been applied to OCR for performance improvement. For example, only specific characters/digits and bib number length will be processed.
There is a lot of work remaining to build an accurate and responsive end-to-end web application in real-time. For example:
● OCR Accuracy: Fine-tune pre-trained OCR model (e.g. PaddleOCR). One way is to train a custom OCR tool using more labeled bib number data to make it more robust to different conditions.
● Frontend: Create a web interface for users to upload their running race photos.
● Backend: Use a backend framework (e.g. Node.js) to handle file uploads. Store the uploaded images in a cloud storage service like AWS S3 or GCP Storage.
● Real-time Updates: Use WebSockets to push updates to the frontend in real-time whenever the model makes a new prediction.
● Deployment: Containerize application using Docker for easy deployment and scalability. Use an advanced app hosting service like Streamlit in Snowflake or Google App Engine with zero server management and zero configuration deployments.
This project is open source and freely accessible under MIT license.
If you have suggestions for improvements or want to report a bug, please open an issue. Thank you for your contribution.