The goal of this project is to design the onboard image segmentation part of a computer vision system for autonomous vehicles.
Image segmentation is the process of classifying each pixel in an image to a particular class (or label). It can be thought of as a classification problem per pixel, but it doesn't distinguish between different instances of the same object; for example, there could be multiple cars in the scene and all of them would have the same label.
- ✔️ Structure the dataset into relevant directories (train, val, test);
- ✔️ Perform Exploratory Data Analysis;
- ✔️ Prepare images processing to be able to segment masks from 30 to 8 main labels;
- ✔️ Generate batches of images (Data Generation);
- ✔️ Choose relevant metrics (and losses);
- ✔️ Train and evaluate different models, from baseline to advanced (including data augmentation);
- ✔️ Deploy model as Flask API on Microsoft Azure.
pip install tensorflow opencv-python flask
- Official CityScapes Github - Marius Cordts and its labels' management;
- Heavy data manipulation : Use of 'Sequence' class with keras, 'Sequence' class with tensorflow, Afshine and Shervice Amidi, Bastien Maurice
- Data augmentation with imgaug or albumentations
- GitHub references for this project : Divam Gupta; Pavel Yakubowskiy; Malte Koche
- arxiv.org and paperswithcode.com : Fully Convolutional Network (FCN); Unet; SegNet; Cityscapes benchmark from 2015 to 2021
- Articles : 2021 guide to Semantic Segmentation - Nanonets, Beginner's guide by Divam Gupta
- Other resources : Simple guide - Bharath Raj, Popular architectures - Priya Dwivedi, Keras Pipeline - Rwiddhi Chakraboty
- Explanation of semantic segmentation metrics and loss functions : Intersection over union; Loss function with Keras and PyTorch
- Flask API deployment on Azure : Microsoft Docs