Team Xion

README.md

@@ -1,17 +1,86 @@
 # intel-oneAPI
 
-#### Team Name -
-#### Problem Statement - 
-#### Team Leader Email -
+#### Team Name -Team Xion (Extreme Identification Of Navigation)
+#### Problem Statement - Object Detection in Autonomous Vehicles
+#### Team Leader Email - [email protected]
 
-## A Brief of the Prototype:
-  This section must include UML Daigrms and prototype description
+#### Demo Video:- https://www.youtube.com/watch?v=ZCrOj5g974A
+
+## A Brief of the Prototype 🎦:
+  #### Developed a road sign 🛑 detection model using the oneAPI framework in conjunction with the openVINO toolkit. 
+  #### Leveraged the oneAPI libraries and tools for efficient implementation and optimization. 
+  #### Utilized a dataset containing German road sign images for training and evaluation. 
+  #### Trained a deep learning model SSD using the dataset within the Intel oneAPI oneDAL(DATA ANALYTIC LIBRARY). 
+  #### Fine-tuned the model to improve its accuracy in detecting Indian road signs and help autonomous vehicles.
+  #### Converted the trained model to the OpenVINO Intermediate Representation (IR) format for deployment. Utilized openVINO's inference engine to perform real-time road sign detection on different hardware platforms. 
+  #### Achieved high accuracy and real-time performance in road sign detection tasks using the combined power of the oneAPI framework and OpenVINO toolkit, showcasing their synergy in computer vision applications.
 
-## Tech Stack: 
-   List Down all technologies used to Build the prototype **Clearly mentioning Intel® AI Analytics Toolkits, it's libraries and the SYCL/DCP++ Libraries used**
+  ### Diagram
+  ![Diagram](https://github.com/Craniace/intel-oneAPI/assets/100042684/be5c1803-083c-4879-8cde-c2d4ec154092)
+
+
+## Tech Stack ⚙: 
+   * Intel SYCL/C++ Library
+   * Intel Distribution for Python
+   * openVINO for Python
+   * Visual Studio Code
+   * Python 3.11
+   * Intel DevCloud Platform
 
-## Step-by-Step Code Execution Instructions:
-  This Section must contain set of instructions required to clone and run the prototype, so that it can be tested and deeply analysed
+## Step-by-Step Code Execution Instructions 👨🏻‍💻:
+   ### Step 1: Install Required Libraries
+
+- Make sure you have Python installed on your system.
+- Install the necessary libraries, such as OpenCV, openVINO, and os, using pip or conda.
+
+### Step 2: Gather and Preprocess Data
+
+- Collect or create a dataset for training your object detection model.
+- Annotate the dataset by labelling the objects of interest with bounding boxes.
+- Split the dataset into training and testing sets.
+ ![image](https://github.com/Craniace/intel-oneAPI/assets/100042684/24732fa4-96bd-4e73-9739-28273e372c65)
+
+
+### Step 3: Trained model with **oneAPI**
+
+- Train the model using intel oneDAL to get better results and faster computation or Select a pre-trained object detection model that suits your requirements.
+![Intel-toolkit-oneAPI-rendering-scaled-960x500_c](https://github.com/Craniace/intel-oneAPI/assets/100042684/ed06ad19-dcc6-4546-96be-1b1e72a5e914)
+
+### Step 4: Fine-tune the Model
+
+- Load the pre-trained model weights.
+- Replace the classification head with a new head suitable for your specific objects.
+- Freeze the initial layers to retain the pre-trained weights.
+- Train the model using the annotated training dataset.
+- Adjust hyperparameters, such as learning rate and batch size, to optimize performance.
+
+### Step 5: Evaluate the Model
+
+- Evaluate the performance of your trained model using the testing dataset.
+- Measure metrics like precision, recall, and average precision to assess the model's accuracy.
+
+### Step 6: Implement Object Detection
+
+- Use the trained model to perform object detection on new images or videos.
+- Preprocess the input by resizing, normalizing, and converting it to the appropriate format.
+- Pass the preprocessed input through the model to obtain predicted bounding boxes and class labels.
+- Apply non-maximum suppression to remove redundant overlapping bounding boxes.
+- Visualize the detected objects by drawing bounding boxes and labels on the input image or video.
+
 
-## What I Learned:
-   Write about the biggest learning you had while developing the prototype
+## What I Learned 💡:
+ **✅Image processing and computer vision techniques:** Road sign detection often involves applying various image processing and computer vision algorithms to identify and locate signs within images or video streams. This includes techniques like image segmentation, feature extraction, and object detection.
+
+**✅Data collection and preprocessing:** Gathering a diverse dataset of road sign images is crucial for training and evaluating the detection model. You would have learned how to collect and preprocess the data, including labeling the signs and handling data augmentation techniques.
+
+**✅Model selection and training:** Choosing an appropriate model architecture for road sign detection and training it using the collected dataset. This involves understanding different deep learning models and their suitability for the task, selecting loss functions, and optimizing hyperparameters.
+
+**✅Integration of oneAPI tools:** oneAPI provides a unified programming model for diverse hardware architectures. You would have learned how to leverage oneAPI tools, such as oneDNN (oneAPI Deep Neural Network Library) and oneVPL (oneAPI Video Processing Library), to optimize and accelerate the road sign detection pipeline on specific hardware platforms.
+
+**✅Performance optimization:** Road sign detection often requires real-time or near real-time processing, especially in autonomous driving applications. You would have explored optimization techniques to improve the inference speed and efficiency of the detection model, such as model quantization, pruning, and parallelization.
+
+**✅Evaluation and accuracy assessment:** Assessing the performance of the road sign detection model through evaluation metrics like precision, recall, and F1 score. This helps measure the accuracy and effectiveness of the model in correctly identifying road signs.
+
+**✅Deployment and integration:** Integrating the trained road sign detection model into larger systems or applications, such as autonomous vehicles or traffic management systems. This involves considering deployment requirements, performance constraints, and compatibility with existing software or hardware components.
+
+

Video and PPT/Video

@@ -0,0 +1 @@
+Demo Video link:- ### https://www.youtube.com/watch?v=ZCrOj5g974A ####

data/ssdlite_mobilenet_v2_coco_2018_05_09/checkpoint

@@ -0,0 +1,2 @@
+model_checkpoint_path: "model.ckpt"
+all_model_checkpoint_paths: "model.ckpt"

data/ssdlite_mobilenet_v2_coco_2018_05_09/pipeline.config

@@ -0,0 +1,181 @@
+model {
+  ssd {
+    num_classes: 90
+    image_resizer {
+      fixed_shape_resizer {
+        height: 300
+        width: 300
+      }
+    }
+    feature_extractor {
+      type: "ssd_mobilenet_v2"
+      depth_multiplier: 1.0
+      min_depth: 16
+      conv_hyperparams {
+        regularizer {
+          l2_regularizer {
+            weight: 3.99999989895e-05
+          }
+        }
+        initializer {
+          truncated_normal_initializer {
+            mean: 0.0
+            stddev: 0.0299999993294
+          }
+        }
+        activation: RELU_6
+        batch_norm {
+          decay: 0.999700009823
+          center: true
+          scale: true
+          epsilon: 0.0010000000475
+          train: true
+        }
+      }
+      use_depthwise: true
+    }
+    box_coder {
+      faster_rcnn_box_coder {
+        y_scale: 10.0
+        x_scale: 10.0
+        height_scale: 5.0
+        width_scale: 5.0
+      }
+    }
+    matcher {
+      argmax_matcher {
+        matched_threshold: 0.5
+        unmatched_threshold: 0.5
+        ignore_thresholds: false
+        negatives_lower_than_unmatched: true
+        force_match_for_each_row: true
+      }
+    }
+    similarity_calculator {
+      iou_similarity {
+      }
+    }
+    box_predictor {
+      convolutional_box_predictor {
+        conv_hyperparams {
+          regularizer {
+            l2_regularizer {
+              weight: 3.99999989895e-05
+            }
+          }
+          initializer {
+            truncated_normal_initializer {
+              mean: 0.0
+              stddev: 0.0299999993294
+            }
+          }
+          activation: RELU_6
+          batch_norm {
+            decay: 0.999700009823
+            center: true
+            scale: true
+            epsilon: 0.0010000000475
+            train: true
+          }
+        }
+        min_depth: 0
+        max_depth: 0
+        num_layers_before_predictor: 0
+        use_dropout: false
+        dropout_keep_probability: 0.800000011921
+        kernel_size: 3
+        box_code_size: 4
+        apply_sigmoid_to_scores: false
+        use_depthwise: true
+      }
+    }
+    anchor_generator {
+      ssd_anchor_generator {
+        num_layers: 6
+        min_scale: 0.20000000298
+        max_scale: 0.949999988079
+        aspect_ratios: 1.0
+        aspect_ratios: 2.0
+        aspect_ratios: 0.5
+        aspect_ratios: 3.0
+        aspect_ratios: 0.333299994469
+      }
+    }
+    post_processing {
+      batch_non_max_suppression {
+        score_threshold: 0.300000011921
+        iou_threshold: 0.600000023842
+        max_detections_per_class: 100
+        max_total_detections: 100
+      }
+      score_converter: SIGMOID
+    }
+    normalize_loss_by_num_matches: true
+    loss {
+      localization_loss {
+        weighted_smooth_l1 {
+        }
+      }
+      classification_loss {
+        weighted_sigmoid {
+        }
+      }
+      hard_example_miner {
+        num_hard_examples: 3000
+        iou_threshold: 0.990000009537
+        loss_type: CLASSIFICATION
+        max_negatives_per_positive: 3
+        min_negatives_per_image: 3
+      }
+      classification_weight: 1.0
+      localization_weight: 1.0
+    }
+  }
+}
+train_config {
+  batch_size: 24
+  data_augmentation_options {
+    random_horizontal_flip {
+    }
+  }
+  data_augmentation_options {
+    ssd_random_crop {
+    }
+  }
+  optimizer {
+    rms_prop_optimizer {
+      learning_rate {
+        exponential_decay_learning_rate {
+          initial_learning_rate: 0.00400000018999
+          decay_steps: 800720
+          decay_factor: 0.949999988079
+        }
+      }
+      momentum_optimizer_value: 0.899999976158
+      decay: 0.899999976158
+      epsilon: 1.0
+    }
+  }
+  fine_tune_checkpoint: "PATH_TO_BE_CONFIGURED/model.ckpt"
+  num_steps: 200000
+  fine_tune_checkpoint_type: "detection"
+}
+train_input_reader {
+  label_map_path: "PATH_TO_BE_CONFIGURED/mscoco_label_map.pbtxt"
+  tf_record_input_reader {
+    input_path: "PATH_TO_BE_CONFIGURED/mscoco_train.record"
+  }
+}
+eval_config {
+  num_examples: 8000
+  max_evals: 10
+  use_moving_averages: false
+}
+eval_input_reader {
+  label_map_path: "PATH_TO_BE_CONFIGURED/mscoco_label_map.pbtxt"
+  shuffle: false
+  num_readers: 1
+  tf_record_input_reader {
+    input_path: "PATH_TO_BE_CONFIGURED/mscoco_val.record"
+  }
+}

intel2.py

@@ -0,0 +1,101 @@
+import cv2
+import numpy as np
+from openvino.inference_engine import IECore
+import sys
+
+# Load the road sign detection model
+model_xml = 'path/to/road_sign_detection_model.xml'
+model_bin = 'path/to/road_sign_detection_model.bin'
+ie = IECore()
+net = ie.read_network(model=model_xml, weights=model_bin)
+exec_net = ie.load_network(network=net, device_name='CPU')
+
+# Define the classes corresponding to road signs
+classes = ['Speed_limit_20_km/h'
+,'Speed_limit_30_km/h'
+,'Speed_limit__50_km/h'
+,'Speed_limit_60_km/h'
+,'Speed_limit_70_km/h'
+,'Speed_limit_80_km/h'
+,'End_of_speed_limit_(80 km/h)'
+,'Speed_limit_100_km/h'
+,'Speed_limit_120_km/h'
+,'No_passing'
+,'No_passing for vehicles over 3.5 metric tons'
+,'Right-of-way_at_the next_intersection'
+,'Priority_road'
+,'Yield'
+,'Stop'
+,'No_vehicles'
+,'Vehicles_over_3.5_metric_tons_prohibited'
+,'No_entry'
+,'General_caution'
+,'Dangerous_curve_to_the_left'
+,'Dangerous_curve-to_the_right'
+,'Double_curve'
+,'Bumpy_road'
+,'Slippery_road'
+,'Road_narrows_on_the_right'
+,'Construction_zone'
+,'Traffic_signal_ahead'
+,'Pedestrian_crossing'
+,'School_zone'
+,'Bicycles_crossing'
+,'Beware_of_ice/snow'
+,'Wild_animals_crossing'
+,'End_of_all_speed_and_passing_limits'
+,'Turn_right_ahead'
+,'Turn_left_ahead'
+,'Ahead_only'
+,'Go_straight_or_right'
+,'Go_straight_or_left'
+,'Keep_right'
+,'Keep_left'
+,'Roundabout_mandatory'
+,'End_of_no_passing'
+,'End_of_no_passing_by_vehicles_over_3.5_metric _tons']
+
+# Initialize video capture
+cap = cv2.VideoCapture(0)
+
+while True:
+    # Read frame from camera
+    ret, frame = cap.read()
+    if not ret:
+        break
+
+    # Preprocess the frame
+    input_blob = cv2.dnn.blobFromImage(frame, size=(300, 300), ddepth=cv2.CV_8U)
+    input_blob = np.transpose(input_blob, (0, 3, 1, 2))
+
+    # Perform inference
+    outputs = exec_net.infer(inputs={net.input_info['data'].input_name: input_blob})
+    detections = outputs[net.output_info['detection_out'].output_name][0][0]
+
+    # Process the detections
+    for detection in detections:
+        confidence = detection[2]
+        if confidence > 0.5:
+            class_id = int(detection[1])
+            class_name = classes[class_id]
+
+            # Get the coordinates of the detected road sign
+            x1 = int(detection[3] * frame.shape[1])
+            y1 = int(detection[4] * frame.shape[0])
+            x2 = int(detection[5] * frame.shape[1])
+            y2 = int(detection[6] * frame.shape[0])
+
+            # Draw bounding box and label
+            cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2)
+            cv2.putText(frame, class_name, (x1, y1 - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (0, 255, 0), 2)
+
+    # Display the frame
+    cv2.imshow('Road Sign Detection', frame)
+
+    # Exit if 'q' is pressed
+    if cv2.waitKey(1) == ord('q'):
+        break
+
+# Release resources
+cap.release()
+cv2.destroyAllWindows()