Computer Vision: Detection, Segmentation, and Applications

Detection and Segmentation

Detection and segmentation are fundamental tasks in computer vision, focusing on identifying objects or regions of interest within an image. These tasks play a pivotal role in applications like autonomous driving, medical imaging, and object tracking. This crucial stage is categorized into two primary tasks:

• Single Object Tasks
• Multiple Object Tasks

Single Object Tasks

Single object tasks focus on analysing or delineating individual objects within an image, with two main objectives:

• Classification: This task involves determining the category or class to which a single object belongs, providing insights into its identity or nature. It is the process of identifying what is in a picture (e.g., identifying that a picture contains a dog).
  • KNN ($K$-Nearest Neighbour) algorithm: May be used for supervised classification.
  • $K$-means clustering algorithm: Can be used for unsupervised classification.
• Classification + Localization: In addition to classifying objects, this task involves precisely localizing the object within the image. This is achieved by predicting bounding boxes that tightly enclose the object to show exactly where it is located.

Multiple Object Tasks

Multiple object tasks deal with scenarios where an image contains multiple instances of objects or different object classes. These tasks aim to identify and distinguish between various objects within the image and include:

• Object Detection:
  • Object detection focuses on identifying and locating multiple objects of interest within the image.
  • It involves analysing the entire image and drawing bounding boxes around detected objects, along with assigning class labels to these boxes.
  • The main difference between classification and detection is that classification considers the image as a whole and determines its class, whereas detection identifies the different objects in the image and classifies all of them.
  • In detection, bounding boxes are drawn around multiple objects and these are labelled according to their particular class.
  • Algorithms typically use extracted features and learning algorithms to recognize instances of an object category.
  • Common algorithms used for object detection include:
    • R-CNN (Region-Based Convolutional Neural Network)
    • R-FCN (Region-based Fully Convolutional Network)
    • YOLO (You Only Look Once)
    • SSD (Single Shot Detector)

Image Segmentation

Image segmentation creates a mask around similar characteristic pixels and identifies their class in the given input image. It helps to gain a better understanding of the image at a granular level.

• Process: Pixels are assigned a class, and for each object, a pixel-wise mask is created in the image. This helps to easily identify each object separately from the others.
• Techniques: Edge detection, which works by detecting discontinuities in brightness, is a technique used in Image segmentation.
• Types of Image Segmentation:
  • Semantic Segmentation: It classifies pixels belonging to a particular class. Objects belonging to the same class are not differentiated. For example, pixels may be identified under the class ‘animals’ but the system does not identify the specific type or individual instance of the animal.
  • Instance Segmentation: It classifies pixels belonging to a particular instance. All objects in the image are differentiated even if they belong to the same class. In this case, pixels are separately masked for each individual object (e.g., separately masking a person, a sheep, and a dog).

High-Level Processing

In the final stage of computer vision, high-level processing plays a crucial role in interpreting and extracting meaningful information from the detected objects or regions within digital images. This advanced processing enables computers to achieve a deeper understanding of visual content and make informed decisions based on the visual data.

• Core Tasks: Tasks involved in high-level processing include:
  • Recognizing objects.
  • Understanding scenes.
  • Analysing the context of the visual content.
• Methodology: Through sophisticated algorithms and machine learning techniques, computers can identify and categorize objects, infer relationships between elements in a scene, and derive insights from complex visual data.
• Purpose: Ultimately, high-level processing empowers computer vision systems to extract valuable insights and drive intelligent decision-making in various applications, ranging from autonomous driving to medical diagnostics.

Applications of Computer Vision

Computer vision principles are already integrated into major products used daily. Key applications include:

• Facial Recognition: Popular social media platforms like Facebook use facial recognition to detect and tag users.
• Healthcare:
  • Assists in evaluating cancerous tumours.
  • Identifying diseases or abnormalities.
  • Facilitating object detection and tracking in medical imaging.
• Self-Driving Vehicles:
  • Makes sense of surroundings by capturing video from different angles around the car.
  • Detects other cars and objects.
  • Reads traffic signals and identifies pedestrian paths.
• Optical Character Recognition (OCR): Extracts printed or handwritten text from visual data such as images or documents (e.g., invoices, bills, articles).
• Machine Inspection:
  • Detects a machine's defects, features, and functional flaws.
  • Determines inspection goals.
  • Selects lighting and material-handling techniques.
  • Identifies irregularities in manufactured products.
• $3$D Model Building: Constructing $3$D computer models from existing objects. This has applications in robotics, autonomous driving, $3$D tracking, $3$D scene reconstruction, and AR/VR (Augmented Reality/Virtual Reality).
• Surveillance: Live footage from CCTV cameras in public places helps to identify suspicious behaviour, identify dangerous objects, and prevent crimes by maintaining law and order.
• Fingerprint Recognition and Biometrics: Detects fingerprints and biometrics to validate a user's identity.