Microsoft Azure AI Fundamentals: Computer Vision

Computer Vision

A core area of AI that focuses on creating solutions that let AI applications “see” the world and make sense of it.

Image

Is an array of numeric pixel values

Filters

Are applied to modify the pixel values of the image to create a visual effect

Kernels

Are an array of pixel values, and are used to define a filter

Convolutional Neural Network (CNN)

An architecture for computer vision which uses filters to extract numeric feature maps from images, and then feed the feature values into a deep learning model to generate a label prediction.

Transformers

A type of neural network architecture which works by processing huge volumes of data, and encoding language tokens (representing individual words or phrases) as vector-based embeddings (arrays of numeric values)

Multi-modal models

Model is trained using a large volume of captioned images, with no fixed labels. It encapsulates relationships between natural language token embeddings and image features

Azure AI vision

A service that provides prebuilt and customizable computer vision models that are based on the Florence foundation model and provide various powerful capabilities.

Azure AI Vision supports multiple image analysis capabilities, including:

• Optical character recognition (OCR) - extracting text from images.

• Generating captions and descriptions of images.

• Detection of thousands of common objects in images.

• Tagging visual features in images

Training custom models

if the built-in models don’t meet your needs, you can use the service to train a custom model for image classification or object detection

Face Detection

Involves identifying regions of an image that contain a human face, typically by returning bounding box coordinates that form a rectangle around the face

Face Analysis

Facial features can be used to train machine learning models to return other information, such as facial features such as nose, eyes, eyebrows, lips, and others.

Facial Recognition

Identifies known individuals from their facial features.

Azure Face

A service that can return the rectangle coordinates for any human faces that are found in an image, as well as a series of attributes related to those faces:

• Accessories: indicates whether the given face has accessories.

• Blur: how blurred the face is, which could indicate how likely the face is to be the focus of the image.

• Exposure: aspects such as underexposure or over exposure; this is applied to the face, not the exposure of the overall image.

• Glasses: can identify whether the person is wearing glasses.

• Head Pose: the face’s orientation in a 3D space.

• Mask: indicates whether or not the person is wearing a mask

• Noise: any virtual noise in the image; meaning any grain that makes the image less clear.

• Occlusion: determines if there may be objects blocking the face in the image.

Azure AI Video Indexer

A service which you can use to detect and identify faces in a video.

Limited Access Policy

A policy that requires customers to submit an intake form to access additional Azure AI Face services such as comparing faces for similarity and identifying named individuals in an image

Optical Character Recognition (OCR)

Enables artificial intelligence systems to read text in images, enabling applications to extract information from photographs, scanned documents, and other sources of digitized text.

Read API

An OCR engine that powers text extraction from images, PDF’s and TIFF files. Is optimized for images with a significant amount of text or has considerable visual noise. It can automatically determine the proper recognition model to use, considering the number of text lines, images that include text, and handwriting.

Results are arranged in a hierarchy:

• Pages - One for each page of text, including information about the page size and orientation.

• Lines - The lines of text on a page

• Words - The words in a line of text, including the bounding box coordinates and text itself.

Vision Studio

A graphical user interface and enables customers to try out Azure AI Vision