MATLAB Data Filtering

Data Cleaning

Data cleaning is the process of identifying and correcting missing, invalid, inconsistent, duplicate, or unusual data before analysis.

Missing Data

Missing data refers to observations for which a value is absent or unavailable in a dataset.

Missing Value

A missing value is a MATLAB representation of unavailable data, whose exact representation can depend on the data type.

ismissing

ismissing returns a logical array identifying elements that contain missing data.

rmmissing

rmmissing removes entries containing missing data from an array or table.

fillmissing

fillmissing replaces missing data using a specified method, such as a constant value, a neighboring value, or an interpolated value.

Missing-Data Imputation

Missing-data imputation is the process of replacing missing observations with estimated or otherwise specified values instead of removing them.

Duplicate Data

Duplicate data consists of repeated observations or records that may need to be identified or removed before analysis.

unique

unique returns the distinct values or rows of an array and can be used to identify or remove duplicate data.

Outlier

An outlier is an observation that differs substantially from the general pattern of the surrounding data.

isoutlier

isoutlier identifies data values that MATLAB considers outliers according to a specified detection method.

rmoutliers

rmoutliers removes detected outliers from data.

filloutliers

filloutliers replaces detected outliers using a specified filling method rather than removing them.

Data Standardization

Data standardization transforms data to a common scale so variables with different magnitudes or units can be compared or analyzed more consistently.

zscore

zscore standardizes numerical data by subtracting its mean and dividing by its standard deviation, producing data expressed in standard-deviation units.

normalize

normalize scales or centers data according to a specified normalization method.

Data Validation

Data validation checks whether data satisfies expected requirements, such as valid ranges, formats, types, or allowable values, before analysis.

Conditional Data Extraction

Conditional data extraction retrieves observations that satisfy specified criteria from a larger dataset.

Range Filtering

Range filtering extracts values or observations that fall within specified lower and upper limits.

Text-Based Filtering

Text-based filtering selects observations according to the contents or patterns of text data.

contains

contains determines whether text contains a specified substring and returns logical values indicating the matches.

Pattern Matching

Pattern matching identifies text that follows a specified textual structure or pattern, allowing matching observations to be located or extracted.

Grouping by Condition

Grouping by condition separates observations according to specified criteria so different subsets of a dataset can be examined independently.