Digital Signals and Time Series Data Fundamentals Summary

Time Series Basics

• Real-world signals (sound, light, temperature) are continuous in value and time/space, meaning they exist at every point in time and space and can take on any value within a certain range.

• Digital signal processing requires sampling to convert continuous functions into numerical arrays, involving quantization in both value and time/space. This process approximates continuous signals to make them computationally manageable.

# Example: Converting a continuous signal to a discrete representation
import numpy as np
import matplotlib.pyplot as plt

# Simulate a continuous signal (sine wave)
fs = 100  # Sampling rate
t = np.linspace(0, 1, fs, endpoint=False)  # Time vector
freq = 5  # Frequency of the sine wave
x = np.sin(2*np.pi*freq*t)  # Sine wave signal

# Plot the continuous signal
plt.figure(figsize=(10, 4))
plt.plot(t, x)
plt.xlabel('Time (s)')
plt.ylabel('Amplitude')
plt.title('Continuous Sine Wave Signal')
plt.show()

Sampling

• Sampling involves regular measurements at fixed time intervals, creating a sequence of measurements x[t]. These measurements form a discrete representation of the continuous signal.

• Time quantization: Capturing values at specific time instants (e.g., daily temperature). This discretization of time is crucial for digital representation.

• Amplitude quantization: Restricting measurements to a fixed set of values (e.g., rounding temperature to the nearest degree). This reduces the infinite possible values to a finite set.

• Sampling rate (fs): Frequency of sampling; \Delta T = 1/fs is the time spacing between samples. The sampling rate must be chosen carefully to avoid losing information.

# Example: Sampling a continuous signal
fs = 10  # Sampling rate (samples per second)
t = np.arange(0, 1, 1/fs)  # Time vector
x = np.sin(2*np.pi*5*t)  # Sine wave signal

# Plot the sampled signal
plt.figure(figsize=(10, 4))
plt.stem(t, x)
plt.xlabel('Time (s)')
plt.ylabel('Amplitude')
plt.title('Sampled Sine Wave Signal')
plt.show()

Importance of Sampling

• Efficiently represents continuous functions as arrays, saving storage and enabling efficient algorithms. Digital storage and computation require discrete representations.

• Allows computational work with approximations of analogue signals (sounds, images). Signal processing techniques can then be applied to these digital approximations.

• Standard array operations correspond to useful signal operations (offset removal, mixing, correlation). This allows for straightforward manipulation and analysis of signals.

Noise and Signal-to-Noise Ratio (SNR)

• Measurements introduce noise, with signal x(t) having a true component y(t) and noise \epsilon(t): x(t) = y(t) + \epsilon(t). Noise can arise from various sources, such as sensor inaccuracies or environmental interference.

• SNR is the ratio of signal amplitude to noise amplitude: SNR = S/N, often measured in decibels: SNR{dB} = 10 \log{10}(S/N) = 20 \log_{10}(y(t)/\epsilon(t)). A higher SNR indicates a cleaner signal.

# Example: Calculating SNR
import numpy as np

# Generate a signal
signal = np.array([0.5, 1.0, 1.5, 2.0, 2.5])

# Generate noise
noise = np.array([0.1, 0.2, -0.1, 0.1, 0.05])

# Calculate signal power and noise power
signal_power = np.mean(signal ** 2)
noise_power = np.mean(noise ** 2)

# Calculate SNR
snr = signal_power / noise_power
snr_db = 10 * np.log10(snr)

print(f"SNR: {snr:.2f}")
print(f"SNR (dB): {snr_db:.2f}")

Filtering

• Removes noise based on assumptions about signal behavior, requiring models to separate signal from noise. Filtering techniques rely on understanding the characteristics of both the signal and the noise.

• Incorrect assumptions can damage the true signal. Overly aggressive filtering can remove important signal components along with the noise.

# Example: Simple Moving Average Filter
import numpy as np

def moving_average(data, window_size):
    if len(data) < window_size:
        raise ValueError("Window size is larger than data size")

    cumulative_sum = np.cumsum(data)
    cumulative_sum = np.insert(cumulative_sum, 0, 0)

    moving_averages = (cumulative_sum[window_size:] - cumulative_sum[:-window_size]) / window_size
    return moving_averages

# Example usage:
data = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
window_size = 3

filtered_data = moving_average(data, window_size)
print("Original Data:", data)
print("Moving Average Filtered Data:", filtered_data)

Amplitude Quantization

• Discretizes signal amplitude, reducing it to distinct values. This process is essential for digital representation but introduces quantization error.

• Coarser quantization saves space but increases noise; levels often quoted in bits (e.g., 8 bits = 256 levels). The trade-off between storage space and signal fidelity must be considered.

Sampling Theory and Nyquist Limit

• Continuous signals can be perfectly reconstructed from samples if the signal's highest frequency is no more than half the sampling rate. This is a fundamental principle in signal processing.

• Nyquist limit: fn = fs / 2. Sampling below this limit leads to irreversible information loss.

Aliasing

• Occurs if signals are not sampled frequently enough, causing high-frequency components to fold over. High-frequency components are misinterpreted as lower frequencies.

• Manifests as artifacts like the wagon wheel effect in video. This distortion can severely affect the quality of the reconstructed signal.

Filtering and Smoothing

• Filtering exploits temporal structure to remove noise and simplify signals. Temporal structure refers to the dependencies between successive data points.

• Simple assumption: True signals change slowly, with noise being independent at each step. This assumption underlies many basic filtering techniques.

Moving Averages

• Averages samples in a sliding window to smooth signals. This reduces the impact of short-term fluctuations.

• A simple model assumes tomorrow will be like today i.e. expects true signals to change slowly. This is a basic form of temporal smoothing.

Sliding Window

• Reduces a signal to fixed-length vectors, processing them as a data matrix. This allows for the application of matrix-based analytical techniques.

• Enables standard vector operations on signals. Operations like PCA and SVD can then be used to analyze the signal.

Nonlinear Filtering

• Filters that are not weighted sums, such as median filters, which use the median value in each window. These filters are useful when noise deviates significantly from normal distribution.

• Robust to outliers and extreme values. Median filters are less sensitive to extreme values than moving average filters.

Convolution and Linear Filters

• Linear filters: Output is a weighted sum of neighboring values. The weights determine the filter's characteristics.

• Achieves wide range of effects, efficiently implemented with multiply and accumulate instructions. Convolution is a fundamental operation in signal processing.

Example: f(x[t]) = 0.25x[t - 1] + 0.5x[t] + 0.25x[t + 1]

Convolution

• Taking weighted sums of neighboring values. This operation combines two signals to produce a third.

• Denoted by a star: f * g. The asterisk notation represents the convolution operation.

• For discrete signals: (x * y)[n] = \sum x[n - m]y[m]. m=-M to M. The summation is over a finite range determined by the length of the convolution kernel.

• Powerful tool for blurring, sharpening, and filtering. Convolution can be used for a wide range of signal processing tasks.

Convolution Kernel

• An array defining the convolution operation; the window shape determines the weights applied. The kernel determines how each sample influences its neighbors.

Convolution Properties

• Commutative: f * g = g * f

• Associative: f * (g * h) = (f * g) * h

• Distributive: f * (g + h) = f * g + f * h

Convolution as a Linear Operation

• Can be seen as a matrix-vector multiply with a specially constructed matrix. This perspective is useful for understanding the mathematical properties of convolution.

Dirac Delta Function

• Zero everywhere except at 0, where it is 1; satisfies \int_{-\infty}^{\infty} \delta(x) = 1. This is an idealized impulse function.

• In discrete time, it is a vector of zeros with a single 1 at the center. This discrete representation is used in digital signal processing.

Convolution with Delta Function

• The convolution of any function with the delta function does not change the original function: f(x) * \delta(x) = f(x). The delta function acts as an identity element for convolution.

• Used in linear system identification by feeding a perfect impulse into a system to recover the convolution kernel. This reveals the system's response to an impulse.

Impulse Response Recovery

• Extracts reverberation of an environment by applying a Dirac delta function in the audio domain. This technique is used to characterize the acoustic properties of spaces.

Frequency Domain

• Signals viewed as a sum of oscillations with different frequencies. This perspective provides insights into the signal's composition.

Pure Oscillation

• A sine wave with a specific period. Sine waves are the fundamental building blocks of many signals.

• Represented as x(t) = A \sin(2\pi\omega t + \theta), with amplitude A, frequency \omega, and phase \theta. These parameters define the sine wave's characteristics.

Fourier Transform

• Decomposes repeating functions into sine waves. This transform reveals the frequency components of a signal.

Correlation between Signals

• Measures how alike two sampled signals are by computing the elementwise product and summing it. This quantifies the similarity between two signals.

Fourier Transform as Correlation

• Generates sine waves of every possible frequency and correlates each with a test signal to find the amplitude and magnitude of response. This illustrates how the Fourier transform identifies frequency components.

Fourier Transform Equation

• Formally defined as: \hat{f}(\omega) = \int_{-\infty}^{\infty} f(x) e^{-2\pi i x \omega} dx , giving a complex value for each frequency. This equation provides a mathematical representation of the Fourier transform.

Discrete Fourier Transform (DFT)

• Discrete version of the Fourier transform for a vector of data:

• F[k] = \sum{j=0}^{N-1} x[j] e^{-2\pi i j/N} = \sum{j=0}^{N-1} [x[j] \cos(-2\pi j/N) + ix[j]\sin(-2\pi j/N)]

DFT Components

• The result of the DFT is a sequence of complex numbers F[k] = a + bi, with magnitude A and phase \theta.

• Frequency is given by the index: X0=0, XN /= fn = fs/2. This relates the index in the DFT output to the corresponding frequency.

• Real frequency is freq = f_N k/N.

Spectra

• Magnitude spectrum: Magnitude of each component as a function of frequency. This represents the strength of each frequency component.

• Phase spectrum: Phase of each component. This represents the phase shift of each frequency component.

Linearity of the DFT

• A linear operation that can be seen as constructing a matrix and multiplying it by a vector. This property simplifies analysis and implementation.

• The Fourier transform is a rotation.

Key Algorithm: FFT

• Fast Fourier Transform (FFT) computes DFT efficiently with O(N log(N)) time complexity. This algorithm drastically reduces computation time for large datasets.

Convolution Theorem

• The Fourier transform of the convolution of two signals is the product of their Fourier transforms:

• FT(f(x) * g(x)) = FT(f(x))FT(g(x))

Filter Types

• Smoothing (lowpass): Reduces high frequencies. This type of filter allows low-frequency components to pass through while attenuating high-frequency components.

• Highpass: Reduces low frequencies. This type of filter allows high-frequency components to pass through while attenuating low-frequency components.

• Bandpass: Reduces frequencies outside a band. This type of filter allows frequencies within a specific range to pass through while attenuating frequencies outside that range.

• Notch (bandstop): Reduces frequencies inside a band. This type of filter attenuates frequencies within