Input Design and Model Validation

Input Design

Achieving a Desired $u(t)$

Several methods can be employed to shape the frequency content of a signal $u(t)$.

1) Filtering

• Types: Low-pass, high-pass, band-pass, band-stop filters.
• Control: The steepness of the filter's cutoff can be controlled by the filter order ($n$).
• End-to-End Example:
  • A white signal passes through a filter.
  • The output is then processed, potentially involving rescaling.
  • Filter order $n$ affects the filter's characteristics (e.g., $n = 2$).
  • $\omega_{cutoff}$ represents the cutoff frequency.

2) Upsampling

• Goal: Retain the low-frequency content.
• Process:
  • Start with a white signal $u(t)$.
  • Upsample the signal (e.g., by a factor of 4).
  • This process generates low-frequency components.

3) Sum of Sinusoids

• Equation: $u(t) = \sum<em>{k=1}^{K} A</em>k \cos(\omega<em>k t + \phi</em>k)$
  • $A_k$: Amplitude of the $k^{th}$ sinusoid.
  • $\omega_k$: Frequency of the $k^{th}$ sinusoid.
  • $\phi_k$: Phase of the $k^{th}$ sinusoid.
• Frequency Domain Representation:
  • $U(\omega) \propto \sum<em>{k=1}^{K} A</em>k e^{j\phi<em>k} \delta(\omega - \omega</em>k) + A<em>k e^{-j\phi</em>k} \delta(\omega + \omega_k)$
  • This shows that the frequency spectrum $U(\omega)$ consists of impulses at the frequencies $\omega_k$.
• Power Spectrum: $|U(\omega)|^2 \propto \sum<em>{k=1}^{K} A</em>k^2 [\delta^2(\omega - \omega<em>k) + \delta^2(\omega + \omega</em>k)]$

4) Swept Sinusoid (Chirp)

• Equation: $u(t) = A \cos(\omega(t)t)$
• Instantaneous Frequency: $\omega(t) = \omega<em>1 + t(\omega</em>2 - \omega_1)$
• Chirp Signal: $u(t) = A \cos\left(\omega<em>1 t + \frac{t^2}{2T}(\omega</em>2 - \omega_1)\right)$

Model Validation

Cross-Validation

• Data Set: Split the data set ${u(t), y(t)}$ into training and testing sets.
• Process:
  1. Train the model on the training data.
  2. Use the trained model to make predictions on the test data: $\hat{y}_{test}$.
  3. Validate the model by comparing the predictions with the actual test data.

Overfitting

• Example: Learning an Autoregressive (AR) model.
• AR Model: $y(t) = \theta<em>0 + \theta</em>1 y(t-1) + \theta<em>2 y(t-1)^2 + … + \theta</em>{p-1} y(t-1)^{p-1} + e(t)$
• NAR(p) Model: $y(t) = f(y(t-1)) + e(t)$
  • p represents the order of the AR model.
• Observations:
  • p=1: Medium training error, medium test error.
  • p=2: Very low training error, high test error (overfitting).
  • Overfitting occurs when the model learns the training data too well, including the noise, and performs poorly on unseen data.

Time-Series Data

• Important: In time-series data, the train-test split must respect temporal continuity.
• Order: Maintain the order of the learned model (number of lags).
• Disjoint Segments: Do NOT attach originally disjoint segments until computing $\hat{y}$ (or $E$) on each segment.