Modeling Wine Preferences by Data Mining from Physicochemical Properties

Study Overview: Modeling Wine Preferences via Data Mining

• Research Objective: The primary goal is to propose a data mining (DM) approach to predict human wine taste preferences based on analytical, physicochemical tests typically available during the wine certification process.
• Significance of Study: Evaluating wine quality is crucial for certification (preventing adulteration), improving production processes, and stratifying wines for pricing (e.g., premium brands).
• Authors: Paulo Cortez, António Cerdeira, Fernando Almeida, Telmo Matos, and José Reis, representing the University of Minho and the Viticulture Commission of the Vinho Verde Region (CVRVV).
• Key Findings: The support vector machine (SVM) outperformed multiple regression (MR) and neural networks (NN) in predicting taste preferences, particularly for white Vinho Verde wines.

The Wine Industry and Certification Context

• Economic Context: Portugal is a top 10 wine exporting country, holding a $3.17\%$ market share in 2005. Exports of Vinho Verde wine increased by $36\%$ between 1997 and 2007.
• Vinho Verde Profile: This unique wine from the northwest (Minho) region of Portugal is known for its freshness and medium alcohol content. It accounts for $15\%$ of Portuguese wine production, with $10\%$ of it being exported (predominantly white wine).
• Quality Assessment Methods:
  • Physicochemical Tests: Objectives tests measuring density, alcohol, pH, acidity, etc.
  • Sensory Tests: Subjective tests relying on human experts. Taste is the least understood human sense, making classification difficult.
• Certification Process: Used to protect human health and ensure market quality. The CVRVV manages sample testing through the iLab computerized system.

Data Mining Methodology and Modeling Techniques

• Classic Approach: Linear/Multiple Regression (MR) is the standard for modeling continuous data due to its interpretability.
• Neural Networks (NN): Based on the multilayer perceptron (MLP) architecture using the backpropagation algorithm (popularized in 1986). It uses hidden layers to model complex, nonlinear relationships.
  • Equation for Prediction:     $\hat{y} = w_{o;0} + \sum_{j=I+1}^{o-1} \frac{1}{1 + \exp(-\sum_{i=1}^{I} x_i w_{j;i} - w_{j;0})} w_{o;j}$
• Support Vector Machines (SVM): A highly flexible approach that finds the best linear separating hyperplane in a high-dimensional feature space.
  • Kernel Function: The Gaussian kernel is used: $K(x, x') = \exp(-\gamma ||x-x'||^2)$, where $\gamma > 0$.
  • Hyperparameters: $\gamma$ (kernel parameter), $\epsilon$ (insensitive loss function tolerance), and $C$ (trade-off between flatness and error fit).
• Regression Approach: The study treats taste preferences as continuous values to preserve order (e.g., predicting 4 when the true value is 3 is better than predicting 7).

Dataset Description and Preprocessing

• Sample Size: The study uses a large dataset compared to previous benchmarks:
  • White Wine: $4898$ samples.
  • Red Wine: $1599$ samples.
• Collection Period: May 2004 to February 2007.
• Physicochemical Attributes:
  1. Fixed acidity: $g(\text{tartaric acid})/dm^3$
  2. Volatile acidity: $g(\text{acetic acid})/dm^3$
  3. Citric acid: $g/dm^3$
  4. Residual sugar: $g/dm^3$
  5. Chlorides: $g(\text{sodium chloride})/dm^3$
  6. Free sulfur dioxide: $mg/dm^3$
  7. Total sulfur dioxide: $mg/dm^3$
  8. Density: $g/cm^3$
  9. pH
  10. Sulphates: $g(\text{potassium sulphate})/dm^3$
  11. Alcohol: vol. $\%$
• Target Variable: Sensory score on a scale of $0$ (very bad) to $10$ (excellent), determined by the median of at least three blind tasting evaluations by experts.

Experimental Design and Model Selection

• Variable Selection: Applied backward selection guided by sensitivity analysis to discard irrelevant inputs. This reduces computational effort and simplifies models.
• Simultaneous Model Selection: Hyperparameters (hidden nodes $H$ for NN and $\gamma$ for SVM) were tuned during the variable selection process via parsimony search.
• Validation Methods:
  • Internal Validation: $2/3$ training and $1/3$ validation holdout split.
  • External Evaluation: $20$ runs of robust $5$-fold cross-validation totaling $100$ experiments.
• Performance Metrics:
  • Mean Absolute Deviation (MAD):     $MAD = \frac{\sum_{i=1}^{N} |y_i - \hat{y}_i|}{N}$
  • Regression Error Characteristic (REC) Curves: Plots error tolerance (x-axis) vs. accuracy (y-axis).
  • Accuracy at Tolerance ($T$):
    • $T=0.50$: Rounds the prediction to the nearest class.
    • $T=1.00$: Accepts predictions within the two closest classes.
  • Kappa Statistic: Measures accuracy relative to a random classifier.

Analytical Results and Statistical Comparison

• SVM Performance: Achieved the best results across all metrics and datasets.
  • Red Wine (SVM): MAD = $0.46 \pm 0.00$; Accuracy ($T=0.5$) = $62.4\% \pm 0.4$.
  • White Wine (SVM): MAD = $0.45 \pm 0.00$; Accuracy ($T=0.5$) = $64.6\% \pm 0.4$.
• Model Complexity:
  • NN: Best models used $H \approx 1$ for red and $H \approx 2.1$ for white wine.
  • SVM: Best models used $\gamma = 2^{0.19}$ for red and $\gamma = 2^{1.55}$ for white wine.
• Classification Depth: Precision for extreme classes (3 and 9) was lower ($0\%$) due to data imbalance, but precision for intermediate classes (4-8) with $T=1.0$ exceeded $81.9\%$.

Knowledge Extraction and Input Importance

• Sensitivity Analysis ($R_a$): Measures variable relevance by analyzing model variance (Va) when inputs are manipulated.
  • $V_a = \frac{\sum_{j=1}^{L} (\hat{y}_{aj} - \bar{\hat{y}}_{aj})^2}{L-1}$
  • $R_a = \frac{V_a}{\sum_{i=1}^{I} V_i} \times 100$
• Ranked Influences:
  • Sulphates: Ranked as the #1 most important factor for both red and white wines, likely related to fermenting nutrition and aroma improvement.
  • Alcohol: High impact on quality; higher alcohol typically correlates with higher preference.
  • Volatile Acidity: Negative impact due to acetic acid (vinegar taste).
  • Type Differences: Citric acid and residual sugar are significantly more important in white wine quality (favouring the freshness-sweetness balance).

Practical Implications and Applications

• Decision Support Systems (DSS): The model can assist human tasters by flagging discrepancies. If an expert's grade deviates more than $T=1.0$ from the model prediction, a re-tasting may be required.
• Oenology Training: Useful for guiding students in identifying characteristics of high-quality wines.
• Production Improvement: Producers can optimize quality by monitoring factors like grape sugar before harvest or managing fermentation to control residual sugar and volatile acidity.
• Target Marketing: Similar techniques could model consumer preferences in specific niche markets (e.g., different countries) during promotional campaigns to design market-specific brands.

Key Technical Implementation Details

• Environment: R environment using the RMiner library.
• Training Algorithms:
  • NN: BFGS algorithm (nnet package).
  • SVM: Sequential Minimal Optimization (kernlab package).
• SVM Hyperparameter Heuristics:
  • $C = 3$
  • $\epsilon = \hat{\sigma} \times \sqrt{\frac{\ln(N)}{N}}$
  • $\hat{\sigma} = \frac{1.5}{N} \times \sum_{i=1}^{N} (y_i - \hat{y}_i)^2$ (using 3-nearest neighbor prediction).