Data Mining & Machine Learning: Classification Algorithms and Techniques

What is the primary theorem used in Naïve Bayes classification?

Bayes theorem

What assumption does Naïve Bayes make about data features?

That all data features are statistically independent of each other.

What are the three parts of evidence in Naïve Bayes for credit scoring?

Debt level (E1), Income level (E2), Marital Status (E3).

How is the probability calculated in Naïve Bayes?

Pr(E1, E2, E3|H) = Pr(E1|H) * Pr(E2|H) * Pr(E3|H)

What does it indicate if Pr(GR|Joe) > Pr(PR|Joe) in Naïve Bayes?

Joe is classified as a good risk applicant.

What issue arises with Bayes theorem if no training samples exist for a class?

It leads to problems in classification for that feature value.

What is the purpose of Laplace Correction in Naïve Bayes?

To handle the problem of zero probability for unseen feature values.

What is one characteristic of Naïve Bayes models?

They have good explanatory power.

How does Naïve Bayes handle missing values in the training dataset?

By not including these instances in the probability estimate calculations.

What is the procedure for constructing Decision Trees?

Top down in a recursive divide and conquer fashion.

What is the criterion for selecting attributes in Decision Trees?

Choose the attribute that results in the smallest tree and produces the purest nodes.

What is the popular purity criterion used in Decision Trees?

Information gain.

How is information gain calculated?

Information before splitting minus information after splitting.

What does entropy measure in the context of Decision Trees?

The information required to predict an event, measured in bits.

What happens if a leaf node in a Decision Tree has instances from multiple classes?

The leaf node is labeled with the majority class.

What are some advantages of Decision Tree classifiers?

Inexpensive to construct, extremely fast at classifying unknown records, and easy to interpret.

What are the two problems that can arise with models developed in Data Mining?

Overfitting and underfitting.

What is underfitting in the context of data models?

When the model has not fully learned all the patterns in the data, resulting in poor prediction accuracy.

What is the significance of the 'purest' nodes in Decision Trees?

They indicate that the instances in the node belong to a single class.

What is the role of the attribute selection heuristic in Decision Trees?

To choose the attribute that produces the greatest information gain.

What does it mean if a Decision Tree is 'stopped'?

The process halts when data cannot be split further or when the minimum number of instances is reached.

What is a potential consequence of having many dependent features in Naïve Bayes?

Accuracy could drop substantially, necessitating the use of a Bayesian Network.

What is underfitting?

Underfitting occurs when a model has not fully learned all the patterns in the data, resulting in poor prediction accuracy.

What causes underfitting in a model?

Underfitting is generally caused by the algorithm's inability to find all patterns in the training dataset, such as a decision tree that is not deep enough.

What is overfitting?

Overfitting occurs when a model learns the training data too well, resulting in high accuracy on the training set but poor performance on new data.

What is a characteristic of a model that is overfitting?

In overfitting, training set accuracy is high while test set accuracy drops significantly.

What can cause overfitting in a decision tree?

Overfitting can be caused by noise in the training dataset or a lack of sufficient data to capture certain patterns.

What is the effect of noise on overfitting?

Noise distorts the decision boundary, making it difficult for the model to generalize to new data.

How can overfitting be addressed in decision trees?

Overfitting can be addressed through post-pruning, where the tree is trimmed in a bottom-up fashion to improve generalization.

What is a DataFrame in Pandas?

A DataFrame is a way to store data in rectangular grids, where each row corresponds to an instance and each column contains data for a specific variable.

What are the three main components of a Pandas DataFrame?

The three main components are Data, Index, and Columns.

How can you select data from a Pandas DataFrame using index?

You can select data using .loc for labels and .iloc for positions.

What is the advantage of using NumPy arrays?

NumPy arrays provide efficient storage and better handling of data for mathematical operations, creating homogeneous n-dimensional arrays.

What function can change the shape of a NumPy array?

.reshape() can change the dimensions of a NumPy array at runtime.

What is Scikit-learn?

Scikit-learn is a machine learning library that provides a wide selection of supervised and unsupervised learning algorithms.

What are some robust algorithms provided by Scikit-learn?

Robust algorithms include regression, clustering, decision trees, neural networks, SVMs, and ensemble methods.

What is the purpose of model selection and validation in machine learning?

Model selection and validation are used to evaluate the performance of different models and select the best one based on accuracy and other metrics.

What is the role of feature manipulation in machine learning?

Feature manipulation involves transforming and selecting features to improve model performance.

What is the significance of tuning parameters in machine learning algorithms?

Tuning parameters can significantly affect the accuracy of machine learning models, and identifying sensitive parameters is crucial for optimization.

What is the relationship between classification accuracy and parameter values?

The relationship can be examined by creating a table or graph that shows how accuracy changes with different parameter values.