Data Science Lecture 9

Which of the following is the main purpose of logistic regression?

Modeling the probability of a binary outcome

The logistic regression model estimates the probability using which transformation?

Logit transformation

In binary logistic regression, the dependent variable is

Binary

Which of the following metrics is typically used to evaluate a logistic regression model?

Log-Loss

Which estimation method is commonly used to fit logistic regression models?

Maximum Likelihood Estimation

A major assumption in logistic regression is:

Linearity between predictors and the logit of the outcome

What is the range of the predicted probabilities in logistic regression?

[0, 1]

The odds of success are defined as:

p / (1 – p)

Which of the following is not a consequence of multicollinearity in logistic regression?

Increased model accuracy

What is the purpose of using a confusion matrix in logistic regression?

Evaluate classification performance

Which method can be used to handle class imbalance in logistic regression?

Oversampling minority class

Regularization in logistic regression is used to:

Penalize complex models and prevent overfitting

What does a coefficient of zero imply in logistic regression?

Predictor has no effect on log-odds

Which metric is commonly used to evaluate the discrimination ability of a logistic regression model?

AUC (Area Under the ROC Curve)

What does the Brier Score measure in logistic regression?

Probability calibration accuracy

The confusion matrix is primarily used to compute which of the following?

Classification Accuracy

Which metric reflects the proportion of true positives among all predicted positives?

Precision

Which metric is used to balance the tradeoff between precision and recall?

F1 Score

What does the ROC curve plot?

True Positive Rate vs False Positive Rate

If a model has a high AUC but low calibration, what does it mean?

The model distinguishes well but probabilities may be inaccurate

Which of the following is not a threshold-dependent metric?

AUC

What is the typical range of the Brier Score?

0 to 1

Which metric would you use to assess the overall probability accuracy of your logistic model?

Log-loss

Which of the following is true about log-loss?

It is minimized when predicted probabilities match actual outcomes

What is the primary limitation of accuracy as a performance metric in imbalanced classification problems?

It overestimates performance when the majority class dominates

Which of the following metrics is based on squared error between predicted probabilities and actual outcomes?

Brier Score

In logistic regression, what is the consequence of using only accuracy for model assessment in skewed datasets?

It may hide poor performance on the minority class

Which statement is true about AUC (Area Under the Curve)?

AUC is robust to class imbalance

Why is imbalanced data a problem in logistic regression?

It can lead to biased predictions toward the majority class

Which of the following metrics is least appropriate for evaluating logistic regression on imbalanced data?

Accuracy

What is a common strategy to handle imbalanced data in logistic regression?

Use resampling techniques such as SMOTE or undersampling

Which method adds synthetic examples to the minority class to help balance the data?

SMOTE (Synthetic Minority Over-sampling Technique)

In a dataset with 95% class 0 and 5% class 1, a logistic regression model that predicts only class 0 will have:

High accuracy but poor class 1 performance

Which of the following can help adjust the decision threshold in logistic regression to address class imbalance?

Modifying the default probability cutoff from 0.5 to a lower value

Which metric specifically evaluates the performance of the minority class in imbalanced data?

F1-score

What is the role of class weights in logistic regression on imbalanced data?

To give equal priority to both classes during training

When using stratified sampling in cross-validation for imbalanced data, it ensures that:

The class distribution is preserved in each fold

Which of the following statements about logistic regression and imbalanced data is false?

Logistic regression coefficients become meaningless when data is imbalanced

What is the purpose of recoding a categorical variable in logistic regression?

To convert it into a numerical format for modeling

Which of the following is a common method for encoding nominal categorical variables in logistic regression?

Dummy variables

Why is one level of a categorical predictor dropped when creating dummy variables in logistic regression?

To avoid multicollinearity

What is the term for the dropped level in simple dummy encoding in logistic regression?

Reference category

Which of the following is true about using unordered categorical variables in logistic regression?

They should be recoded as dummy variables

Which encoding method is typically used when the categorical variable has an ordinal structure?

Integer encoding (label encoding)

Which of the following would NOT be a good strategy for handling a categorical variable with many levels in logistic regression?

Ignoring all levels

Which type of categorical variable should be recoded using simple dummy encoding in logistic regression?

Nominal

If you have a categorical variable with 5 levels, how many dummy variables will be included in the logistic regression model?

4

In logistic regression, what does the coefficient for a dummy variable represent?

The log odds ratio relative to the reference category

Which method is inappropriate for encoding a nominal categorical variable for logistic regression?

Label encoding (1, 2, 3...)

Which of the following coding schemes can be used for categorical variables in logistic regression?

All of the above

What is the risk of using label encoding (1, 2, 3...) on nominal variables in logistic regression?

Misleading interpretation due to artificial ordering

What problem can arise when a categorical variable has a level with very few observations?

Convergence issues or unstable coefficient estimates

Which of the following approaches can help simplify high-cardinality categorical variables?

Grouping similar categories

In logistic regression, when interpreting coefficients of dummy variables, what is being compared?

Each level to the reference category

Which of the following is a classical method used for variable selection in logistic regression?

Forward selection

In forward selection, variables are added to the model based on:

Their p-values or contribution to model improvement

Which variable selection method starts with all predictors and removes the least significant ones iteratively?

Backward elimination

Which of the following criteria is commonly used to guide variable selection in logistic regression?

AIC (Akaike Information Criterion)

Stepwise selection in logistic regression combines which of the following?

Forward and backward selection

Which regularization technique shrinks some coefficients to exactly zero, effectively performing variable selection?

Lasso regression

Why might too many variables in a logistic regression model be problematic?

It may lead to overfitting

Which of the following does not perform variable selection but helps reduce overfitting?

Ridge regression

The use of VIF (Variance Inflation Factor) in logistic regression helps detect:

Multicollinearity among predictors

In practice, variable selection is important in logistic regression because:

It enhances model interpretability and generalization

If the log-odds of an estimator is 0.25, the corresponding probability of this odds is approximately

0.5. (True/False)

False

If the odds of an estimator is 0.25, the corresponding probability of this odds is 0.2. (True/False)

True

In logistic regression, we use the least squares method to solve the likelihood equation. (True/False)

False

The range of log-odds is between 0 and 1. (True/False)

False

The odds of a given probability can be a negative number. (True/False)

False

In logistic regression, we use the maximum likelihood methodology to solve the likelihood equation. (True/False)

True

To evaluate the contribution of a specific predictor to the classification model, three statistical tests can be utilized: the F-test, t-test, and test. (True/False)

False

Wald test statistics (W) follows an asymptotic -distribution with one degree of freedom under the null hypothesis . (True/False)

True

The score test statistic can be calculated as the squared score divided by the Fisher information matrix in logistic regression. (True/False)

True

The Brier score used in logistic regression ranges from 0 to 1, with a lower score indicating better calibration and higher accuracy. (True/False)

True

Lower Brier scores suggest better predictive performance and more accurate probability estimates in logistic regression. (True/False)

True

Oversampling can produce a balanced training sample from imbalanced data using sampling without replacement. (True/False)

False

An imbalanced distribution in the target variable can result in issues such as biased model performance when conducting classification modeling. (True/False)

True

An imbalanced distribution in the target variable can result in issues such as model skewness and overfitting when conducting classification modeling. (True/False)

True

An imbalanced distribution in the target variable can result in issues such as decreased sensitivity when conducting classification modeling. (True/False)

True

Suppose that variable “Source” has twenty-five distinct levels. Dummy variable recoding without a threshold can increase the dimension of the data by twenty-three using simple dummy coding. (True/False)

False

Dimension inflation is not a significant problem in data mining if the number of parameters to be estimated is smaller than the number of dimensions. Therefore, reducing the number of levels of a categorical variable with many levels is not necessary. (True/False)

False

The coefficient of the intercept for a categorical variable using “simple dummy coding” is the average of the response variable. (True/False)

False

The coefficient of the intercept for a categorical variable using “deviation dummy coding” is the average of the response variable. (True/False)

False

The coefficient of the intercept for a categorical variable using “Hermert dummy coding” is the average of the response variable. (True/False)

False

A model that includes all available predictors in the dataset is always the best model. (True/False)

False

Suppose the best model with k variables always has the same k variables, regardless of whether the model selection criterion used is AIC, BIC, SBC, or others. (True/False)

False

The optimal set of k variables chosen through stepwise selection may differ from the set of k variables selected using either the "Backward" or "Forward" selection methods. (True/False)

True

Including an unnecessary predictor in the model is always a good idea since the model is higher after adding an extra predictor. (True/False)

False

A positive coefficient within the logistic regression equation indicates that as the numerical predictor’s value rises, there is a corresponding rise in the log-odds of the event occurring, which consequently results in an elevated probability of the event taking place. (True/False)

True

A positive coefficient within the logistic regression equation indicates that for a specific category, the log-odds are higher compared to the reference category when utilizing simple dummy coding. (True/False)

True