Data Science Quiz 5 - K-means, Logistic Regression and Model Complexity

Which of the following is/are true (select all that apply)

a) computing of new centers sometime can lead to higher total loss

b) At each iteration of the algorithm, the total loss will likely decrease

c) It is possible that total loss will start to diverge after many iterations

d) computing of new centers always likely to lead to lower total loss

e) It is possible that total loss will converge to a fixed value after many iterations

b) At each iteration of the algorithm, the total loss will likely decrease

d) computing of new centers always likely to lead to lower total loss

e) It is possible that total loss will converge to a fixed value after many iterations

Consider the following simulation of the k-means algorithm. 

The image on the left shows the initial choice of three centers (k=3) and the image on the right show the location of centers after several iterations. Which of the following statements are true in regards to observation above? Choose all that apply.

a) This seems to be a normal convergence of centers into the best positions

b) using k-means++ would have given better outcomes than what is observed

c) Another choice of initial centers may have given a better clustering of points

d) This is a classic case of error getting stuck at a local minima

b) using k-means++ would have given better outcomes than what is observed

c) Another choice of initial centers may have given a better clustering of points

d) This is a classic case of error getting stuck at a local minima

Which of the following statements are TRUE about k-means? (select all)

a) The cluster center must always be one of the input points

b) Algorithm can terminate when the loss function does not change much from iteration to iteration

c) It is always possible to know how many clusters are needed for a given unlabeled data set

d) Terminate the algorithm when cluster centers do not change much from iteration to iteration

e) When the difference between pervious error and current error has reached below a threshold, terminate the algorithm

b) Algorithm can terminate when the loss function does not change much from iteration to iteration

d) Terminate the algorithm when cluster centers do not change much from iteration to iteration

e) When the difference between pervious error and current error has reached below a threshold, terminate the algorithm

Given the points (0,0), (0,1), (1,0), (2,2), (3,3) in R2, and choosing (1.1) and (3,2) as initial clusters for k=2, after one iteration, what are the new cluster centers?

a) (3/2, 3/2) and (2/5, 2/5)

b) (1/2, 1/2) and (3/5, 2/5)

c) (1, 1) and (3, 2)

d) (2/3, 2/3) and (5/2, 5/2)

d) (2/3, 2/3) and (5/2, 5/2)

In multi-class logistic regression with 3 classifiers  h₁(θ),  h₂(θ), h₃(θ)), a new observation x resulted in the following probabilities corresponding to categories (k=1, k=2, k=3).

P( y=1 | x, θ)  = g (h₁(θ)) = 0.25

P( y=2 | x, θ)  = g (h₂(θ))  = 0.24

P( y=3 | x, θ)  = g (h₃(θ))  = 0.25         

Which of the following might be true if you decide to choose the category for new x? Choose all that apply.

a) Classifiers seems too weak to differentiate this observation

b) It is likely that new observation x belongs to category 1

c) It is likely that new observation x belongs to category 2

d) choose the class with the highest prior probability

e) It is likely that new observation x belongs to category 3

a) Classifiers seems too weak to differentiate this observation

d) choose the class with the highest prior probability

Which of the following is a common issue with using logistic regression for classification problems? Choose all that apply.

a) It requires a large amount of training data to achieve good performance.

b) It is sensitive to outliers in the data.

c) It is prone to overfitting when the number of features is high.

d) It can only model linear decision boundaries between classes.

b) It is sensitive to outliers in the data.

c) It is prone to overfitting when the number of features is high.

d) It can only model linear decision boundaries between classes.

Consider the following plot of x = mean area versus y = concavity.

Which of the following model is a good fit to data? The function g is the sigmoid function and malignant cells need to be label as 1 and benign cells as 0. 

a) g(0.1 y - 1000 x)

b) g(0.1 y - 1000 x - 1)

c) g(y + 0.10 x - 1000)

d) g(y + 0.0001 x - 0.1)

d) g(y + 0.0001 x - 0.1)

Consider a machine learning model that is used to predict the price of houses. The model is trained on a dataset of historical house sales data. The model has a high bias, and a low variance. Which of the following BEST describes the model performance? Choose all that apply.

 a) the model will be able to learn the true relationships between the features and the target variable, leading to good performance

b) the model will learn the underlying patterns in the training data and is therefore able to generalize to new data points.

c) the model is too simple and is not able to capture the complex relationships between the input features and the target variable.

d) the model is not able to adapt to the specific characteristics of each house and will make similar predictions for houses with very different features.

c) the model is too simple and is not able to capture the complex relationships between the input features and the target variable.

d) the model is not able to adapt to the specific characteristics of each house and will make similar predictions for houses with very different features.

Which of the following is the best approach to choosing the optimal model complexity for a given problem?

 

a) Use the most complex model available and then regularize it to prevent overfitting.

b) Use a simple heuristic to determine the appropriate model complexity based on the size of the dataset.

c) Use the least complex model available and then increase its complexity until it achieves the desired performance.

d) Experiment with a range of model complexities and choose the one that achieves the best performance on a validation or test set.

d) Experiment with a range of model complexities and choose the one that achieves the best performance on a validation or test set.

explanation: Experimenting with a range of model complexities and choosing the one that achieves the best performance on a validation set is generally the best approach to choosing the optimal model complexity for a given problem. This approach allows us to balance the tradeoff between bias and variance and choose a model complexity that generalizes well to unseen data.

Which of the following factors can influence the bias-variance tradeoff in machine learning? Choose only the most likely answers. 

a) Regularization techniques

b) The amount of computational resources available

c) The choice of training algorithm

d) with more data, the model can learn more complex relationships between the input features and the target variable

e) The number and values of the parameters in a model

a) Regularization techniques

d) with more data, the model can learn more complex relationships between the input features and the target variable

e) The number and values of the parameters in a model