ST 531 Exam II Questions

To use the C-statistic to assess model performance, you need to know the cutoff.

True False

False

In Model Studio, when you add any node to a pipeline, the Model Comparison node is added automatically.

True False

False

Which of the following can you specify in the properties panel of the Model Comparison node? (Select all that apply.)

a. the depth to use when applying a lift-based measure

b. the cutoff to use when applying an ROC-based measure

c. the data partition to use for selecting the champion

a. the depth to use when applying a lift-based measure

b. the cutoff to use when applying an ROC-based measure

c. the data partition to use for selecting the champion

When you have an interval target, which of the following fit statistics can you use to select the champion model?

a. area under the curve (C-statistic)

b. average square error

c. cumulative lift

d. misclassification

b. average square error

Which of the following statements about scoring are true? (Select all that apply.)

a. In the SAS Viya environment, score code is a SAS program.

b. You use the validation data again for scoring.

c. You can export score code from SAS Model Manager

a. In the SAS Viya environment, score code is a SAS program.

c. You can export score code from SAS Model Manager

Which Group by methods are available for category or date/datetime data

items?

a. Intervals

b. Distinct values

c. All of the above

c. All of the above

Which graph would help you determine whether a measure is normally

distributed? (Select all that apply.)

a. bar chart

b. box plot

c. histogram

d. time series plot

b. box plot

c. histogram

In model comparison, the best model has the highest value of which of the following measures?

1. Option A

   sensitivity

2. Option B

   misclassification rate

3. Option C

   average squared error

1. Option A sensitivity

The primary considerations for choosing an appropriate model selection statistic, other than the business needs, are the prediction type and the target type.

1. True

2. False

True

The confusion matrix is the foundation for which of the following assessment plots?

1. Option A

   cumulative lift chart

2. Option B

   ROC chart

3. Option C

   cumulative percent hits chart

4. Option D

   LIME plot

1. Option B ROC chart

During the model deployment phase, champion-challenger testing compares the performance of the currently deployed model and a challenger model based on which of the following?

1. Option A

   new, streaming data

2. Option B

   historic data

1. Option B historic data

A confusion matrix helps you classify which type of target?

1. Option A

   binary

2. Option B

   categorical

3. Option C

   interval

1. Option A binary

Which of the following statements is true about an ROC chart?

1. Option A

   True positives are on the x axis.

2. Option B

   Each point on the chart corresponds to a specific fraction of the sorted data.

3. Option C

   The selection value of each point is displayed on the chart.

4. Option D

   For a perfect model, the ROC curve is a straight line from the bottom left corner to the top right corner of the plot.

1. Option B Each point on the chart corresponds to a specific fraction of the sorted data.

A cumulative lift chart shows that a machine learning model has a lift of 2.6 at a depth of 10%. What does this mean?

1. Option A

   For the top 90% of cases, the machine learning model captures 2.6 times more primary outcome cases than secondary outcome cases.

2. Option B

   For the top 10% of cases, the machine learning model captures 2.6 times more primary outcome cases than a random model.

3. Option C

   For the top 90% of cases, the machine learning model captures 2.6 times more primary outcome cases than a random model.

4. Option D

   For the top 10% of cases, a random model captures 2.6 times more primary outcome cases than the machine learning model.

1. Option B the top 10% of cases, the machine learning model captures 2.6 times more primary outcome cases than a random model.

Based on the following C-statistic values, which model has the best classification accuracy?

1. 
   

   Model C-0.73

2. 
   

   Model B-0.65

3. 
   

   Model A-0.54

4. 
   

   Model D-0.82

1. Model D-0.82

Before you score and manage a model in Model Manager, how do you register the model?

1. Option A

   In Model Studio, select Register Models from the Project Pipeline menu on the Pipeline Comparison tab.

2. Option B

   In Model Studio, add a Model Registration node from the Postprocessing group to the pipeline and run the node.

3. Option C

   In Model Manager, select the model in the Projects list and select Register Models from the Models menu.

1. Option A In Model Studio, select Register Models from the Project Pipeline menu on the Pipeline Comparison tab.

Which of the following statements is true about doing a pipeline comparison in Model Studio?

1. Option A

   You can add a challenger model by selecting it in its pipeline.

2. Option B

    You can add a challenger model by selecting it on the Pipeline Comparison tab.

3. Option C

   You cannot add challenger models to a pipeline comparison.

1. Option A You can add a challenger model by selecting it in its pipeline.