Quiz Questions DATASCI 315

Primary goal of supervised learning

To define a mapping from inputs to outputs using labeled data

Inference in supervised learning

The process of predicting outputs from inputs using a trained model

Purpose of the loss function in supervised learning

To quantify the mismatch between predictions and actual outputs

Overfitting

It fits the training data well but performs poorly on test data

Underfitting

The model is too simple to capture the underlying pattern in the data

Separate test dataset purpose

To ensure that the model can generalize to unseen data

Purpose of gradient descent in model training

To minimize the loss function

Least-squares loss function in linear regression

The sum of the squared differences between predicted and actual outputs

Model equation in supervised learning

A family of possible relationships between inputs and outputs

Function represented by a shallow neural network with ReLU activation functions

A piecewise linear function

Maximum linear regions in a shallow neural network with one input, one output, and three hidden units

4

Effect of adding more hidden units to a shallow neural network

It allows the model to approximate more complex functions.

Purpose of the activation function in a shallow neural network

To describe nonlinear relationships between input and output

ReLU (-e^-6)

0

Definition of an activation pattern in shallow neural networks

The combination of active and inactive hidden units for a given input

Parameters in a shallow neural network with 2 inputs, 1 hidden unit, and 1 output

5

For a shallow network with 1 input, 1 output, and 4 hidden units, the number of joints in the output function is...

4

How many parameters are in a shallow network with 1 input, 50 hidden units, and 2

outputs?

202

How many parameters are in a shallow network with 3 inputs, 3 hidden units, and 2

outputs?

20

At most how many linear regions can a shallow neural network with one input, one output, and five hidden units represent?

6

What is ReLU(−10)?

0

How many parameters are in a shallow neural network with 1 input, 1 hidden unit, and 1 output?

4

How many parameters are in a shallow network with 1 input, 10 hidden units, and 2 outputs?

42

What is the main limitation of a shallow neural network with a linear activation function?

It can only model linear relationships between input and output.

Which of the following is NOT a typical feature of a shallow neural network?

Cyclic connections between neurons

Which statement best defines the 'capacity' of a shallow feed-forward neural network?

Capacity refers to the total number of hidden units in the network, which directly influences the complexity of functions it can model.

A deep neural network is characterized by

Having more than one hidden layer

What are hyperparameters in the context of neural networks?

The quantities chosen before training the network, such as the number of layers and hidden units.

What does 'depth' of a neural network refer to?

The number of hidden layers

What does 'width' of a neural network refer to?

The number of hidden units per layer.

Which of the following statements is true regarding depth efficiency?

Some functions require a shallow network with exponentially more hidden units than a deep network to achieve an equivalent approximation.

Which of the following statements best describes the Universal Approximation Theorem in the context of neural networks?

It states that both shallow and deep neural networks, with sufficient capacity, can approximate any continuous function arbitrarily closely.

Which of the following statements accurately describes the use of matrix notation in representing a deep neural network?

In matrix notation, weights are represented by matrices and biases are represented by vectors.

What is the most accurate description of hyperparameters and parameters in neural networks?

Hyperparameters are set before training, whereas parameters like weights and biases are initialized and adjusted through optimization.

Consider a deep neural network with 10 inputs, 20 outputs, and 8 hidden layers, each containing 16 hidden units each. What is the depth of this network?

8

Consider a deep neural network with 2 inputs, 2 outputs, and 2 hidden layers containing 2 hidden units each. How many parameters are in this network?

18

Bias parameters in a deep neural network with 3 inputs, 3 outputs, and 3 hidden layers containing 3 hidden units each

12

Weight parameters in a deep neural network with 1 input, 2 outputs, and 2 hidden layers containing 1 hidden unit each

4

Total parameters in a deep neural network with 1 input, 1 output, and 10 hidden layers containing 1 hidden unit each

22

Total weights in a deep neural network with 1 input, 1 output, and 2 hidden layers containing 3 hidden units each

15

First entry of vector v = [1 2, 3 4] [5 6]

17

Output of the Python code: x = np.array([[1, 2], [3 , 4]]); (2 * x).sum()

20

Output of the Python code: np.arange(4).sum()

6

Output of the Python code: x = np.array([[1, 1]]); y = np.array([[1], [1]]); (x + y).sum()

8

Conditional probability distribution in the context of loss functions

The probability of the output given the input.

Function of the logistic sigmoid in binary classification

It maps the real-valued output of the network to a value between 0 and 1.

Calculation of multiclass cross-entropy loss

Summing the negative log-likelihoods of the correct classes for each training example.

Primary use of the softmax function

Multiclass classification problems

Prediction of a homoscedastic regression model

Only the mean of the output distribution, as the variance is constant.

Result of vector v = [ 1 2, 3 4] [1 1]

3

Result of vector v = [1 1 1, 1 1 1, 1 1 1] [1 2 3]

6

T/f: In maximum likelihood estimation,

assuming that the data are

independent allows the log likelihood

function to be written as a sum of

individual probabilities.

True

T/f: A minimizer of the negative log-

likelihood is also a maximizer of the

log-likelihood AND a maximizer of the

likelihood.

True

T/f: The log-likelihood function is used

instead of the likelihood function in

optimization because logarithmic

transformation preserves the position

of the maximum while converting

products into sums.

True

T/f: The least squares loss function can be

derived from the assumption that the

residuals of a model's predictions

follow a uniform distribution.

False

T/f: Heteroscedastic regression variance assumes variance is constant

False

Softmax function output sums to 1?

True

Loss function definition

Returns a single number that quantifies the mismatch

Bernoulli distribution for multiclass classification

False

SGD always ensures that the loss function decreases each iteration?

False

Gradient descent guarantees a global minimum?

False

Momentum in gradient descent

Averages gradients over iterations, prevents getting stuck in shallow minima

Stochastic gradient descent (SGD) gradient computation

Uses a subset of the training data each iteration

Epoch in model training

A single pass through the entire training dataset.

Nesterov accelerated momentum gradient computation

At a predicted point based on the momentum from the previous step.

Key benefit of stochastic gradient descent (SGD)

It requires less memory by processing one batch at a time.

When training a model, which parameters minimize the negative log-likelihood loss function over the training dataset pairs {xi, yi}?

Network parameters φ̂

For a binary classification problem, which activation function is generally applied to the neural network's output before computing the loss?

sigmoid

When adding two tensors using broadcasting—one of shape (3, 1) and another of shape (1, 4)—what is the total number of elements in the resulting tensor

12

You create a tensor with x = torch.zeros(3, 3, dtype=torch.int32). What is the size (in bytes) of this tensor in memory, assuming each int32 element takes up 4 bytes

36

For a multiclass classification problem, which activation function is generally applied to the neural network's direct output?

softmax

Consider a image classification problem with 4 classes: "car", "squid", "chair", and "octopus". Suppose we've trained a network to solve this problem using the cross-entropy loss function. For a particular image, our network's direct outputs are -1, 4, -2, 4 for these respective classes. According to our model, what is the probability that this image shows an octopus?

.5

Consider a dataset containing 10,000 data points. Suppose we allocate 80% of the dataset for training, 10% for validation, and 10% for testing. During training, we set the batch size to 100 data points and run the optimizer for 10 epochs. How many optimizer steps (iterations) have we performed?

800