ITE303c-NHS

A process or set of rules to be followed in calculations or other problem-solving operations, especially by a computer describes an
A: Troubleshooting problem
B: Set of inputs
C: Algorithm
D: Computer program

Algorithm

To make this algorithm functional, which step would you add to step 4?
1 Scan to find the smallest number
2 Set to 0 in the index in the output array
3 Remove that number from the input array

4 ……………………………………………….

A: Repeat steps 1-3, but subtract the total from the numbers summated
B: Print output in the correct order
C: Divide the array by the index, print the array output
D: Repeat steps 1-3, but add 1 to the index number for each loop

Repeat steps 1-3, but add 1 to the index number for each loop

Take input, get output, use output on next input is an example of a learning algorithm
A: True
B: False

True

Learning algorithms require large datasets, which means storing identifying information about users
A: True
B: False

True

A __________ in machine learning is the idea that the algorithm itself influences the next set of inputs that go into the model. the main takeaway is that algorithms sometimes have more influence than a user's initial input.
A: Feedback loop
B: Large dataset
C: Learning algorithm
D: False dichotomy

Feedback loop

Are anonymous datasets truly anonymous?
A: Yes, thanks to privacy regulations
B: Yes, thanks to machine learning randomness algorithms
C: No, due to lacking regulations
D: No, due to combining data and re-identification

No, due to combining data and re-identification

Which of the following best describes what an algorithm is?
A: A type of computer that calculates problem-solving methods
B: A type of process a human uses to write down what steps need to happen to get a problem solved
C: A recipe that a computer uses to solve problems
D: A list of ingredients a computer uses to generate problems to solve

A recipe that a computer uses to solve problems

An algorithm that takes an input, tries 10 different sorting techniques, and uses the best fit on the next 100 inputs is best described as a
A: Learning algorithm
B: Explicit algorithm
C: Data algorithm
D: Implicit algorithm

Learning algorithm

Which of these steps follows the most logical order for a low-to-high sorting algorithm?
A: Scan to find the smallest number
Set to 0 in the index in the output array
Remove that number from the input array
Repeat steps 1-3, but add 1 to the index number for each loop

B: Scan to find the smallest number
Set the length of the array in the index in the output array
Remove that number from the input
Repeat steps 1-3, but add 1 to the index number for each loop

C: Scan to find the smallest number
Set to 0 in the index in the output array
Remove that number from the input array

D: Scan to find the largest number
Set to 0 in the index in the output array
Remove that number from the input array
Repeat steps 1-3, but add 1 to the index number for each loop

Scan to find the smallest number
Set to 0 in the index in the output array
Remove that number from the input array
Repeat steps 1-3, but add 1 to the index number for each loop

What's the difference between a basic and learning algorithm?
A: A basic algorithm takes an input and gets an output, while a learning algorithm takes multiple inputs and gets multiple outputs
B: An basic algorithm takes an input and gets an output, while a learning algorithm uses the output on the next input
C: A basic algorithm takes an input, gets an output, while a learning algorithm takes multiple inputs and gets multiple outputs
D: A basic algorithm takes an input, while a learning algorithm takes an input and gets an output

An basic algorithm takes an input and gets an output, while a learning algorithm uses the output on the next input

Pseudocode can best be defined as
A: A python library for machine learning
B: A middle ground between code and plain writing that can be fed into a computer
C: A type of javascript that is both human and machine-readable
D: An explainable description of code that is meant for humans, not computers

An explainable description of code that is meant for humans, not computers

What side effect of learning algorithms creates an ethical dilemma for its users?
A: Learning algorithms require government regulation, which is bad for software developers
B: Learning algorithms are costly to run, which drives up prices for consumer services
C: Learning algorithms require large datasets, which means storing identifying information about users
D: Learning algorithms require large amounts of computing power, which is bad for the environment

Learning algorithms require large datasets, which means storing identifying information about users

How do anonymized datasets fall short of their goal of being anonymous?
A: Anonymized datasets can be combined with other datasets, which can re-identify individuals
B: Anonymized datasets can be re-identifyed by anyone holding the hash key
C: Anonymized datasets can be traced back to the individuals by looking at their browsing history in the app
D: Anonymized datasets aren't actually anonymous because many of the data fields can identify a user

Anonymized datasets can be combined with other datasets, which can re-identify individuals

What is a likely outcome for a weather app using a learning algorithm to figure out where to put their future weather stations?
A: Accessing weather forecasts from local broadcasts
B: Storing data in an aws instance with all weather stations in the country
C: Collecting location data every time the app is opened, potentially learning where a user lives, works, etc.
D: Collecting weather data every time the app is opened, knowing the temperature where the app is being used

Collecting location data every time the app is opened, potentially learning where a user lives, works, etc.

Which of the following is a good example of a feedback loop in machine learning?
A: A social media site tracks engagement, uses an algorithm to surface posts you're likely to engage with, which then goes back into the algorithm
B: A social media site surfaces controversial posts, which make users more angry and lead to more angry posts on the network
C: A shopping app tracks your purchases, and recommends new things to buy
D: A shopping app surfaces new items to buy, which is based on dataset from customers fitting a similar profile. when you buy, you go into that dataset

A social media site tracks engagement, uses an algorithm to surface posts you're likely to engage with, which then goes back into the algorithm

A fact of learning algorithms is that
A: Even if you haven't shared an direct datapoint about yourself, with enough related datapoints the algorithm can make an educated guess with alarming accuracy
B: Just because they are capable of improving outputs, they don't need more inputs
C: They only learn when given small amounts of data, and without the proper training sample, the results can be wildly inaccurate
D: They cannot make predictions with current technology

Even if you haven't shared an direct datapoint about yourself, with enough related datapoints the algorithm can make an educated guess with alarming accuracy

A basic learning model can figure out which of the 10 sorting mechanisms works best for this type of input. a complex model
A: Automatically scans all inputs
B: Can figure out up to 150 mechanisms
C: Can figure out up to 50 mechanisms
D: Automatically derives its mechanism

Automatically derives its mechanism

A model's error rate is the ratio of incorrect predictions to total predictions
A: True
B: False

True

The goal of the _ is to get the model's error rate as low as possible. to do this, we repeat a cycle of feeding training data, compare predictions to actual outcomes, and adjust the model as needed.
A: Develop phase
B: Algorithmic phase
C: Training phase
D: Deployment phase

Training phase

As models become more complex, researchers are unable to reason why the decisions are being made. this is called the ___
A: Rationality phase
B: Black box problem
C: Training phase
D: False input problem

Black box problem

can often be caused by predicting what someone may or may not do based on data
A: Irrationality
B: Dataset collection
C: Real harm
D: Retraining scenarios

Real harm

Hedge funds largely rely on predictive models to judge the movement of stocks, bonds, and securities
A: True
B: False

True

What's the difference between a basic and complex learning algorithm?
A: A basic algorithm cannot use computer vision, while a complex algorithm can
B: A basic algorithm cannot process more than 5 steps in a function, while a complex algorithm can process up to 15
C: A basic algorithm has a set amount of choices to optimize for, while a complex algorithm is given the freedom to find its own model
D: A basic algorithm can handle simple inputs like numbers, while a complex algorithm can handle complex inputs like pictures

A basic algorithm has a set amount of choices to optimize for, while a complex algorithm is given the freedom to find its own model

When building a predictive model, what is the goal of the develop phase?
A: To get the model to accept new inputs, train, and repeat training until it finds a better curve
B: To specify the type of algorithm the model should use and make sure the data is cleaned/formatted
C: To plug in 40% of your dataset, testing the model's accuracy
D: To get the model's error function below an acceptable percentage

To specify the type of algorithm the model should use and make sure the data is cleaned/formatted

When building a predictive model, what is the goal of the training phase?
A: To adjust training methods from backpropagation to supervised learning to see how that affects outputs
B: To use the model in real-world scenarios, monitoring performance
C: To adjust the model based on a subset of data, optimizing for a lower error rate
D: To specify the type of algorithm the model should use and make sure the data is cleaned/formatted

To adjust the model based on a subset of data, optimizing for a lower error rate

When building a predictive model, what is the goal of the deployment phase?
A: To specify the type of algorithm the model should use and make sure the data is cleaned/formatted
B: To plug in 40% of your dataset, testing the model's accuracy
C: To use the model in real-life predictions, monitoring the error rate and accuracy
D: To get the model to accept new inputs, train, and repeat training until it finds a better curve

To use the model in real-life predictions, monitoring the error rate and accuracy

What are the attributes of an error function when training a predictive model
A: The percentage of data that is formatted properly
B: The ratio of training data to actual data the model has consumed
C: The ratio of algorithm to curve in a predictive model
D: The percentage of predictions that don't match actual outcomes

The percentage of predictions that don't match actual outcomes

In a complex learning function, we will understand the ____, but not the ____
A: Causal link, correlation
B: Algorithm, cause and effect
C: Inputs/outputs, algorithm
D: Input data, output data

Inputs/outputs, algorithm

What is the black box problem?
A: The problem created when researchers don't create accurate attributes for a model
B: When a model cannot accurately judge shape or color of objects due to missing data
C: The issue of not having enough data to accurately train a model
D: When a model is deployed, but researchers are unable to figure out why it's making decisions

When a model is deployed, but researchers are unable to figure out why it's making decisions

Which of the following is a negative consequence of a predictive model used in real life?
A: A model used by a lab wrongly predicts a person will not be able to pay their credit card
B: A model used by a lab indicates a person is in danger
C: A model used by a bank accurately predicts a person will not be able to pay off a loan
D: A model used by a bank wrongly predicts a person will not be able to pay off a loan

A model used by a bank wrongly predicts a person will not be able to pay off a loan

How are predictive models used in hedge funds?
A: They aid researchers by forecasting financial collapse
B: They provide predictions to shareholders to estimate returns
C: They predict future movement of stocks and find points to exploit the market moving in either direction
D: They predict whether people will be able to pay off loans, and then provide loans

They predict future movement of stocks and find points to exploit the market moving in either direction

What is one possible reason a model may predict a higher crime rate based on datasets used?
A: If a dataset isn't properly formatted, crime may be linked to the error function, outputting false data
B: If crime is down in an area, a model may predict a parabolic curve which estimates crime is due to rise again
C: The model's training curve was not provided enough data
D: If drug arrests are historically high in that area, the model may correlate crime with areas of high drug use based on the datasets

If drug arrests are historically high in that area, the model may correlate crime with areas of high drug use based on the datasets

To measure accuracy, take the number of results and divide over the number of all results
A: False positive and false negative
B: True positive and true negative
C: False positive and true negative
D: True positive and false negative

True positive and true negative

A false negative result is one in which the model predicts a result was negative, and in reality it was ___. it is an ___ prediction
A: Positive, correct
B: Negative, correct
C: Positive, incorrect
D: Negative, incorrect

Positive, incorrect

City and state are correlated data, but a model will measure no variation and the results will not be affected
A: True
B: False

False

A training set relies on running a final accuracy test before deploying a model. an __ training set relies on multiple tests to ensure that a model is free of bias
A: Sample, optimized
B: Test, classic
C: Optimized, classic
D: Classic, optimized

Classic, optimized

An unknown unknown is an example of a cultural reflection of data
A: True
B: False

False

An ethical predictive model needs to be accurate, _, and fair
A: Moral
B: Predictable
C: Truthful
D: Explainable

Explainable

To measure a predictive model's accuracy, you
A: Multiply the number of total predictions by the percentage of correct predictions
B: Divide the number of predictions by the total dataset
C: Measure the ratio of the model's error curve
D: Divide the number of correct predictions by the total number of predictions

Divide the number of correct predictions by the total number of predictions

A predictive model's false negative result can be defined as
A: The predicted result was positive, and the actual result was positive
B: The predicted result was positive, and the actual result was negative
C: The predicted result was negative, and the actual result was positive
D: The predicted result was negative, and the actual result was negative

The predicted result was negative, and the actual result was positive

A predictive model's true positive result can be defined as
A: The predicted result was positive, and the actual result was positive
B: The predicted result was negative, and the actual result was negative
C: The predicted result was negative, and the actual result was positive
D: The predicted result was positive, and the actual result was negative

The predicted result was positive, and the actual result was positive

Model inputs of address with "city + state" as separate inputs from a dataset would violate which accuracy guideline?
A: Domain expertise
B: Objective summarization
C: First principles
D: No correlating data

No correlating data

Once a dataset has been cleaned, which accuracy guideline ensures your model is looking at the problem correctly?
A: Objective summarization
B: Domain expertise
C: First principles
D: Dataset verification

Domain expertise

A good example of cultural reflection in training data is
A: A model fails to recognize cultural differences due to incorrect attributes
B: A predictive model incorporates training data from a variety of sources
C: A model selects for one demographic less often because of their historical representation
D: An image recognition model selects one face over another based on sample data

A model selects for one demographic less often because of their historical representation

A good example of empirical reflection in training data is
A: An image recognition model cannot tell a difference between a photo of a dog and a photo of a photo of a dog
B: An image recognition model selects one face over another based on sample data
C: A true positive result that defies the training data set
D: A model fails to recognize cultural differences due to incorrect attributes

An image recognition model cannot tell a difference between a photo of a dog and a photo of a photo of a dog

A training set based on feeding 60% of data, validating on 20% of data, and then designing multiple tests for the remaining 20% of data is referred to as an
A: False positive set
B: Optimized training set
C: Predictive training set
D: Classic training set

Optimized training set

Our goals for building an ethical predictive model include making sure the results are
A: Precise, methodical, ethical
B: Accurate, precise, fair
C: Precise, explainable, predictable
D: Accurate, fair and explainable

Accurate, fair and explainable

Unknown unknowns refer to
A: Facing unknown empirical data with an incomplete dataset
B: An uncertainty of how the data is gathered
C: Being unsure about the morals of the research team
D: Lack of explainability and what a model is actually looking at to make it's prediction

Lack of explainability and what a model is actually looking at to make it's prediction

Narrow ai (ani) is defined as a specific type of artificial intelligence in which a technology outperforms humans in some defined task.
A: True
B: False

True

An ethical, evolved predictive model would need to mimic a researcher's ability to ___
A: Scrub data
B: Self-learn
C: Parse through datasets
D: Eliminate bias

Eliminate bias

The second evolution of decision-making ai would enable
A: Predictive models to drive cars
B: Predictive models to start companies
C: Predictive models to decide war strategy
D: Predictive models to approve loans

Predictive models to decide war strategy

Researchers believe that a general-purpose ai must be available to as many as possible, making it similar to a
A: Government program
B: Utility
C: Tax plan
D: System of money

Utility

A perverse instantiation is an unintended negative outcome of programming a goal that is too specific given to general intelligence
A: True
B: False

False

For-profit colleges tend to use predictive models
A: To accelerate their research departments
B: To see which candidates are most likely to receive government loans.
C: To see which applicants are most likely to graduate
D: To evaluate the standards of their professors

To see which candidates are most likely to receive government loans.

For a model to clean, parse, and self-train it's own dataset while remaining impartial, the model needs
A: A test for recency bias
B: 10x the amount of data
C: A list of bias and domain tests to run and adjust for
D: More powerful computing algorithms to auto-scrub data

A list of bias and domain tests to run and adjust for

For a model to make decisions that involve human life, the model needs
A: A list of bias tests to run against possible wrong outcomes
B: Enough computing power to make correct predictions 100% of the time
C: A moral code of reasoning and priorities
D: Programmed reflexive decision making ability

A moral code of reasoning and priorities

A type of artificial intelligence that outperforms humans in some defined task is known as
A: Special ai
B: General ai
C: Aei
D: Narrow ai

Narrow ai

A type of artificial intelligence that outperforms humans in all tasks is known as
A: Encompassing ai
B: Outwit ai
C: General ai
D: Specific ai

General ai

In 2019, ____% of equity-futures and cash-equity trades were executed by algorithms
A: 20-30%
B: 80-90%
C: 11-17%
D: 1-5%

80-90%

The optimistic view of general ai could be accurately summarized as ai as a ____
A: Peace-keeping tool
B: Human right
C: Weapon
D: Utility

Utility

The pessimist view of general ai references a scenario in which advancement is _
A: Creating ai for all governments
B: Impossible
C: Winner take all
D: A potential extinction event

Winner take all

An ethical general purpose ai must _ while not harming the safety of humanity
A: Keep those in power responsible
B: Be 100% accurate
C: Benefit as many people as possible
D: Not enact hate

Benefit as many people as possible

"companies have an obligation to their shareholders" is part of a view that sees artificial intelligence as
A: An overall good for humanity, no matter the consequences
B: A harmful tool that will bring about the end of capitalism
C: Just another tool that accelerates research, like online advertising
D: A gimmick for enterprises, unless general intelligence is achieved

Just another tool that accelerates research, like online advertising

An unintended negative outcome of programming a broad goal into general intelligence is known as
A: Perverse instantiation
B: Artificial sanctification
C: An ethical dilemma
D: An enduring output

Perverse instantiation

True or false: the definition of fairness is "just treatment without bias and contempt"
A: True
B: False

False

Statistical parity as a fairness goal makes the most sense when
A: Distributing randomly, ex. tickets
B: Distributing by merit, ex. loans
C: Distributing by gender, ex. tickets
D: Distributing by error rate, ex. loans

Distributing randomly, ex. tickets

Error rate parity means an equal chance of
A: Outcomes for each group
B: Prediction rate for each group
C: Mistakes made for each group
D: Approval for each group

Mistakes made for each group

If you know one group is misrepresented in merit by training data, one way to ensure fairness is to
A: Delete that group's data from the dataset
B: Protect a different group
C: Create a separate threshold for that group
D: Reboot the training data

Create a separate threshold for that group

True or false: fairness in machine learning cannot protect all individuals within protected groups from harm
A: True
B: False

True

Our goal in machine learning fairness is to minimize as long as is obtained
A: Accuracy issues, unfairness
B: Unfairness, equality
C: Error rates, parity
D: Equality, error rates

Error rates, parity

Fairness is best defined as just treatment without ______
A: Prejudice and favoritism
B: Discrimination and prejudice
C: Bias and contempt
D: Favoritism or discrimination

Favoritism or discrimination

Which type of fairness would make sense when dividing tickets evenly between groups?
A: Equality of prediction rate
B: Statistical parity
C: Equality of false positives
D: Error rate parity

Statistical parity

Which type of fairness fails to address merit while maintaining accuracy?
A: Error rate parity
B: Statistical parity
C: Equality of prediction rate
D: Equality of false positives

Statistical parity

A model that prioritizes equality on the outputs uses
A: Equality of prediction rate
B: Equality of assignment rate
C: Error rate parity
D: Statistical parity

Error rate parity

Fairness in machine learning can protect groups from bias, but can still harm
A: Individuals within those groups
B: Future models
C: Researchers
D: Training datasets

Individuals within those groups

A goal of a fair model's accuracy standards is to
A: Minimize the quality metrics as long as the quantity metrics aren't affected
B: Minimize the error rate as long as the training data isn't affected
C: Minimize the error rate as long as parity is obtained
D: Minimize the fairness score as long as the error rate isn't affected

Minimize the error rate as long as parity is obtained

A model that makes more mistakes by moving its decision threshold down 40% of its worthiness metric will be potentially
A: Fairer but less accurate
B: Less fair but more accurate
C: More accurate and fairer
D: Less accurate and less fair

Fairer but less accurate

If one group comprises the majority of the training data, they will skew the dataset and give the model
A: More fairness for that group
B: Less fairness for that group
C: More confidence about that group
D: Less confidence about that group

More confidence about that group

If we know one group's worthiness score has been artificially inflated, one solution for fairness is to
A: Remove that group from the dataset
B: Add the inflation to the other data
C: Balance the error rate by prioritizing the other group
D: Creating separate decision thresholds for each group

Creating separate decision thresholds for each group

An unfair model will by nature
A: Try to balance groups automatically
B: Optimize for making the most errors
C: Optimize for making the fewest mistakes
D: Optimize for making the fewest decisions

Optimize for making the fewest mistakes

True or false: it is practical to protect all possible subgroups in predictive modeling
A: True
B: False

False

In machine learning, a pareto curve helps us
A: Highlight the inequality in our model
B: Pick an optimal threshold for accuracy and error rate
C: Pick an optimal tradeoff between fairness and accuracy
D: Select which model will give the best results

Pick an optimal tradeoff between fairness and accuracy

True or false: a blind attribute model protects group fairness by not including group membership in predictions
A: True
B: False

False

An adversarial algorithm is ____ to identify weaknesses in black box models
A: Given no data
B: Purposefully biased
C: Purposefully fair
D: Trained with large datasets

Purposefully biased

True or false: our analysis revealed that word2vec is not a black box model
A: True
B: False

True

Which step in the fairness process would be most appropriate to introduce an auditing model?
A: Sub-processing
B: Pre-processing
C: Post-processing
D: In-processing

In-processing

A state where resources cannot be reallocated to make one individual better off without making at least one individual worse off is known as a
A: Pareto efficiency
B: Prisoner's dilemma
C: Boron letter
D: Aggregate curve

Pareto efficiency

In machine learning, what do we plot on the x,y axis to determine a pareto curve?
A: Error rate, rejection rate
B: Rejection rate, false-positive rate
C: Rejection rate, subgroup fairness rate
D: Error rate, true positive rate

Error rate, rejection rate

Why is it impractical to protect all possible subgroups in predictive models?
A: Individuals do not need protection from predictive models
B: Fairness scores won't be high enough to be reasonable
C: Accuracy will be lowered beyond a reasonable rate
D: There won't be enough data to reflect each subgroup

Accuracy will be lowered beyond a reasonable rate

A model that equalizes the number of mistakes it makes for each subgroup to reduce harm is deciding on
A: Equality of false negatives
B: Equality of training data
C: Equality of prediction bias
D: Equality of true outcomes

Equality of false negatives

A _ model can still be unfair even though it won't explicitly know which groups are being inputted into the system
A: Single attribute
B: Biased training
C: Blind attribute
D: False-negative optimized

Blind attribute

What tools do researchers have to evaluate the fairness of existing black box models?
A: Evaluate inputs, evaluate data
B: Change training data, evaluate outputs
C: Change inputs, evaluate outputs
D: Change inputs, evaluate training data

Change inputs, evaluate outputs

A "purposefully biased" algorithm used to identify unfair attributes is known as
A: A predictive model
B: An aggregate algorithm
C: A discriminatory algorithm
D: An adversarial algorithm

An adversarial algorithm

In presenting an audit report, a researcher would
A: Prevent the model from launching
B: De-bias the results
C: Score the weight of input attributes on output
D: Re-train the model

Score the weight of input attributes on output

In fixing the word2vec model, we have an advantage over a traditional black box model in that
A: We can decide which inputs to use
B: We have access to the training data
C: We can see the decision-making model
D: We can generate a fairness score

We have access to the training data

An auditing model is an example of a bias mitigation method
A: Sub-processing
B: Pre-processing
C: Post-processing
D: In-processing

In-processing

True or false: the no free lunch theorem states that we cannot have fair models without giving up something else
A: True
B: False

False

Which of the following is a good example of sample bias?
A: Your model is trained to recognize pets, but you only give it photos of dogs
B: Your model is trained to avoid bias, but it contains no samples of that bias
C: Your model is broken because it cannot sample the right attribute
D: Your model is designed to give loans to those who need it, but it is trained with unfair data

Your model is trained to recognize pets, but you only give it photos of dogs

When cleaning/parsing data removes a potentially important attribute, that is referred to as
A: Confirmation bias
B: Automation bias
C: Exclusion bias
D: Observer bias

Exclusion bias

Labeling outputs made by predictive models can avoid which feedback issue?
A: Predictive loop bias
B: Fairness score bias
C: Re-training bias
D: Sample bias

Re-training bias