ISDS 415 Final - Ch 7

What is the goal of deep learning?

To mimic the thought process of humans using mathematical algorithms

What is the advantage of deep learning over classic machine-learning methods?

Ability to automatically acquire the knowledge required to accomplish informal tasks, extract some advanced features that contribute to the superior system performance

Which AI-based learning method does deep learning fall under?

representation learning

These networks are simplified abstractions of the human brain and its complex biological networks of neurons.

Artificial neural networks

In ANNs, neurons are equivalent to…?

processing units/elements (PEs) that perform mathematical operations, push out its own output

T/F: The first step in the development process for an ANN application is to collect, organize, and format the data.

True

What are the three tasks of supervised learning in ANN?

1. Compute temporary outputs.

2. Compare outputs w/ desired targets

3. Adjust the weights and repeat the process

The optimization of performance in the neural network is usually done by an algorithm called ______?

stochastic gradient descent (SGD)

This is the most popular neural network supervised learning algorithm; calculation of network gradients

backpropagation

A central concern in the training of any type of machine-learning model is overfitting, which happens when..?

when the trained model is highly fitted to training data, performs poorly on other datasets

What is the “black box” syndrome?

can solve complex problems, but lack explanation

T/F: Most machine learning techniques, led by ANN, are typically known as black boxes.

True

What is the main characteristic of a convolutional network?

Having at least one layer involving a convolution weight function instead of a general matrix multiplication

T/F: The inputs, weighting functions, and transfer functions in a given network are adjustable; the values of the weights and biases are fixed.

False; The inputs, weighting functions, and transfer functions in a given network are FIXED; the values of the weights and biases are ADJUSTABLE.

The processing element (PE) of an ANN is an _____.

artificial neuron

Each ANN is composed of a collection of neurons that are grouped into layers, often called as _______.

network structure

A _____ layer is a layer of neurons that takes input from the previous layer and converts those inputs into outputs for further processing.

hidden

T/F: Each input corresponds to a single attribute

True

T/F: The output of a network contains the solution to a problem

true

These express the relative importance of each input to a PE and ultimately, to the output

connection weights

summation function

computes the weighted sums of all input elements entering each processing element

What are the nine steps in the development process for an ANN application?

1. College, organize, and format the data

2. Separate data into training, validation, and testing sets

3. Decide on a network architecture and structure

4. Select a learning algorith

5. Set network parameters and initialize their values

6. Initialize weights and start training (and validation)

7. Stop training, freeze the network weights

8. Test the trained network

9. Deploy the network for use on unknown new cases

What are some factors to consider when deciding a network architecture & learning method?

• availability of a particular development tool

• capabilities of the development personnel

• certain problem types have high success rates w/ certain configurations

• exact # of neurons and # of layers

In supervised learning, connection weights are derived from existing cases; the learning process is _____

inductive

What are the three steps of learning process in ANN?

1. Compute temporary outputs

2. Compare outputs w/ desired targets

3. Adjust the weights and repeat the process

What is a widely practiced technique for model explainability and/or variable importance identification in ANNIs?

value perturbation type sensitivity analysis

T/F: Deep networks need small data sets to be trained satisfactorily.

False; need larger data sets

Feedforward multilayer perceptron (MLP) deep networks

• aka deep feedforward networks

• most general type of deep network

• large-scale neural networks that can contain many layers of neurons and handle tensors as their input

• flow of info is always forwarding

Image classification networks traditionally involve what two pipelines?

visual feature extraction

image classification

Recurrent neural network (RNN) is specifically designed to process ____ inputs.

sequential

T/F: A recurrent neural network (RNN) models a dynamic system.

True

T/F: RNNs do not have memory to determine future outputs.

False; RNNs do have memory and apply that memory to determine their future outputs.

ChatGPT falls in a class of machine learning NLP models known as ______.

Large Language Models (LLMs)

LLMs operate by….

1. being pretrained on huge corpora of text data using cutting-edge deep learning algorithms

2. inferring a response for a given user input by breaking down its parts

What makes ChatGPT unique?

Reinforcement Learning from Human Feedback (RLHF)

What are the applications of ChatGPT?

• customer service

• healthcare

• marketing

• computer programming

What are the limitations of ChatGPT?

biases, false answers, obsolete data

cognitive computing

addresses highly complex situations characterized by ambiguity and uncertainty

Based on the Cognitive Computing Consortium, what are the key attributes of cognitive computings systems?

• adaptive: flexible enough to learn as info changes and goals evolve

• interactive: users must be able to interact w/ cognitive machines

• iterative and stateful: asking q’s or pull in additional data as needed

• contextual: must understand, identify, and mine contextual data

What is the main diff b/t cognitive computing & AI?

cognitive computing aimed at helping humans solve complex problems; AI aimed at automating processes that are performed by humans

Cognitive analytics

refers to cognitive computing-branded technology platforms that specialize in processing and analyzing large, unstructured data sets

cognitive search

new general search method that uses AI. toreturn results that are much more relevant to users

How is cognitive search different from traditional search?

• can handle varieyt of data types

• can contextualize the search space

• employ advanced AI technologies

• enable developers to build enterprise-specific search applications

IBM Watson is an example of _____.

cognitive computing

What are some industries where Watson was applied?

• healthcare & medicine

• security

• finance

• retail

• education

• government

• research

T/F: The task undertaken by a neural network does not affect the architecture of the neural network; in other words, architectures are problem independent.

False

Sensitivity analysis is a method for extracting what among the inputs and the outputs of a trained neural network model?
a. detailed quantitative predictive models

b. bias

c. cause and effect relationships

d. underlying constraints

c. cause and effect relationships

_____ is a subset of  artificial intelligence while _____, often used interchangeably with artificial intelligence, is the umbrella term used for technologies that rely on data and scientific methods/computations to make (or help/support in making) decisions. 

Machine learning, cognitive computing

T/F: Deep multilayer perceptron (MLP) and convolutional networks are specialized for processing a sequential grid of values.

False

______, a very popular natural language processing (NLP) application, is trained on an enormous corpus of text data from books, websites, articles, and a variety of other human language sources to answer virtually any question. 

ChatGPT

In developing an artificial neural network, all of the following are important reasons to pre-select the network architecture and learning method, except?

a. Most neural networks need special purpose hardware, which may be absent

b. Some neural network software may not be available in the organization

c. Development personnel may be more experienced with certain architectures

d. Some configurations have better success than others with specific problems

a. Most neural networks need special purpose hardware, which may be absent

_______ are among the key learning elements of an artificial neural networks (ANNs) that represent the relative strength (or mathematical value) of connections between layers.

connection weights

T/F: Cognitive computing addresses highly complex situations that are characterized by ambiguity and uncertainty; in other words, it handles the kinds of problems that are thought to be solvable by human ingenuity and creativity.

True