Data Analysis and Predictive Modeling

0.0(0)
studied byStudied by 0 people
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
Card Sorting

1/55

flashcard set

Earn XP

Description and Tags

Flashcards based on lecture notes about data imputation, regression analysis, neural networks, clustering, text mining, and prescriptive analytics.

Study Analytics
Name
Mastery
Learn
Test
Matching
Spaced

No study sessions yet.

56 Terms

1
New cards

Data Imputation

Assigning a value to missing data, often using methods like mean, mode, or predictive models.

2
New cards

Random Imputation

Choosing a value from existing records to fill in missing data.

3
New cards

Hot Deck Imputation

Selecting a similar record to impute missing data.

4
New cards

Predictive Imputation

Using a predictive model to estimate missing values.

5
New cards

Cold Deck Imputation

Using a separate dataset to find a similar value for imputation.

6
New cards

Value Substitution

Replacing missing values with a predetermined value like zero or a 'Missing' label.

7
New cards

Discretization

Converting continuous values into discrete values, often 1 and 0.

8
New cards

Data Reduction

Discarding variables that are not useful for the analysis.

9
New cards

Supervised Methods

Dividing a dataset into training and testing sets when there is a target variable to predict.

10
New cards

Unsupervised Methods

Methods that seek patterns in data without a target variable.

11
New cards

Overfitting

A model that performs well on training data but poorly on new data.

12
New cards

Imbalanced Dataset

A dataset where one class has significantly more data than another.

13
New cards

Balance Dataset

Adjusting the dataset to have a more even distribution of classes, or generating artificial data to reach data parity.

14
New cards

Regression Analysis

A statistical method used to discover relationships between variables to make predictions.

15
New cards

Simple Linear Regression

Defines the relationship between a dependent variable and a single independent variable; the goal is to find best fit of the regression line.

16
New cards

Model Review

Examining residual values, model coefficients, standard error, R-squared values, adjusted R-squared values, and F-statistics.

17
New cards

Logistic Regression

Used for classification with a binary categorical target variable and numerical independent variables.

18
New cards

Ensemble Learners

Combines multiple models to improve prediction accuracy and robustness.

19
New cards

Advantages of Ensemble Learners

Reduces overfitting and performs well with many or few data points by unifying predictions for different models.

20
New cards

Bagging

Bootstrap aggregating; changes the training set for each model using sampling with replacement.

21
New cards

Random Forests

An ensemble of decision trees that uses random feature selection.

22
New cards

Boosting

Trains base models sequentially, assigning weights to training records and focusing on records difficult to classify.

23
New cards

Neural Networks

Simulates the behavior of neurons for classification and regression tasks.

24
New cards

Uses of Neural Networks

Pattern recognition, computer vision, and natural language processing.

25
New cards

Perceptron

A simple form of neural network with no cycles and two layers.

26
New cards

Activation Function

Decides whether a neuron activates, helping the network learn patterns in the data.

27
New cards

Backpropagation

Calibrates weights based on the error rate of the previous iteration, with initial weights chosen randomly.

28
New cards

Gradient Descent

Used in training to find the optimal combination of weights, adjusting them until the error is acceptable.

29
New cards

Types of Neural Networks

Feedforward, convolutional, transfer learning, recurrent, generative, and autoencoders.

30
New cards

Clustering Means

Uses clustering to solve problems with descriptive statistics.

31
New cards

Clustering

Finds natural groupings in a dataset.

32
New cards

Types of Clusters

Exclusive, overlapping, hierarchical, and probabilistic.

33
New cards

Types of Clustering Algorithms

Prototype-based, density-based, and hierarchical.

34
New cards

K-means

Divides the space into K partitions using measures of proximity.

35
New cards

How K-means Works

Determining K, assigning points to centroids, calculating new centroids, and repeating until all points are assigned.

36
New cards

Evaluation of K-means

Sum of squared errors, Davies-Bouldin index, and cluster-to-cluster distance.

37
New cards

DBSCAN

Identifies clusters by measuring the distribution of density in a space.

38
New cards

DBSCAN Point Types

Core points, border points, and noise points.

39
New cards

Text Mining

Finding patterns and extracting knowledge from unstructured text data.

40
New cards

Uses of Text Mining

Classification, clustering, extracting meaning, and extracting information.

41
New cards

Difficulties of Text Mining

Ambiguity, homonyms, synonyms, misspellings, high dimensionality, and variety of languages.

42
New cards

Difference Between Text Mining and NLP

Focuses on interaction between human language and computers, while text mining focuses on finding patterns and trends.

43
New cards

Process of Text Mining

Collecting, preprocessing, and analyzing data.

44
New cards

Preprocessing Steps in Text Mining

Lowercasing, removing stop words, punctuation, symbols, numbers, and short words; stemming; and considering synonyms and phrases.

45
New cards

Document Representation

Bag of words, TF-IDF, word embeddings, and context-specific vectors.

46
New cards

Visualization and Analysis of Text Mining

Frequencies, word clouds, and dendrograms.

47
New cards

Lexical-Syntactic Processing

Recognizes tokens and normalizes words.

48
New cards

Semantic Processing

Extracting meaning, identifying sentiments, and analyzing emotions.

49
New cards

Types of Sentiment Analysis

Levels, aspect-based, emotion detection, and intention analysis.

50
New cards

Importance of Sentiment Analysis

Market research, customer service, product analytics, and public relations.

51
New cards

Descriptive Analytics

Analyzes what has already happened.

52
New cards

Predictive Analytics

Oriented towards the future.

53
New cards

Prescriptive Analytics

Indicates a decision or suggests the best course of action.

54
New cards

Uses of Prescriptive Analytics

Optimizes supply chain, demand, production costs, delivery times, resource planning, and inventory management.

55
New cards

Components of Prescriptive Analytics

Good quality data, neural networks, hardware, and software.

56
New cards

Types of Prescriptive Analytics Algorithms

Heuristics and exact algorithms.