Flashcards ISBA 1 Chapter 1 LEC & LAB | Quizlet

Data

is the raw material in data analytics that, when processed, provides actionable insights, identifies patterns and trends, and enables informed decision-making, enhanced efficiency, customer understanding, risk management, and innovation.[cite: 3]

Data Analysis

involves the process of inspecting, cleaning, transforming, and modeling data to uncover meaningful insights, patterns, and trends. It typically focuses on examining past data to understand what happened and why it happened. Data analysis often involves descriptive statistics, visualization techniques, and hypothesis testing to conclude the data.[cite: 3]

Data Analytics

encompasses a broader scope and involves the use of advanced techniques and tools to extract actionable insights from data. It goes beyond the examination of historical data and incorporates predictive and prescriptive analytics to anticipate future trends and outcomes, which often involves machine learning algorithms, data mining, and optimization techniques to drive decision-making and strategy.[cite: 3]

Descriptive Analytics

summarizes historical data to understand past performance.[cite: 3]

Diagnostic Analytics

dives deeper into the data to identify the root causes of past events.[cite: 3]

Predictive Analytics

uses historical data and machine learning to forecast future trends and outcomes.[cite: 3]

Prescriptive Analytics

goes beyond prediction to recommend specific actions to achieve desired outcomes.[cite: 3]

Quantitative data

refers to something related to quantities or amounts, expressed in numbers, data that can measure or count.[cite: 3]

Discrete data

consists of countable values, often whole numbers and cannot be broken down into smaller parts. answers the "how many" questions-it's finite and clear-cut.[cite: 3]

Continuous data

involves measurements that can take any value within a given range, which can be measured more precisely with decimals. Unlike discrete data, continuous data can change over time and can be infinitely divided into smaller parts.[cite: 3]

Qualitative data

refers to qualities or characteristics, and is characterized by descriptions. It's non-numeric and focuses on specific qualities or characteristics that can be used to categorize things.[cite: 3]

Nominal data

consists of labels or names that don't have any natural order or ranking.[cite: 3]

Ordinal data

does have a meaningful order or ranking, but there is no consistent or defined difference between the values.[cite: 3]

Structured data

is like a neatly organized spreadsheet where every piece of information is placed into specific rows and columns. Because this data is highly organized, it makes it easier to search and manage.[cite: 3]

Unstructured data

is the opposite of structured data-it's more like a messy drawer filled with various items, some of which might be hard to categorize. Unstructured data includes things like emails, videos, audio files, and customer reviews.[cite: 3]

Semi-structured data

is a bit more flexible. It has some organization, but not everything follows a rigid format.[cite: 3]

Big data analytics

is the process of collecting, organizing, and analyzing large, complex datasets to discover hidden patterns, correlations, and trends, which helps organizations make informed decisions and uncover insights.[cite: 3]

Volume

refers to the quantity of data generated and collected, often measured in terabytes, petabytes, or even exabytes.[cite: 3]

Variety

describes the different types of data, which can be structured (like in databases), semi-structured, or unstructured (like text, images, or videos).[cite: 3]

Velocity

represents the speed at which new data is created, collected, and processed. Examples are high-speed data streams, such as social media feeds or IoT sensor data.[cite: 3]

Veracity

deals with the uncertainty, accuracy, and trustworthiness of the data. It's crucial to ensure the quality and reliability of the data before using it for analysis.[cite: 3]

Value

is the ultimate goal of big data analytics, which is to extract meaningful insights and actionable information from the data to make better decisions, create new products, and improve business processes.[cite: 3]

Supervised learning

is defined by its use of labeled data sets. These data sets are designed to train (supervise) algorithms in classifying data or predicting outcomes accurately using labeled inputs and outputs.[cite: 3]

Classification problems

use an algorithm to assign test data into specific categories

Regression

uses an algorithm to understand the relationship between dependent and independent variables, where models help predict numerical values based on different data points.[cite: 3]

Unsupervised learning

uses algorithms to analyze and cluster unlabeled data sets and discover hidden patterns in data without the need for human intervention (unsupervised).[cite: 3]

Clustering

groups the unlabeled data based on their similarities or differences.[cite: 3]

Association

uses different rules to find relationships between variables in a given data set.[cite: 3]

Dimensionality reduction

is used when the number of features (dimensions) in a given data set is too high. It reduces the number of data inputs to a manageable size while also preserving the data integrity.[cite: 3]

Text mining

is the process of transforming unstructured text into a structured format to identify meaningful patterns and new insights.[cite: 3]

Python

is one of the most popular choices of programming language for data analysts because it is designed to be simple, efficient, and easy to read and write.[cite: 4]

algorithm

is a precise sequence of steps to produce a result.[cite: 4]

program

is an encoding of an algorithm in a language to solve a particular problem.[cite: 4]

syntax

a set of rules for specifying commands.[cite: 4]

Comments

are used by the programmer to document and explain the code, and are ignored by the computer.[cite: 4]

variable

is a name that refers to a location that stores a data value.[cite: 4]

Numeric

represented by integers (int), whole numbers, floats (float), real numbers with a decimal point, and complex numbers (complex), numbers with a real and an imaginary part.[cite: 4]

String

is an immutable sequence of Unicode characters, enclosed within single, double, or triple quotes in its literal representation.[cite: 4]

List

is a built-in data type in Python, which is a sequence of comma-separated items, enclosed in square brackets [].[cite: 4]

tuple

is a sequence of comma-separated items, enclosed in parentheses ().[cite: 4]

Range

represents an immutable sequence of numbers, often used in loops.[cite: 4]

Mapping Type

represented by dictionaries (dict), which are unordered collections of key-value pairs, enclosed in curly braces.[cite: 4]

Set Types

represented by set, which are Unordered collections of unique items, enclosed in curly braces, or frozenset, which are immutable versions of sets.[cite: 4]

Booleans (bool)

represent truth values, either True or False.[cite: 4]

Binary Types

that can be bytes, immutable sequences of bytes, bytearray, mutable sequences of bytes, or memoryview, which allows direct access to the internal data of an object.[cite: 4]

None

that represents the absence of a value.[cite: 4]

Jupyter Notebook

is the original web application for creating and sharing computational documents that offers a simple, streamlined, document-centric experience.[cite: 4]

Pandas

A fundamental library for data manipulation and analysis, providing DataFrames for structured data handling, cleaning, and transformation.[cite: 4]

NumPy

An essential library for numerical computing, offering powerful array objects and mathematical functions for efficient data operations.[cite: 4]

Matplotlib

A widely used library for creating static, interactive, and animated visualizations in Python, enabling the creation of various plot types.[cite: 4]

Seaborn

Built on Matplotlib, Seaborn simplifies the creation of aesthetically pleasing and informative statistical graphics.[cite: 4]

SciPy

Provides a collection of scientific computing tools, including modules for optimization, integration, interpolation, and statistical functions.[cite: 4]

Scikit-learn

A comprehensive machine learning library offering tools for classification, regression, clustering, dimensionality reduction, and more.[cite: 4]

Data cleaning

is the process of identifying and correcting errors or inconsistencies in the data.[cite: 4]

Data integration

involves combining data from multiple sources to create a unified dataset.[cite: 4]

Schema matching

aligns fields and data structures from different sources to ensure consistency.[cite: 4]

Data deduplication

identifies and removes duplicate entries across multiple datasets.[cite: 4]

Data transformation

converts data into formats suitable for analysis, machine learning, or mining.[cite: 4]

Label Encoding

assigns a unique numerical label to each category. Useful when there's an ordinal relationship between categories.[cite: 4]

One-Hot Encoding

creates new binary (0 or 1) columns for each category, indicating the presence or absence of that category. Ideal when there's no inherent order between categories.[cite: 4]

Standardization (Z-score normalization)

Rescales data to have a mean of 0 and a standard deviation of 1. Useful for algorithms sensitive to feature scales, like K-Nearest Neighbors or Support Vector Machines.[cite: 4]

Normalization (Min-Max scaling)

rescales data to a fixed range, typically 0 to 1.[cite: 4]

Data reduction

simplifies the dataset by reducing the number of features or records while preserving the essential information.[cite: 4]

Feature selection

choosing the most important features contributing to the analysis or model's performance.[cite: 4]

Principal component analysis (PCA)

a dimensionality reduction technique that transforms data into a lower-dimensional space.[cite: 4]

Sampling methods

reducing the size of the dataset by selecting representative samples is useful for handling large datasets.[cite: 4]

head

List the first 5 rows in the Dataframe[cite: 4]

tail

List the last 5 rows in the Dataframe[cite: 4]

describe

Shows descriptive statistics, which include central tendency, dispersion, and shape of a dataset's distribution, excluding NaN values.[cite: 4]

info

prints the summary of a DataFrame that includes the data type of each column, RangeIndex (number of rows), columns, non-null values, and memory usage.[cite: 4]

iloc

Returns one or more specified row(s)[cite: 4]

list

Print all the column names of the Dataframe[cite: 4]

to_string

Print the entire DataFrame[cite: 4]

dtypes

List the types of the columns[cite: 4]

columns

List the column names[cite: 4]

axes

List the row labels and column names[cite: 4]

ndim

Number of dimensions[cite: 4]

size

Number of elements[cite: 4]

shape

Return a tuple representing the dimensionality[cite: 4]

values

Numpy representation of the data[cite: 4]

max(), min()

Return max/min values for all numeric columns[cite: 4]

mean(), median(), mode()

Return mean/median/mode values for all numeric columns[cite: 4]

std()

Calculates standard deviation[cite: 4]

count()

Counts the number of non-null values[cite: 4]

sum()

Calculates the sum of values[cite: 4]

var()

Calculates the variance[cite: 4]

sample(n)

Returns a random sample of the data frame[cite: 4]

skew

Sample skewness[cite: 4]

kurt

Kurtosis[cite: 4]

agg

Useful for multiple statistics are computed per column[cite: 4]

dropna()

Drop missing observations[cite: 4]

dropna(how='all')

Drop observations where all cells is NA[cite: 4]

dropna(axis = 1, how='all')

Drop column if all the values are missing[cite: 4]

dropna(thresh = 5)

Drop rows that contain less than 5 non-missing values[cite: 4]

fillna(0)

Replace missing values with zeros[cite: 4]

isnull()

Returns True if the value is missing[cite: 4]

notnull()

Returns True for non-missing values[cite: 4]

duplicated()

Check the rows for duplication[cite: 4]

distplot

histogram[cite: 4]

barplot

estimate of central tendency for a numeric variable[cite: 4]