ELECTIVE MIDMIDDD

DATA ANALYSIS DEFINITION:

·         The process of inspecting, cleaning, transforming, and interpreting data to discover meaningful insights, patterns, and trends.

·         Involves using various techniques, tools, and methodologies to extract valuable information from data.

·         Plays a crucial role in numerous fields, including business, science, social sciences, healthcare, and many more.

DATA COLLECTION:

·         Begins with collecting relevant data from various sources.

·         Data can come from surveys, experiments, sensors, databases, websites, or any other means of data capture.

DATA CLEANING:

·         Raw data often contains errors, inconsistencies, or missing values.

·         Data analysts need to clean and preprocess the data to ensure its quality and reliability.

·         This may involve data imputation, removing duplicates, and handling outliers.

DATA TRANSFORMATION:

·         Data may need to be transformed into a suitable format for analysis.

·         This can include reshaping data, encoding categorical variables, scaling numeric variables, and more.

EXPLORATORY DATA ANALYSIS (EDA):

·         Involves visualizing and summarizing data to understand its characteristics.

·         This step can include creating histograms, scatter plots, box plots, and calculating descriptive statistics.

STATISTICAL ANALYSIS:

·         Statistical techniques are used to analyze data and test hypotheses.

·         This may involve hypothesis testing, regression analysis, ANOVA, and other statistical tests to uncover relationships between variables.

MACHINE LEARNING:

·         Applied to data when predictive modeling or pattern recognition is necessary.

·         Machine learning models can make predictions, classify data, or cluster data points, among other tasks.

DATA VISUALIZATION:

·         Data analysts often use charts, graphs, and other visualizations to present findings in a clear and accessible manner.

·         Data visualization aids in conveying complex information to a broader audience.

INTERPRETATION:

·         After conducting the analysis, data analysts interpret the results and draw conclusions.

·         They provide insights and recommendations based on their findings.

REPORTING:

·         The results of data analysis are typically documented in reports, presentations, or dashboards.

·         Effective communication of findings is crucial for decision-makers and stakeholders.

CONTINUOUS IMPROVEMENT:

·         Data analysis is an iterative process.

·         Analysts may revisit their methods and models, refine their approach, and incorporate new data to improve their analysis continually.

PIVOT TABLE DEFINITION:

·         A data processing tool used in spreadsheet software to summarize and analyze large datasets.

·         Allows users to rearrange and summarize data from one table or spreadsheet into a more compact, organized, and meaningful format for easier analysis and reporting.

·         Particularly useful when you have a lot of data and want to gain insights or make it more manageable.

PIVOT TABLE CHARACTERISTICS AND FUNCTIONS:

·         Data Summarization: Performs operations like sum, average, count, or other calculations on specific columns or fields.

·         Grouping and Categorization: Groups and categorizes data based on different attributes or fields, creating rows and columns that provide a structured view of the data.

·         Aggregation: Aggregates data based on various criteria. For example, groups sales data by product, date, or region and then calculates totals, averages, or other summary statistics for each group.

·         Filtering: Applies filters to the data within a pivot table, allowing you to focus on specific subsets of the data that are relevant to your analysis.

·         Dynamic: Allows you to easily change the arrangement and summary of data by dragging and dropping fields. This flexibility makes them valuable for exploring data from different angles.

·         Data Slicing: Enables you to “slice and dice” data, which means you can view data from various perspectives to uncover trends, patterns, or anomalies.

·         Charts and Graphs: Allows you to create charts and graphs directly from pivot table results, making it easier to visualize and communicate your findings.

MEAN (AVERAGE):

·         A measure of central tendency.

·         Calculated by summing up all the values in a dataset and then dividing that sum by the number of values.

·         Represents the “typical” or “average” value in the dataset.

·         Sensitive to extreme outliers, and its interpretation may be affected by skewed data.

MEDIAN:

·         A measure of central tendency in a dataset.

·         The middle value when the data is arranged in ascending or descending order.

·         If there is an even number of data points, the median is the average of the two middle values.

·         Less affected by extreme outliers than the mean and is often used when the data is not normally distributed or contains outliers.

MODE:

·         The value that appears most frequently in a dataset.

·         Useful for identifying the most common value or category in a dataset, especially in categorical data.

STANDARD DEVIATION:

·         A measure of dispersion or variability in a dataset.

·         Quantifies how much individual data points deviate from the mean.

·         A low standard deviation indicates that the data points are close to the mean, while a high standard deviation suggests that the data points are more spread out from the mean.

VARIANCE:

·         Measures the spread or dispersion of data points in a dataset.

·         Quantifies how far individual data points are from the mean.

·         A higher variance indicates greater variability, while a lower variance suggests that data points are closer to the mean.

RANGE:

·         A simple measure of spread and represents the difference between the maximum and minimum values in a dataset.

·         Provides a quick sense of how spread out the data is.

·         While it’s easy to calculate, the range can be sensitive to outliers and may not provide a complete picture of data dispersion on its own.

POPULATION:

Any specific collection of objects of interest.

SAMPLE:

Any subset or subcollection of the population.

MEASUREMENT:

A number or attribute computed for each member of a population or of a sample.

PARAMETER:

A number that summarizes some aspect of the population as a whole.

STATISTIC:

A number computed from the sample data.

STATISTICS:

The study of data: describing properties of the data, which is called descriptive statistics.

DESCRIPTIVE STATISTICS:

The branch of statistics that involves organizing, displaying, and describing data.

INFERENTIAL STATISTICS:

The branch of statistics that involves drawing conclusions about a population based on information contained in a sample taken from that population.

STEPS IN THE DATA ANALYSIS PROCESS:

Step 1:

·         Decide on the objectives or pose a question

·         Define the purpose and scope of the data analysis

·         Identify the problem or question to be answered

Step 2:

·         What to measure and how to measure

·         Select the variables or attributes of interest

·         Choose the appropriate methods and tools for data collection and measurement

Step 3:

·         Data collection

·         Gather relevant data from various sources

·         Ensure the data quality and reliability

Step 4:

·         Data cleaning

·         Remove errors, inconsistencies, or missing values from the data

·         Preprocess the data to make it suitable for analysis

Step 5:

·         Summarizing and Visualizing Data:

·         Exploratory data analysis helps to understand the data better.

·         Pictures are often more effective than lectures.

·         Measures of Variance indicate the distribution of the data around the center.

·         Correlation refers to the degree to which two variables move in sync with one another.

Step 6:

·         Data Modeling:

·         Build models that correlate the data with your business outcomes and make recommendations.

·         This is where the unique expertise of data scientists becomes important to business success.

·         Correlating the data and building models that predict business outcomes.

Step 7:

·         Optimize and Repeat:

·         The data analysis is a repeatable process and sometimes leads to continuous improvements, both to the business and to the data value chain itself.

DATA MATRIX:

·         A tabular format representation of cases and variables of your statistical study.

·         Each row represents a case and each column represents a variable.

WHEN TO USE WHAT MEASUREMENT OF CENTRAL TENDENCY:

·         If data is categorical (nominal or ordinal), use mode.

·         If data is quantitative, use mean or median.

·         If data has influential outliers or is highly skewed, use median. Otherwise, use mean.

RANGE, INTERQUARTILE RANGE AND BOX PLOT:

·         Range is the difference between the highest and lowest value in a dataset.

·         Interquartile range is the difference between the third and first quartile in a dataset. It leaves out the extreme values.

·         Box plot is a graph that shows the median, quartiles, and outliers of a dataset. It is useful for detecting outliers and comparing distributions.

VARIANCE AND STANDARD DEVIATION:

·         Variance measures the average of the squared differences between each data point and the mean in a dataset.

·         Standard deviation measures how spread out the data points are from the mean in a dataset. It is the square root of the variance.