DSS110 (Lesson 4 & Pandas)

data collection

This is the process of collecting, measuring and analyzing different types of information using a set of standard validated techniques.

• primary data collection

• secondary data collection

These are the two main methods of data collection in research based on the information that is required.

primary data

This refers to data collected from first-hand experience directly from the main source.

primary data

This refers to data that has never been used in the past.

primary data

This is generally regarded as the best kind of data in research

• quantitative data collection

• qualitative data collection

These are the two types of primary data collection methods.

quantitative data collection

This primary data collection method deals with factors that can be counted

qualitative data collection

This primary data collection method deals with factors that are not necessarily numerical in nature

interview

These are a direct method of data collection.

Simply a process in which the interviewer asks questions and the interviewee responds to them.

It provides a high degree of flexibility because questions can be adjusted and changed anytime according to the situation.

observation

In this primary data collection method, researchers observe a situation around them and record the findings.

Surveys and questionnaires

These provide a broad perspective from large groups of people. They can be conducted face-to-face, mailed, or even posted on the Internet to reach respondents from anywhere in the world.

focus group

A ________ is similar to an interview, but it is conducted with a group of people who all have something in common.

focus group

In this method, the data collected is similar to in-person interviews, but they offer a better understanding of why a certain group of people thinks in a particular way.

Oral histories

_________ also involve asking questions like interviews and focus groups. However, it is defined more precisely, and the data collected is linked to a single phenomenon.

Oral histories

It involves collecting the opinions and personal experiences of people in a particular event that they were involved in

Secondary data

________refers to data that has already been collected by someone else. It is much more inexpensive and easier to collect than primary data.

internet

The use of the ______ has become one of the most popular secondary data collection methods in recent times.

government archives

In this secondary data collection method, the most important advantage is that the data in _________ are authentic and verifiable. The challenge, however, is that the data are not always readily available due to a number of factors.

libraries

Most researchers donate several copies of their academic research to _______.

You can collect important and authentic information based on different research contexts.

data preprocessing

This is a component of data preparation that describes any type of processing performed on raw data to prepare it for another data processing procedure.

data preprocessing

This process transforms the data into a format that is more easily and effectively processed in data mining, machine learning and other data science tasks.

preprocessing

Sampling, transformation, imputation, and normalization are all different tools and methods used for ________ data.

sampling

This data preprocessing method selects a representative subset from a large population of data.

transformation

This data preprocessing method manipulates raw data to produce a single input.

denoising

This data preprocessing method removes noise from data

imputation

This data preprocessing method synthesizes statistically relevant data for missing values.

normalization

This data preprocessing method organizes data for more efficient access

feature extraction

This data preprocessing method pulls out a relevant feature subset that is significant in a particular context

data profiling

This is the step in data preprocessing that includes examining, analyzing, and reviewing data to collect statistics about its quality.

data cleansing

This step in data preprocessing aims to find the easiest way to rectify quality issues, such as eliminating bad data, filling in missing data, or otherwise ensuring the raw data is suitable for feature engineering.

data reduction

This step in data preprocessing uses techniques like principal component analysis to transform the raw data into a simpler form suitable for particular use cases.

data transformation

This step in data preprocessing could include things like structuring unstructured data, combining salient variables when it makes sense, or identifying important ranges to focus on.

data enrichment

In this step for data preprocessing, data scientists apply the various feature engineering libraries to the data to effect the desired transformations.

data validation

In this step of data preprocessing, the data is split into two sets.

The first set is used to train a machine learning or deep learning model.

The second set is the testing data that is used to gauge the accuracy and robustness of the resulting model.

Pandas

______ is an open source, BSD-licensed library providing high-performance, easy to use data structures and data analysis tools for the Python programming language.

manipulation

Pandas is for data _______ and analysis.

Matplotlib

This can be used in Python scripts, Jupyter notebook, web application servers, and for graphical user interface toolkits.

Seaborn

This is a Python data visualization library based on matplotlib.

numeric

This strategy to handle missing values in data preprocessing can be applied on a feature which has ______ data. We can calculate the mean, median, or mode of the feature and replace it with missing values.

textual method

This method of data presentation is the simplest one.

The demerit of this method, however, is that one must read the whole text to get a clear picture.

tabular method

This presentation method is characterized by a systematic arrangement of information into columns and rows.

simple table

_______ is a table which contains one property only or depends on one character. This kind of table consist of two lines only.

complex table

_______ is a table which contains more than one property at the same time. This kind of table consists of two lines only.

frequency distribution table

This table is the most essential kind of table.

It is a statistical table showing the frequency or number of observations contained in each of the defined classes or categories.

diagrammatic and graphical data presentation

This is a technique of presenting data through pictograms, cartograms, bar diagrams, and pie diagrams.

histogram

This is the most widely used graph for presentation of quantitative (continuous data)

bar chart

It is similar to a histogram, but there are constant (fixed) spaces between rectangles (bars).

The real class cannot be used to draw a bar chart because there is space between classes.

bar chart

Usually, _________ is used for presenting discrete (separated) or qualitative
data. The rectangles (bars) can be plotted vertically or horizontally.

line graph

This chart uses data points connected by straight lines to show how values change over a continuous interval, often time.

pie chart

It is a circle that is divided into sections according to the percentage of frequencies in each category of the distribution.

pictorial chart

This is also called a pictogram, a pictograph, or a picture chart.

It is a visual representation of data that uses pictograms – icons or pictures in relative sizes – to highlight data patterns and trends.

ogive (cumulative frequency polygon)

A graph showing the cumulative frequency (less than or more than type) plotted against upper or lower class boundaries, respectively

frequency polygon

It is a line graph where the frequency is placed along the vertical axis (Y axis) and Class marks

Pandas

A fast, powerful, flexible, and easy-to-use open-source data analysis and manipulation tool built on top of the Python programming language.

Pandas

It is particularly well-suited for working with tabular data and time series data.

Series, DataFrame

The two primary data structures of Pandas are ç and ________.

Series

These are one-dimensional labeled arrays used in Pandas.

DataFrame

This is a two-dimensional labeled data structure with columns of potentially different types.

Series

It is very similar to a NumPy array, but with an explicit index, which allows for more flexible data alignment and labeling.

index

When creating a Pandas Series, this argument keyword provides a custom index for the Series.

pd.Series()

What is the constructor for creating a Pandas Series object?

pd.DataFrame()

What is the constructor for creating a Pandas DataFrame obkect?