Lesson 5: Explaining Data Intergration and Collection Methods

Transforming data

Act of making the data more meaningful for the purposes of reporting and decision-making

• can involve many different techniques

• in data warehousing, this step of ETL allows business users to access data that has already been cleaned and transformed

Loading data

Process of moving the data into the tool or its target destination, such as a data warehouse

Full load

Method of loading all data into a data system for the very first time

Delta load

Method of loading new data into a data system and updating any existing data that has changed since the last load

Extract, Load, Transform (ELT) process

Allows data to be moved into data storage systems faster because the transformations take place after the data is loaded

• ideal for data lakes because they hold more real-time data that is updated minute-by-minute

• meant to increase data availability and improve processing times

Application Programming Interface (API)

Set of protocols within a computer system that allow two unrelated systems to communicate

• ability to share data across systems

• biggest benefit is the ability to access data from a dedicated system

Web services

Type of API that allows a hosted computer on a network to share data back and forth with a computer in the same hosted environment

• are an API, but not all APIs are this

• key difference between this and most APIs is the use of hosted network

Synchronous web service

System that calls on the web service wants for a response to the request

Asynchronous web service

Other functions can continue so they are not stopped and waiting

Web scraping

Act of pulling information from a website

• can be done manually by hand or a specific tool can be used

• also called data scraping

Machine data

Refers to data that is produced by a machine rather than a human

• biggest value is that we don’t have to enter any of it by hand, it’s built to generate data in various formats for analysis

Surveys

When designed properly and share effectively, they provide a valuable source of data for research, insight, and analysis

Considerations for effective surveys

• whether the provided answer options can elicit an accurate and useful response

• how the results are presented

• free from bias such as leading questions

Bad question design (leading)

“how awesome was your customer service today?”

• extremely awesome, awesome, somewhat awesome

Better question design

“overall, how would you rate the quality of your customer service experience?”

• very positive, positive, neutral, poor, very poor

Types of survey answers

• text-based

• single choice

• multiple choice

• likert