data science final exam 🪩⏱️🪩⏱️🪩⏱️

How is data stored for pivot table analysis?

Flat file structure, all related data must be in the same table to be pivoted.

structured data

well defined fields corresponding to distinct variables

examples of structured data

google sheets, Microsoft excel files, employee name, ids, addresses, etc.

unstructured data

mishmash of semantic entities that can differ from an observation to another

unstructured data examples

emails, text files, social media posts, videos, images, audios, sensor data, etc.

sentiment analysis

Categorizes whether a statement is positive or negative and assigns it a sentiment score accordingly (score = structured data).

Simplest way to do it is with a word library

it’s a form of predictive analysis: predicts sentiment score that human evaluator would give.

benefits of sentiment analysis

Saves time, simplifies process

when can sentiment analysis be innacurate

  Can be hard because statements can include good and bad words: “your class was awfully good”

pipeline sentiment analysis

: data is processed through several applications

analytics

extracting information from data by discovering meaningful patterns.

o   How is analytics related to Big Data?

turns big data info into decisions

descriptive analytics

summarizes and helps understand past data

descriptive analytics example

detect fraudulent behavior, identify top selling products, analyze patient treatment success rates, review player performance states

predictive analytics

predicts values for data points we don’t have.

forecasting

o  form of predictive analysis to predict future data points, example: weather.

predictive analytics examples

predict game outcomes based on past trends, forecast risk of disease for specific patients, predict future customer purchase behaviors, weather forecasting

prescriptive analytics

o   facilitates decision making directly by suggesting an action or automatically triggering it (algorithmic management)

prescriptive analytics examples

whether or not to grant a car loan, reccommend optimal pricing strategies, suggest personalized patient treatment plans, reccommend ticket pricing for events.

confidence/predictor interval

tells us the accuracy of our prediction. Ex: hitting bin with paper ball 99 times, with 100 pieces of crumpled paper – size of bin equals common 95 percent prediction interval.

machine learning

Broad category of techniques to learn model parameters from data so model can be used to predict values or classify items, AKA traditional/discriminative AI.

pivot table analysis

Data summarization tools allowing for quick organization, analyzation, and aggregation of large datasets.

interactive dashboard

Gives multiple perspectives to data rather than just one visualization, allow exploring data from limited set of options to customize visualizations and drill deeper into the data.

How does confidence interval change when choosing 90% level instead of 95%?

Interval becomes narrower and more precise, lower level requires smaller margin of error.

What factors influence the confidence interval of a trend line (i.e., the Tableau exercise)?

Sample size, data variability, chosen confidence level. Larger samples, lower data variability, lower confidence levels can result in more lower, precise intervals.

·       How does sentiment analysis software determine a positive/negative score for a block of text?

Relies on predefined dictionary with labeled words.

·       What is a word frequency analysis? What insight can it give you about a block of text?

Word cloud, can tell you what words are most prominent == largest.