1/26
Covers Introduction PPT and Pandas-DataWrangling
deep learning
subset of machine learning composed of algorithms that permit software to train itself to perform tasks, like speech and image recognition, by exposing multilayered neural networks to vast amounts of data
machine learning
subset of AI that includes abstruse statistical techniques that enable machines to improve at tasks with experience
artificial intelligence
any technique that enables computers to mimic human intelligence, using logic, if-then rules, decision trees, and machine learning (including deep learning)
data set
list of records with values for some predefined fields
x causes y only if all 3 hold
x is correlated with y
x comes before y in time
association isn’t explained by confounding variable
gold standard for establishing causality is a
randomized experiment
t/f: random assignment balances confounders across groups, so any difference must be caused by the treatment
true
why do you convert categorical variables to dummy variables?
categorical variables can’t go straight into a regression
inner join
keeps only keys found in both tables; default
left join
keeps every row in the left table
right join
keeps every row from the right
outer join
keeps all rows from both, provides NaN when no match
what can you do with NaN?
drop them (dropna) —> loses data
fill them (fillna) —> e.g. with mean, can introduce bias
binning/discretization
turns continuous variable into categories
inplace=True
changes original DataFrame; otherwise, Pandas returns a modified copy
issues with AI
adversarial attack
fake content
algorithmic bias
interpretability
algorithmic collusion
violation of privacy
environmental issues
copyright issues
designed system
a personal writes the rules
learned system
the computer finds the rules itself from the data → black box
pivot table
rearranges data into a spreadsheet-like summary
crosstab
special pivot table that counts freqiuencies