Machine Learning Terms

sample space

a set of possible outcomes in your domain.

random variable

a function defined over the sample space S.

Event

a subset of S.

Probability

a real function defined over the events in the sample space

probability has value between 0 and 1.

probability has value between 0 and 1 equation

probability of true is 1

an event in which all outcomes occur; i.e. either heads or tails occurs

probability of false is 0

an event in which no outcomes occur; i.e. neither heads or tails occurs

equation of the probability of A or B

b/c the probability of a&b occurs twice w/in a venn diagram one must be subtracted. the probability of a plus the probability of b minus the prob. of a&b equals the prob of a or b.

Mutually exclusive events

one event doesn’t include the outcome of another event (if a coin lands on heads then it hasn’t landed on tails—two mutually exclusive events).

Probability mass function

all possible events in sample space S should add up to 1. (i.e., P(A)= heads & P(A)=tails would both equal 1 b/c coinds eitehr lande on heads or tails)

P, A_i

P= probability mass function

A_i= a specific value in the discretization of a continuous quantity

Sum Rule

used to prove that P(A)=1-P(notA) and P(A)=P(A&B)+P(A&notB) recall that A and notA are mutually exclusive events; if one occurs the other one doesn’t)

supervised learning

identify, based on previous input-output examples of this function, which class the vector can be mapped onto. standard formulation: classification

unsupervised learning

given a set of inputs w/ no output mapped to them, find a pattern in the inputs via clustering (grouping instances).

semi-supervised learning

labeled and unlabeled functions to generate an appropriate function.

reinforcement learning

algorithm is taught a policy of how to act given an observation. after the agent causes the mechanism there is a feedback mechanism it uses to understand its mistakes and improve future reactions.

Machine learning algorithms have the following in common

representation (how is data represented/what hypothesis space does it belong to?), evaluation (helps algorithm determine if its on the right path/ how its performance can be improved), and optimization (how to optimize algorithms for the data in question).

applications for supervised learning

situations where human experts are hard to find; situations where humans can do the task but can’t describe it; situations where y output changes quickly due to new info; situations where a customer requires a unique function.