Introduction to Structured Data and Statistical Concepts

0.0(0)
studied byStudied by 0 people
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
Card Sorting

1/211

encourage image

There's no tags or description

Looks like no tags are added yet.

Study Analytics
Name
Mastery
Learn
Test
Matching
Spaced

No study sessions yet.

212 Terms

1
New cards

Continuous

Data that can take on any value in an interval.

2
New cards

Discrete

Data that can take on only integer values, such as counts.

3
New cards

Categorical

Data that can take on only a specific set of values representing a set of possible categories.

4
New cards

Binary

A special case of categorical data with just two categories of values (0/1, true/false).

5
New cards

Ordinal

Categorical data that has an explicit ordering.

6
New cards

Data frame

Rectangular data (like a spreadsheet).

7
New cards

Feature

A column in the table.

8
New cards

Outcome

A variable being predicted.

9
New cards

Record

A row in the table.

10
New cards

Mean

The sum of all values divided by the number of values.

11
New cards

Weighted mean

The sum of all values times a weight divided by the sum of the weights.

12
New cards

Median

The value such that one-half of the data lies above and below.

13
New cards

Weighted median

The value such that one-half of the sum of the weights lies above and below the sorted data.

14
New cards

Trimmed mean

The average of all values after dropping a fixed number of extreme values.

15
New cards

Robust

Not sensitive to extreme values.

16
New cards

Outlier

A data value that is very different from most of the data.

17
New cards

Deviations

The difference between observed values and the estimate of location.

18
New cards

Variance

The sum of squared deviations from the mean divided by n-1.

19
New cards

Standard deviation

The square root of the variance.

20
New cards

Mean absolute deviation

The mean of the absolute value of the deviations from the mean.

21
New cards

Median absolute deviation from the median

The median of the absolute value of the deviations from the median.

22
New cards

Range

The difference between the largest and the smallest value in a data set.

23
New cards

Percentile

The value such that P percent of the values take on this value or less and (100-P) percent take on this value or more.

24
New cards

Interquartile range

The difference between the 75th percentile and the 25th percentile.

25
New cards

Boxplot

A plot introduced by Tukey as a quick way to visualize the distribution of data.

26
New cards

Frequency table

A tally of the count of numeric data values that fall into a set of intervals (bins).

27
New cards

Histogram

A plot of the frequency table with the bins on the x-axis and the count (or proportion) on the y-axis.

28
New cards

Density plot

A smoothed version of the histogram, often based on a kernel density estimate.

29
New cards

Mode

The most commonly occurring category or value in a data set.

30
New cards

Expected value

When the categories can be associated with a numeric value, this gives an average value based on a category's probability of occurrence.

31
New cards

Bar charts

The frequency or proportion for each category plotted as bars.

32
New cards

Pie charts

The frequency or proportion for each category plotted as wedges in a pie.

33
New cards

Correlation coefficient

A metric that measures the extent to which numeric variables are associated with one another (ranges from -1 to +1).

34
New cards

Correlation matrix

A table where the variables are shown on both rows and columns, and the cell values are the correlations between the variables.

35
New cards

Scatterplot

A plot in which the x-axis is the value of one variable, and the y-axis the value of another.

36
New cards

Contingency tables

A tally of counts between two or more categorical variables.

37
New cards

Hexagonal binning

A plot of two numeric variables with the records binned into hexagons.

38
New cards

Contour plots

A plot showing the density of two numeric variables like a topographical map.

39
New cards

Violin plots

Similar to a boxplot but showing the density estimate.

40
New cards

Sample

A subset from a larger data set.

41
New cards

Population

The larger data set or idea of a data set.

42
New cards

N (n)

The size of the population (sample).

43
New cards

Random sampling

Drawing elements into a sample at random.

44
New cards

Stratified sampling

Dividing the population into strata and randomly sampling from each strata.

45
New cards

Simple random sample

The sample that results from random sampling without stratifying the population.

46
New cards

Sample bias

A sample that misrepresents the population.

47
New cards

Sample statistic

A metric calculated for a sample of data.

48
New cards

Data distribution

The frequency distribution of individual values in a data set.

49
New cards

Sampling distribution

The frequency distribution of a sample statistic over many samples.

50
New cards

Central limit theorem

The tendency of the sampling distribution to take on a normal shape as sample size rises.

51
New cards

Standard error

The variability of a sample statistic over many samples.

52
New cards

Bootstrap

A method to estimate the sampling distribution by drawing additional samples with replacement from the sample itself.

53
New cards

Bootstrap sample

A sample taken with replacement from an observed data set.

54
New cards

Resampling

The process of taking repeated samples from observed data.

55
New cards

Confidence level

The percentage of confidence intervals expected to contain the statistic of interest.

56
New cards

Interval endpoints

The top and bottom of the confidence interval.

57
New cards

Error

The difference between a data point and a predicted or average value.

58
New cards

Standardize

Subtract the mean and divide by the standard deviation.

59
New cards

z-score

The result of standardizing an individual data point.

60
New cards

Standard normal

A normal distribution with mean = 0 and standard deviation = 1.

61
New cards

QQ-Plot

A plot to visualize how close a sample distribution is to a normal distribution.

62
New cards

Tail

The long narrow portion of a frequency distribution, where relatively extreme values occur at low frequency.

63
New cards

Skew

Where one tail of a distribution is longer than the other.

64
New cards

n

Sample size.

65
New cards

Degrees of freedom

A parameter that allows the t-distribution to adjust to different sample sizes.

66
New cards

Trial

An event with a discrete outcome (e.g., a coin flip).

67
New cards

Success

The outcome of interest for a trial.

68
New cards

Binomial trial

A trial with two outcomes.

69
New cards

Binomial distribution

Distribution of number of successes in x trials.

70
New cards

Lambda

The rate at which events occur.

71
New cards

Poisson distribution

The frequency distribution of the number of events.

72
New cards

Exponential distribution

The frequency distribution of the time or distance from one event to the next.

73
New cards

Weibull distribution

A generalized version of the exponential, in which the event rate is allowed to change over time.

74
New cards

Treatment

Something to which a subject is exposed.

75
New cards

Treatment group

A group of subjects exposed to a specific treatment.

76
New cards

Control group

A group of subjects exposed to no (or standard) treatment.

77
New cards

Randomization

The process of randomly assigning subjects to treatments.

78
New cards

Subjects

The items that are exposed to treatments.

79
New cards

Test statistic

The metric used to measure the effect of the treatment.

80
New cards

Null hypothesis

The hypothesis that chance is to blame.

81
New cards

Alternative hypothesis

Counterpoint to the null (what you hope to prove).

82
New cards

One-way test

Hypothesis test that counts chance results only in one direction.

83
New cards

Two-way test

Hypothesis test that counts chance results in two directions.

84
New cards

Permutation test

The procedure of combining two or more samples together, and randomly reallocating the observations to resamples.

85
New cards

With or without replacement

In sampling, whether or not an item is returned to the sample before the next draw.

86
New cards

P-value

Given a chance model that embodies the null hypothesis, the p-value is the probability of obtaining results as unusual or extreme as the observed results.

87
New cards

Alpha

The probability threshold of "unusualness" that chance results must surpass, for actual outcomes to be deemed statistically significant.

88
New cards

Type 1 error

Mistakenly concluding an effect is real (when it is due to chance).

89
New cards

Type 2 error

Mistakenly concluding an effect is due to chance (when it is real).

90
New cards

t-statistic

A standardized version of the test statistic.

91
New cards

t-distribution

A reference distribution to which the observed t-statistic can be compared.

92
New cards

False discovery rate

Across multiple tests, the rate of making a Type 1 error.

93
New cards

Adjustment of p-values

Accounting for doing multiple tests on the same data.

94
New cards

Overfitting

Fitting the noise.

95
New cards

d.f.

Degrees of freedom.

96
New cards

Pairwise comparison

A hypothesis test between two groups among multiple groups.

97
New cards

Omnibus test

A single hypothesis test of the overall variance among multiple group means.

98
New cards

Decomposition of variance

Separation of components contributing to an individual value.

99
New cards

F-statistic

A standardized statistic that measures the extent to which differences among group means exceeds what might be expected in a chance model.

100
New cards

Sum of squares

Referring to deviations from some average value.