Data Analysis Quiz

Data analysis

sorting, organizing, and summarizing your data so that patterns become visible, then checking those patterns against your research question.

Collection

gathering the raw material: survey responses, interview transcripts, test scores, observation notes, texts, posts, documents.

Analysis

sorting, labeling counting, comparing, and deciding what that all means

Qualitative data

words, stories, images, behaviors, documents
includes interviews, open ended surveys, speeches, and articles
answers How? Why? What is it like? What meanings are made?

Basic qualitative analysis

label chunks of data and group the labels into patterns

Qualitative bias

seeing stories you expect to see

Quantitative data

numbers, counts, scores, measurements
includes test scores, likert scale ratings, minutes psent, frequencys
Answers How much? How many? Is there a difference? Are these connected?

Basic quanitative analysis

summarizing with numbers, comparing groups, looking for relationships

Quantitative bias

Choosing numbers or tests that give the answers you wanted

Mixed methods

combines both quali and quant; you need a plan for each kind of data and a plan for how the two will be combines

Big idea

filter your data with rules not by instinct

Objectivity

using rules that apply the same way for every piece of data, written down before you start

What a good analysis plan answers

What counts?
How will I sort it?
How will i check myself?
How does this answer my research question?

Sample

group or set of sources you collect data from

Variable

something that can change or differ from person to person

Unit of analysis

“thing” you are analyzing (person, post, speech, school)

Code

short label attached to piece of qualitative data

Theme

patterns that appear across many codes

Codebook

written list of codes w definiions and examplesm

mean

arithmetic averages

median

middle most value

mode

most common value

correlation

two things that tend to rise or fall together

causation

one thing actually makes the other happen

reliability

can someone else get the same result using the same steps?V

validity

are you really measuring what you say you are measuring?

Qualitative data analysis

raw data —> code —> themes —> an answer to teh RQ

Qualitative pipeline

prepare data —> read it all once with coding —> build code book —> code everything using the same rules —> group codes into themes —> test your themes —> answer RQ

Qualitative options

thematic analysis, content anlaysis, case/comparative analysis, narrative analysis, document or source analysis (Historical), sentiment analysis, grounded theory-style coding

Intercoder reliability

having a second person code a sample using the codebook to see if you get similar answersin

intracoder reliability

you code the same sample twice, a week apart, and compare results

count and percentage

how many and what share of the total

range

lowest to highest

Standard deviation

on average, how far values sit from the mean (small means tightly clustered and large means spread out)

Graph

shape of the data at a glance

Descriptive stats

summarizes teh data with counts, percentages, averages and graphs
asks for how much or how many

t-test

(compares means of 2 groups) asks whether the gap between two averages is bigger than random change would produce
asks is there a differncebetween two groups

ANOVA

compares three or more averages at once
asks if there is a difference between three or more groups

Chi-square

compares the counts you actually saw against the counts you would expect if there was no connection
is there a pattern in my categories difference from what change would give?

Correlational (pearsons R)

Do two measures rise and fall tgt?
A number from -1 to 1 showing the direction and strength of the connection (no causation)

Simple regression

A line of best fit though a scatterplot
can one measure predict another?

Trend analysis

Graph thevalue in time order, calculate percent change, and describe pattern

How has something changed over time?

Comparing before and after averages

Same sample two time points
Paired t test or percent change

P-value

probability of seeing a difference this big if change alone were at work
Below 0.05 is statistically significant

Action research

before and after measures, reflections, observations

compares before and after (percent change and avergaes), thematic analysis

watch for changing the plans mid cycle without recording why

case study

interviews, docyuments, observations

comparative case matrix, pattern matching, thematic analysis

watch for treating one case as proof for everyone

Content anaylsis

tests, media, posts, documents

codebook with categoru counts, intercoder agreement, chi-sqaure to compare,

watch for a sample of texts picked because it supports claim

Correlational

ttwo or more measured variables

scatterplots, pearsons r, simple regression,

watch for claiming causation

Descriptive research

surverys, meausrements, existing recordings

uses counts percentages, mean, median, graph

Watch for reporting only the average and hiding teh spread

Ethnography

thematic coding of field notes, cultural patterns table

watch for seeing what you expect in a setting you already know well

Experimental

compares group averages (t test, anova) reports size of the difference

watch for groups that differ in ways other than the treatment

Evaluation

compare results to pre set criteria, before and after comparion, thematic analysis
watch for defining success after seeing results

Focus groups

thematic analysis noting which themes came from many voices and from one loud voice
watch for group think and dominant speakers

Grounded theory

uses constant comparison coding with memo notes

watch for entering with a theory already decided

Historical research

uses source evaluation with same questions for every source, timeline or matrix,

watch for using only sources that agree with you

Interviews

uses thematic analysis with a codebook, second coder on a sample

watch for leading questions and selective quoting

Observational studies

pre-set behavior checklist with frequency counts, compare conditions, second observer on a sample

watch for recording only what stands out

Mixed methods

a plan for each data type pluys a stated way of combined them

watch for two separate studies that never meet

Narrative research

uses narrative analysis and thematic comparsion

watch for retelling stories in you own words and changingthe meaning

Needs assessment

gap analysis (what exists vs what is needed), rank needs by how many ppl report them and how serious they are

watch for mistaking priorities for community needs

Phenomenography

group responses into distinct categories of understanding then describe the relationships between categories

watch for ranking some perspectives as superiorSen

Sentiment analysis

defined sentiment took or rubric, comparing across groups or time
watch for tool that misread sarcasm, slang, or dialect

Statistical analysis

uses descriptive states and the test matching the question
watch for leading questions, low or lopsides response rates

Surveys

uses counts and percentages for lcosed items and comparion across groupos, thematic analysis for open ended

Trend analysis

uses time ordered line graph percent change and trend line
watch for a time window chosen to make trend look the way you want

Confirmation bias

noticing and saving the data that fits your hypothesis

Selection bias

choosing which participants, cases, or texts to include in a way that favors the conclusion

Obsevrer bias

recording what you expect to see

Sampling bias

a sample that leaves out certain groups

Interpretation bias

explaining the findings in the way you hoped

publication bias

highliughting onlu positive or expected results

funding biased

letting a sponder or stakeholder shape the conclusions

anchoring bias

giving too much weight to your first impressions or early data

cultural bias

reading other’ words or behavioor through your own norms

Measurement bias

an instrument or scoring rule that distorts results

Social desireability

asnwers that sound too good or too polites

Recall bias

vague or inconsistent memories of past events

Acquiscence bias

many agree answers, regardless of statement

Non response bias

many skipped items or ppl who never answers

Hawthorne effect

behavior that changes when ppl know thet are watched

demand characteristics

participants guessing what you want and gviing it toyou

Slef-selection bias

volunteers who care more about the topic than everyone else

Extreme response bias

answers clustered at 1 or 5

Central tendency bias

answers clustered at 3 or neutral

Recall heuristic bias

frequency estimates driven by wchieven examples come to mind

Turning safeguards into writing

do not just list biases, connect each to a decision

Top scores

require conclusions built on logical sequence of choices, enough evidence, an honest account of limitations, and implications for the field