Applied Data Analysis in Statistics PSYC 3301 Exam 1

research in applied settings

goal to solve a problem. setting naturalistic. low control. low internal validity, high ecological validity. replication is difficult. participants are specific community members. ethical concerns include privacy, potential lack of consent, involved process.

research in academic settings

goal to advance knowledge, test a theory. setting is artificial and controlled. control is high. internal validity is high and ecological validity is low. replication is easy. participants are random. ethical concerns include informed consent, deception, etc

quantitative approach

a priori study design and plan (theory → hypothesis → design → study)

data collection procedures involve an instrument that can be scored numerically, reliably, with little training

data analysis procedure includes using statistical analyses software and doing inferential statistics

qualitative approach

research in naturalistic paradigm where knowledge is constructed based on experiences (observe → data → analyze → knowledge)

data collection subjective. usually gathered from interviews, observations, narrative documents (ex. journals).

data analyses is usually thematic and non inferential.

double barrelled questions

ex. “how much do you like statistics class and also world peace on a scale of one to five?”

asking two qs in one (bad)

survey design strategy for getting quality data

attention checks, timers, force response q’s, captchas, reasonable length, participant reflection

FAIR Data Principles

Findable: doi publicly available

Accessible: dataset codebook for interpretation of data

Interoperable: should be possible to open and run in multiple data analyses tools (ex. csv file not just sav file)

Reusable: should include most/all of data not just the variables you care about so people can study diff factors

how to check for low quality/ bot responses (data auditing/wrangling)

1) Duration: how long did it take them to complete the survey? way short? way longer?

2) IP: If more than three responses from the same IP address that’s fishy

3) Open Ended Written Response: What are they saying? is it gibberish or are they just being quirky w the speech?

why create data composites?

REVIEW THIS DEFINITION it is the mean value for a groups of variables in your dataset. you can use this value to compare values and variables.

before doing check reliability?

old methods of dealing with missing data

1) fill in missing values with overall mean of data (bad because it reduces variability due to artificial consistency)

2) listwise deletion, aka eliminate missing data (bad because reduces sample size)

multiple imputation

a current best practice for dealing with missing data

create multiple datasets with all possible values filled in for every missing datapiece. avoids bias of deletion, preserves power, maintains validity

used in standard inferential stats because it maintains variability

nearest neighbour PCA (principle computer analyses)

a current best practice for dealing with missing data

uses info from the nearest neighbours in the dataset to decide what value is most likely, fills in

used in huge datasets and in predictive stats

data anonymization

remove all personal identifiers from data (ex. names, IP address) and store in a separate and secure file, matched with a unique alphanumeric code in dataset for your personal matching that only you know

data minimization too, only collect what is strictly necessary

technical safeguards (data security)

strong encryption for all data and storage

physical admin safeguards (data security)

locked storage

team training (ethics, privacy, ex, TPCS)

disposal protocols (destruction timeline to securely wipe digital files, usually seven years)

informed consent and compliance

needs both transparent communication (details how data is collected, stored, who will have access) and institutional approval (from a research ethics board)

when to report the median

when data is ordinal OR if data is interval or ratio but skewed

when to report mean

data is interval or ratio and not skewed

when to report mode

when data is nominal (ex. frequency distro)

cronbach’s alpha

single numerical value representing how reliable your scale is

assesses whether multiple items meant to asses the same construct have the same answers

0.70 is strong for us. clinical psyc want 0.80 or 0.90