Research: Midterm 2

Descriptive statistics

Brief descriptive coefficients that summarize a given data set. Can be either a representation of the entire population or a sample of a population.

1. Broken down into measures of central tendency (mean, median, mode): describe center of data set

2. Measures of variability (spread): describe dispersion of data within the set (range, variance, standard deviation)

3. Frequency distributions: table, chart, figure; shows intervals; used to display & describe data

Inferential statistics

Hypothesis testing

1. Probability: infers info about population

2. Based on an assumption of normal distribution, hypothesis testing, likelihood of an event occurring denoted by “p”

3. Allows you to make predictions (inferences) from that data

Take data from samples and make generalizations about a population

Correlational statistics

Method of assessing a possible two-way linear association between two continuous variables 

1. Correlation: an association, connection, or any form of relationship, link or correspondence. Measured by a statistic called the correlation coefficient, which represents the strength of the putative/supposed linear association between the variables in question

Play type one error/false positive

When a true null hypothesis is incorrectly rejected, leading to false conclusion that an effect/difference exists

1. False positive relationship between research variables

Play type two error/false negative

• When a researcher fails to reject a false null hypothesis in the population

• concluding there is no significant effect or difference, even though one actually exists often due to small sample size or low statistical power 

What is a null hypothesis?

1. No significant difference between specified populations

2. Any observed difference due to sampling or experimental error

3. What is happening due to chance (no relationship)

4. H0: µ1 = µ2

What are the 4 levels of measurement/type of data (from least to most quantitative?)

nominal < ordinal < interval < ratio

Measurements translate what’s been observed into…

numerical values, representing an underlying concept empirically

Nominal (categorical) measurement/data

• Categories describing traits/characteristics participants can check

• Attributes w/ names (ex: gender, ethnicity, religion)

• No ranking of data

Ordinal (categorical) measurement/data

• Data ranked from smallest to largest values (ex: SES, education levels)

• Interval between the data may NOT be equal (ex: rating scale - strongly agree, agree, disagree, etc.)

Interval (continuous) measurement/data

equal intervals between data levels or categories (ex: date, temperature)

Ratio (continuous) measurement/data

an interval scale w/ an absolute zero (ex: weight, income)

Credibility (true value) - Qualitative

• Has the researcher represented multiple realities revealed by informants as adequately as possible?

• Ensures findings represent participant views/experiences accurately, often through member checking.

• vs Internal Validity: ensures quantitative results are not due to bias.

• Done thru triangulation, member checking

EX: Researcher checks themes with participants

Dependability (Reliability) - Qualitative

• Can the variability in the study findings be ascribed to identify sources?

• The findings would be consistent if the inquiry were replicated with the same subjects, across researchers, or in a similar context.

• Consistency of research process

• Done via Audit trail

• EX: Documenting coding steps

vs Reliability: in quantitative study, focuses on identical replication of results.

Confirmability (neutrality, objectivity) - Qualitative

• Neutrality of data, participants’ voices, objective research is seen as scientifically distant

• Ensures findings reflect participants' experiences rather than the researcher's biases, often shown through an audit trail.

• Findings based on data, not bias

• Done via Reflexivity, multiple

  coders

• EX: 2 researchers compare codes

vs Objectivity: through standardized, impersonal measures.

Transferability (External Validity) - Qualitative

• Degree to which the findings fit into contexts outside the study situation that are determined by the degree of similarity or goodness of fit between the 2 contexts

• Qualitative research uses "thick description" to allow readers to apply findings to similar contexts

• EX: Detailed participant context

vs External validity: Quantitative research uses statistical sampling to infer results to a broader population

Trustworthiness - Qualitative (vs Reliability/Validity)

• Merit of qualitative inquiry, with 4 criteria (credibility, transferability, dependability, confirmability)

• Ensure confidence in findings rather than reducing error through measurements (in reliability/validity).

• Overall quality and rigor of a

  qualitative study

• ex: A study uses multiple strategies

  (e.g., triangulation, audit trail,

  member checking) to ensure

  strong, trustworthy findings

11 Methods to enhance Trustworthiness

1. Prolonged (breadth) and persistent engagement (depth)

2. Time sampling: systematize informant contacts, sample all possible situations, times, groups

3. Reflexivity: diary, notes, bracketing, self-interrogation, peer interview, group participation

4. Triangulation: convergence of multiple perspectives to ensure that all aspects of a phenomenon have been investigated. Triangulated sources are cross-checked: multiple methods, data sources, theory, investigators

5. Member checking: ask participants to review and react to study data and emerging themes and conceptualizations

6. Interview techniques

7. Peer examination

8. Structural coherence: interpretation and analysis explains apparent contradictions

9. Authority of the researchers

10. Representatives of the participants

11. Audit trails: review raw data, process notes, data reduction, etc

Validity

• Establishing strength of a relationship between a measurement indicator & the underlying concept

• Are we measuring what we say we’re measuring? (hitting the target)

Construct validity

Definition & conceptual model of the attribute being measured (e.g. independence, co-occupation, happiness, aging in place)

External validity = Transferability (Qualitative)

• Affects degree to which you can generalize from the study to the larger population (which the study sample’s supposed to represent)

• Strengthen external validity in designs: constant replication

Threats to external validity

1. Too many exclusion criteria (overly specific study characteristics that do not represent other populations)

2. Hawthorne effect (observer effect bias): Refers to tendency of subjects in a study to behave or act differently (i.e., work harder) when they know they’re being watched

3. Rosenthal effect (bias): The investigator’s expectations about the outcome of a given study affect the actual study outcome.

Internal validity

1. The correspondence of conceptual & operational definitions

2. Are we measuring/manipulating what we want to measure/manipulate?

Ways to strengthen internal validity in designs

1. Randomization: equalize groups

2. Crossover: exposed to treatment in different order

3. Homogeneity: as much as possible people in groups are similar on the variables, ie, gender, age

4. Stratification: eliminate effects of one variable, eg. separate groups for gender

5. Matching: people’s characteristics equally represented in each group or assessing the confounding variables and matching participants on those variables.

6. Statistical control: enables comparison of various effects and ability to “remove” influence of confounding variables, eg. ANOVA, ANCOVA

Threats to Internal Validity

1. Confounding factors: a variable that creates an artificial relationship or that masks a real relationship between study variables (ex: ice cream and drowning deaths)

2. Selection bias: error introduced when the study population does not represent the target population due to some selection preference (see Types of Biases for details)

Statistical conclusion validity

• Degree to which conclusions about the relationship between variables, based on data, are reasonable and accurate

• Ensures that observed effects are real, not due to chance, low statistical power, or invalid assumptions

• Focuses on whether the correct statistical tests were used and properly applied.

Beneficience: Above all, DO NO HARM!

1. Freedom from harm & exploitation

2. Maximizing benefits to participants & society

3. Maintaining an appropriate risk/benefit ratio

4. Possible harm minimized. Good outcome with as little risk to participants possible.

Nonmaleficience "do no harm"

Individuals to refrain from providing ineffective treatments or acting with malice

Confidentiality "Respect for Persons”

• Ensuring researchers protect participant privacy, data, and identities

• Personal information obtained during research is managed safely to uphold autonomy, directly supporting ethical standards alongside informed consent and justice

Social/Distributed Justice

• Right to fair treatment & privacy (confidentiality, anonymity)

• Equals ought to be treated equally

Steps in the measurement development process in order

Conceptualization → operationalization → reliability & validity

Measurement Development Process: Conceptualization

• Initial, iterative process of defining research goals

• Refining abstract ideas into concrete concepts

• Developing research questions or hypotheses

Measurement Development Process: Operationalization

• Process of strictly defining abstract concepts into measurable, observable variables, bridging theory with empirical research.

• Converts theoretical constructs (e.g., "intelligence") into specific, testable procedures (e.g., "IQ test score"), ensuring validity and reliability

• This step is critical for data collection and replication

Measurement Development Process: Reliability (consistency/dependability)

Consistency or repeatability of the measures 

1. Degree to which random error exists in a measurement instrument

2. Composed of the relationship among: O = T+E 

   1. O = Observed score

   2. T = “true” score

   3. E = error score

   4. Error = Observed - True 

   5. This is usually expressed as a correlation coefficient (Pearson r, Spearman r, ICC, Cronbach’s alpha) with r = .80 or above generally considered acceptable.

Measurement Development Process: Reliability - Stability

Test-retest: consistency of repeated measures of attribute(s)

Measurement Development Process: Reliability - Internal consistency

Assesses the correlation between multiple items in a test that are intended to measure the same construct

• Testing homogeneity; uses split half consistency (split test in half, even odd, random selection of half)

Measurement Development Process: Reliability - Equivalence

Agreement between two inter-rater, alternate forms used in pen-pencil tests

Measurement Development Process: Validity

Establishing strength of a relationship between a measurement indicator & the underlying concept: are we measuring what we say we’re measuring? (hitting the target)

1. The measuring procedure represents the intended only! What’s being measured is a true reflection of the underlying concept.

Likert Scale

Level of agreement (favorable-unfavorable) indicated; usually represented by 5-7 responses 

1. EX: OCTH 245 is the most interesting class I’ve ever taken. 

2. 1 = strongly agree, 2 = disagree, 3 = undecided, 4 = agree, 5 = strongly agree 

Threats to reliability

Random error, nonsystematic mistakes in measurement

• EX: misreading a questionnaire item, observer interpret the results incorrectly, nonsystematic misinterpretations of a behavior, coding errors/data entry errors, ambiguous instructions, fatigue (of the interviewer or respondent)

Semantic Differential Scale

Attitudes toward objects, events, concepts. Allows rating along a bipolar continuum (3-7 choices)

1. Good ________ Bad 

2.            3210123

Guttman scale

Cumulative scaling of increasing intensity to establish a 1-dimensional continuum for a concept. Respondents check all items with which they agree.

• EX: Bogardus Social Distance Scale - Please check each statement you agree with: Are you willing to have immigrants in your town? Would you let your child marry an immigrant?

Ratio scale

Highest level of measurement in research, featuring quantitative variables with equal intervals and a true zero point, representing a total absence of the variable

• Allows for precise comparisons, such as "twice as much" or "half as long”

• Supports all statistical analysis, including mean, median, mode, & geometric mean

What is triangulation?

Use of multiple methods or data sources, sites, investigators or ways of analysis & interpretation in qualitative research to develop comprehensive understanding of phenomena 

1. Convergence of multiple perspectives to ensure that all aspects of a phenomenon have been investigated

2. Triangulated sources are cross-checked: multiple methods, data sources, theory, investigators

3. EX: Interviews & Observations

Focus groups

Medium; guided discussion with a small group (often 6-12 participants).

1. Data Type: Qualitative; focuses on group dynamics and collective meanings.

2. Pros: Efficient (multiple people at once), generates dynamic discussions.

3. Cons: Risk of participant bias, groupthink, complex to analyze.

4. Best For: Market research, exploring complex social issues, gathering feedback.

Unstructured groups

Low; conversational with few or no predetermined questions

1. Data: Qualitative; detailed, in-depth, and subjective

2. Pros: High flexibility, comfortable, explores topics deeply.

3. Cons: Hard to compare, time-consuming, high risk of bias.

4. Best For: Exploring new topics, assessing cultural fit, creative roles.

Structured groups

High; rigid, predetermined questions with set order.

1. Data Type: Quantitative; easy to compare across subjects.

2. Pros: Efficient, reliable, reduces interviewer bias.

3. Cons: Low flexibility, impersonal, cannot probe for deeper info.

4. Best For: Skill assessments, high-volume recruitment, consistent data collection.

Phenomenology Methods (Bracketing - Multiple Methods Done)

Aims to describe the "essence" or "universal meaning" of a lived experience shared by a group.

1. Focus on the individual experience. Interviews are the main method of data collection. Observations. Studies individual experiences. May not require as much time as ethnographic study.

2. Research product: Description of the essential structure of breast cancer experience

3. Analytic strategy: Phenomenological reduction; hermeneutic analysis

4. Research question: What is the lived experience of having breast cancer?

Phenomenology Validity/Scope

Knowledge is valid when it captures the deep internal processes of consciousness across multiple individuals (typically 5–25 participants).

Grounded theory

A methodology that involves developing theory through data analysis. Aims to develop theories in relation to collective data

1. Researchers don’t consult literature before analyzing data, since it may influence their findings

2. Theoretical sampling technique used

3. Research product: Theory regarding basic social processes involved in coping with breast cancer and factors that might account for variations

4. Analytic strategy: Constant comparative analysis

5. Research question: How do women w/ breast cancer cope with changes to body image?

Grounded theory Validity/Scope

Relies on "theoretical saturation," where no new information emerges from data (typically 20–60 participants).

Narrative analysis 📖

Story provides understanding of experience. How the individual makes sense of the experience is found in the narrative

1. Analysis derived from construction of the “story”: Narrative approach to the COPM; individual story used in the Occupational Performance History Interview (OPHI). 

2. Doesn’t have a specific approach = Weakness

3. Inherently dependent upon subjective interpretation & the interpretation occurs through collaboration w/ the individual under investigation 

4. Addresses how the individual “makes sense” of the situation

5. Analysis derived from the construction of the “story”

   1. Think of the individual story that is used in the Occupational Performance History Interview (OPHI)

   2. Narrative approach to the COPM

6. NO specific approach and that is considered a weakness

7. Inherently dependent upon subjective interpretation and the interpretation occurs through collaboration with the individual under investigation

8. Research product: Narrative accounts of women’s explanations for their breast cancer experiences

9. Analytic strategy: generating, interpreting, representing women’s stories in narrative form

10. Research question: How do women w/ breast cancer come to know their experience?

Narrative analysis 📖 Validity/Scope

Focused on 1 or 2 individuals; valid knowledge resides in the depth and authenticity of the story and its meaning to the teller.

Life history

Qualitative method collecting a person's entire experience or specific segments, often through in-depth interviews, to understand the relationship between individual lives, social structures, and historical context

• Captures subjective, longitudinal, and detailed personal narratives, frequently highlighting the lives of individuals within their cultural or social settings.

• Common techniques: in-depth, semi-structured interviews (often multiple, lasting 1–1.5 hours each), longitudinal studies, oral histories, and analysis of personal documents like letters or diaries.

Life history Validity/Scope

Valid when it successfully connects a unique individual's timeline to larger cultural or social movements

Reflexivity (self-awareness)

• Awareness of researcher

  influence

• Done via journaling

• EX: researcher reflect on bias

• process of consciously examining one's own subjective point of view and how it might impact the research's outcomes

Member checking (participant validation)

process of returning to participants to verify that the researcher's findings, interpretations, and themes represent their reality accurately

Audit trail (transparency)

detailed, chronological record of all decisions, procedures, and methodological steps taken during the research process

Structural coherence (logical consistency)

ensuring that the research results are consistent, logical, and supported by the data without inner contradictions

Codes (part of thematic analysis)

Specific labels

1. What are these? Be precise, what label would you use? What are the inclusionary requirements and exclusionary aspects? Where would you find these?

2. Categories: grouping of codes w/ similar attributes

Narrative transcription

To understand the human experience as a whole story.

1. Maintaining the context and sequence of the original narrative.

2. Result: Reconstructing the participant's story to highlight key plot points, setting, and themes.

3. Best for: Life histories, in-depth interviews, experiential studies

Frequency distribution

table, chart, or figure that shows intervals and used to display & describe data, showing the number of times (frequency) each value or range of values occurs in a dataset

Histograms

1. graphical tool to visualize the frequency distribution of continuous numerical data by grouping data points into ranges, or "bins"

2. Displays the shape, center, and spread of data, showing how often values fall into specific intervals

3. Key types include frequency (count) and density (percentage) histograms

Pie charts

static, simple part-to-whole proportions (percentages) with few categories

Line charts

display trends, shifts, or continuous changes over time

What are confidence intervals (CI) and their purpose?

1. A measure of variability

2. How confident are you?

3. Estimate to indicate reliability of estimate

4. A range of values, derived from sample data, that is likely to contain the value of an unknown population parameter (e.g., mean or proportion)

5. Acts as a safety net for estimates, defining the uncertainty around a measurement, with a commonly used 95% confidence level indicating that 95% of similar samples would produce intervals containing the true population parameter.

Correlation coefficients

• Measures the relationship between two variables rather than the agreement between them, THUS commonly used to assess relative reliability or validity

• A more positive correlation coefficient (closer to 1) is interpreted as greater validity or reliability.