research design final

Bentley et al 2019 article

-real-time monitoring of the SCED research

-SECDs offer a lot of potential: high internal validity, repeated assessment offers statistical analysis, allows for a fine-grained picture of how and why behavior occurs, allows researchers and clinicians to tailor manipulations of IV in an ongoing process

-real-time monitoring (ambulatory assessment): ecological momentary assessment/experience sampling (active) and passive sensing

-combo of EMA and passive sensing may strengthen conclusions

-SCED may be readily employed in clinical practice given relative cost-effectiveness

-Examining whether and how quickly an intervention is working may inform clinical decision-making and elucidate components that account for change

-Clinicians may efficiently receive information that can be used to inform treatment tailoring (e.g., contact the client between sessions)

-identify dynamic processes and symptoms that drive or maintain difficulties

-increased ecological validity of real time monitoring may increase the generality of SCED research

-opens up a lot of ethical concerns (e.g., how do you report self harming behavior, is this harming the therapeutic relationship, etc. )

ecological momentary assessment/experience sampling

-active assessment

-usually entails responding to prompts or structured or semi-structured questions or entering self-report data into a device (ex: phone or tablet)

-data collected at fixed or random time intervals or may be event contingent

-information typically time-stamped

passive sensing

-Collecting data unobtrusively, without participants engaging in active data entry (e.g., smartphones usuage patterns, sleep duration, quality, steps, taken)

challenges of real time monitoring

participant compliance and retention

-balance desire for maximum data points and participant burden

-consider effects of sesnory sampling rates on battery life

-compensate for compliance (while considering implications for gnerality)

privacy and confidentiality concerns

-select platforms that meet best practice seurity guidelines

-consult experts in IT and related

-discuss priacy, related pratices, and potentially utlity in informed consent process

real time data management,ent/analysis

-ensure qualified staff and related resources for data analysis

-exercise caution when drawing conclusions from passive sensing data

potential for reactivity

-consider factors that may influence reactvity (e.g., behavior and monitoring complexity)

-closely monitor for reactivity during non-intervention and extend baseline as needed

lack of universal device ownership

-provide low cost study smartphone or wearable device if feasible

responding to risk

-establish protocol for monitoring and responding to incoming data (IRB collab). this is an on going process

-be transparent about protocols for responding to risk during informed consent

mediator

intervening variable that may account (statistically) for relationship between IV and DV

mechanism

basis for effect; process or event responsible for change (i.e., reason why change occurred or how it came about). deeper more conclusive way of saying how change happened. similar to mediator but the design if diff

moderator

characteristic that influences direction or magnitude or relationship between IV and DV

kazdin 2007: Reason to study mediators/mechanisms

-wealth of treatments in use: understanding mechanisms help provide parsimony and order to the breadth of knowledge available.

-elucidate (clarify) connections between diverse outcomes: therapy can alter physical conditions like pain or blood pressure, increase quality of life, improve recovery from illness/surgery

-potential to better optimize therapeutic change: helps us understand which components are important to the intervention. ex: how it can strengthen IV to see how the therapeutic change stems from our treatment

-helps to optimize the generality of treatment effects from research to practice: to generalize treatments, investigate why treatment works must be implemented first. by understanding the mechanisms for treatment and change, we can later replicate interventions that are studied in research. Looking at what may make the intervention/treatment stronger or weaker can better inform how the intervention can be generalized, and further, how it can be changed/redesigned for different contexts to optimize efficacy

-Understanding how treatment works can help identify moderators of treatment: understanding the therapy can help identify variables that can influence the treatment outcome or effectiveness. This gives us an idea of who may or may not respond to treatment.

-Understanding mechanisms has implications beyond the therapy context; can help understand more general processes of change. We have to look at the individual mechanisms to understand the context fully. By understanding that mechanisms have implications beyond therapy. look at social interactions and biological factors. This method can provide a better understanding of human functioning. Also, processes of change may be relevant to change via activities outside of psychotherapy (e.g., physical activity, community involvement, journal writing)

kazdin 2007: Requirement for demonstrating mediation

-strong association: The initial requirement of demonstrating a strong association between the intervention and hypothesized mediator. Then, the association between mediator and therapeutic change needs to be shown. Ex. Psychological flexibility as hypothesized mediator in relationship between ACT and change in quality of life, it has to show strongly association with ACT (intervention) and quality of life (therapeutic change)

single case experimental design: key characteristics

-continuous assessment

-baseline assessment

-stability of performance

-use of different phases

-commonly used designs are ABAB (reversal) and multiple baseline (concurrent and nonconcurrent), and changing criterion designs

continuous assessment

observations on multiple occasions prior to and during period in which intervention is administered

baseline assessment

assessment for period of time before intervention is implemented; serves descriptive and predictive functions

Stability of Performance

attention to variability, trend (i.e., slope) over time

Use of different phases

periods of time (days, weeks) in which a particular condition is implemented and data collected

ABAB/reversal design

Two separate phases alternated over time: baseline phases with no intervention (A) and intervention phases (B)

multiple baseline design (MBD; 1)

MBDs demonstrate effect by showing that behavior change accompanies the introduction of the intervention at different points in time; within (MBD 1). we implement at different phases because there is more possibility for rival hypothesis

multiple baseline design (MBD; 2)

MBDs demonstrate effect by showing that behavior change accompanies the introduction of the intervention across individuals; same behavior different people. A concurrent multiple baseline design means multiple patients are being assessed at the same time. Nonconcurrent is when each indiviudal would be treated separately. Pay attention to how quickly you see change and how dramatic the change is

changing criterion designs

Causal relation demonstrated if behavior matches constantly changing criterion for performance over the course of treatment; specific levels of performance chosen as criterion; reinforcement given when criterion is met. one person, one behavior, one intervention, but criterion for reinforcement intervention. ex: at the ICD, there was a staying in seat goal. start at 30 secs and then progressively add time.

criteria for data evaluation

-changes in means

-change in level

-change in slope

-latency in change

-consider immediacy of treatment effects, presence of trends, variability within and between phases

-very useful but no formal decision rules

changes in means

mean rate of behavior shows a change from phase to phase in expected direction

change in level

change in behavior from last day of one phase and the first day of next phase; an abrupt shift facilitates data interpretation

change in slope

direction of slope changes from phase to phase (e.g., horizontal level to accelerating slope)

latency of change

speed with which change occurs once conditions change (faster change supports causal explanation)

Methodological standards in SCED: Raising the bar (Ganz & Ayres, 2018)

-SCEDs more widely accepted in recent years, especially in those with low-incidence conditions, including intellectual and developmental disabilities

Methodological characteristics of studies quite variable (see Table 1)

Higher levels of rigor in conducting and reporting SCEDs would allow for clearer determination of valuable interventions

Iterative process of developing and revisiting standards for these studies, based on needs of applied settings, national institute statements, and related legislation

One aim of article is to provide standards for Research developmental Disabilities and related publications

methodological standards in SCED: select standards (Ganz & Ayres, 2018; part 1)

Participant descriptions

Essential

Adherence to APA guidelines of referring to participants (e.g., person first language)

Acknowledgment of informed consent

Disability diagnosis/category (if any) and associated assessment directly relevant to the study

Nature of relationship between researcher/interventionist and participant

Educational or related context

Aspirational

Diagnostic tests used to identify disability and screening tests used

Recruitment process and pool of potential participants

Inclusion/exclusion criteria

Settings and arrangements

Essential

Description of other people present and ratio of staff to participants/clients/students

Approximate dimensions and arrangement of space(s) where study took place

Description of environmental context in which procedures took place

Aspirational

Participant familiarity with setting prior to study

Materials

Essential - size, color, dimensions, quality; manufacturer and model; description of how materials worked (e.g., software)

Aspirational - if materials not subject to copyright and can be shared, links to copies or examples.

methodological standards in SCED: select standards (Ganz & Ayres, 2018; part 2)

Interventionist

Essential - description of person who implemented intervention

Aspirational - interventionist's occupation, relationship with participant(s), educational level, relevant training and experience

Response definitions and measurement

Essential

DV observable and measurable behavior

Inter-observer agreement (IOA; for 2+ assessors) collected in at least 20%of sessions in each condition, for each participant

When IOA falls below accepted standards, explain likely reason for low value and efforts to mitigate

Aspirational

When discontinuous assessments are used (e.g., ecological momentary assessment), report information re: percentage of total observations

Video recorded sessions

Qualifications of data collectors and associated training

Report and graph IOA broken down by various parameters of study

Maintenance and generalization data

Essential

Data collected on generalization of measures to other contexts, people, stimuli

Data collected on maintenance of measures following cessation of intervention phase, as appropriate in face of research question(s)

Aspirational

Generalization data collected 3+ times per phase

Maintenance data collected 3+ times and at least 4 weeks following intervention cessation

Procedural descriptions

Essential

Replicable precision in description for each condition's procedures

Procedural fidelity data collected

Session length, frequency, intensity described

Aspirational

Procedural fidelity checklists provided

Integrity data collected on at least 20% of sessions in each phase

Integrity scores acceptable (80% or better)

IOA data provided for integrity and met standards

methodological standards in SCED: select standards (Ganz & Ayres, 2018; part 3)

Research design

Essential

Designs structured to evaluate 3+ demonstrations of effect at 3+ points in time

With rapid alternation of multiple treatments, 4+ contrasts of each condition pairing

Each phase consists of 3+ data points

Trends in data clear and distinctly contrasted from other conditions/phases

Aspirational

Accurate identification of design with citation to major text in field

Each phase consists of 5+ data points (though different number of points are also useful to guard against cyclic variability)

In multi-treatment comparison, include intra-participant replication of treatment superiority across multiple behaviors

Social validity

Essential

Socially significant DVs and associated measures

Behavior change significant in face of study criteria or stakeholder report

Intervention includes natural environment component (e.g., generalization to natural setting with person in this environment serving as interventionist)

Aspirational

Intervention efficient and cost effective

All key stakeholders (e.g., parents, teachers) report satisfaction intervention and outcome

Reporting results

Essential

Data displayed in line graph with specific proportions (2:3 vertical to horizontal axis)

Three or fewer data paths on one graph

Data paths broken so path does not cross a phase change

Labeling clear and specific

Report results from all participants given intervention (not only those who responded to it)

Aspirational

When reporting participants for whom the primary intervention did not result in behavior change, apply another intervention to show that behavior was malleable

Both visual and statistical analyses are provided (the latter based on current standards of literature)

Assessment plans for evaluating potential mediators/mechanisms (Kazdin, 2007, table 3)

-this is what timeline is refered to

-at 3 during assessment, this doesn't tell muh still about mechanisms because we don't know directionality (is the mechanism or other outcome occurred first)

-4 shows just a snap shot of time. if mechanism has changed from pretest to midway test, and the outcome has not changed, then you know the mechanism happened before the outcome

-4 and 5 let us see directionality

-when only pre and post test are being assessed and not during, the mediator is treated like an outcome

-5 tests multiple times throughout like in a single case design. this allows us to see change along the way and how the mediator changes.

-"meticulous description blurs line between description and explanation; 'what' can be sufficiently fine grained to convey the 'how'"

Requirements for demonstrating mediation

-timeline

-gradient

strong association

-specificity

-consistency

-experimental manipulation

-gradient (dose response)

-plausible coherence

timeline

An established timeline is necessary to infer a mediator of change. A timeline helps us dive into specific causes and effects. We need to be able to see the timeline of change and the order in which things occur. Not being able to assess directionality is a key limitation when determining the mediator and it's outcome. "outcome" needs to be assessed early to ensure mediator has changed before the outcome. Kazdin calls this Achilles heel of treatment studies.

Ex: can not wait until the middle of therapy session to measure how therapeutic relationship might effect how therapy is working. You would need to assess the therapeutic relationship before therapy has began to establish that the mediator has occurred before the "outcome". assess at mulitple times a long the way.

gradient

How much a value is changing and in what direction (dose-response relation: stronger doses or greater activation of mediator is associated with greater change in the outcome). when i assess mechanism and outcome, do i see more the mediator changes, the more the outcome changes? if i do, then there is a dose response relationship

Ex: There is a dose-response between passive cigarette smoke (i.e. exposure to second-hand smoke) and coronary heart disease

strong association

The initial requirement of demonstrating a strong association between the intervention and hypothesized mediator. Then, the association between mediator and therapeutic change needs to be shown.

Ex. Psychological flexibility as hypothesized mediator in relationship between ACT and change in quality of life, it has to show strongly association with ACT (intervention) and quality of life (therapeutic change)

specificity

Arguing that one construct accounts for therapeutic change makes a stronger argument for the success of the intervention rather than multiple plausible constructs.

consistency

Looking into consistency of findings across different studies can help point out which specific mediators were mechanisms of change. "when consistency across studies is obtained, this greatly facilitates drawing inferences about whether a particular mediator may be involved" (Kazdin, 2007, p. 5).

Ex. Looking at different studies to evaluate if psychological flexibility is a consistent mediator for ACT interventions.

experimental manipulation

A common demonstration of direct manipulation, such as an RCT, helps identify causal relationships. Uncommon demonstrations are experiments that manipulate the mediator or mechanism to show how it affects the outcome. The experimental evidence gathered helps establish that the proposed mediator is the true cause of the change in the outcome being analyzed.

plausibility/coherence

Observing patterns or other explanations in the history that might contribute to the current state; ensuring your conclusion makes the most sense. also parsimonious.

Ex. Child abuse: even though parent might claim injuries occurred through a bike accident, and the story gets more unrealistic or unusual, pattern of injuries plausibly may better suggest physical abuse

back to methods for studying mediators and mechanisms in general (kazdin, 2007)

Even more elegant, complex statistical modeling (e.g., bootstrapping,structural equation modeling [SEM]) cannot establish that one variable preceded another. need multiple assessments over time

Even when timeline established, mediation does not necessarily =mechanism of action

Proposed mediator might be proxy variable for another construct or bebroader construct that includes multiple components useful to consider independently

Need for more fine-grained analyses. just because soemthing is a mediator does not mean we know processes of change

Language used to describe analyses/findings lends itself to misconceptions about the timeframe

e.g., predictors/IVs, outcomes/DVs. using IV and DV makes us seem like we know more than we do

Distinction between antecedent, outcome variables in steps of analyses and printouts is arbitrary

% of variance accounted for does not speak to mediators/mechanisms

clinical significance

the practical value or importance of the effect of an intervention (i.e., whether it makes any "real" difference to the clients or to others in their functioning and everyday life (Kazdin, 2017; p. 312). statistical isgnificance does not tell us the impact of change on that person's life. does this change matter is the domain of practical importance and social validity.

Jacobson & Truax, 1991

Where client ends up following a regimen of therapy

a - functioning falls outside range of the dysfunctional population (two standard deviations beyond population mean)

b - functioning falls within range of the functional/normal population (within two standard deviations of mean)

c - level of functioning places client closer to mean of the functional population than mean of the dysfunctional population

"Best" criterion When norms available, b or c is preferable to a as the cutoff point

When functional, dysfunctional populations overlap, c is preferable to b

When distributions are nonoverlapping, b is cutoff point of choice

When norms not available, a only option for the cutoff point

How much change has occurred during the course of therapy - reliable improvements

-the reliability change index reflects the amount of improvement from pre- to post-treatment on the outcome measures. Within-subject comparison is drawn against the individual themselves (their pre-test level to their post-test level).

-magnitude of change

ways to evaluate clinical significance of change

-falling within normative levels of fucntioning

-magnitude of change from pre- to post- treatment

-no longer meeting diagnosis criteria

-subject evaluation

-clinicl problem no longer present

-recovery

-quality of life

-impairment and disability

-qualitative assessment

-social impact

falling within normative levels of functioning

-Extent to which client's fall within a normative sample after treatment. Client is presumed to have a disparity in terms of functioning in specific domains (anxiety, depression, etc.) which is what bring them in for treatment. It is then likely that after treatment the client is indistinguishable from others who are within said normal range of functioning - citing clinically important change. limitations include: This measure may not account for personal experience or history when comparing to what is considered "normative" No two experiences are ever the same.

magnitude of change

Determine if magnitude of change for individual patients treated is statistically reliable. limitations include: Regression to the mean, Test sensitization, Maturation, Symptoms reduce from severe to mild, still presenting dysfunctional symptoms. There is a reliable change, but the client is still outside the normative range of functioning.

Example: Change from pretreatment scores where an individual's scores depart 1.96 standard deviations from the mean of their pretreatment scores would indicate an important change in improvement.

no longer meeting diagnostic criteria

Individuals are recruited and screened based on whether they meet criteria for psychiatric diagnosis. If treatment has achieved a sufficient change, the individual would no longer meet criteria for the diagnosis. limitations include: The absence of a diagnosis doesn't always equal to a full recovery or restoration of function.

Example: Diagnostic criteria for OCD previously met (Os and/or Cs; time consuming; distressing & impairment) but now assessing after treatment Billie no longer finds her obsessions distressing and reports obsessing less than 1 hour a day, thus no longer meeting criteria for OCD

subjective evaluation

Refer to the use of individual perceptions and opinions from the client and those who interact with them (family) to assess whether changes made during treatment result in a palpable difference in the client's function. limitations include: This can relate to inconsistencies or biases. Because this method uses third-party raters like family members, responses can be influenced by personal feelings and differing interpretations of what "palpable difference" entails. This can lead to potentially unreliable data.

clinical problems no longer present

The "problem" is no longer present, and a change of clinical significance can be observed. limitations include:

- while it is useful to consider if a problem is completely eliminated, most changes aren't that stark, changes occur in a matter of degrees: researchers need to decide if the degree of change really makes a difference in participants' lives

- the degree of change in symptoms is more relevant to specific changes in an individual's life, not the removal of symptoms entirely

-not only do you not reach diagnostic criteria, you no longer feel you have the problem

-important to understand this change.

recovery

Recovery is defined as a process of change through which individuals improve their health and wellness, live self-directed lives, and strive to meet their full potential. There are many dimensions of "spheres" that contribute to recovery, such as a social support system, a sense of purpose, and a safe place to live, etc. multi dimensional beyond sympotms. limitations include: -recovery is not so clear for psychological distress

- for individuals who have long-life mental or physical impairment, "recovery" may seem misleading.

quality of life

Quality of life refers to overall well-being, how clients view of multiple facets of their life as useful, purposeful, or important (

e.g., relationships and standard of living). Not recovering from anything per se. Encompasses mental and physical health. Some overlap with recovery, but is more "how do I feel about these domains in my life?" much more subjective than recovery. measure using the quality of life inventory. in this there are 13 domains of life (e.g., education, peers, family, etc.). It asked how important each domain is to you and how satisfied you are in each domain. limitations include:

-Many different ways to operationalize, a very broad concept.

-Based on subjective experience, may be difficult to measure.

-Someone may not need to be impaired by symptoms in order to have a low quality of life

impairment and disability

An impairment is the extent to which a person's social, occupational, and academic functioning is impaired. This impairment can hinder the ability to meet the demands in each of these domains. The experiences of one with an impairment will look different from person to person. On the other hand, a disability is more clinical and it's impacts are more formally evaluated in research. Limitations include:

-Can be measured through qualitative assessment which allows us to give weight to client views, but it can be labor intensive.

-due to scope of impairment, clients themselves may not be able to comment well on their own impairment.

Qualitative Assessment

Qualitative assessment refers to the understanding of themes, context, and emotional factors that may be contributing to the distress of the client. more invitation to share about experiences and do not need to feel compelled to put a number to experiences. Limitations include:

- lack of systemization,

-examiner bias,

-difficulty in distinguishing cause-effect relations, especially when considering a timeline

social impact

Social impact measures refer to the outcomes that are assessed in everyday life that are relevant to society in general. These measure the larger context of how a treatment may affect broad outcomes (ex. Crime rates, health emergencies, etc.) that can be beneficial for society. Limitations include:

-commodification of attention to cope with distress.

-There is often an emphasis on a group outcome. treatments can't be easily shown to have a social impact unless the group is large enough to show statistical significance

Decuir-Gunby, 2020

-Purpose of article to discuss how researcher's positionality and inquiry worldview influence methodological choices

- Context of research - lack of focus on race and race-related constructs in educational psychology

-talks a lot about positionality: talk about her own experince of growing up creole in the south. when she went to church, she realized intersectionality of racism and having a low socioeconomic class. when no one wanted to shake her hand at a predominantly white church, she realized that she will always face discrimination but only in her professional life but personal as well. she also speaks of her experince where she grew up speaking spanish and when she went into the field of psychology, it was predominatly white and she did not end up seeing many racial differences.

-this paper is the heart of qualitative research compared to quantitative research. this is why we need to understand the research's lense and not just present on the data

-goes over critical race theory as a world view

critical race theory

-legal theory that emphasizes challenging racism and systems of power

-Founded as means of challenging often slow progress of civil rights legislation and racial reform as well as creation/maintenance of white supremacy within legal system

-Strong advocacy component - centers on creating change for marginalized groups

-Principles include centrality of race and racism; challenge to dominantideology; property rights of Whiteness; intersectionality; myth of meritocracy (people have what they have because they work hard rather than racism and other experinces acting as road blocks); centrality of experiential knowledge; historical/contextual perspective; commitment to social justice; interdisciplinarity; reinterpretation of civil rights outcomes/interest convergence

critical race Mixed methodology (CRMM)

combination of traditional mixed methods designs with CRT (race and critiquing power structures are central areas of focus). using qualitative and quantitative research and how they complement each other

benefits:

-Ability to center race within mixed methods

-Balances out emphasis on qualitative research within CRT-based work in education

-Process of data integration (e.g., via joint displays and other visuals)

challenges:

-IRB approval - may be difficult given sensitivity of topic of race

-Can be challenging to find participants and organizations willing to participate - re-living one's racial experiences can be emotionally taxing

-Common for school administration to want to have control over process of data collection and/or data interpretation

-Educational psychology open to mixed methods but less so race-related constructs and theories including CRT - may be difficult to publish

-because of these challenges, having an interdisciplinary team approach; members should have simialr or compatible views on race and equity

CRMM sequential explanatory study

racial microaggressions in Higher Education

Quantitative data collected first, then qualitative

Although not ideal, published separately

Quantitative study

Examined racial microaggressions, coping, and racial identity as predictors of jobsatisfaction

Racial microaggressions and detached coping emerged as significant predictors forsample as a whole and for females

Qualitative study

Involved 17 African American higher education instructors and administrators

Data, coding process defined (p. 249)

A priori coding - based on tenets of CRT

In vivo coding - emerged from data if appeared within multiple interviews and were consistent with CRT

Themes and contextualization - themes included microinsults, coping approaches considered positive and negative

Better understanding and countering racial microaggressions in education context is critically important

Lund et al, 2023

-Need for research on professional psychology trainees and their training experiences

May help answer questions about topics including admissions and attrition, internship match rates, representation, and research productivity

Critical for addressing DEI issues including intersections among marginalized identities

Data may guide program decisions and policies, foster awareness of strengths and growth areas, and support data-driven, DEI-informed approach to training

-Emerging and under-researched area of inquiry - purpose of article to discuss critical methodological considerations -Emphasis on trainees with disabilities

Intersection with rehabilitation psychology - focuses on advocating for those from marginalized groups, especially those with disabilities

Providing evidence-informed training, supervision, and other forms of support may increase diversity and representation of rehabilitation psychology workforce

-talks about 3 main approached to data collection:

Direct quantitative survey data collection

Direct qualitative data collection

Analysis of archival data

-when selecting an approach, consider mixed methods approaches and tradeoffs, integrate participatory frameworks, and find balance between protecting confidentiality and allowing for analyses with multiple identities

direct quantitative survey data collection

Goal typically to generalize findings from sample to population

Research evolved to use more nuanced and sophisticated statistical approaches

Eludicates challenges and promotes opportunities to advance field through identifying complex relationships and longitudinal trends, including with program- and field-wide interventions

Benefits include researchers being able to develop questions and collect complex data from relatively large samples or to emphasize study of specific populations; allows for examination of complex models (e.g., with moderation and/or mediation) and sophisticated approaches to missing data; many not yet used in this population Limitations include response bias and participant self-selection and limited representation of those with multiple marginalized identities

May combine results from multiple samples, but have to attend to cohort effects; may oversample certain (sub-)populations

direct qualitative survey data collection

Options include open-ended questions in survey, focus groups, interviews

Allows for collection of rich and nuanced data; useful to inform questions to be asked and to examine experiences of small populations

Generalizability typically not a goal; transferability (i.e., ability of readers to transfer findings to own practice/lives) may be goal

Benefits include depth of data and generating solutions from direct input

Limitations include recruitment bias and time costs, difficulty with comparative analysis, and coder bias

May build in flexibility, across or within modalities of participation (e.g., allowing participants to opt for in-person, phone, or virtual method of assessment, if all in-person, may assist with travel, if all virtual, may select platform that is most accessible)

analysis of archival data

Data examined from large archival data sets collected by regulatory or professional organizations (e.g., APA, APPIC)

May be publicly available or requested to address particular questions

Data typically quantitative in nature

Benefits include much larger, more representative samples and less likelihood of suffering from serious self-selection or reporting bias

Limitations include access to fewer variables and their intersections (due to privacy and confidentiality concerns), item and response wording variables (across surveys) and out of researchers' hands, variation in items and response options over time

ethics in research (kazdin, 2024)

Affect all aspects of research process

Need to be considered from the beginning

Culturally sensitive research involves active participation on part of cultural group being studied

Protection of human participants is investigators' responsibility; IRB an important resource in that process

Deception

-Weighing risks and benefits

-Critical to consider expectations related to honesty, trust in relationships, particularly professional ones

invasion of privacy. ex: know you participated in the research. in Kazdin, there is an example of how 41-50% of the Inupiat community in Alaska meet critieria for alcoholism. Even though they did not say who was in the 41-50%, it is an invasion of privacy because it highlights a whole group.

confidentiality

when identifying info can not be linked to responses

aninmity

when you never collect information to begin with

privacy

protecting your identity as a participant in the study. ex: can we give people a private place to do their research so no one at all has to know they were in the research.

Hansen and Heu, 2020

Human behavior embedded in social contexts, but psychologicaltheories often talk about "people" as if theories are universal

People from WEIRD societies represent 96% study participants but 12% of world's population; little research conducted in "Global South" (southern hemisphere)

Conducting valid and meaningful research across cultures is effortful and time-consuming; researchers often 'copy-and-paste'materials and procedures. this means they use the same methods as in a WEIRD population which may not reflect the fame values

Value of integrating etic and emic perspectives

Recommendations for conducting meaningful cultural research

- Establishing egalitarian and respectful partnership with representatives of local community. shared responsibility between researchers and local partners in terms of control over ideas, processes, and outcomes

- Ensuring research constructs carry the same meaning and are relevant. if there is a meaningful term, it should be used as a "dictionary approach" (look at the literally dictionary for the language of the community and see if the word is there). for example, if you want to look at self esteem in China, see how it is defined there or if there even is a concept of self esteem there.

- Preparing cultural-sensitive research materials and procedures. material can be adjusted to colloquial terms and linguistic level. attend to topics that are stigmatizing or otherwise culturally insesative. ex: need to think about privacy because some cultures may not have a notion of privacy. Emphasize concrete examples instead of abstract ones. behavioral observations are important.

emic

we can understand people outside of their cultural context

etic

speaks to people broadly without context

program evaluation (orcher)

stages of program evaluation

-needs assessment (1st aspect, usually by questionnaire, extent to want to have type of program, program goals and activities, for new program areas you may use qualitative methods for needs of proposed clients)

-formative evaluation (2nd aspect, implementation of program, auditing state to see if they are consistent with what is described and activities are timely)

-summative evaluation (clinical significance and statistical significance)

communication of research findings

• in preparing the manuscript, the author invariably wishes to make

a statement (conclusion). The strength of that conclusion is based

on the extent to which the study addresses issues highlighted in

prior chapters (p. 668)

-Dissemination process is reflected in all topics and issues we explore (trade-offs of methodology)

-when thinking about disseminating, look at the journal would be a good fit for your journal, look at the notes to the authors

Communication of research findings involves three interrelated

tasks

 Description

 Explanation

 Contextualization – e.g., ...how do these findings add to research and

support, refute, or inform current theory? (p. 667)

Three tasks of the results and discussion section

-description: hypothesis testing, non tecnical terms describing what was found. how can we explain wgat we found?

-explanation

-contextualization: fitting my study into the current cintext of my literature. is it consistent with what is currently there or is it not? put this set of data points into context. talks about limitations, future directions

bushman and anderson, 2023

Amount of violent media seems to be increasing (e.g.,acts of gun violence in PG-13 films more than 3x thatthey were in 1985)

Numerous professional and scientific organizations issue statements about potential harmful effects of violent media exposure, yet some scholars publishing and publicly proclaiming about null results (saying media violence isn't that bad)

Null effects obtained when explained variance is relatively small and/or unexplained variance is relatively large (F = MS effect/MS error) (F is the main statistic and if it is large enough then it will yield statistically significant results) (systematic variability/unsystematic variability)

-attempt to address the challenge that research is telling the whole truth about the world (all research may not be reflective of whats actually going on)

Many "failed replication" studies may be due to problems discussed in article (also keep in mind that key question is not who is right and who is wrong, but why; critical importance of contextual factors and Truth vs. truth - pragmatic truth criterion)

Offers great overview of many statistics topics and highlights the many researcher degrees of freedom at play in most studies!

examples that may contribute to the null effect

Including inappropriate and/or too many covariates in models; appropriateness based on research questions (trying to account for everything)

Low statistical power (sample size is too low)

Demand characteristics (compelling cover stories and suspicion checks recommended)

Inappropriate control conditions and/or measures (not active enough control)