CED 501

In the research process to become a critical thinker you will need to:

1. Ask questions

2. Define the problem 

3. Examine available evidence

4. Assess assumptions and biases 

5. Avoid emotional reasoning 

6. Separate facts from opinion 

7. Avoid oversimplifying 

8. Consider alternative explanations

9. Tolerate uncertainty 

What are the objectives of science? 

1. Describe

2. Explain 

3. Predict

4. Control 

What is the best predictor of future behaviors?

Past behaviors

What are the ethical principles?

1. Beneficence and Nonmaleficence

2. Fidelity and Responsibility

3. Integrity

4. Justice

5. Respect for people’s rights and dignity

Beneficence and Nonmaleficence

Psychologists strive to benefit those with whom they work and take care to do no harm. 

Fidelity and Responsibility

Psychologists establish relationships of trust with those with whom they work. They are aware of their professional and scientific responsibilities to society and to the specific communities in which they work. 

Integrity

Psychologists seek to promote accuracy, honesty, and truthfulness in the science, teaching, and practice of psychology. In these activities psychologists do not steal, cheat, or engage in fraud, subterfuge, or intentional misrepresentation of fact.

Justice

Psychologists recognize that fairness and justice entitle all persons to access and benefit from the contributions of psychology and to equal quality in the processes, procedures, and services being conducted by psychologists.

Respect for people’s rights and dignity

Psychologists respect the dignity and worth of all people, and the rights of individuals to privacy, confidentiality, and self-determination.

What are the common ethical issues?

1. Violation of confidentiality 

2. Potential physical or mental harm 

3. Coercion to participate 

4. Invasion of privacy 

5. Lack of adequate informed consent

Elements of an informed consent

1. Purpose of the study 

2. Description of the study’s procedures including time for completion 

3. Possible risks and benefits 

4. Statement informing participants they can refuse to participate without penalty 

5. Principal investigator contact information

6. Signature line 

What is confidentiality? 

A participant’s data may be used ONLY for purposes of the research and not divulged to others.

What is the easiest way to maintain confidentiality?

To ensure participant’s responses are anonymous. 

What is debriefing? 

1. Clarifies the nature of the study to participant 

2. Remove any negative consequences 

3. Obtain participant feedback about study methods

4. Helps participants leave their study feeling positive

What are the elements of a research manuscript?

1. Title page

2. Abstract

3. Introduction

4. Method

5. Results

6. Discussion

APA Style Citation

Author, A. A., Author, B. B., & Author, C. C. (Year). Title of article. Title of Periodical, volume number (issue number), pages. https://doi.org/xx.xxx/yyyy

What is an independent variable?

Is the variable in an experiment that is manipulated by the researcher.

What is a dependent variable?

A variable that is measured to see whether the independent variable had an association on our IV.

What is a moderator?

1. A variable that may impact the association between the IV and DV. 

   1. Can increase, decrease, or even reverse the relationship between the IV and the DVs.

What is a mediator?

A variable that is assumed to be a predictor of one or more dependent variables and, at the same time, is predicted by one or more independent variables.

When is a research question used?

When there is NO empirical evidence support to offer a directional hypothesis (H).

What is a hypothesis?

Are empirically supported statements about the potential associations between variables. 

Grounding Questions in Theories 

Repeatable, probabilistic explanations that are supported by verifiable data of why and how things (variables) interact to influence each other.

Repeatable

Can be applied over and over to “correctly” explain a phenomena.

Hypothesis Testing

1. Have a theory 

2. Create a hypothesis 

Considerations for hypothesis testing

Avoiding type 1 + 2 errors

Power

The ability to detect effects/significance or not (i.e. demonstrate an association between variables)

1. Want to have power 

Ways to increase power?

1. Sample 

2. Design

3. Measurement

Operationally defining variables 

Defining a concept in terms of how it will measure concept. 

What is reliability?

The extent to which a measure produces the same result when repeated.

What is validity?

The extent to which a measure actually measures what it is supposed to measure. 

Observational

1. Behavior 

2. Duration: How long a behavior occurred

3. Frequency: The # of times a behavior occurred 

4. Time sampling: How long the behavior occurred in a fixed period of time. 

5. Continuous: What behaviors occurred over a specific time period. 

   1. Need to have a clean operational definition of behavior 

Cons of observational

1. May be limited to smaller sample size

2. Different factors affecting 

3. Hawthorne effect → Knowing 

Pros of observational

Can be considered “the true form” 

Physiological

1. Association between biological processes and behavior. 

2. Involves specialized equipment to measure heart rate, brain activity, hormonal changes, and other bodily perspectives.

Cons of physiological

1. Very expensive 

2. Miss cognitions

Pros of Physiological

Allows us to examine behavior through physiological data and recordings 

Self-report

1. Measures designed to measure thoughts, feelings, and behaviors 

2. Commonly completed online via survey platform (e.g. qualtrics) 

3. Where most of our research comes from

Test-retest reliability

1. A method for evaluating the consistency of a measure over time.

   1. How consistent are the results from T1 to T2.

Interrater reliability

1. A method used to determine the reliability of observational measures

   1. How well do 2 or more raters reach an agreement on X.

Parallel forms reliability

How reliable are the scores across two different measures.

Internal Consistency relability

1. How well do the scores across items in a measure relate to one another. 

   1. .90 greater

Construct validity

How well the measure aligns with what you are intending to measure.

Content validity

How well does the measure align with the definition of the construct.

Face validity

The extent to which a measure seems to be reasonable.

Criterion validity

How well is the measure aligned with another measure assessing the same construct. 

yet unreliable

A measure cannot be valid

but valid

A measure unreliable

yet not reliable

A measure can be reliable

Population

Entire group will focus study on.

Sample

Small/selected population will conduct study. 

Recruitment Methods

1. Flyers

2. Online listserv

3. Professional networks

4. Professional recruiting services (professional match + prolific: expensive requires grant funding) 

5. Social media 

6. Ads + newsletter 

What is the grade level the informed consent should be written in? 

5th grade reading level 

Snowball sampling

Passing info through networks

Sampling error

The extent to which characteristics of individuals selected for the sample differ from those of the population. 

Convenience

Non-probability sampling; Selecting research participants that are “convenient” to access. 

Quota

Non-probability sampling; Selecting research participants based on a predetermined subgroup (some # of the group) 

Purposive

Non-probability sampling; Selecting research participants based on specific characteristics. 

Simple Random

Probability sampling; Randomly selects participants from the population. 

Systematic

Probability sampling; Selecting research participants in a random manner to minimize error.

Stratified

Probability sampling; Dividing participants into a group (strata) and then randomly selecting. 

1. Ex: PhD Vs. MC 

Cluster

Probability sampling; Similar to stratified, yet you divide participants into clusters.  

Qualitative

• Use it when we do not have enough research/framework. Depth of participant experience.

1. Fewer words

2. Interview

3. Focus group

4. Case studies  

Quantitative

• Looking at people (breath)

1. Test hypothesis 

2. Surveys

3. Online 

Archival records

Provides descriptive data about an organization.

Focus group

Formed to address specific issues

Four functions of a focus group? 

1. Gather information 

2. Generate insight 

3. Understand decisions 

4. Encourage interactions

Goals of focus group

1. Understand different perspectives 

2. Uncover influential factors 

3. Open an inquiry/range

Advantages of focus group

1. Efficient 

2. Ask direct questions  

3. Straightforward

4. Share/compare

5. Provoke memories

6. Support groups

7. Less structure 

8. Safe place to share

Limitations of focus group

1. Created self-report 

2. Remain Hidden 

3. Chatty → difficult to facilitate 

4. Less time to share 

Case studies advantages

1. Limited focus allows detailed exam of subject

2. Best way to gather detailed info about subjects

3. Can suggest directions for future research

Case studies disadvantages

1. Time consuming

2. Generalizability

3. Bias (subject to bias to observing and recording data, cannot demonstrate cause and effect relationships, and limited generalizability)

Single Subject Design (SSDs)

Charts the progress to determine the effectiveness of an intervention.

1. How you analyze data

2. Behaviors

3. Evaluate subjects behavior

Target Behaviors

Specific observable behaviors/series of behaviors that indicate effectiveness.

Experimental Design

1. Randomly selects subjects from the population.

2. Includes a “control” group

3. Includes an “experimental” group

Maturation

Changes due to natural development.

History

Uncontrolled outside influences on participants during an experiment.

ABAC Design

Intervention B is counseling + intervention Cl’s Behavior modification

A: Pure baseline

B: Intervention

C: Behavior modification

Threats to internal validity

Accuracy of concluding that the outcome of an experiment is due to the IV.

Regression

The tendency for extreme scores to move toward more typical performance when retested.

Mortality

• Changes in the group composition because some participants have left the study.

• participants drop out of the study

• Affect results validity

Experiment Effects

Effects due to the presence of the experimenter.

Key characteristics of Quasi-experimental design

• No control over group assignment

• Useful when manipulating the IV Is not possible/ethical

Examples of Quasi-experimental design

• Single subject(s) designs

• Longitudinal designs

• Cross-sectional method

Quasi-experimental: Cross sectional Advantages

• Short time span

• Less expensive than longitudinal

• Requires little cooperation between researchers and participants

Quasi-experimental: Cross sectional Disadvantages

• Groups may not be strictly comparable.

• People of same age may be at different maturation levels

• Little directional predictability

Important for consideration of longitudinal studies

1. Funding

2. Attrition (drop out)

3. Mortality

Longitudinal Methods

Examines change over time (I.e. developmental changes)

Disadvantages of longitudinal designs 

• Mortality 

• Expensive 

How to conduct Single Subject Design (SSD)

1. Define the behavior

2. Choose your scale of measurement

3. Choose your time increment

Manual thematic analysis

1. Familiarization with the data

2. Initial coding

3. Theme development

4. Theme review and refinement

5. Defining + naming themes

6. Reporting findings

interpretative phenomenological analysis

1. Reread analysis translation

2. Identify themes

3. Summary table of themes

4. Summary of themes

Program Steps

1. Establish stakeholders

2. Explain what the program is about

3. Select the design of the evaluation

4. Collect data

5. Generate conclusions based on data analysis.

6. State fundings and provide recommendations.

Program Theory

Answers questions about program conceptualization and service delivery.

Process Evaluation

Answers questions about the program operations, implementation, and service delivery.

Outcome Evaluation

Answers questions about the program effectiveness and tests it’s.

Efficiency Assessments

Answers questions about program’s cost and cost-effectiveness.

Within Subject/Comparison

Differences in groups

Between subject/comparison

Difference in control/treatment

Random Assignment

• Ensures all in study have an equal chance to take part in the study.

• To ensure all procedures are considered fair across study groups.

• To test group differences.