Thinking Critically About Research

Objectives

Develop critical thinking skills: Enhance the ability to analyze, evaluate, and form judgments about information.
Understand science and the scientific method: Learn the foundational principles and processes of scientific inquiry.
Know the purposes of science: Comprehend the goals of scientific investigation, including description, prediction, improvement, and explanation.
Explain scientific theory: Define and clarify the nature and role of scientific theories in explaining phenomena.
Outline types of scientific logic: Differentiate between inductive and deductive reasoning in scientific contexts.
Describe how we gain information: Compare and contrast scientific and non-scientific methods of acquiring knowledge.
Illustrate constraint levels in research: Understand the varying degrees of control in different research designs.
Explain basic versus applied research: Distinguish between research aimed at expanding knowledge and research focused on practical applications.
Clarify replication research: Understand the importance and methods of replicating research findings.

Understanding Research

Becoming a critical research consumer is essential: Empowers individuals to evaluate research claims effectively.
Critical thinking: ability to consider all sides and make informed decisions. Involves questioning assumptions, evaluating evidence, and considering alternative perspectives.
Professionals often skip critical analysis, accepting conclusions as fact or dismissing them based on personal biases: Highlights the need for rigorous evaluation in professional practice.
How research is conducted impacts findings: Emphasizes the importance of methodology in shaping research outcomes.
Modifications in procedures can produce different results: Illustrates the sensitivity of research to procedural changes.
Researchers may be invested in their procedures and overlook alternative explanations: Acknowledges potential biases in research interpretation.
Teach professionals how to be critical research consumers to improve education and psychological practices: Aims to enhance evidence-based practices through critical evaluation.
The primary purpose of teaching research is to develop critical thinking skills: Underscores the educational value of research training.
Chapter aims to help readers think critically about research methods, findings, and conclusions: Sets the scope and objectives of the chapter.

Importance of Critical Thinking

Francesco Sizzi failed to recognize Galileo's discovery due to conventional thinking: Shows how preconceived notions can hinder scientific progress.
Sizzi's mistake highlights the importance of critical analysis: Reinforces the necessity of questioning established beliefs.

Elements of Becoming a Critical Research Consumer

Familiarity with research methodologies: Understanding different research approaches and their strengths and limitations.
Understanding why different designs are used and what research questions they answer: Knowing how research questions guide the selection of appropriate methodologies.

Key Elements

Science: Understand what science is, how it works, and why it works: Grasp the systematic and empirical nature of scientific inquiry.
Scientific Theory: Know what constitutes a scientific theory: Understand the components and characteristics of robust scientific theories.
Types of Logic: Understand the different types of logic used in science: Differentiate between inductive, deductive, and abductive reasoning.
Sources of Information: Differentiate between scientific and nonscientific sources of information: Distinguish between evidence-based and non-evidence-based information.
Research Methods: Discriminate between research methods with varying control levels: Compare experimental, quasi-experimental, and observational methods.
Theoretical vs. Applied Research: Understand the differences between theoretical and applied research: Recognize the goals and applications of each type of research.
Replications: Learn what replications are, why they are conducted, and how they advance knowledge: Understand the role of replication in validating research findings.

Science and the Scientific Method

Science: a systematic approach to seeking knowledge about the natural world. Emphasizes empirical evidence and rigorous testing.
Scientific research: a continual process of rigorous reasoning with dynamic interplay among methods, theories, and findings. Highlights the iterative and interconnected nature of scientific inquiry.
Science: The search for understanding the world around us and finding order and lawful relations in the world. Aims to identify patterns and principles that govern natural phenomena.
Science: testing of our ideas in a public forum. Promotes transparency and peer review in the scientific process.

Steps in the Scientific Method

Identify a problem: Awareness of a gap in knowledge. Recognizing an area where understanding is incomplete or lacking.
Define the problem: Clarify the exact nature of the problem. (e.g., "disruptive behavior in the classroom" needs specific definition). Specifying the variables and context of the issue.
Formulate hypotheses or research questions: Hypotheses = statements about tentative relations between concepts. (e.g., if a teacher attends to disruptive behavior, then it will get worse). Research questions = queries about events, not predictions. Developing testable statements or inquiries to guide the investigation.
Determine observable consequences: The concrete outcomes expected if the hypothesis is true. (e.g., teachers who attend to appropriate behavior will have better classroom management). Identifying measurable results that would support the hypothesis.
Test the hypothesis: Collect and analyze data to determine if the hypothesis is correct. (e.g., data shows that teachers who attend to disruptive behaviors experience higher levels of disruptions). Gathering evidence through observation or experimentation to evaluate the hypothesis.

Critical research consumers must recognize that errors can occur: Acknowledges the possibility of flaws in research studies.
Isolated observable events can support conflicting theories: Demonstrates the complexity of interpreting empirical evidence.
We can never prove a theory, only provide support for it: Highlights the tentative nature of scientific conclusions.
Observable data can come from chance: Recognizes the role of random variability in research outcomes.

Purposes of Science

Describe phenomena: Provide detailed accounts of events or occurrences.
Predict events or outcomes: Forecast future occurrences based on current knowledge.
Improve practice: Enhance the effectiveness of interventions or strategies.
Explain phenomena: Develop theories to understand underlying mechanisms.

Description

Attempt to describe an event or phenomenon: Providing comprehensive details about what is observed.
Piaget described child development stages with specific abilities: Illustrates descriptive research in developmental psychology.
Descriptions allow us to draw conclusions: Enable the formulation of inferences based on observed patterns.
Description can lead to future research questions and important discoveries: Serves as a foundation for further scientific inquiry.

Prediction

Make predictions about what will happen in the future. (e.g., standardized tests predict success in school): Forecasting outcomes based on identified relationships.
Identify students likely to do poorly (e.g., low SES backgrounds and Head Start Programs): Using predictive research to target interventions.
Predictions increase the probability of outcomes, but do not guarantee them: Acknowledges the probabilistic nature of scientific predictions.
The importance of making predictions is to provide interventions: Facilitating proactive measures based on anticipated outcomes.

*As with descriptive research, most research

methods allow the prediction of something in the future.

Improvement

Determine the best interventions to bring about desirable changes: Identifying effective strategies for achieving specific goals.
Determine variables that control how children respond (independent variables): Manipulating factors to influence outcomes.
Responses to independent (cause) variables are called dependent variables (effect): Defining the relationship between manipulated and measured variables.
Question: How can we control responses to bring about desired changes?: Guiding inquiry into effective intervention strategies.

Research

Experimental research requires manipulation of the independent variable to determine effects on the dependent variable to determine cause-and-effect relationships: Establishing causality through controlled experimentation.
Causal-comparative methods compare on an existing condition: Examining differences between groups based on pre-existing characteristics.
Correlational research measures relationships between variables: Assessing the strength and direction of associations between variables.

Explanation

Explain why a phenomenon occurs: Uncovering the underlying mechanisms and causes of events.
Researchers develop theories: Constructing models to describe and understand phenomena.
Some argue fully explaining is impossible due to infinite possible reasons: Acknowledges the complexity of causal explanations.
We can demonstrate relationships among variables but never truly know the definitive cause: Highlights the challenges in establishing causality.
Explain requires description, prediction, and the ability to change a phenomenon: Integrates multiple aspects of scientific inquiry.

Scientific Theory

An explanation of observed phenomena about constructs and their relationships: Providing a framework for understanding events and their interconnections.
Theoretical construct: a concept inferred from observing events. (e.g., verbal fluency, depression, intelligence): Defining abstract concepts based on empirical observations.
Example: Teaching alternative behaviors decreases problem behavior, consistent with functional communication training: Illustrating how theories explain practical outcomes.
A variable: a quantitative expression for a construct: Operationalizing constructs for measurement and analysis.

Theories

Identify commonalities: Darwin's theory linked commonalities across species: Recognizing unifying patterns across diverse phenomena.
Enable prediction and control: Scientists predict weather, space travel, behavior: Facilitating forecasts and interventions based on theoretical understanding.
Explanatory system: Describes data gathered before theory development.- Theories should not rely on guessing or speculation in the absence of observation.
- Theories are derived from experimentation and grounded in data.

Scientific vs. Non-Scientific Theories

Scientific theory is falsifiable/refutable and can only be disproven but not proven: Emphasizing the testability and revisability of scientific theories.
Errors are always possible in scientific research: Acknowledging the inherent limitations of scientific inquiry.
Do not say that we will prove a theory, rather that we have confidence in the theory: Highlighting the probabilistic nature of scientific validation.

Popper

Confirmations are easy to obtain if we look for them: Critical reflection on the confirmation bias.
Confirmations should result from risky predictions: Stressing the importance of bold and falsifiable hypotheses.
Good theories forbid certain things from happening: A testable theory makes specific predictions that can be proven wrong.
A non-refutable theory is nonscientific: If a theory can not be proven wrong through testing then it is not a scientific theory
Genuine tests attempt to falsify the theory: Experiments should always be meant to prove the theory wrong instead of right.
Confirming evidence counts when resulting from a test: Only data collected through rigorous experiments should contribute to evidence
Protecting theories with ad hoc assumptions lowers scientific status: Creating excuses to explain away falsifications diminishes the theory

Critical thinking means finding alternative explanations for research findings.

Good vs. Bad scientific theories

Accurately reflects the facts of the real world: Correspondence with empirical evidence.
Is stated clearly and understandably: Clarity and coherence in theoretical formulation.
Explains past events and predicts future events accurately: Explanatory and predictive power.
Offers practical guidance in solving daily problems: Applied relevance and utility.
Is internally consistent: Logical consistency within the theory's framework.
Founded on few unproven assumptions: Parsimony and reliance on minimal unsupported premises.
Can be disproven: Falsifiability and testability.
Has convincing evidence: Empirical support and validation.
Accommodates new data: Adaptability and openness to new information.
Provides a novel or original view: Innovation and uniqueness in perspective.
Offers reasonable answers to all educational phenomena: Applicability across various contexts.
Stimulates new research techniques and knowledge: Generative capacity and influence.
Attracts attention and adherents for an extended period: Enduring impact and recognition.
Fits with one's way of viewing the world: Personal resonance and acceptance.

Types of Scientific Logic

Scientific theories

Constructed after investigations lead to theory development (Induction).
Constructed, then used to predict future events (Deduction).
Scientific logic takes two basic forms: induction and deduction.

Induction

Moving from specific to general.

(e.g., students in this class learned critically; therefore, all future students will too).

Theories are grounded in collected supporting data.

Theories grounded in the data, and investigations are not guided by the theory = Inductive logic.

Deduction

Moving from the more abstract to the specific.
Constructing a theory >> making predictions >> testing parts of the theory.

Combination of Logic

Induction and deduction exist in combination.
Science involves a combination.
Engaging in deductive logic from an inductive source.

Critique of Both Deduction and Induction

Neopositivist approach

The epistemological doctrine that physical and social reality are independent of those who observe it.

Postpositivist approach

Science is viewed as more of sociocultural activity than a technical one.

Ways of Gaining Information

Five ways to consider

Three are nonscientific: tenacity, intuition, authority.
Two are scientific: empiricism, and rationalism.

Tenacity

Persistence of a certain belief or way of thinking for a long period of time.
Arguments include "I was spanked, my parents were spanked,"
Recalling that for centuries the population believed that the world was flat.
Tenacity is not considered a source of scientific information.

Intuition

A "feeling" about a topic.
A decision based on intuition comes from past and current experiences from rather than extrasensory perception.
Decisions are supported by previous information on similar topics.
Feedback may or may not occur, either leading to accuracy or inaccuracy.
Has scientists develop research questions to confirm or refute information.

Authority

Accepted belief based on the person providing the information.
Authorities may be wrong in their information.
This active attempt can gather supporting or refuting information.
Authority is a source of research questions but provide information to confirm or refute a scientific theory.

Empiricism

Gaining information through our senses.
Inferring emotional state based frown, a lowered head, or a quiet demeanor.
Empiricism is the foundation on which science functions, "Show me that it works."
Repeating something over time; otherwise it runs the risk of making incorrect conclusions.

Rationalism

The interpretation and understanding of the world through reasoning.

Examples

Deductive reasoning.
All students learn how to think critically.
The student just finished reading the book.
Then that student can too.

Rationalism and Empiricism can be combined in three ways

Deductive reasoning.
Inductive reasoning.
Fusing induction with deduction.

Constraint Levels in Research

Constraint Level: is the degree of control one has over the research process

Level does not make a research method "good" or "bad".
Cause can mean something resulted due to something else.

*Cause can be determined to mean there is a functional between variables, such

such as a Y occurs reliably followed by X.

High constraint research

Advantage

cause-and-effect determinations about specific variables.

Disadvantage

Makes context extremely artificial, limiting generality.

Low constraint research

Advantage

Can tell us about what will happen in an applied context.

Disadvantage

Cannot provide the effect of variables.

Five levels of constraints

Experimental
Experimental (Single-case)
Causal comparative
Correlational
Case study
Naturalistic/descriptive

Experimental Research

Highest level of constraint is through attempt to isolate the causal variable.
Conducted in a laboratory setting, the sights, sounds, smells, and so forth, can

be controlled by those performing the research thus creating minimal exposure to

outside variables.

Single-Case Methods

Single-case research involves one or a limited number of participants using

within-participant comparisons. Used to identify causal relationships between

variables.

Causal-Comparative Research

Causal-comparative research looks like experimental research, with the

main differences being participants are not assigned, and the independent

variable is not manipulated.

Two-reasons

Independent variable of interest cannot be manipulated >> instructional

technique for students with or without Down syndrome.

Cannot assign Down syndrome to individual

When it is unethical to manipulate the independent variable >> Effects on students

who proper or not proper nutrition.

Correlational Research

Relies on there being just one group.
There isn't any intent to directly manipulate the effects.

Case Study Research

Qualitative: Studying a single participant or group of participants in depth.

Naturalistic or Descriptive Research

Natural research is associated the ecological credibility of research.
Not as much of a definition of an indepedent variable being used >> as not

taking anything for causal study.

Ethnographic Research intense and in-depth

study of the culture. Multiple methods of data such as observation and interviews are analyzed.

Basic vs. Applied Research

Basic research

Focuses on developing new theories or refining existing practices.

Applied research

Attempts to solve real-life problems or to study phenomena that directly impact

practitioners or laypeople, using a theory developed through basic research

*Action research (application of scientific method to everday classroom issues)

Teachers are directly involved in the implementtaion of research for thier benefit.
The setting that is being researched benefits due to the focus being on the action

arising from results.

Replication Research

Replication is the method to determine the usefulness as well reliability of findings >> helping scientists identify mistakes. Replicating involves doing research more then once.

Reasons for replication

Reliability consistency of previous research >> did anything weaken the study.
Generalizability transferability >> One study is not the same as many studies.

Types of Replications

Direct

Conducting in the same manner

Systematic

A systematic replication makes this process better in the form of a

different researcher making the conclusion, as well as individuals with different

backgrounds from the original research used.

Clinical

Clinical replications combine

two different variables to a an existing package, in order to for results that have

Okay, so this chapter is all about how to think critically about research. Basically, it wants us to not just blindly accept anything we read or hear.

Thinking Critically About Research

Objectives

Learn to think critically: You know, question stuff, analyze it, and make your own judgments.
Understand science: Get the basics of what science is and how it works.
Know why we do science: Figure out the goals, like describing things, predicting stuff, improving things, and explaining why things happen.
Explain scientific theory: Understand what a scientific theory really is.
Types of scientific logic: Spot the difference between inductive and deductive reasoning.
How we get information: Compare science-y ways of learning to non-science-y ways.
Constraint levels in research: Get how much control we have in different types of research.
Basic versus applied research: Know the difference between research for knowledge and research for real-world problems.
Replication research: Why it's important to do studies again to see if they're legit.

Understanding Research

Gotta be a critical consumer: Don't just believe everything; check it out yourself.
Critical thinking: Look at all sides before deciding anything. Question, think, evaluate.
Pros skip this: People in jobs sometimes just believe stuff or dismiss it without thinking. Bad!
How you do research matters: The way you do it changes what you find.
Change the steps, change the results: Small tweaks can make a big difference.
Researchers can be biased: They might like their own ideas too much and miss other explanations.
Teach people to think critically: Helps make education and psychology better.
Main goal: To help us think critically about research.
This chapter: Helps us question research methods, what they find, and what they mean.

Importance of Critical Thinking

Guy didn't get Galileo because he was stuck in old ways: Shows why you gotta be open-minded.
That mistake: Shows why we need to question stuff.

Elements of Becoming a Critical Research Consumer

Know your research methods: Understand different ways of doing research and what they're good and bad at.
Know why we use different designs: Understand what research questions they answer.

Key Elements

Science: What it is, how it works, why it works: Get the basics.
Scientific Theory: What makes a good one: Know the parts and what makes them strong.
Types of Logic: Different types we use in science: Like inductive and deductive.
Sources of Information: Science vs. non-science sources: What's based on evidence and what's not.
Research Methods: How much control we have: Experimental, quasi, etc.
Theoretical vs. Applied Research: Research for knowledge vs. research for solving problems.
Replications: Doing studies again: Why it matters.

Science and the Scientific Method

Science: A way to learn about the world using evidence and testing.
Scientific research: Always testing, always changing. Uses methods, theories, and findings.
Science: Finding order in the world.
Science: Testing ideas in public.

Steps in the Scientific Method

Identify a problem: See something you don't understand.
Define the problem: Get specific. Like, what exactly is "disruptive behavior?"
Formulate hypotheses or research questions: Make educated guesses. "If X, then Y."
Determine observable consequences: What should happen if your guess is right?
Test the hypothesis: Do the experiment and see what happens.

Errors happen: Gotta remember that.
One thing can support different ideas: It's complicated.
Can't prove anything, just support it: Science is always open to change.
Chance happens: Sometimes things just happen randomly.

Purposes of Science

Describe stuff: Explain what's going on.
Predict stuff: Guess what will happen.
Improve practice: Make things better.
Explain stuff: Figure out why things happen.

Description

Explain what we see: Give details.
Example: Piaget and child development: Describing what kids can do at different ages.
Descriptions help us figure things out: Draw conclusions.
Leads to more questions: Sets up more research.

Prediction

Guess what will happen: Standardized tests predict school success.
Find students who might struggle: Help them early.
Not always right: Just increases the chances.
Helps us intervene: Take action.
Most research helps predict.

Improvement

Find the best ways to make things better: Interventions.
Find what controls how kids act: Independent variables (things we change).
Dependent variables: What happens because of the changes.
Question: How do we control things to make them better?

Research

Experiments: Change one thing to see what happens.
Causal-comparative: Compare existing conditions.
Correlational: Measure relationships between things.

Explanation

Explain why things happen: Find the reasons.
Develop theories: Models to explain stuff.
Hard to know everything: Always more to learn.
Can't know the real reason: Just relationships.
Need description, prediction, and ability to change things.

Scientific Theory

Explains what we see: How things connect.
Theoretical construct: An idea we get from watching.
Example: Teaching new behaviors reduces problem behavior.
Variable: A number for an idea.

Theories

Find common things: Darwin and species.
Predict and control: Predict weather, space travel, behavior.
Explain stuff: Based on data, not guessing.

Scientific vs. Non-Scientific Theories

Scientific theory: Can be proven wrong: Always testable.
Errors happen: Always possible.
Don't say "prove", say "confidence": Science is always open to change.

Popper

Easy to find what you want to see: Confirmation bias.
Should be risky predictions: Bold guesses that can be wrong.
Good theories stop things from happening: Testable.
Can't be proven wrong = not science: Gotta be testable.
Try to prove it wrong: Not right.
Only data from experiments counts: Rigorous testing.
Excuses lower the status: Don't make excuses.

Critical thinking means finding other ways to explain things.

Good vs. Bad scientific theories

Fits the real world: Evidence.
Clear: Easy to understand.
Explains past, predicts future: Power.
Helps solve problems: Useful.
Makes sense: Consistent.
Few assumptions: Simple.
Can be disproven: Testable.
Evidence: Support.
New data: Flexible.
Original: Unique.
Explains everything: Applies widely.
New research: Generates ideas.
Lasts long: Impactful.
Fits your views: Personal.

Types of Scientific Logic

Scientific theories

After research (Induction).
Predict future (Deduction).
Two types: induction and deduction.

Induction

Specific to general.
Example: This class learned critically, so all will..
Based on data.
Theories come from data = Inductive logic.

Deduction

General to specific.
Theory >> predictions >> testing.

Combination of Logic

Induction and deduction together.
Science uses both.
Deductive from inductive.

Critique of Both Deduction and Induction

Neopositivist approach

Reality is independent.

Postpositivist approach

Science is sociocultural.

Ways of Gaining Information

Five ways

Three non-scientific: tenacity, intuition, authority.
Two scientific: empiricism, and rationalism.

Tenacity

Stick with what you know.
Example: "I was spanked, my parents were spanked."
Example: World is flat.
Not scientific.

Intuition

A feeling.
From past experiences.
Feedback may or may not occur, either leading to accuracy or inaccuracy.
Helps form questions.

Authority

Believe it because of who said it.
Authorities can be wrong.
Helps gather information.
A source of research questions but provide information to confirm or refute a scientific theory.

Empiricism

Use your senses.
Example: Inferring emotion.
Show me it works.
Repeat over time.

Rationalism

Use reasoning.

Examples

Deductive reasoning.
All students learn how to think critically.
The student just finished reading the book.
Then that student can too.

Rationalism and Empiricism can be combined in three ways

Deductive reasoning.
Inductive reasoning.
Fusing induction with deduction.

Constraint Levels in Research

Constraint Level: How much control you have.

Doesn't make it good or bad.
Cause = something causes something else.
Cause can be determined to mean there is a functional between variables, such
such as a Y occurs reliably followed by X.

High constraint research

Advantage

See cause and effect.

Disadvantage

Artificial, not real life.

Low constraint research

Advantage

Real-life context.

Disadvantage

Can't see the effect of variables.

Five levels

Experimental
Experimental (Single-case)
Causal comparative
Correlational
Case study
Naturalistic/descriptive

Experimental Research

Most control, isolate the cause.
Lab setting, control everything.

Single-Case Methods

One person, see what changes.
Single-case research involves one or a limited number of participants using
within-participant comparisons. Used to identify causal relationships between
variables.

Causal-Comparative Research

Like experiments, but can't assign people or change things.

Two-reasons

Can't change the independent variable >> instructional
technique for students with or without Down syndrome.
Cannot assign Down syndrome to individual
When it is unethical to manipulate the independent variable >> Effects on students
who proper or not proper nutrition.

Correlational Research

One group, no manipulation.
There isn't any intent to directly manipulate the effects.

Case Study Research

Studying one person or group closely.
Qualitative: Studying a single participant or group of participants in depth.

Naturalistic or Descriptive Research

Real-world research.
Not as much of a definition of an indepedent variable being used >> as not
taking anything for causal study.

Ethnographic Research intense and in-depth
study of the culture. Multiple methods of data such as observation and interviews are analyzed.

Basic vs. Applied Research

Basic research

New theories or improve old ones.
Focuses on developing new theories or refining existing practices.

Applied research

Solve real problems.
Attempts to solve real-life problems or to study phenomena that directly impact
practitioners or laypeople, using a theory developed through basic research
Action research (application of scientific method to everday classroom issues)
Teachers are directly involved in the implementtaion of research for thier benefit.
The setting that is being researched benefits due to the focus being on the action
arising from results.

Replication Research

Do the study again to see if it's right.
Replication is the method to determine the usefulness as well reliability of findings >> helping scientists identify mistakes. Replicating involves doing research more then once.

Reasons for replication

See if it's consistent >> weak study?
See if it applies to other situations >> One study is not the same as many studies.
Reliability consistency of previous research >> did anything weaken the study.
Generalizability transferability >> One study is not the same as many studies.

Types of Replications

Direct

Do it the same way
Conducting in the same manner

Systematic

Different researcher repeats, different backgrounds from the original research used.
A systematic replication makes this process better in the form of a
different researcher making the conclusion, as well as individuals with different
backgrounds from the original research used.

Clinical

Combine two things.
Clinical replications combine
two different variables to a an existing package, in order to for results that have

Okay, so this chapter is all about how to think critically about research. Basically, it wants us to not just blindly accept anything we read or hear.

Thinking Critically About Research

Objectives

Learn to think critically: You know, question stuff, analyze it, and make your own judgments.
Understand science: Get the basics of what science is and how it works.
Know why we do science: Figure out the goals, like describing things, predicting stuff, improving things, and explaining why things happen.
Explain scientific theory: Understand what a scientific theory really is.
Types of scientific logic: Spot the difference between inductive and deductive reasoning.
How we get information: Compare science-y ways of learning to non-science-y ways.
Constraint levels in research: Get how much control we have in different types of research.
Basic versus applied research: Know the difference between research for knowledge and research for real-world problems.
Replication research: Why it's important to do studies again to see if they're legit.

Understanding Research

Gotta be a critical consumer: Don't just believe everything; check it out yourself.
Critical thinking: Look at all sides before deciding anything. Question, think, evaluate.
Pros skip this: People in jobs sometimes just believe stuff or dismiss it without thinking. Bad!
How you do research matters: The way you do it changes what you find.
Change the steps, change the results: Small tweaks can make a big difference.
Researchers can be biased: They might like their own ideas too much and miss other explanations.
Teach people to think critically: Helps make education and psychology better.
Main goal: To help us think critically about research.
This chapter: Helps us question research methods, what they find, and what they mean.

Importance of Critical Thinking

Guy didn't get Galileo because he was stuck in old ways: Shows why you gotta be open-minded.
That mistake: Shows why we need to question stuff.

Elements of Becoming a Critical Research Consumer

Know your research methods: Understand different ways of doing research and what they're good and bad at.
Know why we use different designs: Understand what research questions they answer.

Key Elements

Science: What it is, how it works, why it works: Get the basics.
Scientific Theory: What makes a good one: Know the parts and what makes them strong.
Types of Logic: Different types we use in science: Like inductive and deductive.
Sources of Information: Science vs. non-science sources: What's based on evidence and what's not.
Research Methods: How much control we have: Experimental, quasi, etc.
Theoretical vs. Applied Research: Research for knowledge vs. research for solving problems.
Replications: Doing studies again: Why it matters.

Science and the Scientific Method

Science: A way to learn about the world using evidence and testing.
Scientific research: Always testing, always changing. Uses methods, theories, and findings.
Science: Finding order in the world.
Science: Testing ideas in public.

Steps in the Scientific Method

Identify a problem: See something you don't understand.
Define the problem: Get specific. Like, what exactly is "disruptive behavior?"
Formulate hypotheses or research questions: Make educated guesses. "If X, then Y."
Determine observable consequences: What should happen if your guess is right?
Test the hypothesis: Do the experiment and see what happens.

Errors happen: Gotta remember that.
One thing can support different ideas: It's complicated.
Can't prove anything, just support it: Science is always open to change.
Chance happens: Sometimes things just happen randomly.

Purposes of Science

Describe stuff: Explain what's going on.
Predict stuff: Guess what will happen.
Improve practice: Make things better.
Explain stuff: Figure out why things happen.

Description

Explain what we see: Give details.
Example: Piaget and child development: Describing what kids can do at different ages.
Descriptions help us figure things out: Draw conclusions.
Leads to more questions: Sets up more research.

Prediction

Guess what will happen: Standardized tests predict school success.
Find students who might struggle: Help them early.
Not always right: Just increases the chances.
Helps us intervene: Take action.
Most research helps predict.

Improvement

Find the best ways to make things better: Interventions.
Find what controls how kids act: Independent variables (things we change).
Dependent variables: What happens because of the changes.
Question: How do we control things to make them better?

Research

Experiments: Change one thing to see what happens.
Causal-comparative: Compare existing conditions.
Correlational: Measure relationships between things.

Explanation

Explain why things happen: Find the reasons.
Develop theories: Models to explain stuff.
Hard to know everything: Always more to learn.
Can't know the real reason: Just relationships.
Need description, prediction, and ability to change things.

Scientific Theory

Explains what we see: How things connect.
Theoretical construct: An idea we get from watching.
Example: Teaching new behaviors reduces problem behavior.
Variable: A number for an idea.

Theories

Find common things: Darwin and species.
Predict and control: Predict weather, space travel, behavior.
Explain stuff: Based on data, not guessing.

Scientific vs. Non-Scientific Theories

Scientific theory: Can be proven wrong: Always testable.
Errors happen: Always possible.
Don't say "prove", say "confidence": Science is always open to change.

Popper

Easy to find what you want to see: Confirmation bias.
Should be risky predictions: Bold guesses that can be wrong.
Good theories stop things from happening: Testable.
Can't be proven wrong = not science: Gotta be testable.
Try to prove it wrong: Not right.
Only data from experiments counts: Rigorous testing.
Excuses lower the status: Don't make excuses.

Critical thinking means finding other ways to explain things.

Good vs. Bad scientific theories

Fits the real world: Evidence.
Clear: Easy to understand.
Explains past, predicts future: Power.
Helps solve problems: Useful.
Makes sense: Consistent.
Few assumptions: Simple.
Can be disproven: Testable.
Evidence: Support.
New data: Flexible.
Original: Unique.
Explains everything: Applies widely.
New research: Generates ideas.
Lasts long: Impactful.
Fits your views: Personal.

Types of Scientific Logic

Scientific theories

After research (Induction).
Predict future (Deduction).
Two types: induction and deduction.

Induction

Specific to general.
Example: This class learned critically, so all will..
Based on data.
Theories come from data = Inductive logic.

Deduction

General to specific.
Theory >> predictions >> testing.

Combination of Logic

Induction and deduction together.
Science uses both.
Deductive from inductive.

Critique of Both Deduction and Induction

Neopositivist approach

Reality is independent.

Postpositivist approach

Science is sociocultural.

Ways of Gaining Information

Five ways

Three non-scientific: tenacity, intuition, authority.
Two scientific: empiricism, and rationalism.

Tenacity

Stick with what you know.
Example: "I was spanked, my parents were spanked."
Example: World is flat.
Not scientific.

Intuition

A feeling.
From past experiences.
Feedback may or may not occur, either leading to accuracy or inaccuracy.
Helps form questions.

Authority

Believe it because of who said it.
Authorities can be wrong.
Helps gather information.
A source of research questions but provide information to confirm or refute a scientific theory.

Empiricism

Use your senses.
Example: Inferring emotion.
Show me it works.
Repeat over time.

Rationalism

Use reasoning.

Examples

Deductive reasoning.
All students learn how to think critically.
The student just finished reading the book.
Then that student can too.

Rationalism and Empiricism can be combined in three ways

Deductive reasoning.
Inductive reasoning.
Fusing induction with deduction.

Constraint Levels in Research

Constraint Level: How much control you have.

Doesn't make it good or bad.
Cause = something causes something else.
Cause can be determined to mean there is a functional between variables, such
such as a Y occurs reliably followed by X.

High constraint research

Advantage

See cause and effect.

Disadvantage

Artificial, not real life.

Low constraint research

Advantage

Real-life context.

Disadvantage

Can't see the effect of variables.

Five levels

Experimental
Experimental (Single-case)
Causal comparative
Correlational
Case study
Naturalistic/descriptive

Experimental Research

Most control, isolate the cause.
Lab setting, control everything.

Single-Case Methods

One person, see what changes.
Single-case research involves one or a limited number of participants using
within-participant comparisons. Used to identify causal relationships between
variables.

Causal-Comparative Research

Like experiments, but can't assign people or change things.

Two-reasons

Can't change the independent variable >> instructional
technique for students with or without Down syndrome.
Cannot assign Down syndrome to individual
When it is unethical to manipulate the independent variable >> Effects on students
who proper or not proper nutrition.

Correlational Research

One group, no manipulation.
There isn't any intent to directly manipulate the effects.

Case Study Research

Studying one person or group closely.
Qualitative: Studying a single participant or group of participants in depth.

Naturalistic or Descriptive Research

Real-world research.
Not as much of a definition of an indepedent variable being used >> as not
taking anything for causal study.

Ethnographic Research intense and in-depth
study of the culture. Multiple methods of data such as observation and interviews are analyzed.

Basic vs. Applied Research

Basic research

New theories or improve old ones.
Focuses on developing new theories or refining existing practices.

Applied research

Solve real problems.
Attempts to solve real-life problems or to study phenomena that directly impact
practitioners or laypeople, using a theory developed through basic research
Action research (application of scientific method to everday classroom issues)
Teachers are directly involved in the implementtaion of research for thier benefit.
The setting that is being researched benefits due to the focus being on the action
arising from results.

Replication Research

Do the study again to see if it's right.
Replication is the method to determine the usefulness as well reliability of findings >> helping scientists identify mistakes. Replicating involves doing research more then once.

Reasons for replication

See if it's consistent >> weak study?
See if it applies to other situations >> One study is not the same as many studies.
Reliability consistency of previous research >> did anything weaken the study.
Generalizability transferability >> One study is not the same as many studies.

Types of Replications

Direct

Do it the same way
Conducting in the same manner

Systematic

Different researcher repeats, different backgrounds from the original research used.
A systematic replication makes this process better in the form of a
different researcher making the conclusion, as well as individuals with different
backgrounds from the original research used.

Clinical

Combine two things.
Clinical replications combine
two different variables to a an existing package, in order to for results that have

In research, the constraint level indicates the degree of control a researcher exerts over the study. Higher constraint levels facilitate cause-and-effect determinations but may reduce the

Clinical: Combines two things.

Clinical replications combine
two different variables to a an existing package, in order to for results that have