Critical Thinking About Research

Introduction

Research evaluation? Think detective work. Spot clues to see if the research answers the question.
Ask yourself:
- Does the research design really tackle the question?
- Do the results give us real answers?
- What's missing? What did the researchers miss? Spotting these gaps is key.

Key Elements for Effective Research

The book gives us seven tools for smart research reading.
Think of them as your research toolkit.
The goal? To do research that hits the mark in your field.

What is Science?

Science: Finding patterns and rules in the world.
Psychology: Describe, explain, and guess behavior.
Good research helps us:
- See what people do or think.
- Figure out what they might do in the future.
- Understand why they do what they do.

Conceptualizing Science

Think of science as a special pair of glasses for looking at the world.
It's about being careful, thoughtful, and keeping things in check.
"Control" doesn't always mean a fancy controlled experiment.

Research Questions vs. Hypotheses

Here's where things get a bit different depending on who you ask. Some fields love research questions, others want hypotheses.
Research questions: Great for exploring new stuff.
Hypotheses: Use them when you're ready to make a prediction. They might be what's expected to show that your research is top-notch.

Purposes of Research

Research should aim to:
- Paint a picture.
- Make predictions.
- Explain why things happen.
- Make things better (especially when it comes to fixing problems).

Scientific Theory

A theory is like a story that explains what we've seen and how things connect.
It's a way to make sense of all the facts we have.
Theories help us see what's the same and what's different in various situations.
A good theory should be something you can try to prove wrong.

Falsifiability

Theories have to be testable. We need to see if they can be proven wrong.
Big theories (like Bandura's or Bronfenbrenner's) can explain almost anything, which makes them tough to test.
Smaller theories are better because they zoom in on things we can actually test.

Grand Theories

Big theories from folks like Freud or Piaget aren't as popular as they used to be because they often don't hold up under the science microscope.

Why Have Theories?

Theories help us find common threads.
They're great for guessing what's going to happen.
They give us ways to explain things that are based on real stuff.
Coming up with good theories is a big deal in the research world.

Resisting Falsification

Sometimes, researchers get too attached to their own ideas. They don't want to admit they might be wrong.
This can slow down progress and keep us from accepting new info.
For instance, Piaget's ideas about stages of development are still taught, even though we know they're not quite right.
Old ideas stick around because they feel familiar and make sense to us.

Standards of a Good Theory

Table 1.1 in the book gives us 14 things to look for in a good theory:

Real-World Reflection: Does it match what's really going on?
Clarity: Is it easy to get?
Predictive Power: Can it explain what's happened and what will happen?
Practical Guidance: Does it help us solve real problems?
Internal Consistency: Do all the pieces fit together?
Few Unproven Assertions: Does it keep the guesswork to a minimum?
Disprovability: Can we test it to see if it's wrong?
Convincing Evidence: Is there solid proof to back it up?
Handling New Data: Can it handle new info that comes along?
Novelty: Does it give us a fresh way of looking at things?
Reasonable Answers: Does it make sense?
Stimulates Research: Does it make us want to dig deeper?
Enduring Attention: Does it stay interesting over time?
Alignment with Worldview: Does it jive with how you see the world?

Logic in Research

Instead of just saying something "logically follows," make sure your writing actually makes sense.
There are two main types of logic:
- Inductive Logic: Start with what you see and build a theory.
- Deductive Logic: Start with a theory and use it to make sense of things.
- Interactive Logic: Mix and match the two.

Hypotheses vs. Questions

Hypotheses go with deductive logic.
Research questions go with inductive logic.

Sources of Information

Non-Scientific Sources

Tenacity: Sticking to beliefs just because they've been around for a while, even if there's no proof. Like the idea that everyone has a "learning style," even though it's not really true.
Intuition: Thinking something is true just because it feels right.
Authority: Believing something because someone you trust said it, without checking it out yourself.

Scientific Sources

Empiricism: Getting info by using your senses and watching what happens.
Rationalism: Using your brain to make sense of things.
Most researchers like to use a bit of both.

Continuum of Research

Researchers are okay with different amounts of uncertainty.
If you control things more in your research, you can be more sure about what's causing what.
But more control can also mean your research doesn't really apply to the real world.
Finding the right balance is key.
Getting research from lots of different people can give us a better picture.

Hierarchy of Research Designs

Experimental Studies:
- Randomized Controlled Trials (RCT): The gold standard for control.
- Simple Experimental: Might not be as controlled.
- Quasi-Experimental
Mixed Methods
Causal Comparative Studies:
- Comparing groups that are already different.
- For example: Comparing kids who move a lot to those who don't, or kids in charter schools to those in public schools.
- Can't really prove cause and effect.
Correlational Studies
- Simple Correlational
- Regression: Not always about cause and effect.
Qualitative Data
Descriptive Data:
- Just tells you what's there, not why.
- Gives you basics like how many people were studied and what the averages were.

Forms and Style

Basic Research

Helps us build theories.

Applied Research

Uses theories to solve real-world problems.

Action Research

Super specific, usually done by teachers in classrooms.
It's about tracking how well certain fixes are working.
Control can be a problem.
It's hard to know if it's the fix that's working or just the person using it.

Replication

Types of Replication

Direct Replication: Doing the exact same study again.
Systematic Replication: Changing things like where you do the study or who's on the team, but keeping the main ideas the same.

Value of Replication

Helps us see if the findings work in different places.
Shows us that the results aren't just a fluke.

Challenges in Replication

Lots of famous studies have issues. Like the Milgram study or the marshmallow study (which only worked once).
Even things like the word length effect don't always work with every word.

Publication Bias

Studies that show big effects are more likely to get published.
Studies that don't show anything often get ignored.

Generalization and Diverse Populations

Doing studies with all sorts of people is super important.
A lot of old studies only looked at certain groups (like white, middle-class men).
Culture can change everything. Like how test anxiety is different in different countries because the