Chapter 2: Reading and Evaluating Scientific Research

Objective Measurements

the measure of an entity or behaviour that, within an allowed margin of error, is consistent across instruments and observers

Variable

the object, concept, or event being controlled, manipulated, or measured by a scientist

Operational Definitions

statements that describe the procedures (or operations) and specific measures that are used to record observations

Validity

the degree to which an instrument or procedure actually measures what it claims to measure

Reliability

measure that provides consistent and stable answers across multiple observations and points in time

Test-retest Reliability

examines whether scores on a given measure of behaviour are consistent across test sessions

Alternate-forms Reliability

examines whether different forms of the same test produce the same results

Inter-rater Reliability

when the raters arrive at very similar conclusions 

Generalizability

the degree to which one set of results can be applied to other situations, individuals, or events

Population

the groups that researchers want to generalize about

Sample

a select group of population members

Random Sample

a sampling technique in which every individual of a population has an equal chance of being included

Convenience Sample

samples of individuals who are then most readily available

Ecological Validity

the results of a laboratory study can be applied to or repeated in the natural environment

Researcher Bias

an unintentionally introduced form of bias from the researchers

Subject/Participant Bias

when the test subject (person or animal) introduce their own bias

Hawthorne Effect

a behaviour change that occurs as a result of being observed

Social Desirability

Research participants respond in ways that increase the chances that they will be viewed favourably by the experimenter and/or other participants

Placebo Effect

a measurable and experienced improvement in health or behaviour that cannot be attributable to a medication or treatment

Demand Characteristics

inadvertent cues given off by the experimenter or the experimental context that provide information about how participants are expected to behave

Anonymity

each individual’s responses are recorded without any name or other personal information that could link a particular individual to specific results

Confidentiality

the results will be seen only by the researcher

Single-blind Study

the participants do not know the true purpose of the study, or else do not know which type of treatment they are receiving

Double-blind Study

a study in which neither the participant nor the experimenter knows the exact treatment for any individual

Peer Review

a process in which papers submitted for publication in scholarly journals are read and critiqued by experts in the specific field of study

Replication

the process of repeating a study and finding a similar outcome each time

Publication Bias

when successful results are published and studies that showed no effect are not

Falsifiable

the hypothesis is precise enough that it could be proven false

Anecdotal Evidence

an individual’s story or testimony about an observation or event that is used to make a claim as evidence

Data Selection Bias

when people only present the data that support their views

Appeal to Authority

the belief in an “expert’s” claim even when no supporting data or scientific evidence is present

Appeal to Common Sense

a claim that appears to be sound but lacks supporting scientific evidence

Research Design

a set of methods that allows a hypothesis to be tested

Descriptive Research

answers the question of “what” a phenomenon is; describes its characteristics

Qualitative Research

involves examining an issue or behaviour without performing numerical measurements of the variables (interviews)

Quantitative Research

involves examining an issue or behaviour by using numerical measurements and/or statistics

Case Study

an in-depth report about the details of a specific case

Phineas Gage

iron rod through the head shows that head trauma causes personality change

Naturalistic Observations

they unobtrusively observe and record behaviour as it occurs in the subject’s natural environment

Unobtrusively

the individuals being observed shouldn’t know that they are being observed

Self-reporting

a method in which responses are provided directly by the people who are being studied, typically through face-to-face interviews, phone surveys, paper-and-pencil tests, and web-based questionnaires

Correlation Research

measuring the degree of association between two or more variables

Scatterplot Direction

• If the correlations are positive it means that the two variables change values in the same direction (both increase)

• If the correlations are negative it means that as the value of one variable increases, the value of the other decreases

Scatterplot Magnitude

• Describes the correlation coefficient

• Correlation of 0 means there is no relationship 

• +1 is the most positive correlation coefficient possible 

• -1 is the most negative correlation coefficient possible 

Third Variable Problem

the possibility that a third, unmeasured variable is responsible for a well-established correlation between two variables

Illusory Correlations

relationships that exist only in the mind, rather than in reality

Random Assignment

a technique for dividing samples into two or more groups in which participants are equally likely to be placed in any condition of the experiment

Confounding Variable

a variable outside of the researcher’s control that might affect or provide an alternative explanation for the results

Independent Variable

the variable that the experimenter manipulates to distinguish between two or more groups (images being viewed)

Dependent Variable

the observation or measurement that is recorded during the experiment and subsequently compared across all groups (stress response)

Between-subjects Design

an experimental design in which we compare the performance of participants who are in different groups

Experimental Group

the group in the experiment that receives a treatment or stimuli targeting a specific behaviour

Control Group

the group that does not receive the treatment or stimuli targeting a specific behaviour; this group serves as baseline to which the experimental group is compared

Within-subjects Design

an experimental design in which the same participants respond to all types of stimuli or experience all experimental conditions

Quasi-experimental Research

a research technique in which the two or more groups that are compared are selected based on predetermined characteristics, rather than random assignment

Converging Operations

when a theory’s predictions hold up to dozens of tests using a variety of designs; using multiple research methods to analyze the same question

Research Ethics Board (REB)

a committee of researchers and officials at an institution charged with the protection of human research participants

1. Committee weighs potential risks to the volunteers against the possible benefits of the research

2. Requires that volunteers agree to participate in the research

Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (2nd edition):

a set of requirements created by the Government of Canada’s Panel of Research Ethics

Informed Consent

A potential volunteer must be informed and give consent without pressure and must know:

• The topic of the study

• The nature of any stimuli to which they will be exposed

• The nature of any tasks they will complete

• The approximate duration of the study

• Any potential physical, psychological, or social risks involved

• The steps that the researchers have taken to minimize those risks

Deception

misleading or only partially informing participants of the true topic or hypothesis under investigation

Debriefing

meaning that the researchers should explain the true nature of the study, and especially that nature of and reason for any deception

Scientific Misconduct

when individuals fabricate or manipulate their data to fit their desired results 

Andrew Wakefield

published a paper incorrectly linking vaccines to autism