Essentials of Research Design and Methodology

Flashcards from Lecture Notes on Research Design and Methodology

What are the defining characteristics of scientific research?

Testing hypotheses, careful observation and measurement, systematic evaluation of data, drawing valid conclusions

What is the goal of correlational research?

To determine whether two or more variables are related, without necessarily implying causation.

What is the primary goal of experimental research?

To compare two groups on one outcome measure to test a hypothesis regarding causation.

How can science be defined?

A methodological and systematic approach to acquiring new knowledge.

What is scientific knowledge based on?

Objective data that were reliably obtained in the context of a carefully designed research study.

What is the scientific method?

An approach to acquiring new knowledge that distinguishes science from non-science by providing clear guidelines for gathering, evaluating, and reporting information.

What does the empirical approach rely on?

Direct observations and experimentation in the acquisition of new knowledge.

Why is an operational definition important in research?

Researchers ensure that everyone is talking about the same phenomenon, providing clarity and avoiding ambiguity.

What is a hypothesis?

An educated and testable guess about the answer to a research question, often phrased as an 'if-then' statement.

What does the null hypothesis state?

Predicts that there will be no differences between the groups being studied.

What does the alternate (experimental) hypothesis state?

Predicts that there will be a difference between the groups being studied.

What did Karl Popper state about hypotheses?

That hypotheses must be falsifiable; researchers must be able to demonstrate that the hypothesis is wrong.

What does accuracy refer to in the context of experiments?

Whether the measurement is correct.

What does reliability refer to in the context of experiments?

Whether the measurement is consistent.

What is the purpose of using statistical techniques when analyzing data?

To minimize the likelihood of reaching an erroneous conclusion about the relationship between the variables being studied.

What is a Type I error?

Concluding that there is a difference between the groups being studied when, in fact, there is no difference (a 'false positive').

What is a Type II error?

Concluding that there is not a difference between the two groups when, in fact, there is a difference (a 'false negative').

What does replication mean in the context of research?

Conducting the same research study a second time with another group of participants to see whether the same results are obtained

What are the key features of quantitative research?

Involves studies that make use of statistical analysis to obtain their findings, using formal and systematic measurement.

What are the key features of qualitative research?

Involves studies that do not attempt to quantify their results through statistical summary or analysis, often using interviews and observations without formal measurement.

What is the nomothetic approach?

Uses the study of groups to identify general laws that apply to a large group of people, aiming to identify the average member or average performance.

What is the idiographic approach?

The study of an individual.

What is a sample?

A subset of the population, used because it is often not feasible to study an entire population.

What is a population?

All individuals of interest to the researcher.

What does description refer to in research:

Defining, classifying, or categorizing phenomena of interest.

What is a positive correlation between two variables?

Both variables change in the same direction (either both increase or both decrease).

What is a negative correlation between two variables?

As one variable increases, the other variable decreases; the variables change in opposite directions.

What are the three prerequisites for drawing an inference of causality between two events?

There must be a relationship between two events, the cause must precede the effect, and alternative explanations for the relationship must be ruled out.

What is a theory in the context of research?

A conceptualization or description of a phenomenon that attempts to integrate all that we know about the phenomenon into a concise statement or question, serving as a rich source of hypotheses.

What is the purpose of a literature review?

To help researchers become familiar with the work that has already been conducted in their selected topic areas.

What three criteria must a research problem meet?

It must describe a relationship between two or more variables, take the form of a question, and be capable of being tested empirically.

What does the Null Hypothesis always predict?

Predicts that there will be no differences between the groups being studied, or that there is no relationship between the variables being studied.

When are directional hypotheses used?

Used when researchers believe that the groups being studied will differ AND they have a belief regarding how the group will differ.

What is the independent variable?

The factor that is manipulated or controlled by the researcher; the 'cause'.

What is the dependent variable?

A measure of the effect (if any) of the independent variable; the 'effect'.

What are categorical variables?

Variables that can take on specific values only within a defined range of values (e.g., gender, marital status).

What are continuous variables?

Variables that can theoretically take on any value along a continuum (e.g., age, weight, income).

What is random selection?

a procedure through which a sample of participants is chosen from the population of interest in such a way that each member of the population has an equal probability of being selected to participate in the study.

What is 'random assignment'?

The most effective method of assigning participants to groups

What is 'random assignment'?

Assigning study participants to groups within the study in such a way that each participant has an equal probability of being assigned to any of the groups within the study

Why is 'random assignment' important?

It is an effective way of ensuring that the groups within a study are equivalent

How are multicultural considerations distinct?

Multicultural considerations often have a considerable effect on a researcher’s choice of research question and research design and are important in the selection and composition of the sample of participants used in particular research studies

Multiculturalism and researchers

The researcher often benefits from consulting with members of diverse and traditionally underrepresented cultural groups, and researchers also need to be aware of multicultural considerations when deciding on assessment techniques and instruments for their studies

What does validity refer to in research?

The conceptual and scientific soundness of a research study or investigation.

What is internal validity?

The ability of a research design to rule out alternative explanations for the results.

What is external validity?

The generalizability of the results of a research study to other conditions, participants, times, and places.

What is construct validity?

The congruence between the study’s results and the theoretical underpinnings guiding the research.

What is statistical validity?

Aspects of quantitative evaluation that affect the accuracy of the conclusions drawn from the results of a study.

What is experimenter bias?

The potential for researchers themselves to inadvertently influence the behavior of research participants in a certain direction.

What is the double-blind technique?

The most powerful method for controlling researcher expectancy and related bias

What are Participant effects?

A source of artifact and bias stemming from a variety of factors related to the unique motives, attitudes, and behaviors that participants bring to any research study

What is the Hawthorne effect?

Participants often change their behavior merely as a response to being observed and to be helpful to the researcher.

What is randomization?

A control method that helps to ensure that extraneous sources of artifact and bias will not confound the validity of the results of the study

What is random selection?

Random selection is a control technique that increases external validity

What is a 'sample of convenience'?

A sample of convenience is simply a potential source of research participants that is easily accessible to the researcher

What is another way to control extraneous variables?

Another selection approach for controlling extraneous variables requires the researchers to hold the extraneous variable in question constant by selecting a sample that is very uniform or homogeneous on the variable of interest.

What is a 'mean'?

an average score

What is a standard deviation?

a measure of variability indicating the average amount that scores vary from the mean.

What is a 'T-test'?

for two groups

What is ANOVA?

Analysis of variance

What is ANCOVA?

Analysis of co-variance

What is 'Partial Correlation'?

In essence, a partial correlation is a correlation between two variables after one or more variables have been mathematically controlled for and partialled out.

Why is 'Measurement' so important?

It is often viewed as being the basis of all scientific inquiry, and measurement techniques and strategies are therefore an essential component of research methodology

What is the difference between Non-metric Data vs. Metric data?

Nonmetric data cannot be quantified and are predominantly used to describe and categorize. Metric data are used to examine amounts and magnitudes.

What are Nominal scales?

Used only to qualitatively classify or categorize not to quantify.

What are Ordinal scales?

Categorize a variable and its relative magnitude in relation to other variables.

What are Interval scales?

Build on ordinal measurement and Provide information about both order and distance between values of variables.

What are Ratio scales?

Identical to the interval scale, except that they have an absolute zero point.

What does reliability refer to?

The consistency or dependability of a measurement technique.

What is referred to as a 'True score'?

The score that would have been obtained if the measurement strategy were perfect and error free

What is meant by 'Measurement error'?

The portion of the score that is due to distortion and imprecision from a wide variety of potential factors, such as a poorly designed test, situational factors, and mistakes in the recording of data

What is meant by the term 'Test-retest reliability'?

Refers to the stability of test scores over time and involves repeating the same test on at least one other occasion

What does 'Split-half reliability' refer to?

referrs to the administration of a single test that is divided into two equal halves

What is 'Alternate-form reliability'?

expressed as the correlation between different forms of the same measure where the items on each measure represent the same item content and construct

What is 'Interrater reliability'?

is used to determine the agreement between different judges or raters when they are observing or evaluating the performance of others.

What does validity focus on?

focuses on what the test or measurement strategy measures and how well it does so.

What is 'Content-related validity'?

refers to the relevance of the instrument or measurement strategy to the construct being measured

What is 'Criterion validity'?

is determined by the relationship between the measure and performance on an outside criterion or measure

What does 'Construct validity' measure?

assesses the extent to which the test or measurement strategy measures a theoretical construct or trait

What is 'Interviewing'?

is a form of self-report that is a relatively simple approach to data collection

What is 'Global ratings'?

a form of self-report that is commonly used as a data collection technique in research and quantifies a construct by asking the participant to rate his or her response to a summary statement on a numerical continuum

What is 'Observation'?

This approach relies on the direct observation of the construct of interest, which is often some type of behavior

What are 'Biological measures'?

This approach is common in medical and psychobiological research andit often involves measuring the physiological responses of participants to any number of potential stimuli

What is a true experimental design?

a one in which study participants are randomly assigned to experimental and control groups

What is a random numbers table used for?

A random numbers table is nothing more than a random list of numbers displayed or printed in a series of columns and rows

What is the primay purpose of the Randomized two-group design?

the Randomized two-group design is in order to demonstrate causality— that is, to determine whether a specific intervention (the independent variable) causes an effect (as opposed to being merely correlated with an effect)

Why is random assignment important in the Randomized two-group design???

Random assignment ensures, to some degree, that the two groups are equivalent before treatment so that any post-treatment differences can be attributed to the treatment.

What is meant by the term, 'Attrition'?

the loss of participants during the course of the study

What is the principal advantage of the Solomon four-group design?

The key advantage of the Solomon four-group design is that it controls for the potential effects of the pretest on posttest outcomes

What does a 'Factorial design' allow us to do?

the factorial design allows us to look for interactions between independent variables which means it permits the simultaneous examination of more than one independent variable

Are experimental designs perfect?

Threats to validity

When does 'Compensatory equalization of treatments occur'?

occurs when one of the groups is perceived by participants as better or more desirable than the other

When does 'Compensatory rivalry' occur?

occurs when there are substantial differences in the implementation of the experimental and control conditions.

When are Quasi-experimental designs necessary?

When randomized designs are not feasible, researchers must often make use of quasi- experimental designs because they attempt to select groups that are as similar as possible.

In the nonequivalent groups posttest-only

In the nonequivalent groups posttest-only design, one group (the experimental group) receives the intervention while the other group (the control group) does not.

What is the time-series design?

is the time-series design perhaps best described as an extension of a one-group pretest- posttest design—the design is extended by the use of numerous pretests and posttests

Single-subject experimental designs

the single subject design seeks to (1) establish that changes in the dependent variable occur following introduction of the independent variable and (2) identify differences between study conditions

What are case studies?

Non-experimental or qualitative designs

What are the disadvantages of case studies?

First, like all non-experimental approaches, they merely describe what occurred, but they cannot tell us why it occurred and Second, they are likely to involve a great deal of experimenter bias

What is the primary advantage of naturalistic observation?

The primary advantage of the naturalistic observation approach is that it takes place in a natural setting, where the participants do not realize that they are being observed.