Research Methods

Theory

A principle or an idea that explains or solves a given issue; they typically tend to address a set of problems

TEACUP (theories)

T: Is the theory testable? 

E: Evidence - A discussion of the type of evidence. Studies are a strength of a theory because they are considered as evidence to support the theory. Discuss the type of evidence that is used. Evidence may have limitations. 

A: How has the theory been applied?

C: Which constructs may be difficult to measure?

U: Is there a cultural or gender bias to the theory?

P: Is the theory more predictive or explanatory?

GRAVE (studies)

G: Generalizability 

R: Reliability 

A: Application

V: Validity (internal and ecological)

E: Ethics

High heuristic validity

When a theory has a lot of applications

Reductionism

Describing a complex phenomenon in terms of simple explanations or singular variables

Holism

An approach to understanding the mind of humans and the behavior that focuses on looking at things as a whole; this term is often used to contrast reductionism

Representative sample

A sample of research done that represents a specific population

Population validity

Type of external validity that describes how well the sample used can be generalized to a population as a whole

Target population

A specific group of people researchers are interested in conducting their study on

Opportunity sampling

Studying a group that already exists

Strength: Easy to access

Limitation: Likely to have less diversity

Random sampling

Every member of the target population has an equal chance of being selected

Strength: Easily generalized

Limitation: Difficult to perform with super large populations

Self-selected sampling

Made up of volunteers

Strength: Easy to obtain with high motivation in participants

Limitation: Rarely reflects general population, not generalizable

Snowball sampling

When participants recruit other participants to be a part of the study

Strength: Gains trust

Stratified sampling

Samples subpopulations of a population

Strength: Prevents overrepresentation

Haphazard sampling

Studying people based on the researcher's whims

Limitation: Extremely likely to be biased

WEIRD bias

W: Western 

E: Educated

I: Industrialized

R: Rich 

D: Democratic

Participant variability

The extent to which participants may share a common set of traits that can cause biases in the outcome of the study

Ethics

What is good or bad to do

Protection from undue stress or harm

Participants may not be harmed in any way: physical, fear, embarrassment, anxiety, etc.

Informed consent

The participants are debriefed on the aim and procedure of the study and are asked to give a formal consent agreement with acknowledgement of these outcomes

Deception

Lying or misleading participants in some way, making them have agreed to a study that they did not agree to; negative but there are cases when it can be justified to avoid demand characteristics

Right to withdraw

Participants have the right to leave the experiment at any time and/or to withdraw their data

Debriefing

Participants are made aware of the study’s aim, procedure, etc. if they had been deceived previously for the sake of unbiased data

Anonymized

Nobody outside of the experiment may know the names of the participants or the results from each participant

Lab study

An experiment conducted under highly controlled conditions, where accurate measurements are possible

Strengths: It is easier to replicate, they allow for precise control of extraneous and independent variables

Limitations: The artificiality of the setting may produce unnatural behavior that does not reflect real life, demand characteristics or experimenter effects may bias the results

Field study

Research in a naturalistic setting

Strength: Advantage of being observed in “real life”

Limitation: Harder to replicate with a variety of variables that are unable to be controlled

Retrospective study

The researcher is asking the participant about previous experiences (childhood memories, etc.)

Strength: The best research method to investigate a relationship or correlation between two variables

Limitations: Reliant on the participant’s memory (not always reliable), there is not enough information to support these participant recollections

Prospective study

The researcher measures a variable and watches its effect over a period of time

Strength: The researchers are not dependent on the participants' memories

Limitations: Prospective studies take a lot longer to carry out and there is always the possibility that many of the participants will drop out of the study over time

Longitudinal research

Researchers repeatedly examine the same individuals to detect any changes that might occur over a period of time

Strengths: They are effective in determining variable patterns over time, they can ensure clear focus and validity

Limitations: They require huge amounts of time, they risk gathering data that is not 100% reliable, they require a large sample size

Cross-sectional research

A researcher analyzes data of a certain population at a specific point in time

Strength: Takes less time to carry out and fewer participants decide to drop out (in comparison to longitudinal research)

Limitation: The temporal link between the outcome and the exposure cannot be determined because both are examined at the same time

Application

How a theory or empirical study is used in the real world

Validity

Whether the research/results does what it claims to do (two main types: internal and external)

Extraneous variable

A variable that has not been outlined to be a part of the independent variable that will affect the results

Construct validity

To what extent can the concept being studied be defined and measured

External validity

The generalization of results, asking “whether a causal relationship holds over variation in people, settings, treatments, and outcomes”

Ecological validity

The generalizability of the study to other settings or situations outside of the laboratory

Mundane realism

The level to which the situation represents a real life situation

Reliability

The consistancy of a research study or measuring test

Internal validity

How well an experiment is done, especially whether it avoids the influence of outside or extraneous variables on the outcome of the study

Independent variable

Characteristic of a psychology experiment that is manipulated or changed by researchers, not by other variables in the experiment

Dependent variable

A measure of a certain aspect of a participant's behaviour

Operationalized

Turning abstract conceptual ideas into measurable observations

Standardized

The process of making a test uniform or setting it to a specific standard

Experimental hypothesis

Predicts what change(s) will take place in the dependent variable when the independent variable is manipulated

Null hypothesis

There is no relationship between the two variables being studied (one variable does not affect the other)

Control condition

A condition that does not involve exposure to the treatment or intervention under study

Extraneous variables

Any variable that you're not investigating that can potentially affect the outcomes of your research study

Demand characteristics

A subtle cue that makes participants aware of what the experimenter expects to find or how participants are expected to behave

Expectancy effect

The phenomenon whereby a person or group’s expectation for the behavior of another person or group serves actually to bring about the prophesied or expected behavior

Screw you effect

Where the participant will deliberately try to have a negative effect on the researcher’s experiments

Social desirability effect

The tendency of some respondents to report an answer in a way they deem to be more socially acceptable than would be their "true" answer

Researcher bias

Any unintended errors in the research process or the interpretation of its results that are attributable to an investigator's expectancies or preconceived beliefs

Double-blind control

An experimental procedure in which neither the participant nor the experimenter are aware of which group (i.e. experimental or control) each participant belongs to

Positive correlation

A relationship between two variables in which both variables move in the same direction

Negative correlation

A relationship between two variables in which the value of one variable increases as the value of the other decreases

Bidirectional ambiguity

It is impossible to know if x causes y, y causes x, if they interact to cause behavior, or whether it is just coincidental and no relationship truly exists

Confirmation bias

The seeking or interpreting of evidence in ways that are partial to existing beliefs, expectations, or a hypothesis in hand

Reflexivity

The examination of one's own beliefs, judgments, and practices during the research process and how these may have influenced the research

True experiment

Any randomized experiment; all studies with at least 1 independent variable that is experimentally manipulated and with at least 1 dependent variable or outcome

Strengths: tighter control of variables (easier to comment on cause and effect), relatively easy to replicate, enables use of complex equipment, often cheaper and less time-consuming than other methods

Weaknesses: demand characteristics (​​participants aware of experiment, might change behavior), artificial environment (low realism), may have low ecological validity, experimenter effects/bias

Field experiment

Natural environment with independent variables manipulated by researchers

Strengths: people may behave more naturally than in a laboratory experiment (higher realism), easier to generalize results

Weaknesses: often only weak control of extraneous variables (difficult to replicate), can be time-consuming and costly

Quasi experiment

The researcher manipulates an independent variable but does not randomly assign participants to conditions; designed like a true experiment except the participants aren’t randomly assigned

Strengths: Useful when it is unethical to manipulate the IV, studies the ‘real effect’ which increases ecological validity, can be carefully controlled

Limitations: Non-uniform comparison groups can limit generalization of findings, threat to internal validity if the groups aren’t equal, potential for low validity, must wait for IV do occur, can only be used in natural experiments

Natural experiment

Natural changes in independent variable are used - it is not manipulated

Strengths: used in situations in which it would be ethically unacceptable to manipulate the independent variable, less chance of demand characteristics or experimenter bias interfering

Weaknesses: the independent variable is not controlled by the experimenter, no control over the allocation of participants to groups (random in a ‘true experiment’)

Correlational

A type of non-experimental research in which the researcher measures 2 variables and assesses the statistical relationship between them with little or no effort to control extraneous variables

Strengths: calculating the strength of a relationship between variables, useful as a pointer for further, more detailed research

Weaknesses: cannot assume cause and effect, strong correlation between variables might be misleading, lack of correlation may not mean there is no relationship (it could be non-linear)

Independent measures

Randomly allocated to one condition

Strength: Order effects are controlled, demand characteristics are less likely, less likely to guess hypothesis, same materials can be used for all conditions, can draw a conclusion more easily

Limitation: Participant variability may influence results, more participants are required

Repeated measures

One sample of participants receives each condition of an experiment; sometimes conditions are tested at the same time; can use counterbalancing: one group starts with condition A and another group starts with condition

Strengths: Individual is only compared to themselves, participant variables are controlled, fewer participants are needed

Limitation:

• Participants may demonstrate order effects if they take part in multiple conditions

  • Practice effect (if they get better as they do the experiment more)

• Participants may demonstrate demand characteristics (interpretation of experiment’s purpose and subconsciously changing behavior to fit the interpretation)

  • Expectancy effect by doing the interpretation

  • Screw you effect by trying to disprove hypothesis

• often not possible to use same materials for both conditions

Matched-pairs design

Not randomly allocated, pre-tested with regard to variables. Allocated into groups based on results of the pretest so that each condition has a range of participants

Strengths: guarantees each condition has full range of ability, less participation variability in results

Inductive approach

Research that involves the search for patterns from observation and the development of explanations/theories for those patterns through a series of hypotheses; starts by making broad generalizations from specific observation, then a theory/hypothesis is established

Deductive approach

Research method often considered to be associated with scientific investigation; the researcher studies what others have done, reads existing theories of whatever phenomenon they are studying, and then tests hypotheses that emerge from those theories; starts out with a general statement/hypothesis and examines the possibility of reaching a logical conclusion

Emic approach

Researchers start by immersing themselves in the culture to be studied; they spend time observing the culture to identify behaviors and trends, then propose a hypothesis and work with the local community to find a way to test the hypothesis

Etic approach

Attempts to test established theories cross-culturally with the goal of determining the extent to which a behavior is universal

Structured interview

Interviewer has tight control over the questions and answers; questions are close ended, pre-coded, or fixed choice; interviewer is given strict instruction to ask the questions provided to them and nothing else, they can not deviate and ask follow up questions; questions are standardized, precise, said word for word, same order and tone of voice used for all

Strengths:

• Quicker than a semi or unstructured interview, can include more participants

• Easy to quantify, analyze, snd compare results; highly standardized

Limitations:

• Can not ask clarifying questions to understand more about participants’ responses

• Low ecological validity due to structure and nature of interview

• Time consuming during planning process

Semi-structured interview

An interview with loose structure; the researcher will have prepared a list of open ended questions and they can ask follow up questions to dig deeper into responses

Strengths:

• Flexibility in types of questions and allows for follow up questions to clarify answers or dig deeper into responses

• The open-ended questions ensures all topics are covered while allowing flexibility in responses

Limitations:

• One‐on‐one approach is not ecologically valid

• Data analysis is time‐consuming

• Quality of data can depend on the skill of the interviewer

• Interviewer can give out unconscious signals that can affect participant's answer

Unstructured interview

Researcher prepares an interview guide that lists themes that should be explored during the interview; this guide helps to ensure that the same information is obtained from all the participants in the study; there is a great deal of flexibility in that the order of the questions and the actual wording of the questions are not determined in advance

Strengths:

• Researchers are not restricted, they can ask the interviewee to elaborate on his or her answers

• Enables the researcher to make interventions, asking participants either to clarify or to expand on areas of interest

• Uses an inductive approach which believes that data may emerge from an interview and provide information that was not expected.

Limitations:

• The way that questions are asked is subject to the skill and biases of the researcher, the one-to-one situation is somewhat artificial, issues with ecological validity

• Data analysis is very time-consuming as each participant that is interviewed has a different set of questions

• Not highly standardized, making a comparison of the data difficult in some cases

Focus group

A group interview that is used to study what a specific group of people think or feel about a topic; the members often have a common characteristic that is relevant for the topic of investigation; group processes can help people to explore and clarify their views in ways that would be difficult to achieve in one-to-one interviews; uses open-ended questions that encourage the participants to explore the issues of importance to them

Strengths:

• It is a quick and convenient way to collect data from several individuals simultaneously

• Provides a setting that is natural, higher ecological validity than the one-to-one interview

• The researcher plays less of a central role and the conversations happen among the participants; members of the group may help to prompt relevant ideas and opinions of others that might not come out in a one-on-one interview

Limitations:

• Not appropriate for all research questions; ethical concern that other participants in the group may not keep information confidential

• The presence of other participants may result in group dynamics such as conformity

• They can be difficult to facilitate and the data is more difficult to analyze

Overt observation

Participants know that they are being observed

Strength: Minimal ethical considerations because participants can give informed consent

Limitations:

• Behavior of participant can change due to social desirability bias (not always representative of actual behavior)

• Demand characteristics may cause expectancy effect (when participants act they way that the researcher is expecting them to react)

Covert observation

Participants are not aware that they are being studied

Strengths:

• Investigator effects are unlikely to affect results meaning that the participants behavior may be more representative/genuine

• Avoids demand characteristics meaning that participants are less likely to change behavior because they know they are being observed

Limitations:

• Participants cannot give informed consent

• Researchers have to deceive participants

Participant observation

Researcher becomes part of the group they observe (participants are usually deceived)

Strengths:

• Provides detailed knowledge of a topic that cannot be obtained through other methods

• Avoids researcher bias because researchers seek to understand how and why what they are observing is the way that it is instead of imposing their own biases to it

• Provides holistic interpretation of topic because researcher takes into account many aspects of group of people they are observing

Limitations:

• Difficult to record data promptly and objectively

• Time-consuming and demanding

• Researchers may lose objectivity during the time they spend with the participants

Non-participant observation

Researcher is not part of group they observe

Strengths:

• Faster and more efficient way to gather data

• Investigator effects and evaluation apprehension are less likely since the researcher is not visible to participants

Limitations:

• Observations may not reflect how people behave naturally (reactivity may invalidate the data)

• Researchers may overlook or miss behaviors of interest to the investigation since they are not as close to the participants

Case study

An in-depth investigation of human experience; can be on one person, families, social groups, an event, organization, etc.; design can be focused on one case or multiple cases that are to be compared; these types of studies often take place in the participants’ natural environment

Strengths:

• Provide rich data

• Uses triangulation which increases the credibility of the findings

• Gives researchers the possibility to investigate cases that could not be set up in research laboratories

• May contradict existing theories and aid in developing new ones

Limitations:

• Cannot be replicated and a single case study cannot be generalized to a wider population

• There is a potential risk for researcher bias in that the researcher’s own beliefs can influence the way the data are collected and analyzed

• Qualitative research often depends on people’s perception and memory which could be subject to distortion; participants may also change their story to sound more socially acceptable

Intrinsic case study

A type of case study often focused on resolving problems or focusing on one specific case

Instrumental case study

A case study that represents more general phenomena of interest (e.g losing a child, being homeless, etc…); the goal is to draw conclusions that may be beneficial outside of the original study

Qualitative data

Data gathered through direct interaction with participants - for example, through one-to-one or group interviews, or by observations in the field; may be analyzed with regard to existing theory or the data can generate theory

• Provide rich data - that is, in-depth descriptions of individual experiences based on concepts, meanings and explanations emerging from the data

• Useful for investigating complex and sensitive issues, such as coping with illness, human sexuality, homelessness or living in an abusing relationship

• Explain phenomena - that is, go beyond mere observation of phenomena to understand what lies behind them e.g. why do people become homeless?)

• Identify and evaluate factors that contribute to solving a problem. (e.g. What initiatives are needed to successfully resettle people who are homeless?)

• Generate new ideas and theories to explain and overcome problems.

• People are studied in their own environment, which increases validity

Limitation: data is difficult to analyze

Quantitative data

Data in the form of numbers that are easy to summarize and submit to statistical analysis; meant for generalization beyond the sample from which the data was drawn

Representational generalization

When the findings from a research study can be applied to the general population that the studies sample was drawn from

Inferential generalization (transferability)

When the findings from a study can be applied to a broader range of settings (the findings can be applied to situations and settings outside of the setting in which the study took place)

Theoretical generalization

When the theoretical concepts that were derived from the original study can be utilized to further develop the theory

Triangulation

A kind of cross-checking of information and conclusion in research; it involves the use of different perspectives, methods, researchers, and sources of data to check if the interpretation of data can be supported

Data triangulation (source triangulation)

When a researcher collects data from different sources; by using this strategy, you can confirm the findings of one set of data with another set of data

Method triangulation

Involves comparing data that come from the use of different methods (qualitative and quantitative); by using this method, the researcher determines whether it was simply the choice of research method that led to the findings

Theory triangulation

Involves looking at the data using different theoretical approaches - for example, a biological, psychological, and sociocultural approach

Survey

A data collection tool used to gather information about individuals

Strength: unlike in questionnaires, quantitative data is obtained, which means that the data can be analyzed using statistical methods.

Limitations: the validity of the Likert scale attitude measurement can be compromised due to social desirability, there are cultural differences in the approach to Likert Scales

Questionnaire

Any written set of questions

Strengths: easy to administer and can generate a lot of data; leads to rich, qualitative data.

Limitations: participants may choose the "best answer" that only partially reflects what they think