Field Methods and Experimental Research Designs Study Guide
Order Effects and Counterbalancing Techniques
Order Effect: A confounding effect caused by experiencing one experimental condition prior to another, such as practice effects or fatigue.
Asymmetrical Order Effect: An order effect that exerts greater strength in one particular condition sequence than another, rendering standard counterbalancing ineffective.
Methods for Managing Order Effects:
Counterbalancing: A technique where performance improvements due to prior experience are controlled by splitting participants; for example, having half the participants perform alone first and the other half perform in front of an audience first.
Complex Counterbalancing:
ABBA Counterbalancing: Designed specifically to balance asymmetrical order effects.
Multiple Condition Design: Used when an experiment contains conditions (three levels of an independent variable). Participants are randomly assigned into distinct sequence groups (, , , , , ).
Reverse Counterbalancing: Assigning half of the participants to complete conditions in a specific sequence and the remaining half to complete conditions in the exact reverse sequence (e.g., –).
Randomization of Condition Order: Arranging a series of conditions in a completely random sequence so that subsequent conditions are unpredictable from prior ones (e.g., using a computer to generate a random sequence of numbers from to for each participant and administering conditions matching that sequence).
Randomization of Stimulus Items: Presenting participants with a single compiled list containing all stimulus items from all conditions mixed together at random (e.g., combining conditions—, , and —into a single randomized sequence like ).
Standardized Procedures: Administering measurements or testing behavior using an identical formalized routine across all participants (e.g., ensuring every participant is tested in the same room, at the same temperature, and under the same time limit).
Elapsed Time: Allowing sufficient time to elapse between experimental conditions so that learning effects or fatigue can fully dissipate (e.g., administering Drug A today and Drug B next week).
Using Alternative Designs (Vignettes): Utilizing descriptions or stories across conditions. A vignette refers to a scenario, story, or description presented to all participants where specific details are systematically altered between conditions.
Comparison of Research Designs
Comparative Analysis across Seven Key Aspects:
Participant Testing:
Repeated Measure Design: Each participant takes part in all experimental conditions.
Independent Sample Design: Each participant takes part in only one experimental condition.
Order Effects:
Repeated Measure Design: Highly vulnerable to order effects.
Independent Sample Design: Completely avoids order effects.
Participant Dropout:
Repeated Measure Design: Losing a single participant results in losing data scores across all experimental conditions.
Independent Sample Design: Losing a single participant only results in losing a single score from one experimental condition.
Demand Characteristics:
Repeated Measure Design: High risk; participants are more likely to deduce the aim of the study and alter their natural behavior.
Independent Sample Design: Lower risk; participants are less likely to guess the research aim because they only encounter a single condition.
Time & Scheduling:
Repeated Measure Design: Testing requires more time per participant due to potential fatigue or practice factors.
Independent Sample Design: Faster execution overall, as testing can occur simultaneously across different condition groups.
Participant Differences:
Repeated Measure Design: Completely eliminates individual participant differences because the exact same individuals are tested in every condition.
Independent Sample Design: Performance differences between groups may stem from individual participant differences.
Sample Size Required:
Repeated Measure Design: Requires a smaller total number of participants.
Independent Sample Design: Requires double the total number of participants to obtain the same number of scores per condition.
Matched-Pairs Design:
Matched Pairs: A design where each participant in one condition group is individually paired on specific matching variables with a participant in another group.
Example: Measuring the effect of music on student focus by matching Participant studying with music to Participant studying without music based on pre-existing focus levels.
Single-Participant and Small-N Designs:
Single-Participant Design: Testing a single participant across several experimental trials at all levels of the independent variable.
Unrelated Design: A design in which individual scores in one condition cannot be logically paired or linked to individual scores in another condition (e.g., pairing separate, distinct individuals).
Related Design: A design in which individual scores in one condition are directly paired with individual scores in another condition (e.g., testing the exact same individual before and after an intervention).
Small-N Design: A research design involving only a small number of participants, commonly utilized in clinical or counseling settings.
Factorial Design: An experimental framework where more than one independent variable is manipulated simultaneously.
Experimental, Quasi-Experimental, and Non-Experimental Designs
Experimental Research Overview: Research conducted using a formal scientific approach manipulating independent variables to observe effects on dependent variables.
Major Types of Experimental Designs:
True Experiments: Considered the gold standard of experimental research; defined by deliberate manipulation of the independent variable (IV), random assignment of participants, and the inclusion of a control group.
Quasi-Experiments:
Quasi-Experimental Research: Involves manipulating an independent variable without utilizing random assignment of participants.
Quasi-Experiment Definition: An experiment where the researcher lacks complete control over all central confounding variables.
Time Series Design: Testing single individuals or small samples repeatedly across extended time intervals.
Interrupted Time Series Design: A quasi-experimental method where a group is measured repeatedly before and after a specific event or intervention to evaluate whether the intervention interrupts an established baseline trend.
Example: A municipal government investigating if a new speed bump reduces traffic accidents on a busy road. Researchers measure accident rates before installation, introduce the speed bump as the interruption, and re-measure accident rates after months.
Research Settings and Ecological Considerations:
Naturalistic Design: Investigating participant behavior directly inside everyday real-world environments (e.g., observing and analyzing child behavior at home, at school, and during play).
Correlational Study: Examining the natural relationship between variables without experimental manipulation, typically measured outside laboratory environments.
Experimental Realism: The degree to which an experiment is engaging, attention-grabbing, and realistic to participants, counteracting laboratory artificiality or demand characteristics.
Example: Hofling's hospital nurse study (), in which hospital nurses experienced genuine psychological involvement and pressure during an unannounced operational trial.
Natural Experiment: Studies observing events outside the direct control of researchers where independent and dependent variables can still be identified.
Observational Study: A non-experimental research design focused purely on watching and recording baseline behaviors.
Field Experiment: An experimental study executed directly within a real-world, naturalistic setting.
Group Difference Study: Comparing pre-existing measurements across contrasting demographic or inherent participant groups (e.g., sex, gender, or ethnicity).
Non-Equivalent Groups: A comparative research structure where participants belong to existing groups without random assignment.
Mundane Realism: The extent to which an experimental task physical mirrors real-life daily tasks, regardless of how engaging it is.
Example: Comparing reaction time when driving an actual automobile on a street (high mundane realism) versus pressing a computer space bar when a screen flashes red (laboratory setting).
Post Facto Research: Research evaluating pre-existing, non-manipulated participant characteristics or past events to identify group differences or correlations.
Example: Comparing the academic performance of current students who attended preschool against those who did not attend preschool (where preschool attendance occurred in the past).
Core Components of Experimental Control:
Independent Variable (IV): The factor manipulated by the researcher to determine its causal effect on outcomes.
Dependent Variable (DV): The variable measured to assess outcome effects; cannot be directly manipulated by the experimenter.
Experimental / Treatment Group: The participant group receiving the active manipulation or experimental treatment.
Control Group: The participant group withheld from treatment, providing baseline measurement data for direct comparison.
Random Assignment: Assigning participants such that every individual has an equal probability of placement into either treatment or control conditions.
Single-Blind Study: A procedure where participants are unaware of their specific group assignment, while the experimenter retains full knowledge.
Double-Blind Study: A procedure where neither the participants nor the experimenters interacting with them know who belongs to the treatment or control conditions.
Observational Methods and Observational Techniques
Observational Foundations:
Observational Technique: Any procedure incorporating systematic observation, which may function independently or as part of a broader experiment.
Observational Design: Research relying exclusively on observational data collection without experimental manipulations.
Participant vs. Non-Participant Observation:
Participant Observation: Observation wherein the researcher actively joins or plays a role within the observed group.
Example: A researcher working as a barista in a coffee shop to study the internal culture and social behaviors of baristas and regular customers.
Non-Participant Observation: Observation conducted from a distance where the researcher aims to exert zero influence over the observed behaviors.
Disclosure: Formally informing participants that they are under observation.
Structured and Systematic Observation:
Structured / Systematic Observation: Utilizing explicit coding schemes and frameworks to record precise behaviors.
Example: Observing playground sharing behavior by tallying exact instances of a child sharing toys, taking turns, or actively refusing to share.
Data Gathering Instruments:
Visual recording devices
Still cameras
Audio equipment (recording verbal dialogue)
Handwritten notes, numerical ratings, or real-time checklist coding grids
Behavioral Coding Protocols:
Code (Coding): Quantifying qualitative behavior by assigning standardized symbols to specific actions (e.g., marking
"H"for hand raising,"S"for speaking out of turn, and"W"for walking around a classroom).Event Coding: Tallying pre-specified behavioral events each time they occur (e.g., placing a tally mark every single time a child takes another child's toy during a -minute recess).
Interval Coding: Recording whether specific target behaviors occur within set time blocks (e.g., dividing a -minute observation period into four -second blocks and checking off blocks in which hand-raising occurs).
Time Sampling: Conducting interval coding exclusively during non-continuous, scheduled periods (e.g., observing a classroom for minutes every hour to count mobile phone usage, rather than continuously monitoring all day).
Observational Reliability and Environments:
Inter-Observer Reliability: The degree of agreement between independent observers coding the exact same behavior.
Example: Two independent researchers watching a video recording of playground behavior and achieving matching tallies on out of aggressive acts.
Controlled Observation: Observation conducted within controlled environments such as observation suites or laboratories.
Naturalistic Observation: Observing individuals in their natural settings without environmental intervention.
Evaluator and Observational Biases and Role-Play Dynamics
Cognitive Biases in Evaluation:
Halo Effect: First conceptualized by Thorndike in ; the tendency to evaluate an individual's specific traits favorably based on an overall positive first impression or an observed central positive trait.
Example: An interviewer judging an attractive, well-dressed, friendly job applicant as highly intelligent, competent, and hardworking prior to reviewing actual qualifications.
Reverse Effect (Horn Effect): Evaluative bias where an initial negative impression leads to negative ratings across unrelated traits.
Example: A manager noticing an error in an employee's document and immediately concluding the employee is unproductive and untrustworthy.
Role-Play and Simulation Frameworks:
Role Play: Research where participants act out assigned characters or scenarios.
Example: Assigning participants to act out doctor and patient roles to study communication strategies during bad news delivery.
Simulation: Recreating realistic social interactions within constructed settings.
Example: Building a mock prison environment (as seen in the Stanford prison experiment) to observe social interactions between assigned guards and prisoners.
Active Role: Requiring active participation inside a simulated social interaction (e.g., initiating conversation to get to know a stranger).
Non-Active Role: Participants observe simulated role-plays or performances and subsequently report reactions, feelings, or evaluations.
Diary Method: Data collection where participants maintain periodic (often daily) logs of events, thoughts, or behaviors (e.g., logging daily stress ratings and coffee intake every night before sleep for weeks).
Degrees of Participant Observation:
According to Patton (), participant observation exists along a continuum (e.g., a researcher studying a local soccer club moving along a spectrum from sitting quietly in the bleachers to joining weekly team practices as an active player).
Continuum Categories:
Full Participant: Research identity is completely undisclosed; the observer acts as a genuine group member (e.g., an undercover police officer posing as a gang member to collect daily evidence).
Participant as Observer: Research role is known but kept unobtrusive (e.g., a sociologist joining a fire department, wearing the uniform and training alongside firefighters while open about gathering research notes on teamwork).
Observer as Participant: The primary role is explicitly that of an observer, accepted by the group (e.g., an investigator conducting open interviews with hospital nurses during shifts while acting purely as a researcher).
Full Observer: A completely uninvolved, non-participant observer.
Case Study Method: Intensive study of a single individual, group, or organization using multiple gathering methods. Note: Individual case studies carry an inherently high degree of unreliability.
Indirect Observation & Archival Metrics:
Archival Data: Existing public records evaluated as research evidence (e.g., analyzing official municipal marriage and divorce records).
Indirect / Archival Observation: Collecting behavioral evidence from existing documentation rather than live human observation (e.g., evaluating past government voting tally sheets to identify historical voting patterns).
Verbal Protocols and Observer Bias:
Verbal Protocol: Audio-recording participant speech when instructed to think aloud while executing tasks (e.g., recording a student speaking their thought process aloud while solving complex calculus problems).
Observer Bias: Invalidity introduced into observational data caused by observer expectations or personal characteristics (e.g., a researcher expecting male aggression rating neutral playground play as "hostile" solely because the child is male).
Interview Methods and Interpersonal Dynamics
Self-Report Foundations:
Self-Report Method: Any assessment procedure requiring participants to explicitly provide information about themselves.
Self-Report Scales: Quantitative psychological instruments designed to measure attitudes and personality traits.
Structural Dimensions of Interviews:
Structure: The degree to which interview items and operational routines are standardized across participants (e.g., closed-question surveys).
Structured Interviews: Administering identical questions in an identical order to every respondent.
Semi-Structured Interview: Utilizing an outline of pre-set topics while allowing an informal, conversational tone where the interviewer adapts to the conversational flow.
Interpersonal Variables in Interview Settings:
Gender: Potential bias or negative attitudes linked to gender exclusivity.
Ethnicity: Racial or ethnic dynamics altering communication behavior between interviewer and respondent.
Formal Roles: Exaggerated influence of gender or ethnicity when the interviewer is perceived as an authority figure.
Personal Qualities & Evaluative Biases: Interviewee discomfort stemming from interviewer traits (halo/horn effects).
Social Desirability: Tendency for interviewees to mask true views or behaviors to present socially acceptable answers.
Evaluative Cues: Unfamiliarity or discomfort experienced by respondents when asked for personal opinions in non-judgmental, non-critical settings.
Taxonomy of Interview Formats:
Non-Directive Interview: Format where the interviewee speaks freely on any topic without directional prompting, while the psychologist provides reflective, non-judgmental support.
Informal Interview: Data-gathering interview directed only to the extent of keeping the participant on topic.
Semi-Structured Interview (Informal but Guided): Flexible approach keeping procedures informal by avoiding fixed question ordering.
Structured but Open-Ended Interview: Delivering pre-set questions in a fixed sequence to every respondent while permitting open-ended verbal answers.
Fully Structured Interview: Completely standardized, pre-set questions administered in strict order.
Face-to-Face: Direct conversation conducted in the physical presence of both participant and researcher.
Clinical Method (Clinical Interview): Flexible hypothesis-testing interview utilizing structured baseline questions tailored dynamically based on prior responses.
Conducting Semi-Structured Interviews and Survey Design
Execution Guidelines for Semi-Structured / Open Interviews:
Giving Information: Relieve participant anxiety by clearly explaining research objectives at the outset.
Anonymity and Confidentiality: Reassure respondents regarding privacy to encourage candid disclosures.
Achieving and Maintaining Rapport: Display positive interviewer characteristics to put respondents at ease.
Language Alignment: Adapt to the participant's natural language style, ensuring they feel valued and validated.
Neutrality: Maintain neutral reactions to non-verbal behaviors and speech to reinforce a non-judgmental atmosphere.
Active Listening: Practice careful listening to time questions appropriately and demonstrate genuine interest.
Demonstrating Interest: Ensure participants feel their time and information are valuable.
Non-Verbal Sensitivity: Monitor non-verbal signals subtlely without creating social awkwardness.
Natural Questioning: Maintain a natural conversational flow to yield authentic answers.
Interview Guide Development and Sequence:
Only Ask What Is Needed: Formulate items directly aligned with core research aims; eliminate superfluous queries.
Ensure Answerability: Verify that participants possess the knowledge required to respond.
Ensure Truthfulness: Rephrase topics bound to strong social norms that invite dishonest responses.
Minimize Refusals: Handle sensitive items thoughtfully to prevent refusal to answer.
Logical Sequencing: Arrange questions in a logical, intuitive sequence for the respondent.
Avoid Problematic Items: Eliminate double-barreled, overly complex, ambiguous, leading, or emotive questions.
Open-Ended Questions: Items allowing detailed, unstructured verbal answers.
Closed Questions: Items restricting responses to pre-determined option sets (e.g., Yes/No choices).
Probes and Prompts:
Prompt: A pre-set request used in semi-structured interviews to elicit required details not offered spontaneously.
Manage Emotional Reactions: Avoid starting interviews with controversial or emotionally charged topics.
Helpful Feedback: Keep respondents informed regarding interview progress and upcoming topic transitions.
Interview Recording Protocols:
Note-Taking: Inefficient and disruptive to conversational flow during live sessions.
Audio Recording: Highly accurate, though visible microphones may inhibit some respondents.
Video Recording: Live cameras can dominate the environment and disrupt the informal atmosphere of open interviews.
Transcription: Verbatim written conversion of spoken recordings, preserving pauses, intonations, and non-verbal speech nuances.
Surveys, Panels, and Focus Groups:
Survey: Structured questioning administered to large samples (e.g., distributing a Google Form to university students to measure average daily sleep hours).
Survey Design: Operational framework covering sample selection, questioning modes, and item formulation.
Census: A comprehensive survey covering an entire population.
Panel: A stratified, pre-selected group consulted repeatedly over time to track opinion changes.
Focus Group: A small group meeting to discuss specific topics within a guided collective interview format (e.g., convening high school teachers for a -hour guided discussion on classroom distractions).
Interview Media: Distribution channels including face-to-face, telephone, postal mail, email, internet platforms, and SMS text messaging.
Electronic Surveys & Discussion Forums: Utilizing web tools, email, or online community forums to capture survey data and analyze natural conversations.
Survey Research Methods and Quantitative Analysis
Survey Research Principles: Structured questionnaires or statements administered to groups to measure values, beliefs, attitudes, or behavioral tendencies.
Varieties of Survey Administration:
Interviews: Direct verbal questioning (e.g., the Kinsey report, a famous historical sex survey).
Phone Surveys: Administering questionnaires via telephone calls.
Electronic Surveys: Web-based and email data collection platforms.
Written Surveys: Traditional paper-and-pencil questionnaires.
Survey Biases:
Non-Response Bias: Systemic bias occurring when individuals who decline to complete a survey differ significantly from respondents.
Social Desirability Bias: Participant tendency to answer based on perceived societal expectations rather than true beliefs.
Advantages of Online Surveys:
Accuracy: Reduced error rates via direct digital input choices.
Speed & Analytics: Rapid real-time data collection and automated analysis.
Participation Ease: High participant convenience via email links.
Branding: Opportunities to format surveys with organizational brand identity.
Flexibility & Honesty: Concise online questionnaires boost response rates and elicit honest feedback.
Templates: Access to expert-designed survey templates.
Designing Quality Surveys:
Define Objectives: Establish clear aims and distribution plans prior to survey deployment.
Manage Length: Limit question quantity and eliminate redundant items.
Simple Phrasing: Utilize easily understandable language.
Appropriate Question Types: Match formats to research utility and participant clarity.
Consistent Scales: Standardize rating scales across all survey sections.
Flawless Logic: Implement clear survey logic to avoid technical routing failures.
Survey Implementation and Advanced Quantitative Analysis:
Sampling Frameworks:
Probability Sampling: Random selection methods where every population unit has a known, equal selection chance.
Non-Probability Sampling: Non-random sample selection based on researcher discretion or availability.
Measurement Scales: Incorporating nominal, ordinal, interval, and ratio scales within multiple-choice formats.
Survey Logic (Skip Logic & Branching): Programmed conditional or unconditional branching routing respondents past irrelevant items based on prior answers.
Demographics: Standard identification metrics used to categorize respondents.
Wording Guidelines:
Avoid Leading Questions (phrasings pushing specific choices) and Double-Barreled Questions (combining two distinct issues into a single question).
Maintain simplicity, write complete sentences, eliminate abbreviations, slang, colloquialisms, and jargon, avoid negative phrasing, and present balanced choices.
Quantitative Analysis Techniques:
Cross Tabulation: Arranging raw survey data into comparative side-by-side matrices to analyze parameter relationships.
Trend Analysis: Tracking aggregated response metrics across time to plot longitudinal trends.
MaxDiff Analysis: Also termed the "best-worst" scaling method; quantifies feature preferences across multiple choices.
Conjoint Analysis: Evaluates underlying trade-offs and feature weighting driving consumer decisions.
TURF Analysis: (Total Unduplicated Reach and Frequency) Measures communication reach frequency and audience saturation.
Gap Analysis: Matrix evaluations comparing practical performance against planned targets.
SWOT Analysis: Categorizes survey outcomes into Strengths, Weaknesses, Opportunities, and Threats.
Text Analysis: Advanced statistical and natural language tools that convert open-ended, qualitative text into structured quantitative metrics.