Psychology - observations and pilot studies

Observation:

where a researcher observes and records participants’ behaviour, but does not manipulate any variables.

observations can be used as part of a…

experiment

example of an observation being used as a part of an experiment:

I get participants to solve a puzzle in a hot or cold room, then I observe them to see how stressed they look. I’m using an observation as part of my experiment (manipulating an IV to measure the effects on a DV). If I just use an observation to gather data on how stressed people are, with no manipulation or conditions, then I am carrying out observation research, not an experiment.

different types of observation:

• naturalistic

• controlled

• overt

• covert

• participant

• non participant

Naturalistic:

This technique involves observing participants in their natural environment. It’s often used where it would be unethical to manipulate variables.

Strengths:

• High ecological validity: Since the observation occurs in a real-world setting, the behavior observed is more likely to be genuine and reflective of how people act in everyday life. This improves the generalizability of the findings.

• Useful for obtaining observations in situations where intervention would be unethical. For example, if you were investigating behaviours in domestic abuse, you would not be able to create conditions for domestic abuse! But you may observe couples outside a night club on a Saturday night.

• Unbiased behaviour: Because participants are often unaware they are being observed, they are less likely to change their behaviour due to demand characteristics.

Weaknesses:

• Lack of control: Researchers have little to no control over the environment and external variables. Factors beyond the researcher’s control may influence the observed behaviour.

• Difficulty in replicating: Because naturalistic observations occur in real-world settings that are constantly changing, it can be difficult or impossible to replicate the exact conditions of the study, limiting the reliability of findings.

• Difficult to ensure reliability of data collection. This is because the observation is in a natural setting, therefore, people could block the eyeline of the observation, noise could distract the observers etc.

Controlled Observation:

This technique involves a situation being slightly controlled by the researcher, but with no IV.Usually conducted in a laboratory type setting. For example the Milgram study. The picture below is of a study by Milgram which was a controlled observation.

Strengths:

• Data recording is likely to be reliable because there is a specific focus, and the controlled environment allows for the data to be collected in a consistent manner. Controlled observations makes it easier for other researchers to replicate the study, which strengthens the validity of the findings.

• Extraneous variables can be controlled as the observation has been set up in a controlled environment e.g. noise or obstruction of the observers can be avoided.

• By controlling the setting and stimuli, researchers can focus specifically on the behaviours of interest without distractions, allowing for detailed examination.

Weaknesses:

• Low ecological validity: Since controlled observations occur in artificial or structured environments, the setting may not reflect real-life situations. Participants might behave differently than they would in natural environments, limiting the generalizability of the findings.

• Demand characteristics: Participants may become aware that they are being observed and alter their behaviour to meet perceived expectations, leading to less accurate results.

• Not suitable for complex social phenomena: Complex behaviors that emerge in dynamic, real-world situations may be difficult to observe in a controlled setting, as the structured environment may not allow for the same interactions or influences found in natural environments.

Overt Observation:

Participants know that they are being observed. For example the TV Big Brother or any reality TV shows would be an example of an overt observation.

Strengths:

• Ethical transparency: Since participants know they are being observed, it ensures ethical practices, particularly around informed consent. This is especially important in research where privacy and consent are crucial.

• Ease of data collection: Because participants are aware of the observation, researchers can interact with them more easily. They can ask questions, clarify behaviors, or gather additional data if needed without breaching ethical guidelines.

Weaknesses:

• Increase chance of demand characteristics or observer effects, as the participant is aware of the researcher, they may change their behaviour in order to fit in with what they think the researcher wants to see.

• Lack of natural behavior: Knowing they are being watched may cause participants to behave unnaturally or in a socially desirable way, which reduces the authenticity of the data collected.

• Limited access to sensitive behaviours: Some behaviors, particularly sensitive or socially undesirable ones (e.g., aggression, dishonesty), may be suppressed when participants know they are being observed, leading to an incomplete picture of the phenomenon under study.

Covert: 

Participants do not know they are being observed.

Strengths:

• Natural behavior: Since participants do not know they are being observed, they are more likely to behave naturally and authentically, providing more valid data. This is especially useful for studying behaviors that might change under observation.

• Less demand characteristics as the participant isn’t aware of the researcher, there would be less chance of them changing their behaviour to fit in with the researchers expectations.

• Access to sensitive behaviors: Covert observation allows researchers to study behaviours that might not be expressed if participants were aware of being watched, such as socially undesirable actions (e.g., aggression, dishonesty, or substance use).

Weaknesses:

• It creates ethical issues as the participant has not consented to being observed. This makes it difficult to follow other ethics such as right to withdraw, debrief etc.

• Limited Scope for Follow-up: Since participants are unaware of the observation, it’s difficult or impossible to follow up with them for clarification or to probe deeper into their behaviors or motives, limiting the depth of analysis.

• Risk of Discovery: If participants discover that they are being observed, it can lead to distrust and may affect future interactions. This discovery can also cause a change in behavior (observer effect), undermining the purpose of the covert observation.

Participant observation:

The researcher is involved with the people they are observing.

Strengths:

• Rich, in-depth data: By participating directly in the group’s activities, the researcher gains a deeper, more nuanced understanding of the behaviours, interactions, and cultural context. This method provides detailed, qualitative data that might be missed through other observation methods.

• Builds trust and rapport: By being part of the group, the researcher can build trust and rapport with participants, leading to more open and honest communication. This can help reveal behaviours or information that participants might otherwise withhold in non-participant or more formal settings.

• Realistic setting: Unlike controlled observations, participant observation occurs in natural settings where behaviours are more likely to be genuine and less influenced by experimental conditions, improving the ecological validity of the findings.

Weaknesses:

• Harder to remain objective as the researcher may get lost in the moment and begin to interpret behaviour at a personal level (building of relationships) which may be more opinionated rather than what is actually happening.

• Difficulty maintaining a dual role: Balancing the roles of both participant and observer can be challenging. The researcher needs to engage in group activities while simultaneously collecting and analyzing data, which can be overwhelming and lead to incomplete or inaccurate data recording.

• Small sample size: Because of the intensive nature of participant observation, studies often focus on small groups or specific cases, making it difficult to generalize the findings to a larger population.

Non participant observation:

The researcher remains separate from the person they are observing

Strengths:

• Easier data recording: Researchers can focus solely on observing and recording behaviors without the distraction of participating in activities. This can lead to more comprehensive and systematic data collection.

• Access to larger groups: Non-participant observation can often be conducted with larger groups or settings, allowing researchers to gather data from a wider range of participants and increase the generalizability of their findings.

• Objectivity: The researcher maintains a degree of detachment from the participants, which can help reduce bias and enhance the objectivity of the observations. This allows for a more impartial analysis of behaviors and interactions.

Weaknesses:

• Limited depth of understanding: Since the researcher remains detached from the group, they may not fully understand the participants’ perspectives, emotions, or the meaning behind their behaviours. This can lead to superficial insights compared to participant observation.

• Potential for misinterpretation: Without active involvement, the researcher might misinterpret behaviours, actions, or social cues, especially if they lack familiarity with the group’s culture or context. This can lead to inaccurate conclusions.

• Less flexibility: Since the researcher is only observing from the outside, they may have limited ability to follow up on emerging behaviours or explore certain aspects in depth, compared to the flexibility available in participant observation.

observation sampling methods:

• event sampling

• time sampling

Event Sampling:

A data collection technique that uses a checklist of possible activities, which are tallied as they occur.

Strengths:

• Records are easy to obtain and analyse as researchers can clearly see the total number of behaviours for each event. This can make analysis extremely quick and easy, especially when looking for most or least common behaviours.

• More reliable observations as the events are already planned, therefore it could be easily replicated to measure consistency of observational behaviours.

• Observing for the entire time, so more accurate reflection of the entire event.

Weaknesses:

• Can miss important behaviours due to having set events already planned, other behaviours that were not considered are missed – reducing validity.

• Because you are observing the whole time, if many events occur at once it may lead to behaviours not being recorded – reducing validity.

• It gives no indication of the amount of time spent on each behavioural category, therefore it can sometimes lead to less valid conclusions about behaviour.

Time Sampling:

Behaviour, as specified on a predetermined checklist, is observed and recorded at specific time intervals (e.g. every 10 minutes for a period of 15 seconds)

Strengths:

• Less likely to miss behaviours as the researcher usually has a short time to focus on recording behaviour, therefore is more likely to be accurate.

• Time sampling may be easier to manage for observations where there is likely to be many behaviours occurring throughout the set observation time.

• Useful for observations where you are trying to understand how behaviours change over a longer period of time.

• You can understand how long a behaviour occurs for. In comparison to event sampling which only indicates how frequently the behaviour occurs.

Weaknesses:

• Behaviours that occur outside the time intervals are not accounted for, therefore may reduce validity as important behaviours may be missed.

• Reduced Detail: Because observations are taken at set intervals, the method may not capture the full context or nuances of a behaviour. This can lead to a superficial understanding of complex actions or interactions.

Inter-rater reliability:

Researchers observing the same behaviour, the sample people, at the same time, and coding the behaviour in the same way. If there is low inter-rater reliability it suggests:
–the behaviour categories are vague and lack clarity.
–they’re not observing the same event, or there are issues with observer bias.

does having multiple providers provide inter-rater reliability?

no, they have to agree first

whats the only way to improve inter-rater reliability?

carry out a pilot study

Behavioural categories

clearly defined behaviours are identified, which can be observed and recorded. These may be placed on a checklist and tallied every time that behaviour occurs.

Strengths of observations in general:

• See how people behave rather than how they say they behave.

• Allows us to study variables that would be unethical to manipulate e.g. behaviour in prisons.

• Observations can be useful as a pilot to generate hypothesis for future research.

Weaknesses of observations:

• Sometimes observations can be difficult to replicate, particularly those in natural settings.

• Does not provide us with thoughts or feelings, only behaviour.

• Non-experimental observations do not manipulate variables, so can’t establish cause and effect.

• Issues with observer bias and observer effect.

To increase validity of observations:

• Carry out a covert observation so participants don’t change their behaviour (observer effect).

• Double blind observations to reduce observer bias.

• Use clearly operationalised behavioural categories.

To increase reliability of observations:

• Clearly operationalised behavioural categories.

• Check inter-rater reliability by having multiple observers watching the same participants, at the same time, using the same behavioural categories.

• Train researchers to use the same behavioural categories to ensure there is a consistent understanding of the behavioural categories.

• Conduct a pilot study to check behaviour categories are clear.

what is a pilot study?

a small-scale, preliminary version of a larger research project that tests the logistics and procedures before the main study is conducted. It helps researchers identify and fix potential issues with their methodology, such as data collection, participant recruitment, and equipment, to ensure the full-scale study is more efficient and effective. By running a trial version, researchers can avoid costly errors, improve their approach, and determine the viability of their plan.