Observations and pilot studies

what is observation

where a researcher observes and records participants’ behaviour, but does not manipulate any variables.

Different types of observation

Naturalistic, controlled, overt, covert, participant and non participants

Naturalistic observation definition + weakness and strength

This technique involves observing participants in their natural environment. It’s often used where it would be unethical to manipulate variables.

• High ecological validity: Since the observation occurs in a real-world setting, the behavior observed is more likely to be genuine and reflective of how people act in everyday life. This improves the generalizability of the findings.

• Lack of control: Researchers have little to no control over the environment and external variables. Factors beyond the researcher’s control may influence the observed behaviour.

Controlled observation

This technique involves a situation being slightly controlled by the researcher, but with no IV. Usually conducted in a laboratory type setting.

• Extraneous variables can be controlled as the observation has been set up in a controlled environment e.g. noise or obstruction of the observers can be avoided.

• Low ecological validity: Since controlled observations occur in artificial or structured environments, the setting may not reflect real-life situations. Participants might behave differently than they would in natural environments, limiting the generalizability of the findings.

• Demand characteristics: Participants may become aware that they are being observed and alter their behaviour to meet perceived expectations, leading to less accurate results.

Overt observation

Participants know that they are being observed. For example the TV Big Brother or any reality TV shows would be an example of an overt observation eg. Love Island

• Ethical transparency: Since participants know they are being observed, it ensures ethical practices, particularly around informed consent. This is especially important in research where privacy and consent are crucial.

• Ease of data collection: Because participants are aware of the observation, researchers can interact with them more easily. They can ask questions, clarify behaviours, or gather additional data if needed without breaching ethical guidelines.

• Increase chance of demand characteristics or observer effects (trying to fit in with what they think the researcher wants to see)

• Lack of natural behaviour

Covert observation

Participants do not know they are being observed.

• They are more likely to behave naturally and authentically, providing more valid data.

• Access to sensitive behaviors: Covert observation allows researchers to study behaviours that might not be expressed if participants were aware of being watched, such as socially undesirable actions (e.g., aggression, dishonesty, or substance use).

• Less demand characteristics

• It creates ethical issues as the participant has not consented to being observed. This makes it difficult to follow other ethics such as right to withdraw, debrief etc.

Participant observation

The researcher is involved with the people they are observing.

• the researcher can collect more detailed in depth data that may be missed by any other observation method as they are up close and interacting with the participants.

• realistic setting → improves ecological validity of findings

• Difficulty maintaining a dual role: Balancing the roles of both participant and observer can be challenging. The researcher needs to engage in group activities while simultaneously collecting and analyzing data, which can be overwhelming and lead to incomplete or inaccurate data recording.

Non participant observation

The researcher remains separate from the person they are observing.

• Easier data recording: Researchers can focus solely on observing and recording behaviors without the distraction of participating in activities. This can lead to more comprehensive and systematic data collection.

• Objectivity: The researcher maintains a degree of detachment from the participants, which can help reduce bias and enhance the objectivity of the observations. This allows for a more impartial analysis of behaviors and interactions.

• Potential for misinterpretation of eg. behaviours

Observation sampling methods

event sampling and time sampling

Event sampling

A data collection technique that uses a checklist of possible activities, which are tallied as they occur.

• easy to record and analyse

• Observing for the entire time, so more accurate reflection of the entire event.

• Can miss important behaviours due to having set events already planned, other behaviours that were not considered are missed – reducing validity.

• Because you are observing the whole time, if many events occur at once it may lead to behaviours not being recorded – reducing validity.

• It gives no indication of the amount of time spent on each behavioural category, therefore it can sometimes lead to less valid conclusions about behaviour.

Time sampling

Behaviour, as specified on a predetermined checklist, is observed and recorded at specific time intervals (e.g. every 10 minutes for a period of 15 seconds).

• Behaviours that occur outside the time intervals are not accounted for, therefore may reduce validity as important behaviours may be missed.

• Less likely to miss behaviours as the researcher usually has a short time to focus on recording behaviour, therefore is more likely to be accurate.

Inter-rater reliability

Researchers observing the same behaviour, the sample people, at the same time, and coding the behaviour in the same way. If there is low inter-rater reliability it suggests:
–the behaviour categories are vague and lack clarity.
–they’re not observing the same event, or there are issues with observer bias.

THE ONLY WAY TO IMPROVE INTER-RATER RELIABILITY IS A PILOT STUDY

Behavioural categories

clearly defined behaviours are identified, which can be observed and recorded. These may be placed on a checklist and tallied every time that behaviour occurs.

Increasing validity of observations

• Carry out a covert observation so participants don’t change their behaviour (observer effect).

• Double blind observations to reduce observer bias.

• Use clearly operationalised behavioural categories.

Increasing reliability of observations

• Clearly operationalised behavioural categories.

• Check inter-rater reliability by having multiple observers watching the same participants, at the same time, using the same behavioural categories.

• Train researchers to use the same behavioural categories to ensure there is a consistent understanding of the behavioural categories.

• Conduct a pilot study to check behaviour categories are clear.