key science skills: analysing data

true value

True Value: the value or range of values you obtain if your measurement is perfect without any error

But there are always all types of errors! True value is something we believe to exist and aspire to, but it is also something that we are unlikely to ever know for sure. 

accuracy

Accuracy is how closely a measurement is to the ‘true’ value of the quantity being measured

True value = The value, or range of values that would be found if the quantity could be measured perfectly.

It is described in qualitative terms, such as being more accurate or less accurate

For example, if a student’s true value of their height is 172 cm, but they measure their own height as 176 cm by using a damaged ruler, then their value has poor accuracy.

Precision

Precision is how closely a set of measurement values agree with each other. Unlike accuracy, precision does not indicate how close the measurements are to the true value.

If a study was replicated and similar results were obtained this indicates that the results are precise and that our method is reliable.

For example, a fridge thermometer is checked every day for a week, and obtains the following results: 3.1, 3.2, 3.1, 3.1, 3.2, 3.2, 3.1

These results can be considered to be precise as the values are close together and quite consistent. 

Repeatability

Repeatability: the ability to obtain the same data values again under the same experimental conditions by the same observer.

• A good measure for precision

• NOT good measure for accuracy, NOT

Reproducibility

Reproducibility: the ability to obtain the same data values again under slightly different experimental conditions, such as with a different measuring instrument, in a significantly different experimental environment, or with different experimenters.

• Requires clear experimental methods and well-defined variables

• A good measure for accuracy, as well as precision

validity

In general, validity denotes how well a measurement measures what it claims to be measuring.

• If I want to measure the fitness of a group of individuals, but I only collect data about their height and weight to calculate each individual’s BMI, do you think that is a valid measurement?

• Do you think VCE exam is a valid measurement of students’ learning?

Similarly, we use qualitative terms, low/medium/high to describe the validity of a measurement.

internal validity

Internal Validity: an assessment of how well an investigation actually measures what it was designed to measure. That is, was it the IV that caused the change in the DV, or was it something else?

Factors that can influence internal validity: 

• confounding and extraneous variable

• research design

• sampling and allocation methods used

A lack of internal validity suggests that the results are inaccurate, and that no conclusion can be made. 

If a study has low internal validity, then external validity is irrelevant.

external validity

External Validity: an assessment of the generalizability of your findings to a greater population.

• Generalizability: how much of your findings obtained from the experiment participants still hold true on other target populations?

• e.g, Replication crisis & W.E.I.R.D.: Western, Educated, Industrialized, Rich, Democratic, e.g., the Müller-Lyer illusion

random errors

Random errors are unpredictable errors that affect the precision of a measurement.

how can random errors occur

• They can occur as a result of: 

  • environmental factors

  • imprecise or unreliable instruments

  • variations in procedure

  • Example: During a reaction time test, a participant is momentarily distracted by a sudden noise
    

how can random errors be eliminated

• When you have a random error, if you measure the same thing multiple times, the scores will cluster around the true value. However, the average of these scores is quite close to the true value.
  
  

• Random errors cannot be eliminated from an experiment.

• Random errors may be reduced by:

  • repeating and conducting more measurements

  • calibrating measurement tools correctly

  • refining measurement procedures

  • controlling any other extraneous variables

  • increasing the sample size of participants

systematic errors

Systematic errors result in a consistent or proportional difference between the observed value and true value each time a measurement is taken.

reduce accuracy of a measurement

sources of systematic errors

• They can occur as a result of:

  • observation errors

  • environmental interference

  • inaccurate instrument calibration eg, using an inaccurate ruler

how can systematic errors be eliminated

• They can be reduced by being familiar with the instruments being used and using them correctly.
  
  

• Repetition of measurements will NOT reduce systematic errors.

uncertainty

Uncertainty refers to the inherent (inbuilt) lack of knowledge of what the true value is.  I.e how on earth do we accurately measure the exact amount of stress someone is feeling?? 

For example, a study may aim to test ‘positive mood’ and use a range of measures that attempt to assess it. However, given the blurred boundaries of what ‘positive mood’ truly is, researchers would still have some uncertainty in its assessment. The uncertainty of measurement reflects the lack of exact knowledge regarding the true value and less quantifiable nature of what is being measured.

sources of uncertainty

Sources of uncertainty:

• Measured construct not clearly defined, e.g., how to measure a subjective feeling? Does the same data value represent the same actual value?

• No standardized tools, e.g., three thermometers give three different readings, how do we make sure what the room temperature is?

• No standardized procedure, e.g., a mixture of self-report and observation

• Incomplete data, e.g., only collecting 50% of intended responses, does the data still representative for our sample?

• Contradictory data, e.g., suggesting possible confounding factors.

  When evaluating personally sourced or provided data, students should be able to identify contradictory (incorrect data) and incomplete data (missing data – questions without answers or variables without observations), including possible sources of bias

outliers

data point not like the other data points

do we omit outliers?

no you must consider if they are genuine values and reasons for outliers

what method do you use to measure and ensure there’s no outliers

the median, the middle value.

causes of outliers

• An accident during the experiment or measurement

• Data entry errors

• Naughty participants

• Some unaccounted underlying mechanism

mitigation of outliers

• Multiple measurements to rule out random or personal errors

• Search for confounding variables or previously ignored mechanism if the same outlier repeatedly occurs