Conclusion and Evaluation

How to evaluate methods and findings?

How to evaluate the reliability of data collection method?

• Site 

  • repeat data collection over multiple sites, not just 1 or 2

• Time

  • repeat data collection across different timings within a day and a year

• Day 

  • repeat data collection over different days of the week, including weekends 

• Sampling Methods 

  • ensure a suitable method is used
    
    Probability                   Non-Probability
    1) simple random          2) convenience
    3) stratified random       4) quota

• Sampling Size 

  • at least 30

  • large enough for the data to be representative of the overall population

What are some examples of the evaluation of reliability of data collection method?

Reliable:

• Sample size is large enough to be representative of the opinions of residents in the neighbourhood → 80 residents were interviewed 

• Data collection was carried out over 5 sites, which are spread throughout the park → sufficient to be representative of the cleanliness condition @ the park

• Simple random sampling method was used to pick samples → removing the bias of a student’s own selection

Unreliable:

• Sample size of 10 respondents is too small to generalise what 1000 Deyians think about canteen improvements

• Data collection was carried out only on 1 weekday → misses out on people who visit the sports park on weekends

How to improve?

1) Increase no. of sites/ days/ timing

2) Ensure data is collected on both weekdays and weekends in order not to miss out on representative groups of people

3) Use probability sampling methods while ensuring sample size is large enough

How to evaluate how well the data supports the hypothesis?

• Data

  • does the majority of data support the hypothesis or not

• Evidence

  • does the answer support the hypothesis or not

E.g., Supports

Yes, the data supports the hypothesis because…

(state data; From fig 2.1,…)

(provide evidence)

How to evaluate the validity of findings/conclusions?

• Data

  • does data answer the research question/ hypothesis

  • provide evidence

• Method used for collection of data

  • are the survey questions/ mental maps appropriate for the investigation

  • are there limitations in the types of data used

• Reliability

  • day

  • site

  • time

  • sampling method

  • sampling size

What are some examples of evaluating the validity of findings/conclusions?

Valid:

• The questionnaire captured the inputs of both variables in the hypothesis

  • e.g., Q1 asked for the age of the respondents, while Q2 asked respondents how many times they recycle a week

• The questionnaire features >1 Qn about what respondents can do about recycling → reliable data

  • e.g., Q3 asks respondents about the locations of recycling facilities in the neighbourhood and Q4 asks them to identify the types of recyclables

Invalid:

• sample size of 20 respondents in each age group is too small → can’t generalise whether other Singaporeans in that age group recycle

• questionnaire survey was only carried out on Wednesday mornings

  • responses of working adults and students aren’t captured

What are the aspects of validity and reliability?

Validity:

• Analysis of Data - does the data answer the hypothesis/ research question + provide evidence

• Data Collection Method 

  • are the survey questions/mental maps appropriate?

  • are there limitations in the types of data used?

• Consider the 4 aspects of reliability 

Reliability:

• Sampling method

• Day

• Time

• Site