SCI 201 Chapter 3 - Data Collection (Questions)

What is the difference between random error and systematic error?

Random error occurs naturally, not caused by external forces or bias. Errors tend to cancel out when calculating means in a large sample due to nature

Systematic error occurs as a result of some outward bias (ex. convenience, voluntary, equipment error, etc.). Errors consistently occur in a definite direction and do not cancel

The sample distribution is used to estimate ___.

Population distribution

How does increasing sample size affect precision and accuracy, respectively?

Increases precision of the measurements and power of the test, because random error is lowered

May not necessarily increase the accuracy, as instrument error or other bias does not disappear upon increased sample size and measurement errors remain

What sample parameter allows you to estimate precision?

Standard deviation

High SD = imprecise (measurements spread widely)

Low SD = precise (measurements contained within small range)

What sample parameter allows you to estimate accuracy?

Sample mean (accuracy of sample mean)

Sample mean close to actual mean = accurate

Sample mean far from actual mean = inaccurate

What terms describing error are synonymous in natural and social sciences?

Precision = reliability

Accuracy = dependability

What are the ways that reliability of a study can be increased?

Clear definitions, multiple indicators/methods to measure a variable, pretesting methods through pilot studies, using standard/proven methods already demonstrated to be reliable

What aspects of study design are involved when determining statistical validity?

Appropriate/best hypothesis test based on research design, deciding whether data is parametric or non parametric, whether or not to transform the data such that a parametric test can be performed, checks that all assumptions are met.

In general, choosing the most appropriate and powerful test while ensuring all conditions of that test are met

How can we tell that power of the test has increased?

P value is smaller

Describe the advantages that can be gained from conducting a pilot study.

Identify possible extraneous variables that can be properly controlled during the experiment

Test questionnaires or interview schedules to see if they are properly designed (social sciences), test data recording techniques (natural sciences)

Determine the sample size necessary to test certain hypotheses

Determine how long an experiment may have to run

Help adjust level of treatment, etc. to be more likely to show significant results

What are the types of questions asked in interviews and questionnaires? Describe them briefly.

Demographic questions: surrounding race, location, gender, etc.

Knowledge questions: asking what people know about a certain phenomena

Attitude questions: about opinions, views, feeling about certain phenomena, events, people, or aspects of life

Behaviour questions: about personal conduct or activities such as hobbies, usually with reference to duration or time

What are the main advantages and disadvantages of mail out questionnaires?

Advantages: fast and efficient (able to get a large sample size quickly), moderate cost, no possibility of interviewer bias OR social desirability bias by the respondent, visual aids/charts may be sent out to help the respondent

Disadvantages: easy for a potential respondent to not reply (low response rates and high voluntary response bias), no possibility of asking clarifying questions, does not reach homeless people/no fixed address people

What are the main advantages and disadvantages of internet surveys?

Advantages: efficient and lowest cost, can be linked with computer software to analyze the data quickly, no possibility of interviewer bias OR social desirability bias by the respondent

Disadvantages: very high likelihood of a participant not responding, unknown sampling frame thus not possible to obtain random sample (information lacks reliability and validity)

What are the main advantages and disadvantages of face to face interviews?

Advantages: interviewer can build trust with client so that they are more comfortable giving responses (eye contact, facial expression), high response rate reduces voluntary response bias, interviewer can clarify questions or ask additional information, interviewers can note certain behaviour that can’t be seen online or remotely, clarifying aids can be shown

Disadvantages: time consuming (interviewers are actually taking out time), more expensive (resources needed), sample size likely to be smaller, not anonymous → participants may get social desirability bias, potential for interviewer bias

What are the main advantages and disadvantages of telephone interviews?

Advantages: less time consuming and expensive than in person interviews, interviewer can clarify questions or ask additional information, more anonymous than face to face interviews → less social desirability bias

Disadvantages: more time consuming and expensive than questionnaires, lower response rate than in person interviews → higher voluntary response bias, potential for interviewer bias, excludes those without phones

Explain the different circumstances in which you would use unstructured vs structured questionnaires or interview schedules.

Structured: facilitate data analysis instead of content analysis

Unstructured: explore respondent creativity, exploratory research in which possible answers may not be known

What are the principles of writing good questions in a questionnaire or interview?

• Ask questions regarding the variables in the study (on topic)

• Checked for clarity and completeness via a pilot study

• Give sufficient background information and define terms clearly

• Written concisely in complete sentences

What are the potential errors of writing questions in a questionnaire or interview?

• Avoiding jargon, abberviations, acronyms, etc. that might not be understood by the respondent

• Avoiding ambiguity, define all time periods and be specific

• Avoid bias by leading questions, emotional language, or loaded questions

• Avoiding double-barreled questions because there may be different responses for each individual question

• Avoid negative syntax questions

What interview style and structure is usually used for key informants?

Face to face, unstructured