Chapter 4: Alternatives to Experimentation

Survey research

• obtains data about opinions, attitudes, preferences, and behaviors using questionnaires or interviews.

• allow us to gather data about experiences, feelings, thoughts, and motives that are hard to observe directly.

• can be particularly useful for collecting data on sensitive topics because they can be given anonymously, making it more likely that subjects will answer truthfully.

• can be useful for making inferences about behavior, although they do not allow us to test hypotheses about causal relationships directly.

• allow us to gather large amounts of data efficiently.

• Low manipulation/low/high imposition

Written questionnaires and face-to-face interviews

Two most common survey techniques in psychology research

Identify the research objectives

1st step in constructing surveys

Design the survey items

2nd step in constructing surveys

Close-ended questions

• can be answered using a limited number of alternatives and have a high imposition of units.

• structured units

• High imposition

Open-ended questions

solicit information about opinions and feelings by asking the question in such a way the person must respond with more than a yes, no, or 1 – 10 rating.

Content analysis

• To quantify answers to open questions, however, a system must be designed to evaluate and categorize the content of each answer. This process, called

• is similar to coding behaviors using systematic observational techniques.

Double-barreled or compound questions

Avoid ____, meaning they ask for responses about two (or more) different ideas in the same question.

Exhaustive response choices

Use ____ , meaning they need to contain all possible options.

Nominal scale

differentiates between items or subjects based on categories. It is the simplest level of level of measurement.

Ordinal scale

allows ranking of objects or observations; data consisting of spectrum of values.

Interval

• measures magnitude or quantitative size using measures with equal intervals between the values.

• has no true zero point.

Ratio scale

• has equal intervals between all values and true zero point, which means that the values can be compared with each other with zero as a reference point.

• The highest level of measurement.

Nature of variable and measurement precision

The best type of scale to use will depend on two things:

Continuous dimension

the concept that traits, attitudes, and preferences can be viewed as a ____ and each individual could fall at any point along each dimension.

Interval

Most used level of measurement in psychological research

Semantic Differential Scale

a survey method that measures attitudes by asking respondents to rate a concept or object on a scale anchored by opposing adjectives (e.g., "good" vs. "bad"). 

Likert scale

• a rating scale that asks respondents to indicate their level of agreement or disagreement with a statement.

• It's a common tool in surveys and social science research. 

Graphic Rating Scale

a performance appraisal method that uses a visual scale (like a numbered or descriptive continuum) to evaluate employee performance against specific criteria, providing a simple and quantifiable way to assess various aspects of work. 

Non-verbal scale

refers to a method of assessing or measuring something without relying on spoken language,

Relevant to the central role, easy to answer, interesting, answerable by most respondents, close format

According to Czaja and Blair (1996) the first survey question should have the following characteristics:

Value laden

In addition, make sure your questions are not ____ – not worded to make them positive or negative.

Response styles

are tendencies to respond to questions or test items in specific ways, regardless of the content.

Willingness to answer

comes into play whenever questions require specific knowledge about facts or issues. When unsure, some people will leave questions blank; others will take a guess. often a problem in survey research.

Position preference

when in doubt about answers to multiple-choice questions, some people always select a response in a certain position such as C.

Manifest content

the plain meaning of the words or questions that actually appear on the page.

Yea-sayers (response acquiescence)

tend to agree with the questions.

Nay-sayers (response deviation)

tend to disagree with the questions.

Context effects

• sometimes the position of a question – where it falls within the question order can influence how the question is interpreted.

• 7 blanks

Framing

refers to changing the context of question to become acceptable

Reverse scoring

to reduce yea-sayers and nay-sayers response style

Buffer items

• to reduce context effects

• questions unrelated to either of the related questions.

Self-administered questionnaire

is a survey designed for respondents to complete independently, without researcher or interviewer assistance

Mail surveys

• polite and professional cover letter (and include a stamped, self-addressed envelope).

• questionnaire and return procedures protect subjects anonymity.

• Typical response rates - 45% and 75%

Computer and Internet Survey

• Considering adding your survey to the many Web-based.

• The quality of data obtained using these methods has not yet been thoroughly evaluated, and for now we consider these methods to be extensions of the written versions of questionnaires.

Telephone Surveys

The response rates for ____ usually fall between 60% and 90% when surveys takers keep trying to reach someone at a randomly

Interview

• One of the best ways to gather high-quality survey data

• in terms of time and money, this is the most expensive method for collecting survey data,

• will need to spend time winning the subject’s confidence and establishing trust.

Structured interview

• a standardized assessment method where all candidates are asked the same questions in the same order, and responses are evaluated using a predefined rating system, aiming for fairness and objectivity

• limited responses

• deviations are not permitted

• more usable quantifiable data

Semi-structured interviews

• strikes a balance between having a set of predetermined questions and allowing for flexibility and open-ended exploration. 

Unstructured interview

• is a conversational approach where questions are not pre-determined, allowing for a more flexible and exploratory discussion

• free to explore interesting issues

• information may not be usable for statistical analysis

Diagnostic interview

a structured conversation between a health professional and a patient to explore the patient's current situation, presenting problem, and background, ultimately aiming to formulate a diagnosis, prognosis, and develop a treatment plan

Job/Personnel Interview

a formal meeting where a hiring manager or HR representative assesses a candidate's suitability for a position by asking questions about their skills, experience, and personality. 

In-depth interview

a qualitative research method that involves a one-on-one, open-ended conversation with a participant to gather detailed information about their experiences, perspectives, and feelings on a specific topic. 

Survey interviews

a formal method of gathering data by questioning individuals to assess opinions or conditions. It involves collecting, analyzing, and interpreting the views of people from a specific target population through interviews, questionnaires, or focus groups.

Females

What gender is more successful in conducting interviews?

Focus Groups

Usually a small group of people with similar characteristics who are brought together by an interviewer, called a facilitator, who guides the group in a discussion of specific issues.

Reliability

• is the extent to which the survey is consistent and repeatable.

• responses should be consistent

• very similar responses

Validity

• is the extent to which a survey actually measures the intended topic.

• can predict actual behavior

• give the same results to the topic

• can fairly capture all aspects of the topic

Reliable

The test should be ___ always. It should be higher.

Sampling

deciding who the subjects will be and then selecting them.

Population

consists of all people, animals, or objects that have at least one characteristic in common

Sample

is a group that is subset of the population interest.

Representativeness

how closely the sample mirrors the larger population.

Probability sampling

is a sampling technique in which all members of the population have an equal opportunity of being selected as participants in the study.

Random sampling

meaning that any member of the population has an equal opportunity to be selected.

Systematic random sampling

a variation of random sampling in which a researcher selects every nth person from the population.

Stratified random sampling

randomly sampling from people in each subgroup in the same proportions s they exist in the population.

Cluster sampling

a form of probability sampling in which a researcher samples entire clusters, or naturally occurring groups that exist within the population.

Nonprobability sampling

is a sampling technique in which not all members of the population have an equal chance of participating in the study.

Quota sampling

researchers select samples through predetermined quotas that intended to reflect the makeup of the population.

Convenience sampling

is obtained by using any groups who happen to be available.

Purposive sampling

the selection of nonrandom samples that reflect a specific purpose of the study.

Snowball sampling

another form of nonprobability sampling, a researcher locates one or a few people who fit the sample criterion and asks these people to locate or lead them to additional individuals.