Psyc3302 Week 2 L.2 - Item Development Considerations

What are closed-ended questions?

Questions that ask respondents to choose from a fixed set of response alternatives, such as yes/no, multiple choice, or numerical rating scales.

What are open-ended questions?

Questions that do not provide response alternatives, allowing respondents to answer in their own words.

What is the most widely used response format in surveys?

The Likert scale.

What are the characteristics of Likert scales?

Use words to anchor numerical ratings, range from 2 to 11 points, most effective with 4–7 points.

What is an advantage of a 6-point Likert scale over a 5-point scale?

Provides more response variability, better for statistical analysis.

What is the debate around mid-points in Likert scales?

Some argue 'Neither Agree Nor Disagree' may let respondents avoid giving a true opinion.

What did Simms et al. (2019) find about response scale lengths?

6 to 7-point response scales produced the best results.

What is a potential issue with 5-point scales?

They are often used like 3-point scales because respondents avoid extremes.

Why are verbal anchors problematic?

They can be ambiguous (e.g., what does 'sometimes' mean?), reducing comparability.

What are advantages of closed-ended questions?

Fast and easy for respondents, easy to analyze statistically, help keep responses on-topic.

What is a specific memory benefit of closed-ended questions?

They can remind people of events they may have forgotten.

What are disadvantages of closed-ended questions?

May not reflect true feelings, can force opinions, may frustrate respondents.

What are advantages of open-ended questions?

Allow precise expression, richer data, help with sensitive topics, and provide qualitative insights.

How can open-ended questions support qualitative reporting?

They offer quotes and deeper insights managers often appreciate.

What are disadvantages of open-ended questions?

Time-consuming to complete and analyze, not quantitative, quality depends on respondent effort.

What is an alternative to fully open-ended questions?

Asking respondents to list a fixed number of reasons, e.g., top 3 reasons.

What is a common misuse of open-ended responses?

Respondents may vent rather than answer the question.

What are negatively keyed items?

Items where agreement implies denial of a trait, used to reduce response bias.

What are double-barrelled items?

Items that ask two things at once, leading to ambiguity in responses.

What is an example of a double-barrelled item?

I found the training program pleasant and useful.

Why should simple language be used in items?

To ensure clarity, ideally at or below an eighth-grade reading level.

Why should items be short?

Long items risk losing attention; more than 15 words or multiple commas often indicate problems.

What’s a general rule for identifying long or complex items?

Items with more than one comma are likely too complex.

Why do people sometimes ignore parts of a response scale?

They may cluster responses around the middle, effectively reducing scale granularity.

What is the main purpose of item clarity and simplicity?

To ensure reliable and valid measurement across diverse respondents.