CMN 102v- Taylor- Final- lectures 12-17

At their core, surveys are:

asking people questions and looking for relationships among their answers

What is the 'super power' of survey as a research method?

flexibility

Which type of validity does survey do better than controlled experiments?

external validity

Which of the following is NOT one of the things that contributes to ensuring high-quality data for a survey?

people who take the survey each have pre-existing biases

Why is it important that all survey respondents receive exactly the same questions asked in exactly the same way?

validity

Which of the following survey methods is arguably the most expensive on a per-respondent basis?

in-person interview

True or false: One of the reasons researchers prefer in-person interviews for survey data collection in spite of its cost is that it ensures that there is no systematic bias introduced by the survey researcher(s).

false

Interview methods in general have advantages over questionnaire methods that arise largely from:

social and conversational norms

Which of the following methods tends to have the highest response rate?

in-person interview

True or false: Large, national, important, recurring surveys always use in-person interviews for data collection.

false; the Michigan Consumer Sentiment Index is a counter-example

True or false: there is no large, well-funded, professionally-administered survey research conducted through mail questionnaire.

false; the Nielsen company is a counter-example

Which of the following is NOT one of the drawbacks to mail questionnaires as a survey research method?

interviewer bias

In-person questionnaires can be advantageous when:

you need to study a small population that is geographically close together

Online questionnaires are particularly susceptible to:

people providing bad data

We talked about the General Social Survey earlier in the lecture. Same set of questions (more or less) asked of a representative sample selected anew every two years. This is an example of a(n):

longitudinal study

The Nurses' Health Study is best described as an example of a(n):

panel survey

Which of the following are important elements of writing questions for a survey?

when using terms that could possibly have more than one meaning, define those terms; don't use highly specialized terms or vocabulary; avoid questions that point to only one 'right' answer

When wording the questions, the wording has to reflect the constructs being measured. This is a question of:

validity

Let's say your hypothesis is that the physical attractiveness of a person increases the level of attention given to them by a conversation partner. You want to test this, so you're doing a survey. You will measure people's attractiveness by having a team of judges rate everyone on a scale from 1 to 10. Then, you plan to ask them, "Do you think that people pay more attention to more attractive people?" What do you think?

Pretty bad. There is no measure of the DV

If I am interested in getting responses that are relatively less biased by researcher assumptions, I might want to go with:

open-ended questions

If I want to make sure the responses to my survey questions are reliable, I probably want to use:

closed-ended questions

True or false: If you give people open-ended questions, research suggests they will give the same answer they would give on a closed-ended question, as long as that option is one of the closed-ended choices.

false

True or false: The use of numeric response options (e.g., How attractive are you on a scale from 1 to 10?) takes the subjectivity out of survey responses.

false

True or false: The response options people get can affect the responses they give, even for things that should be questions of fact.

true

In one study we discussed, people were asked how successful they were on an 11-point response scale, either -5 to +5 OR 0 TO 10. Which produced ratings of success that were higher relative to the midpoint of each scale?

-5 to +5

Someone who tends to gravitate towards choosing the highest or strongest agreement on numerical response scales have which response style?

acquiescence

Study participants who tend to think about their answers in terms of how they are related to the scale midpoint have which response style?

moderation

Acquiescence as a response style tends to be more likely to occur among which of the following groups?

folks high in collectivism; people who are comfortable with uncertainty; women (relative to men)

Understanding different response styles is likely to be most important when:

comparing responses across different cultural groups

Why does the order of questions affect the answers people give?

in survey research, normal conversational norms still apply; cognitive priming

Which of the following can survey researchers do to protect against the hazards we've discussed in these lectures?

when changing from one topic to another, make a very clear, obvious transition; measure things more than one way, even in the same study; randomize the order in which your participants see the questions on the survey; between topics or questions, make participants do something to distract them

a distraction task can prevent bias due to:

question order effects

Measuring the same concept in different ways can help reduce bias from:

response range; response style

When reading survey research as a research consumer, you should pay attention to details such as:

the order in which questions were asked; possible sources of bias; methods of measurement

As a research consumer, the best way to overcome the limitations of specific questionnaire and question design is:

to use multiple studies with different methods

The most common category of stuff advertised during children's television programming is:

high-calorie foods

The issues with studying the impact of children's food requests on parental purchases are mostly about:

external validity

Why did the researchers in the grocery store study that Dr. Taylor talked about use different grocery store chains and different neighborhoods for their observations?

external validity

Based on the totality of research Dr. Taylor discussed in this lecture, which of the following is a claim supported by scientific research?

children's food requests affect parental purchasing; advertising food products to children affects parent buying

Unobtrusive measures:

don't involve researchers interfering with participant behavior

When people collectively, over time, wear a path in the grass on the quad, that path is called:

a desire path

Which of the following is an example of analysis of trace evidence?

identifying developing cholera outbreaks through analysis of wastewater

Which of the following is/are examples of digital trace data?

the weekly app usage report from your phone; location data from your mobile phone; the browsing history from your laptop

The most important issue of data quality, specifically data validity, with trace evidence is probably:

what do the traces observed actually mean

unobtrusive data:

often look cool, real, authentic; often have unknowable or weak reliability; are not always valid

True or false: Most unobtrusive observation involves making video recordings of people in public places.

false

In the No-Tel/Uni-Tel/Multi-Tel study, trained observers made observations of:

children playing on the playground

Extensive training of coders/observers who are going to participate in an unobtrusive observation study is important to ensure:

reliability; validity; unobtrusiveness

Compared to purposively-collected survey or experimental data, existing data that is used in analysis of existing data is more likely to be:

aggregate

True or False: Although a lot of data are available for analysis of existing data, it is descriptive in nature. Analysis of existing data cannot be used to test causal hypotheses.

false

According to studies using analysis of publicly-available existing data, what is the impact of sexual content in a movie on the profitability of that movie?

movies w/ more sex are less profitable

Content analysis is a particular type of:

unobtrusive measurement

Dr. Taylor discussed some perplexing results for a poll that measured perceptions of political partisan of the news, one that suggests that survey research is an ineffective way to answer the question. This is a function of:

respondent bias

Content analysis is an approach to describing media content in a way that is:

not subject to respondent bias

true or false: Laws have been passed in a number of states that restrict textbook choices for elementary and high schools based on assumptions about messages about race in those books.

true

Content analysis applies the tools of _____________ the the study of ___________.

science, messages

Which of the following types of message can be studied with content analytic methods?

television shows, movies, Instagram images, journal entries

Which of the following hypotheses is/are likely to be more effectively tested with content analysis than other methods?

Areas in which TV shows featuring more violence are more common will have more permissive gun laws; Members of the Orange Party are more uncivil in their online communications than members of the Green Party; Movies with a higher total number of actors also contain more frequent depictions of relationship aggression

Which of the following approaches to describing or studying messages has the potential/capacity to produce reliable, valid descriptions?

content analysis

Imagine several different studies of the text "Green Eggs and Ham" by Dr. Seuss. Which of the following is reasonably likely to be the results of a content analysis rather than a critical or interpretive analysis?

the more words a character uses to address another character, the more words there are in the response; there are only 50 unique words in the entire text

True or false: representative sampling is less important for content analysis than for controlled experiments.

false

Sampling for content analysis begins by:

clearly defining the overall body of content that you're actually trying to describe

Stratified sampling is often important in content analysis because:

variation in messages is unlikely to be random

When Dr. Taylor talked about Twitch streamers and why a simple random sample might be problematic, a good solution to the problem he raised would probably be what kind of sampling?

quota sample

Which of the following describes a case of "fragmenting" in sampling for a content analysis?

researchers interested in comedy randomly select words from situation comedies and analyze which words are most likely to have the laugh track accompany them; researchers studying support-seeking messages analyze the titles of chat room posts, but not the contents of the posts themselves

The heart of validity in a content analysis lies in:

conceptual and operational definitions

Which of the following represent(s) latent content in an analysis of song lyrics?

whether the song is about a serious ex-boyfriend; whether the singer expresses affection

During coding for a content analysis, if two coders consistently disagree about how to code something, then:

they are not reliable

True or false: If a researcher just does all the coding for a content analysis him- or her- or themself (singular), that's probably fine, because they are the expert in the area they are studying.

false

What percentage agreement is necessary for two coders to be judged reliable enough for a good content analysis?

none of these (not 80%/70%/60%)

Which of the following statistical tools can account for agreement by chance?

Cohen's kappa, Scott's pi, Krippendorf's alpha

Digital trace data refers to:

traces of behavior left in digital environments

Conceptually and practically, which of the following is the nearest analogue to doing research with digital trace data?

content analysis

What type of digital trace gets used most often in communication research?

words

Which of the following is arguably the most common type of digital trace we LEAVE?

click-stream data

Which of the following is a type of computer program that takes content from web pages it is pointed to and downloads that content into a database?

web scraper

true or false: In order to use web crawling and web scraping to get digital trace data, you really need to be a reasonably proficient computer programmer.

false

An API is a tool that makes it easy for people to:

get digital trace data from a specific application or site

Google Trends is an example of:

an API

A key limitation of using an API is that the data you get:

has been curated or controlled by the API

True or False: The only kind of data you can get through data donation is things like social media behavior, where the company that holds the data is obligated by law to provide it upon request. General digital trace data, like what web sites you visited, is not available through data donation.

false

true or false: Using tracking for collection of digital trace data is unethical, violating the principle of autonomy.

false

When analyzing text digital trace data, we generally use:

automated content analysis

Many of the analysis approaches for text digital trace data is, at its core:

counting words

When researchers talk about "natural language processing," what's natural?

the language

Sentiment analysis is an AI approach for analyzing text digital trace data. (true or false)

false

Using an AI approach, if I want to use a trained classifier to determine whether each message, say each Tweet, fits into some predetermined categories, the first categorization takes place:

through human coding

When Dr. Taylor poses the question of whether online behavior is behavior, he is questioning what with regard to digital trace data?

external validity

Which of the following introduces systematic biases into digital trace data?

old people use Facebook, young people use Insta; poor people need money; people concerned about privacy aren't on social media; liberals don't use Truth Social

Do I, as a person who acts online, have access to all the digital trace data that I leave behind?

no

true or false: Digital trace data has become very popular over recent years because many social media companies are making it easier for researchers to collect such data, providing access to their APIs and their data free to academic researchers.

false

The Gartner Hype Cycle applies to:

innovative technology

Focus group research is:

empirical but not quantitative

True or False: Focus group research is used in many different professions, though not generally communication professions.

false

How many people should there typically be in a focus group?

about 9

About how long should a focus group last?

about an hour

Which of the following underlies the real 'superpower' of focus groups?

interpersonal dynamics

True or false: Because there is no survey, informed consent and IRB reviews are not required for focus group research.

false

The most important part of a focus group is:

developing valid questions

Questions in a focus group should:

be written ahead of time