Chapter 4: Nonexperimental Methods: Descriptive Methods, Correlational Studies, Ex Post Facto Studies, Surveys and Questionnaires, Sampling and Basic Research Strategies

What are descriptive research methods?

Research methods that do not involve the manipulation of an independent variable.

Give examples of descriptive research methods.

Archival data, naturalistic observation, correlational studies, ex post facto, surveys, questionnaires, tests, and inventories.

Naturalistic Observation

Seeking answers to research questions
by observing behavior in the real world.

Reactance, Reactivity Effect, or Hawthorne effect

The finding that participants respond differently when they know they are being observed.

Time Sampling

Making observations at different time periods.

Situation Sampling

Observing the same behavior in different
situations.

Interobserver Reliability

The extent to which observers agree.
- (Number of times observers agree/Number of opportunities to agree) x 100 = Percentage of agreement
- Want 85%+

Correlational Study

Determination of the relation between two variables.

Positive Correlation

As scores on one variable increase, scores on the second variable also increase.

Negative Correlation

As scores on one variable increase, scores on the second variable decrease.

Zero Correlation

Two variables under consideration are not related.

Archival of Previously Recorded Sources of Data

Use of archival data avoids biasing participants' responses, however this approach suffers from lack of generalizability, selective deposit, and selective survival.

What is the reactance effect? How is it avoided by the use of archival sources of data?

The reactance or reactivity effect refers to the biasing or influencing of the participants' scores or responses because they know they are being observed. Archival research avoids the reactance effect because the researcher does not observe the participants and the data of interest are recorded before they are used in the research project

Selective deposit is a problem associated with

a. case studies
b. naturalistic observation
c. cause-and-effect research
d. archival research

d. archival research

You want to write a history of the psychology department at your school, so you look through old catalogs, departmental correspondence, and musty file folders you find in the basement. You are conducting

a. arcival research
b. a case study
c. an experiment
d. a particiapnt observation study

a. arcival research

The extent to which experimenters agree about a particular bit of data best describes
a. archival symmetry
b. interobserver reliability
c. concordance of naturalistic observation
d. participant-observer ratio

b. interobserver reliability

Why are time sampling and situation sampling employed?

Situation sampling and time sampling are used to provide greater generality for the research project

What is interobserver reliability? How is it calculated?

The extent to which two observers agree is called interobserver reliability. It is calculated by dividing the number of times the observers agree by the number of opportunities for agreement and multiplying by 100. These calculations result in the percent agreement

Which of the following would probability be negatively correlated?

a. numbers of days present in class and exam grades
b. height and weight of college students
c. number of difficult classes taken and semester GPA
d. overall college GPA and graduate school GPA

c. number of difficult classes taken and semester GPA

Ex Post Facto Study

A study in which the variable(s) to be studied are selected after they have occurred.

Descriptive Survey

Seeks to determine the percentage of the population that has a certain characteristic, holds a particular opinion, or engages in a particular behavior.

Analytic Survey

Seeks to determine the relevant variables and how they are related.

Pilot Testing

Preliminary, exploratory testing that is done prior to the complete research project.

Yes-No Questions

The respondent answers yes or no to the items.

Forced Alternative Questions

The respondent must select between two alternative responses.

Multiple-Choice Questions

The respondent must select the most suitable response from among several alternatives.

Likert-Type Scales

The individual answers a question by selecting a response alterna-
tive from a designated scale. A typical scale might be the following: (5) strongly agree,
(4) agree, (3) undecided, (2) disagree, or (1) strongly disagree.

Open-Ended Questions

A question is asked to which the respondent must construct his or her own answer.

Demographic Data

Information about participants' characteristics such as age, sex, income,
and academic major.

Validity

The extent to which a test or inventory
measures what it is supposed to measure.

Content Validity

The extent to which test items actually represent the type of material they are supposed to represent.

Interrater Reliability

Degree of agreement among judges concerning the content validity of the test
or inventory items.

Concurrent Validity

The degree to which the score on a test or inventory corresponds with another
measure of the designated trait.

Criterion Validity

Established by comparing the score on a test or inventory with a future score on another test or inventory.

Reliability

The extent to which a test or inventory is consistent in its evaluation of the same individuals.

Test-Retest Procedure

Determination of reliability by repeatedly administering a test to the same participants.

Split-Half Technique

Determination of reliability by dividing the test or inventory into two subtests and then comparing the scores made on the two halves.

Achievement Test

Designed to evaluate an individual's level of mastery or competence.

Aptitude Test

Designed to assess an individual's potential ability or skill in a particular job.

Personality Test or Inventory

Measures a specific aspect of the individual's motivational state, interpersonal capability, or personality.

Population

The complete set of individuals or events.

Sample

A group that is selected to represent the
population.

Random Sample

A sample in which every member of the population has an equal likelihood of
being included.

Random Sampling without Replacement

Once chosen, a score, event, or participant
cannot be returned to the population to be selected again.

Random Sampling with Replacement

Once chosen, a score, event, or participant can be returned to the population to be selected again.

Stratified Random Sampling

Random samples are drawn from specific subpopulations or strata of the general
population.

Single-Strata Approach

Gathering data from a single stratum of the population of interest.

Cross-Sectional Research

Comparison of two or more groups
during the same, rather limited, time period.

Longitudinal Research Project

Obtaining research data from the same group of participants over an extended period.

Cohort

A group of individuals born during
the same time period.

Descriptive Survey

A. Defines participants born in the same time period
B. affects the ability to predict the outcome
C. May include age, sex, and annual income
D. indicates the percentage having a certain characteristic
E. compares scores on two separate measures
F. tries to determine what the relevant variables are
G. does testing or evaluating in advance of the complete research project.

D. indicates the percentage having a certain characteristic

Analytic Survey

A. Defines participants born in the same time period
B. affects the ability to predict the outcome
C. May include age, sex, and annual income
D. indicates the percentage having a certain characteristic
E. compares scores on two separate measures
F. tries to determine what the relevant variables are
G. does testing or evaluating in advance of the complete research project.

F. tries to determine what the relevant variables are

Demographic Data

A. Defines participants born in the same time period
B. affects the ability to predict the outcome
C. May include age, sex, and annual income
D. indicates the percentage having a certain characteristic
E. compares scores on two separate measures
F. tries to determine what the relevant variables are
G. does testing or evaluating in advance of the complete research project.

C. May include age, sex, and annual income

Pilot Testing

A. Defines participants born in the same time period
B. affects the ability to predict the outcome
C. May include age, sex, and annual income
D. indicates the percentage having a certain characteristic
E. compares scores on two separate measures
F. tries to determine what the relevant variables are
G. does testing or evaluating in advance of the complete research project.

G. does testing or evaluating in advance of the complete research project.

Cohort

A. defines participants born in the same time period
B. affects the ability to predict the outcome
C. May include age, sex, and annual income
D. indicates the percentage having a certain characteristic
E. compares scores on two separate measures
F. tries to determine what the relevant variables are
G. does testing or evaluating in advance of the complete research project.

A. defines participants born in the same time period

Concurrent Validity

A. defines participants born in the same time period
B. affects the ability to predict the outcome
C. May include age, sex, and annual income
D. indicates the percentage having a certain characteristic
E. compares scores on two separate measures
F. tries to determine what the relevant variables are
G. does testing or evaluating in advance of the complete research project.

E. compares scores on two separate measures

Criterion Validity

A. defines participants born in the same time period
B. affects the ability to predict the outcome
C. May include age, sex, and annual income
D. indicates the percentage having a certain characteristic
E. compares scores on two separate measures
F. tries to determine what the relevant variables are
G. does testing or evaluating in advance of the complete research project.

B. affects ability to predict the outcome

Describe the steps involved in creating a good survey.

The steps involved in developing a good survey include (a) determining how the information you seek is to be obtained and what type of instrument will you use, (b) determining the nature of the questions that will be used, (c) writing the items for your survey, (d) pilot testing your survey or questionnaire, (e) considering what demographic data are desired, and (f) specifying the procedures that will be followed in administering the survey or questionnaire.

"Working with IVs that the experimenter does not manipulate" best describes

a. a case study
b. naturalistic observation
c. participant observation
d. an ex post facto study

d. an ex post facto study

How can the low return rate of mail surveys be improved?

The low return rate of mail surveys can be increased by (a) including a letter that clearly summarizes the nature and importance with the initial mailing (a self-addressed, pre-paid return envelope should be included with the initial mailing) and (b) sending additional mailings at two- to three-week intervals.

Why is the use of personal interviews declining?

The use of personal interviews is declining because (a) the expense involved in conducting them has increased greatly, (b) the fact that an individual administers the surveys increases the possibility for bias, and (c) the appeal of going from door to door is decreasing because of unavailability of respondents and high crime rates in large metropolitan areas.

Distinguish between achievement and aptitude tests.

Achievement tests are used to assess an individual's level of mastery or competence. Aptitude tests are used to assess an individual's skills or abilities.

_____ are used to assess a specific attribute or ability.
a. Surveys
b. Questionnaires
c. Pilot studies
d. Inventories

d. Inventories

The general group of interest is the ____. The group that is selected to represent the general group is a ____.

population, sample

A test can be ____ and not be _____.
a. valid; accurate
b. reliable; valid
c. valid; reliable
d. split; halved

b. reliable; valid

What is random sampling? With replacement? Without replacement?

Random sampling occurs when every member of the population has an equal chance of being selected. Random sampling with replacement occurs when the chosen items are returned to the population and can be selected on future occasions. When random sampling without replacement is used, the chosen item is not returned to the population.

What is stratified random sampling, and why is it used?

Stratified random sampling involves randomly selecting participants from a single layer or strata in the population (e.g., a limited age range). When stratified random sampling is used, group homogeneity is increased. The more homogeneous the sample, the fewer chances there are for nuisance variables to operate. The fewer chances there are for nuisance variables to operate, the smaller the within-group variability.

Distinguish among the single-strata, cross-sectional, and longitudinal approaches to research.

The single-strata research approach attempts to secure research data from a single, specified segment of the population of interest. The cross-sectional research approach involves the comparison of two or more groups of participants during the same rather limited time span. The longitudinal research approach involves gathering information from a group of participants over an extended period of time.

Qualitative

Descriptive records, often narrative.

Quantitative

Numerical measurements and analysis.

Advantages and Disadvantages of Mail Surveys

Advantages: Reach a large number of participants without the researcher being present

Disadvantages: Uncertainty about who completes the survey. The potential for respondents to answer questions out of order. Low return rates can lead to sample bias. The typical response rates range from 25% to 30%; rates of 50% or higher are considered good.

Advantages and Disadvantages of Personal Interviews

Advantages: High response rate, interviewers can clarify ambiguous questions

Disadvantages: Time consuming, costly, susceptible to interviewer bias, and safety concerns

Advantages and Disadvantages of Telephone Interviews

Advantages: Address many issues associated with personal interviews and mail surveys. Can use random-digit dialing for random sampling

Disadvantages: Screening and blocking technology, lower response rates, inability to use visual aids and lack of nonverbal cues make it harder to clarify questions

Advantages and Disadvantage of Archival Research

Advantages: Avoids biasing participants' responses

Disadvantages: Lack of generalizability, selective deposit, and selective survival