State Mandated Research Methods Quiz

A statement of specific relationship between a study's variables that the researcher expects to observe if a theory is accurate.

What is a hypothesis?

It is testable - verifiable through observation

It is rational - follows the rules of logic

It is empirical - objective and systematic

What makes a good hypothesis?

Shows the ability to yield the same results again and again with different participants, in potentially different situations.

What is the importance of replication?

It increases "the weight of evidence" for phenomena.

Conceptual and Replication-with-Extension are useful for external validity.

What does replication increase?

Pseudo-science is "false science" based on theories put forth as scientific when they are not. (e.g. astrology, phrenology)

Real science is empirical, rational, testable, parsimonious, general, tentative, and rigorously evaluated.

What is the difference between pseudo-science and actual science? Give examples of pseudo-science.

Science is based on objective, systematic observations. It is not based on belief.

What does it mean that science is empirical?

Science follows the rules of logic and is consistent with known facts.

What does it mean that science is rational?

Science is verifiable through observation and can be disproved.

What does it mean that science is testable?

Scientific explanation provides the simplest explanation using the fewest possible assumptions.

What does it mean that science is parsimonious?

Scientific explanation applies beyond the original observations on which they are based (external validity).

What does it mean that science is general?

Science is never accepted as absolutely correct and researchers never say that they have "proven" anything, rather that the data does or does not support the theory or hypothesis.

What does it mean that science is tentative?

1: Observe a phenomenon and identify variables related to the behavior you are interested in

2: Formulate a tentative explanation (hypothesis)

3: Further Observation/Experimentation by testing the hypothesis and collecting and analyzing data, then drawing conclusions

4: Refine and retest your explanation by either refining your supported hypothesis and testing it further, OR reworking and restesting a disconfirmed hypothesis

Explain the steps in the scientific method.

In actual science, you can replicate studies with different people, and yield the same results. This shows that there is no Type 1 Error because it establishes generalizability.

Science abandons theories and ideas that are not supported by evidence.

There is no conflict of interest because they are not making money off of research.

Dissemination - where we read it - is also more trusted and credited in scientific literature and peer-reviewed journals.

Why is science better than pseudo-science?

ONLY in a true experiment. This is where the experimenter manipulates at least one variable, randomly assigns Ps to groups, and controls for extraneous variables.

When can you say "cause"?

You can describe a relationship, but you cannot identify the direct cause.

Since there is no direct manipulation, the can be no causal conclusion.

Since there is no randomization of participants, you can only make weak causal claims.

Why can we not say "cause" with correlational and quasi-experimental studies?

Correlations/Quasi-Experiments are often misinterpreted

Carefully evaluate research design by asking, "Did the researchers implement different levels of an IV? Did the researchers randomly assign participants to levels?"

How do you know if you can say "cause"?

These are variables the influence the results of the study, but they are not desired variables of the actually study. They influence the relationship between the variables actually being studied.

Ex: 3rd Variable problem

What is an extraneous variable? Provide an example.

A threat to internal validity in an experiment in which a second variable happens to vary systematically along with the IV and therefore, is an alternative explanation for the results.

Ex: When testing subjects with different versions of the test, one test may be more difficult than the other.

What is a design confound? Provide an example.

Think about confounds in advance and turn them into control variables instead.

This will help establish internal validity and make it more possible to make a causal claim.

How can researchers avoid design confounds?

A set of statistics used to organize and summarize the properties of a set of data.

What are descriptive statistics?

A value that the individual scores in a data set tend to center on.

Types: mean, median, mode

What is central tendency descriptive statistic? What are the different types?

The average of the data collected

What is the mean?

The most common score in a set of data

What is the mode?

The middlemost score, when all scores are distributed in equal halves.

What is the median?

Usually the mean, except when there are few extreme scores

Which is the best: mean, median, mode?

mean = mode = median

The data is not skewed to one extreme.

What does normal distribution mean?

Standard deviation, which is the average deviation of each score from the mean.

What is a spread/variable descriptive statistic?

A symmetrical, bell-shaped distribution having half the scores about the mean and half the scores below the mean.

Distances from the mean in a normal distribution can be measured in standard deviation units.

Describe the graph of a normal distribution?

In a factorial design, the overall effect of one IV on the DV, averaging over all levels of the other IV.

What is a main effect?

When the effect of one IV depends on the levels of another IV.

What is an interaction?

Non-parallel lines

What signals an interaction on a graph?

It measures the magnitude and direction of an association between two variables.

Ranges -1.0 - 1.0

What does the correlation coefficient (r) tell us about data?

There is a strong, positive correlation between the two variables.

What does r = 0.8 tell us?

There is a weak, negative correlation between the two variables.

What does r = -0.3 tell us?

The spread in a set of numbers (R^2).

What is variance?

R^2 = 0.01 x 100 = 1% is explained.

How much variance can be explained by the predictor, with the correlation of r = 0.1?

R^2 = 0.25 x 100 = 25% is explained.

How much variance can be explained by the predictor, with the correlation of r = 0.5?

A variable whose levels are categories (male/female).

Also called "nominal" variable.

What is a categorical variable?

A variable whose levels are measured along a scale.

What is a continuous variable?

When the independent variable is categorical and the dependent variable is categorical.

When would a researcher use a Chi-square test?

When the independent variable is categorical and the dependent variable is continuous with only 2 categories.

When would a researcher use a t-test?

When the independent variable is categorical and the dependent variable is continuous with 3 or more categories.

When would a researcher use an ANOVA?

When the independent variable is continuous and the dependent variable is categorical.

When would a researcher use logistic regression?

When the independent variable is continuous and the dependent variable is continuous.

When would a researcher use correlation or regression?

Basic categories such as gender, major, race, etc.

What is a nominal scale of measurement? Provide an example.

Categories have different names AND are organized sequentially. There is NO set interval between categories.

Ex: Ranking favorite movies from 1st to last. Order of which people finished a race.

What is an ordinal scale of measurement? Provide an example.

Categories are organized sequentially with equal distances between each, but NO absolute zero.

Ex: Temperature, intelligence

What is an interval scale of measurement? Provide an example.

Consists of equal, ordered categories, anchored by a non-arbitrary zero point.

Ex: Number of negative thoughts in a day. Speed of a car.

What is a ratio scale of measurement? Provide an example.

The results are consistent.

What does it mean that data is reliable?

Results are highly correlated with other things that it should be correlated with (bullseye).

What does it mean that data is valid?

Face - looks good

Content - has all parts of construct

Criterion - extent to which a measure is correlated with a behavior or concrete outcome that it should be related to

Construct - An indication of how well a variable was measured or manipulated

What are the different ways to assess validity?

Concurrent - correlates with some behavior now

Predictive - correlates with some behavior later

What are the two types of criterion related validities?

Convergent - correlates with other measures of this

Divergent - small/negative correlation with different construct

What are the two types of construct validity?

It is possible for a measure to be RELIABLE, but NOT VALID.

Ex: When stepping on a scale, the weight may be reliable and the same every time, but the number may not be correct.

What is the relationship between validity and reliability? Provide an example

Criterion-related and construct

What are considered to be the scientific and objective ways of assessing validity?

We need to make sure our sample is REPRESENTATIVE of the population

What is necessary of our sample, if we wish to generalize our results?

Avoid sampling bias by using matched pairs, a weight list, flipping a coin, etc.

How do we make sure that our sample is representative?

Everyone has the same chance of being chosen for the study.

What is random sampling?

Abstract - summary of entire paper

Introduction - Background and hypothesis

Method - You participants, your stimuli, how you collected data

Results - Statistics, graphs, data tables

Discussion - what your results mean, how they fit with previous work

References - works cited

What are the different sections of an APA paper, and what goes into each?

The Institutional Review Board, that receives federal funding to approve research conducted at an institution.

Members are scientist and non-scientist

Concerned primarily with ethical treatment of participants - potential costs and benefits of research

What is the IRB and who are its members? What are its main concerns?

The right of research participants to learn about a research project, know its risks and benefits, and decide whether to participate.

What is informed consent?

If participant is 17 year old or younger, must obtain consent for parent or legal guardian.

The participant is free to leave at any time with no penalty.

What is necessary to obtain informed consent?

To inform participants afterward about a study's true nature, details, and hypothesis.

What is debriefing?

Smaller sample sizes lower confidence in the results from that sample.

Conclusions drawn should be considered with this in mid, though in some cases small samples are unavoidable.

What can be problematic about a small sample size?

Prevents any bias by the subjects because they do not know if they are in the test or the control group.

Double-blind studies - the researcher doesn't even know what group the P is in, which reduces observation bias.

What is blind-testing and why is it beneficial?

This involves selecting data from experiments which supports the conclusion of the research whilst ignoring those that do not. If a research paper draws conclusions from a selection of its results, not all, it may be cherry-picking.

What does it mean to "cherry-pick" results?

Many companies employ scientists to carry out and publish research - whilst this does not necessarily invalidate research, it should be analyzed with this in mind, Research can also be misrepresented for personal or financial gain.

What is a conflict of interest?

Speculations from research are just that - speculation. Be on the look out for words such as "may", "could", "might", and others, as it is unlikely the research provides hard evidence for any conclusions they precede.

What is speculative language?

Using a sample of people who are readily available to participate.

What is convenience sampling?

When the sample contains only people who volunteer to participate, and can cause a problem for external validity.

How can there be bias through self-selection to participate in research?

Random sampling

Every member of the population of interest has an equal chance of being selected for the sample, regardless of whether they are close by, easy to contact, or motivated to respond

What is probability sampling?

Clusters of participants within a population of interest are randomly selected, and then all individuals in each selected cluster are used.

What is cluster sampling?

Two random samples are selected (a random sample of clusters, then a random sample of people within those clusters)

What is multistage sampling?

The researcher selects particular demographic categories on purpose and then randomly selects individuals within each of the categories.

What is stratified random sampling?

A variation of stratified sampling, in which the researcher intentionally overrepresent one or more groups.

When is oversampling?

Using a computer or a random number table, the researcher starts by selecting two random numbers.

What is systematic sampling?

With random sampling, researchers create a sample using some random method, so that each member of the population has an equal chance of being in the sample. This enhance EXTERNAL VALIDITY

Random assignment is used only in experimental designs, when researchers want to place participants into different groups. This enhances INTERNAL VALIDITY.

What is the difference between random sampling and random assignment?

An IQ test really is measuring IQ rather than motivation or persistence.

What is an example of a measure being valid?

A threat to internal validity that occurs when some cue leads participants to guess a study's hypothesis or goals.

What is a demand characteristic?

A situation in which it is unclear which variable in an association came first

What is the directionality problem?

Respect for Persons: Individuals participating in research should be treated as autonomous agents (informed consent) and they should be free to make up their own minds about whether they wish to participate in a research study.

Beneficence: Researchers must take precautions to protect participants from harm and to ensure their well-being. Must consider harm and benefit.

Justice: Calls for fair balance between the kinds of people who participate in research and the kinds of people who benefit from it. Researchers might first ensure that the participants involved in a study are representative of the kinds of people who would also benefits from the results.

What are the three main principles of the Belmont report, and what do each mean?