lesson 2 : research design & data collection methods [fmpsy]

Qualitative Variables

• Categorize data based on qualities or characteristics, such as gender, religion, or type of error.

• They describe differences in kind.

• Descriptive, relating to words and language.

• Describes certain attributes, and helps us to understand the “why” or “how” behind certain behaviors.

• Dynamic and subjective, open to interpretation.

• Gathered through observations and interviews.

• Analyzed by grouping the data into meaningful themes or categories.

Quantitative Variables

• Measure data based on numerical values, such as height, weight, or number of errors.

• They describe differences in amount.

• Countable or measurable, relating to numbers.

• Tell us how many, how much, or how often.

• Fixed and universal, “factual.”

• Gathered by measuring and counting things.

• Analyzed using statistical analysis

Discrete Variables

• are those that can only take on specific, whole number values.

• there are no values between these whole numbers.

  • examples include the number of children in a family or the number of errors on a task.

Continuous Variables

• can take on any value within a range, including decimals.

• there are infinitely many possible values between any two given points.

  • examples include time and blood alcohol level.

Independent Variables (IV)

• are believed to cause changes in other variables.

• in research, they are manipulated to observe their effects.

• also called: Exposure Variable, Control Variable, Explanatory Variable, Manipulated Variable

Dependent Variables

• are the outcomes or results that are influenced by Independent Variables.

• researchers measure these to determine if the Independent Variable had an effect.

• also called: Outcome Variable, Controlled Variable, Explained Variable, Response Variable

Situational Variables

Characteristics or factors within the environment that can influence behavior or outcomes (physical & social environment, task-related factors)

Subject Variables

Personal characteristics or attributes of individuals that can influence behavior or outcomes (demographic info, personality traits, abilities)

Constructs

• These are abstract ideas or concepts that represent underlying mental or physical processes.

• These are not directly observable but are inferred from measurable behaviors or outcomes.

• In essence, they are the theoretical building blocks of research.

Hypothetical

A key characteristic of construct whereby they exist as theoretical concepts rather than tangible entities.

Inferred

A key characteristic of construct whereby their presence is deduced from observable data

Complex

A key characteristic of construct whereby they are often multi-dimensional and encompass various related concepts.

Moderator Variables

• Explains the process through which two variables are related.

• Acts as a Middleman, carrying the effect of the Independent Variable.

• Answer the question: How does one variable influence another?

Conceptual Definition

• A general, abstract description of a variable or concept.

• Purpose: to convey the underlying meaning and theoretical basis of the concept.

• Example: Intelligence is the ability to acquire and apply knowledge and skills.

Operational Definition

• A specific, concrete description of how a variable will be measured or manipulated in a study.

• Purpose: to translate the abstract concept into measurable terms.

• Example: Intelligence is measured by the score on the Wechsler Adult Intelligence Scale (WAIS).

Measurement

The systematic process of assigning numerical or categorical values to represent the attributes of something.

Nominal Scale

Different scale values only represent different qualities.

Ordinal Scale

Scale values represent quantitative ordering.

Interval scale

Equal scale intervals represent equal quantitative differences

Ratio Scale

Equal scale intervals represent equal quantitative differences, and there is a true zero point.

Accuracy

means true to intention and is also the ability to hit a target.

Precision

means true to itself and is also the ability to achieve the same results over and over

Test-retest Reliability

• Measures the consistency of results over time.

• Method: Administer the same test to the same group of participants on two different occasions and correlate the scores.

• Example: A personality test administered today and again in two weeks should yield similar results.

Inter-rater Reliability

• Measures the consistency of ratings between different observers or raters.

• Method: Multiple raters independently assess the same data or subjects and compare their ratings.

• Example: Multiple judges scoring a gymnastics competition should agree on the scores.

Parallel Forms Reliability

• Measures the consistency between two equivalent forms of the same test.

• Method: Create two different versions of a test that measure the same construct and administer both versions to the same group of participants.

• Example: Two different IQ tests administered to the same group should yield similar results.

Internal Consistency Reliability

• Measures the consistency of items within a single test or scale.

• Method: Assess how closely related items are within a test.

• Example: A questionnaire measuring depression should have items that are highly correlated with each other.

Face validity

Tool measures content according to the lay-person

Content Validity

Tool measures content according to an expert based on theory

Criterion Validity

• The criteria of the tool match other tools

• Focus: How well the measure correlates with other measures (criteria) of the same construct or a related construct.

Predictive (criterion) Validity

• Does the measure predict future outcomes related to the construct?

• Tool predicts significant differences among different groups

Concurrent (criterion) Validity

• Does the measure correlate with other measures of the same construct taken at the same time?

• Tool gives similar scores as other tools on the same subjects

Convergent (criterion) Validity

• Does the measure correlate with other measures of the same or similar constructs?

• Tool gives similar scores as other tools on the same subjects

Discriminant (criterion) Validity

• Does the measure not correlate with the measure of unrelated constructs?

• Tool gives high scores for well people and low scores for sick people

Construct Validity

• Tool measures what is supposed to .

• Focus: The accuracy of a measurement tool in representing the construct it intends to measure.

• Level: Measurement level.

• Example: A depression scale accurately measures depression symptoms.

Internal Validity

Measurement results warrant a causal conclusion

Inference Validity

• Focus: The accuracy of drawing conclusions (inferences) from the research findings.

• Level: Study level.

• Example: A study accurately concludes that a new drug effectively treats depression based on the research findings.

Translation Validity

• Focus: Whether the measure adequately represents the underlying construct it aims to measure.

• Face validity: Does the measure appear to measure the intended construct at face value?

• Content validity: Does the measure cover all relevant aspects of the construct?

IV

The factor manipulated by the researcher.

DV

The outcome or response measured.

Control group

A group not exposed to the Independent Variable for comparison.

Randomization

Assigning participants to group randomly to reduce bias.

Replication

Repeating the experiment to ensure reliability of results.

Pre-Experimental Design

• One-shot case study, One Group Pretest-Posttest, Static Group

• These designs offer minimal control over extraneous variables and lack a control group, making it difficult to establish causal relationships.

One-shot case study

• Pre-experimental design

• A single group is exposed to a treatment, followed by a measurement. offers the least control and is often used for exploratory purposes.

• Example: Introducing a new teaching method to a class and measuring their performance afterwards.

One-group pretest-posttest design

• Pre-experimental Design

• A single group is measured before and after a treatment. while it provides a baseline, it's susceptible to threats to internal validity like maturation and history effects.

• Example: Measuring students* math scores before and after implementing a new math curriculum.

Static group comparison

• Pre-experimental Design

• Compares two existing groups, one exposed to a treatment and one not. Lack of random assignment weakens causal inferences.

• Example: Comparing the reading levels of students in two different schools

True-experimental Design

• Pretest-Posttest Control Group, Pretest: Only Control Group, Solomon Four-Group, Static Group

• These designs involve random assignment of participants to experimental and control groups, providing strong control over extraneous variables and enhancing causal inferences.

Pretest-posttest control group design

• True-experimental

• Randomly assigned groups are measured before and after the treatment. allows for comparison between groups.

• Example: Randomly assigning participants to either a medication or placebo group, measuring their symptoms before and after treatment.

Posttest-only control group design

• True-experimental

• Randomly assigned groups are measured after the treatment. efficient but lacks baseline data.

• Example: Randomly assigning students to either a new teaching method or traditional method, measuring their test scores at the end of the term.

Solomon four-group design

• True-experimental

• Combines the previous two designs, providing information about the impact of the pretest. offers strong control but is complex and time-consuming.

• Example: Randomly assigning participants to four groups: two with pretests and two without, with one group in each pair receiving the treatment.

Statistical Design

• Time series, Multiple Time series, Static Group

• These designs involve statistical techniques to analyze data and control for extraneous variables.

Time series design

• Statistical

• Similar to the quasi-experimental time series design, but with a stronger focus on statistical analysis to identify patterns and trends.

• Example: a researcher might track the monthly sales of a product over several years to identify seasonal trends, economic impacts, or the effectiveness of marketing campaigns.

Multiple time series Design

• compares two or more time series to assess the impact of a treatment.

• Example: to evaluate the effectiveness of a new smoking cessation program, researchers might track the smoking rates of two similar cities over several years. One city implements the program, while the other serves as a control group. By comparing the trends in smoking rates between the two cities, researchers can assess the program's impact.

Quasi-Experimental Design

• Randomized Blocks, Latin Square, Factorial Design, Static Group

• These designs lack random assignment but attempt to control for extraneous variables through matching or statistical techniques

Randomized Blocks

• Quasi-experimental

• participants are grouped based on a relevant variable (e.g., age, gender) and then randomly assigned to treatment conditions within each block.

• Example: grouping students by their math ability level before randomly assigning them to different teaching methods.

Latin square

• Quasi-experimental

• controls for multiple variables by arranging participants in a matrix, ensuring each treatment condition appears once in each row and column.

• Example: testing four different fertilizers on four plots of land, ensuring each fertilizer is used once in each row and column.

Time Series Design

• multiple measurements are taken before and after a treatment to establish a pattern and assess the impact of the intervention.

• Example: measuring a company's sales over several months before and after launching a new marketing campaign.

Correlational Research

This examines the relationship between two or more variables without manipulating them. While it doesn’t establish causation, it can identify patterns and trends.

Naturalistic Observation

• Type of Correlational Research based on Data Collection methods

• Involves observing subjects in their natural environment without interference.

• Example: Studying the behavior of chimpanzees in the wild to understand social interactions.

Survey Research

• Type of Correlational Research based on Data Collection methods

• Collects data through questionnaires or interviews.

• Example: Conducting a survey to assess the relationship between education level and income

Archival Research

• Type of Correlational Research based on Data Collection methods

• Analyzes existing data, such as records or databases.

Cross-Sectional Studies

• Type of Correlational Research based on Study Design

• Collect data from a sample population at a specific point in time.

• Example: Comparing the smoking habits of different age groups in a single year.

Longitudinal Studies

• Type of Correlational Research based on Study Design

• Collect data from the same group of participants over an extended period.

• Example: Tracking the cognitive development of a group of children from infancy to adulthood.

Cohort Studies

• Type of Correlational Research based on Study Design

• Follow a group of people with a shared characteristic over time. a cohort shares a common characteristic (e.g., age, occupation, exposure to a specific factor).

• Example: Studying the health outcomes of people exposed to a particular environmental factor.

Case-control studies

• Type of Correlational Research based on Study Design

• Compare people with a specific condition (cases) to those without (controls) to identify potential risk factors.

• Example: Comparing people with lung cancer to people without lung cancer to identify potential causes.

Positive Correlation

• based on the direction change of variables

• As one variable increases, the other also increases.

• Example: Height and weight. taller people tend to weigh more.

Negative Correlation

• based on the direction change of variables

• As one variable increases, the other decreases.

• Example: Hours of study and exam anxiety. More study hours often lead to less anxiety

Linear correlation

• based on the number of variables studied

• The relationship between two variables can be represented by a straight line.

• Example: The relationship between age and height in children.

Non-linear correlation

• based on the number of variables studied

• The relationship between two variables is not linear but curved.

• Example: The relationship between the amount of fertilizer used and crop yield, which often follows an S-shaped curve.

Simple correlation

• based on the constancy of the ratio of change bet. variables

• Measures the relationship between two variables.

• Example: The relationship between hours of exercise and height loss.

Multiple correlation

• based on the constancy of the ratio of change bet. variables

• Measures the relationship between one dependent variable and two or more independent variables.

• Example: The relationship between a student's GPA and their hours of study and IQ.

Partial correlation

• based on the constancy of the ratio of change bet. variables

• Measures the relationship between two variables while controlling for the effect of one or more other variables.

• Example: The relationship between height and weight while controlling for age.

Descriptive Research

• a research method used to describe the characteristics of a population or phenomenon.

• it aims to accurately portray the subject under study without manipulating variables. in essence,

• it answers the questions “what," "where," "when," and "how," but not "why."

Parameter

The measurable quality of a population

Statistic

The measurable quality of a sample

Population

• a complete set

• reports are a true representation of opinion.

• it contains all members of a specified group.

Sample

• a subset of the population

• reports have a margin of error and confidence interval

• it is a subset that represents the entire population.

Sampling

This is the process of selecting a subset of individuals from a larger population to estimate characteristics of the whole population.

Probability Sampling

Every member of the population has a known chance of being selected for the sample.

Simple random sampling

• Probability sampling

• Every member of the population has an equal chance of being selected.

• Example: Drawing names from a hat.

Clustered sampling

• Probability sampling

• divide the population into clusters, randomly select clusters, and then sample all individuals within selected clusters.

• Example: Randomly selecting schools and surveying all students within those schools.

Systematic sampling

• Probability sampling

• Select individuals at regular intervals from a list of the population.

• Example: choosing every 10th person on a customer list.

Stratified random sampling

• Probability sampling

• Divide the population into subgroups (strata) based on specific characteristics and then randomly select from each stratum.

• Example: Stratifying a student population by grade evel and randomly selecting students from each grade.

Non-probability Sampling

The selection of individuals is not based on random choice.

Convenience Sampling

• Non-probability Sampling

• Select individuals based on availability and accessibility.

• Example: Surveying students in a college cafeteria

Consecutive sampling

• Non-probability Sampling

• Recruit all participants who meet the inclusion criteria until a sample size is reached.

• Example: Enrolling the first 100 patients who meet the study criteria at a hospital.

Quota Sampling

• Non-probability Sampling

• Create a sample that reflects the characteristics of the population based on specific quotas.

• Example: Interviewing a predetermined number of people from different age groups:

Purposive or Judgemental sampling

• Non-probability Sampling

• Select individuals based on specific criteria determined by the researcher.

• Example: Interviewing experts in a particular field.

Snowball sampling

• Non-probability Sampling

• Participants recommend other potential participants, creating a chain of referrals.

• Example: Studying a rare disease by asking patients to refer others with the same condition.

Instrument Development

This is a systematic process of creating a tool (Questionnaire, Interview Guide, Observation Checklist, etc.) To measure specific variables or constructs. It involves several stages

Item creation

This is the initial phase of developing instrument items. It involves the literature review, survey, in-depth interviews.

Item selection

Choosing from the pool of generated items, a subset is chosen for the final instrument.

Ranking exercise

This is done in item selection wherein experts or researchers arrange items based on their relevance, clarity, and importance.

Item revision

Selected items are refined and improved based on feedback.

Index card sorting test

This is done in item revision wherein participants sort items into categories, providing insights into item clarity and grouping.

Instrument Validation

This process assesses the instrument's accuracy, reliability, and validity.

Instrument preparation

The phase wherein the instrument is being finalized for use. This involves pre-tests, pilot tests, and instrument translation.

Instrument application

Using the finalized instrument to collect data from the target population, also known as field survey.

Pre-test

Administering the instrument to a small sample to identify any issues or ambiguities.

Pilot Test

Conducting a larger-scale test to assess reliability and validity.