formin methods

FORMULATING THE METHODS

Outline

  • How to present methods

  • Why planning is important

  • Two principles for planning experiments

  • Describing participants

  • Selecting and describing instruments

  • Describing procedures

  • Describing design and analysis

  • Establishing cause and effect

  • Interaction of participants, measurements, and treatments

HOW TO PRESENT METHODS

  • The purpose of the methods section is to explain how the study was conducted.

  • The description must be thorough enough that a competent researcher could replicate the study.

  • A typical methods section is divided into four parts:

    • 1) Participants

    • 2) Instruments or apparatuses

    • 3) Procedures

    • 4) Design and analysis

WHY PLANNING IS IMPORTANT

  • Careful planning is crucial to eliminate any alternative or rival hypotheses.

  • Correct study design leads to predictable results, implying the only explanation is the research conducted.

  • Example: Testing a hypothesis where shoe size is positively related to mathematics performance in elementary children (grades 1-5).

    • Students' shoe sizes are measured alongside standardized math scores.

  • The plotted data shows each dot representing a single student, indicating a trend: larger shoe size correlates with better math performance.

  • Critical reflection: "Could this be true? What have we overlooked?"

MAXICON PRINCIPLE

  • Remember the MAXICON principle which consists of:

    • Maximize true variance: Increase the odds of discovering the real relationship or explanation.

    • Minimize error variance: Reduce any mistakes that may obscure the true relationship.

    • Control extraneous variables: Ensure that rival hypotheses do not serve as real explanations for observed relationships.

TWO PRINCIPLES FOR PLANNING EXPERIMENTS

  1. Less is more:

    • Refers not to the number of participants but the number of independent and dependent variables.

    • Avoid adding extra variables solely for exploratory purposes.

  2. Simple is better:

    • Simplicity in treatment design, analysis, and displaying data is essential.

    • Overly complex studies can impede interpretation of results, even for the researchers.

DESCRIBING PARTICIPANTS

  • This section outlines how participants were selected and the pertinent characteristics for the study:

    • Are participants with special characteristics necessary?

    • Age: Specify categories such as children, adolescents, young-middle-aged adults, older adults (provide age in years).

    • Sex: Male, female, or both.

    • Level of training: Trained vs. untrained.

    • Level of performance: Experts vs. novices.

    • Size: Weight, adiposity, etc.

    • Special types: Defining attributes such as athletes, cyclists, sedentary individuals.

  • Considerations:

    • Can necessary permission and cooperation from participants be obtained?

    • Is it feasible to find enough participants?

REPORTING PARTICIPANT CHARACTERISTICS

  • Clearly include the exact number of participants, often noted as N = .

  • List participant characteristics crucial for the research, defined as needed.

  • For reporting, express characteristics statistically (M ± SD).

SELECTING AND DESCRIBING INSTRUMENTS

  • Selecting instruments, apparatuses, or tests for data collection requires careful planning.

  • Critical questions to consider include:

    • What are the validity and reliability measures? (Validity: Does it measure accurately? Reliability: Does it provide consistent measurements?)

    • How difficult is it to obtain the measures?

    • Do you have access (budget considerations) to the instruments/tests needed?

    • Are you capable of administering the tests or using the equipment?

    • Can you evaluate test performance adequately?

    • Will the tests yield a reasonable range of scores for selected participants?

    • Are participants willing to commit the necessary time for test administration?

EXAMPLE OF INSTRUMENT SELECTION

  • In a sport psychology study on how college athletes respond to lectures on steroid use:

    • Administer three tests:

    • Steroid knowledge test

    • Attitude about responsible drug use

    • Trait personality measures.

    • Describe reliability and validity of each test with citations and explain scoring methods.

DESCRIBING PROCEDURES

  • Detailed description of data acquisition (testing procedures and data analysis).

  • Organization should typically be chronological.

  • Include details such as:

    • Who administered the tests?

    • The testing situation and participant preparation.

    • Specific instructions provided to participants.

POINTS TO CONSIDER IN PROCEDURES

  • When collecting data, clarify:

    • When, where, and duration of tests.

    • Use of pilot data to demonstrate researcher skill with tests/equipment and understanding of participant responses.

    • A well-designed scheme for data acquisition, recording, and analysis is imperative.

TREATMENT PLANNING

  • Determine treatment specifics:

    • Duration, intensity, and frequency of interventions.

    • Methods for assessing participant adherence to treatments.

    • Use pilot data to preemptively gauge participant responses and researcher administration capabilities.

    • Ensure treatments are appropriate for the participant type.

REPLICATION IN WRITING

  • Procedures must detail an order of steps taken:

    • Timing of study, duration of procedures, and gaps between procedures.

    • Instructions delivered to participants requiring replication.

    • Include briefings, debriefings, and necessary safeguards.

DESCRIBING DESIGN AND ANALYSIS

  • Designing research is crucial for outcome control.

  • Independent variables must be manipulated to examine their impact on the dependent variable.

  • A well-designed experiment ensures that changes in the dependent variable solely arise from the independent variable treatment.

STATISTICAL ANALYSES

  • Statistical techniques should be explained:

    • Descriptive statistics such as means ± SD for each variable.

    • For correlational techniques, variables must be named alongside specific analysis methods used.

    • In experiments, statistics detailing between-group differences must also be presented.

ESTABLISHING CAUSE AND EFFECT

  • Establishing cause-and-effect relationships involves more than just design and statistics.

  • Statistical tests aim to either retain or reject the null hypothesis:

    • If rejected, what remains is the research hypothesis, though proving this can be challenging.

    • Criteria for establishing cause and effect:

    1. Method of agreement: An effect occurring when both A and B exist, with C in common likely indicates C as the cause.

    2. Method of disagreement: An effect absent in E and F when C is the only common absent element suggests C is the cause.

MANIPULATION EFFECTS IN EXPERIMENTATION

  • In experimental research, treatment involvement occurs regularly.

    • Example: Participants in a home exercise program are expected to exercise daily for a minimum of 40 minutes.

  • Assurance of participant adherence may require manipulation checks.

MANIPULATION CHECKS

  • Example manipulation check: requiring participants to wear activity monitors as validation of exercise engagement.

  • Identify challenges: Fraudulent reporting may occur if participants ask non-participants to wear devices.

  • Laboratory settings facilitate easier checks compared to real-world settings.

FATAL FLAWS OF RESEARCH

  • Evaluating studies for fatal flaws is essential.

  • Consider the following questions:

    • Does the study lack characteristics that lead to automatic rejection for publication regardless of methodology or outcomes?

    • Are hypotheses logically consistent with the theory and study characteristics?

    • Are assumptions about the study rational?

    • Is there an adequate number of "right" participants selected?

    • Are treatments sufficiently intense and lengthy to effect desired changes?

ADDITIONAL EVALUATION QUESTIONS

  • Are extraneous variables being adequately controlled?

  • Are dependent variables effectively characterizing participant responses to treatments?

  • Are all measurements both valid and reliable for the context?

  • Are data collection and storage procedures meticulously planned and executed?

INTERACTION OF PARTICIPANTS, MEASUREMENTS AND TREATMENTS

  • In both correlational and experimental studies, participant selection, measurement, and treatment choice is vital.

  • For correlational studies, measurements should capture the critical characteristics and yield an appropriate range of scores to facilitate discovering important relationships.

    • Example: To study the relationship between anxiety and motor performance, participants need a range of anxiety levels and motor skills to draw valuable conclusions.

  • In experimental studies, this principle also applies when evaluating the impact of a treatment (independent variable) on dependent variables.

    • Example: Determining the influence of a resistance training program on jumping performance requires diverse baselines in participants’ training backgrounds and skills to enable observation of treatment effects.