Research Methods and Validity: Ethical Data Collection, Testing, and Reporting

Why must all data in a research study be collected in an ethical way?

To protect participants and ensure research integrity, including informed consent, respect for the right to privacy, discreet data collection, and secure data storage.

What are two key participant rights that must be protected during data collection?

Informed consent and the right to privacy.

What is meant by a 'test' in test selection?

An instrument or tool used to make a particular measurement, which may be oral, written, mechanical, and can assess cognitive, psychomotor, or affective domains.

What is measurement in the context of research?

Usually a quantitative process of assigning a number to a performance or attribute of a person.

What is reliability in relation to a test?

The consistency of a test, meaning it should obtain the same or very similar results each time it is administered.

What is validity in relation to a test?

The degree to which a test measures what it claims to measure; the 'truthfulness' of the measurement.

How are validity and reliability related?

For a test to have a high degree of validity, it must also have a high degree of reliability; a test must be reliable to be valid.

What is a validity coefficient and how is it interpreted?

A correlation coefficient that provides evidence for validity; the closer it is to +1.00, the more valid the test.

What validity coefficient is typically recommended when a test is used as a substitute for another validated test?

A validity coefficient of about 0.80 or higher.

How is sensitivity of a diagnostic test defined and calculated?

It is the ability of the test to correctly identify those with the disease and is calculated as True Positives / (True Positives + False Negatives).

How is specificity of a diagnostic test defined and calculated?

It is the ability of the test to correctly identify those without the disease and is calculated as True Negatives / (True Negatives + False Positives).

What is criterion-referenced measurement?

A type of measurement used when individuals are expected to perform at a specific level of achievement, comparing performance to a set standard rather than to other people.

What is one limitation of pass/fail criterion-referenced standards?

They do not indicate how good or poor an individual's ability is and the success standard is often set arbitrarily.

What is norm-referenced measurement?

A type of measurement that compares an individual's performance with the performance of others using norms such as z-scores, T-scores, or percentiles.

What factors should be considered when using norms in norm-referenced measurement?

The sample size used to determine the norms, the characteristics of the population (e.g., age and experience), and the date the norms were established.

What is content validity for a norm-referenced test?

The extent to which a test measures all skills and subject matter that have been presented and is clearly related to the objectives of the class or presentation.

What is the difference between predictive validity and concurrent validity?

Predictive validity estimates future performance by correlating a test with a later criterion; concurrent validity indicates current performance by correlating test scores with a criterion measured at approximately the same time.

What is construct validity evidence?

Evidence that a test reflects the degree to which an individual possesses a theoretical trait or construct such as anxiety, intelligence, motivation, cardiovascular fitness, or tennis skill.

What is the test-retest method for estimating reliability?

Administering the same test twice to the same group and calculating the correlation between the two sets of scores, with potential error coming from changes in individuals over time.

What is the split-half method for estimating reliability?

Splitting a test into two halves (e.g., odd vs. even items), correlating scores on the halves, and recognizing that reliability generally increases as the total length of the test increases.

What do credibility and transferability in qualitative research correspond to in quantitative research?

Credibility corresponds to reliability, and transferability corresponds to validity.

Name two strategies used to enhance credibility and dependability in qualitative research.

Identifying researcher biases (reflexivity), collecting multiple forms of data (triangulation), separating observations from interpretations (memos), seeking contradictory evidence, spending considerable time on site, using member checking, and maintaining audit trails.

What are the main elements of a research report?

A clear understanding of the problem, methods description, precise data presentation, data interpretation, identification of weaknesses, and summarization of findings.

What is the purpose of the introduction and discussion sections in a research report?

To state the main purpose, create a connection between writer and readers, and link the discussion back to the introduction.

What key components should be included when explaining the research problem?

The statement of the problem and sub-problems, the setting, importance of the problem, definitions of key terms, and a review of related literature.

What is the primary goal of the methods section in a research report?

To describe the methods in enough detail for replication by another researcher.

What specific elements should be included in the methods section?

General design, research setting, inclusion and exclusion criteria, sample, assessment instruments, and procedures.

What should the data and analysis section of a report contain?

Rationale for statistical approaches, summary of data related to each problem, unexpected findings, and supporting figures, tables, or examples.

What is the main purpose of the data interpretation section?

To explore the intrinsic meaning of the data, fully utilize what the data reveal, and draw conclusions based on findings.

Why should researchers avoid using the word 'prove' when interpreting data?

Because research supports or fails to support conclusions rather than proving them.

Under what condition should researchers draw conclusions about causation?

Only when the study design allows for legitimate causal conclusions.

What kinds of weaknesses should be identified in a research report?

Assumptions or biases, delimitations of the research, and limitations that may cast doubt on findings.

What is the purpose of the summary and connections-to-broad-context section?

To summarize findings, answer the 'so what' question, and explain the study's contribution to collective knowledge.

List three general guidelines for effective research writing.

Choose an appropriate style, use a logical structure with headings, and be clear and precise.

What does maintaining academic integrity require regarding others' ideas?

Appropriately crediting others' words and ideas through accurate citation.

How should researchers handle confidentiality and participants' privacy?

By maintaining confidentiality and protecting participants' rights to privacy.

What information about participants should be described to maintain academic integrity?

Participants dropped and reasons, limitations of measurement instruments, and a comprehensive report of all findings.

Who should be listed as coauthors on a research report?

Only those who have made significant intellectual contributions.

What is the suggested way to use reviewers' critiques in the writing process?

Seek critiques, review them carefully, and revise the report accordingly.

What are typical components of the front matter in a research article?

Title page, signature/copyright page, abstract, acknowledgments, and table of contents.

What are footnotes and endnotes used for in a report?

To provide sources, permissions information, and supplemental information.

What should a complete reference list include for each source?

Author, date of publication, title of the work, and publication information.

What key questions should a well-written abstract answer?

What the project intended to accomplish, the research problem addressed, general design and methodology, results, and conclusions.