Research Methods and Ethics

Credential-Based Knowledge : Authoritative Knowledge

• Relies on deferring to experts for accurate information, trusting their credentials and training

• Society often shows strong trust in authority figures

• Expert advice can be wrong. E.g. Weather forecasts, election predictions and box office success illustrate misjudgement by professionals

Common Sense and Intuition as a Source of Knowledge

• Common sense relies on personal and observed experiences to form “practical knowledge”

• Intuition is “direct access” knowledge or gut feelings guiding decisions without conscious reasoning

(-) both common sense and intuition, like traditional and authoritative knowledge, encourage uncritical acceptance of information

Science as a Trustworthy Way of Knowing

• Scientific knowledge provides the highest quality information and allows us to be critical consumers of information

• Science is distinct because it uses methodical processes designed to reduce or control errors

• Systematical, methodical rules are used to gather Empirical Evidence (evidence gained through observation)

Empirical Evidence

• Science does not accept assertions about the world at face value

• Assertions need ot be supported by concrete, objective evidence

• Empirical evidence includes sensory data that can be seen, heard or smelled

Methodological Rules

• Science uses standardised procedures to reduce errors and guide the search for accurate information

• Rules govern how to establish casual connections between events or between characteristics and behaviour (criteria for internal validity)

• Rules determine whether research findings can be generated beyond the study (criteria for external validity)

• Rules exist for developing and assessing methods

Replication

• Science requires that findings occur more than once to be considered true and reliable.

• Replication safeguards against premature or false conclusions

Exploratory Research

• Aims to increase understanding of new or little studied phenomena, groups or settings

• Uses small samples for in-depth, first hand information

• often produces qualiatative data through one-to-one interviews or participatory study.

Descriptive Research

• Provides a detailed, accurate picture of a social phenomenon, group, or setting.

• Describes what is happening, who is involved, and the size of the group.

• Focuses on accurate measurement and sampling.

• Often uses quantitative methods and statistics to summarize data.

Explanatory Research

• Seeks to understand causes and processes behind social phenomena (e.g. domestic violence)

• Answers the “why” and “how” questions

→ wants to know why some and not others resort to this family dynamic? How do violent events occur or unfold?

• Has a strong focus on causal analysis and establishing cause-effect relationships

Evaluation Research

• Judges the effectiveness or impact of social programs, policies or interventions

• Concerned with the outcomes and practical results

• Important for accountability and securing funding

Central Role of Ethics

• Research must follow standards of professionalism and honesty

• Researchers must aim to earn respect and trust of participants and the public

Ethics in Social Science

• Usually less physically harmful than in natural science

Ethical Principles

Do Not Harm :

→ Researchers must safeguard the physical, psychological and emotional well-being of participants

→ some harm is unpredictable and can occur in an experiment that seems safe at face value.

→ Researchers can’t predict all consequences but must still anticipate likely risks as well as take steps to reduce harm and maximise benefits.

→ This can be done through debriefing sessions in which the researcher explains fully the study to participants and gathers feedback on their experiences and any negative after effects

Informed Consent

The right of individuals to decide for themselves whether to participate in research or not.

Participants must be fully informed about all aspects of the research

4 Elements of Informed Consent

1. Competence : Informed consent can only be given by competent individuals, capable of deciding if participation is in their interest. E.g. children, low cognitive abilities, under duress/against their will etc…

→ passive consent increases participation but does not ensure genuine, informed agreement,

2. Voluntarism : participants are free to say yes or no without any pressure or coercion.

→ some situations make true voluntarism difficult to achieve, especially in institutional settings like hospitals, schools or prisons.

→ egalitarian relationships between researcher and participant are ideal but often unreliable as participants are usually seen as research subjects.

3. Full Information : Research participants must receive all relevant information needed to make an informed choice.

→ sharing every detail may confuse participants or bias study results

4. Comprehension : individuals must understand the information they receive to give informed consent.

→ information should be presented in clear, non-technical, lay language

Respecting Subjects’s Privacy

The right to privacy allows individuals to control when and how others access their personal information

3 Privacy Issues Researchers Must Pay Attention To

Sensitivity of Information ; any personal information can be potentially threatening for some respondents. E.g. if the survey contains threatening questions, the respondent should be forewarned and reassured about the steps taken to ensure their privacy.

→ the more sensitive the topic, the more safeguarding that is needed.

Research Setting : social research occurs in diverse settings; homes, schools, bars, workplaces, malls etc…

→ researchers must consider the boundary between private and public spaces and how to protect a participants safety.

→ the extent to which a setting is private or public is not really all that obvious

Disseminating Research Finding :

• Privacy: Researchers must protect participants' personal information when publishing findings.

• Anonymity: The researcher cannot link data to specific individuals (e.g., not collecting names or addresses).

• Confidentiality: The researcher knows participants' identities but promises not to reveal their information publicly.

• Protecting data: Use case numbers, secure personal records, and destroy documents containing identifiers when appropriate.

• Limits of confidentiality: Courts may require researchers to disclose data through subpoenas.

• Certificates of Confidentiality: Can help protect sensitive research information from forced disclosure.

• Researchers' responsibility: Clearly explain the limits of confidentiality and take steps to protect participants' privacy.

Researchers Should Avoid Conflicts of Interest

• Conflict of interest: Personal or financial interests may influence research.

• Corporate funding: Sponsors may influence the direction, scope, publication, or findings of research.

• Transparency: Researchers should disclose their sponsors, allegiances, and competing interests.

• Research independence: Funding should not compromise objectivity or the accuracy of findings.

Ethical Reporting: The Whole Truth and Nothing but the Truth

• Innumeracy: Difficulty understanding or critically evaluating numerical information and statistics.

• Misuse of statistics: Researchers may selectively present statistics to support a particular argument or sponsor.

• Biased reporting: Choosing which findings to highlight can create different interpretations of the same data.

• Truthful reporting: Researchers must report findings fairly, accurately, honestly, and responsibly.

Science and Valid Knowledge

• Science has the advantage of verifying the truth

• Valid knowledge (empirical) is best achieved through scientific research methods

• Science is not infallible, as errors can occur in research execution, interpretation, or judgement

Measurement Validity

Are we accurately measuring what we’re claiming to measure? Using the right tools?

Examples :

Polygraph (lie detector) tests

• Advocates argue that polygraphs detect truth or deception

• Courts reject polygraphs due to lack of measurement validity; inadmissbale as evidence

IQ Tests

• Claims to measure innate intelligence

• Critics argue that the test measures the quality of education and social experiences someone has, not innate ability

• IQ tests are biased against people of colour (originally made to suit white, western males)

Fear of Crime Survey

• GSS asks “Is there any area within a mile where you’d be afraid to walk alone alone at night?”

• Critics question whether this actually measures ‘fear of crime’ or only fear of walking alone at night.

Internal Validity

Can we trust that ‘A’ definitely cause ‘B’?

• Correct causal identification is crucial as false conclusion could lead to misguided policies made in response.

To establish causality, there are 3 criteria:

• Temporal Order → the independent variable must precede the dependent variable

• Association → the independent variable and dependent variable must be connected in a patterned way/are connected

• Non-spuriousness → all rival explanations for the dependent variable must be ruled out.

Research design is crucial for internal validity, especially ones that satisfies the three causality criteria.

External Validity

The generalisability of research findings to other settings, groups or conditions.

• Experiments are especially prone to external validity issues because they occur in contrived, controlled settings.

• Researchers must consider whether experimental conditions influence participant behaviour and whether findings can be generalised beyond the study context

Survey findings must be evaluated for their reach beyond the original source:

• Sample generalisability

• Cross-population generalisability

Critical assessment of survey data ensures that conclusions are not overstated or misapplied

Selecting Participants and External Validity

• Experimental research often relies on volunteers and selection protocols, which can weaken external validity and therefore generalisability of results

• Sampling is the process of studying a small group to learn about a larger group, its success depends of selecting a sample that accurately represents the population

External Validity and Replication

• To determine if experimental findings apply to other groups or settings, researchers must use replication

• Studies must be repeated to increase confidence in scientific knowledge

Concepts

• Concepts are mental images, abstraction or terms that symbolise persons, things or events

• In Natural Science, concepts are often expressed in symbols or notations

• In Social Sciences, concepts are usually expressed in words

Importance of Concepts

• They are essential for communication, allowing efficient exchange of information

• They make it possible to describe complex topics such as weather or politics

Concept and Research

• Research focuses on the concrete and empirical while concepts are mental abstractions

• Despite this, concepts are central to research because they enter through theory

Concept and Theory

• Theories are sets of logically related or linked ideas about how the world or processes work

• Theories consist of statements about the relationship between concepts

E.g: Structural Functionalism → uses concepts such as social stability, integration and consensus to explain the world

Conflict Theory → uses concepts such as dissensus, coercion, power and social control to explain social reality

Theory and Research

Theory offers ideas about how the world works; research empirically documents those ideas as correct

Two Research Paths:

→ Deductive Research : tests established theory

→ Inductive Research : starts in the empirical realm to generate new theories

Good research either ends in or begins with concepts and theories

From Concepts to Measurement

Problem : Concepts are abstract, research demands concrete, empirical evidence

Solution : translation; convert concepts into empirical counterparts through measurement

Measurement : transforming the abstract into the concrete

Measurement Validity and Scientific Standards

Scientific Requirement : concept-to-variable translations must satisfy measurement validity

E.g: Stomping feet might indicate anger and not happiness

Warm puppy is a poor happiness measurement for someone afraid of dogs

A researcher’s duty is to ensure an error-free translation process and provide evidence of adequacy for measures

Conceptualisation

Concepts can have different meanings for different people, so researchers must clarify their intended meaning

E.g: one researcher may define violence as physical action while another may include verbal attacks

Theoretical/Nominal Definitions

Sources of Nominal Definitions:

• standard dictionaries and sociology dictionaries

• E.g: Anxiety - emotional state of extreme apprehension without clear focus

• Reviewing relevant literature helps information conceptualisation and provides multiple perspectives

Operationalistion

This is the process of finding the best empirical counterpart for a concept once conceptual clarity is achieved

Researchers aim to create strong connections between theoretical definitions and observable measures

Challenges in Measurement

Some measurement ideas may work well while others must be discarded

E.g: Conservatism may be measured by agreement or disagreement with statements on social innovations (voting for a female president, supporting gays in the military)

Researche may need to refine their theoretical definitions or specify dimensions of complex concepts

E.g: Alienation is a multi-dimensional concepts ( a loss of involvement in the activity of

working; an estrangement from fellow creatures etc…)

Levels of Measurement

There are 4 levels:

• Nominal → Data is placed into discrete categories or names that have no natural order or ranking.

• Ordinal → Data is placed into categories that have a meaningful, natural order or rank. (Likert Scale 1-5)

• Interval → Data has a meaningful order and the intervals (gaps) between values are equal and known. (Temperature in Celsius or Fahrenheit; a ruler)

• Ratio → Data has all the properties of an interval scale, plus a true, absolute zero point (where zero means none or complete absence of the trait)

Operational Definitions

Finalise the measurement process by specifying the exact steps and procedures for measuring a concepts, functioning like a ‘recipe’ for measurements.

They translate abstract concepts into concrete, measurable forms and ensure clarity and consistency in research

What makes an operational definition thorough?

A: It explains exactly how to measure a concept, including:

• The type of questions and response options (closed-ended or open-ended).

• How multiple questions/items will be combined to calculate a final variable value.

Allows for research replication/realiability

Assessing Measures

Measuring concepts is like translating languages, the results may not match the original meaning

Operationalisation risks distortion when indicators fail to capture the construct

E.g: exam grades may not capture an individuals intelligence

Face Validity

A measure must appear to make sense on the surface

If the ‘empirical translator’ does not seem appropriate, it lacks face validity.

E.g: Technological savvy measured by asking if an individual knows how to use a phone

Subjectivity of Face Validity

Face validity is highly subjective; different people may judge the same measure differently.

Because of this, face validity is not considered a very convincing test of validity by itself

Content Validity

Content Validity assesses the fit between between nominal (theoretical) and operational (practical) definitions

Does the tool cover all parts of the concept it is supposed to measure?

E.g: A mental health test for depression must ask about all major symptoms (like mood, sleep, and appetite) rather than just asking about sleep

Criterion Validity

Uses empirical evidence to demonstrate the validity of a measure; if a measure accurately matches or predicts a real-world outcome

Types of Criterion Validity

1. Predictive : measures the ability to predict a logically related future outcome.

E.g: Leadership Ability Index - a ten item measure predicts that individuals scoring high will occupy leadership roles after 12 months

Accurate prediction serves as objective evidence that the measure is valid

2. Concurrent : how well a new measure matches an older, already-proven test when both are given at the same time

Both tests are taken together or right next to each other

If the two sets of scores match closely (a high positive correlation), the new test has good concurrent validity

Construct Validity

The degree to which a measurement tool accurately captures an abstract, unobservable concept (like intelligence, anxiety, or creativity) rather than something else.

E.g: An intelligence test that relies heavily on dense reading passages might accidentally measure English reading comprehension or speed rather than true logical reasoning

Reliability

A measures ability to produce consistent results over repeated use.

A measure is reliable if it yields the same results each time, assuming the variable has not changed.

Test-Retest Reliability

Measuring the same thing twice over a period of time, and then checking if the results are consistent

High correlation between the two sets of results indicates reliability

Timing Considerations of Test-Retest

• For humans, immediate retesting can produce artificially consistent results due to memory of previous answers

• Long delays, however, introduce the possibility that the variable (i.e anxiety), has changed

Multiple Forms Method

Used when researchers only have one time access to respondents (e.g surveys)

Involves creating alternate versions of a measure.

Reliability is established by comparing results from each form; nearly identical results indicate good reliability

E.g: asking for a person year of birth in one form and in another asking for their age in years.

Easily used for single-item measures like age as opposed to multi-item measures like anxiety

Single-item vs. Multi-item Measures

Single-Item Measures : easy to use multiple forms method by creating a seconf question asking for the same information

Multi-Item Measures : harder to create a functionally equivalent form as there are multiple aspects of a concept that need to be addressed.

Spilt-Half Technique (Multi-Item Measures)

This technique checks if all items in a measure are equivalent and consistently measure the same concept.

E.g: a ten-item anxiety index can be split into two sets

Scores for each set are calculated and correlated; high correlation indicates reliability

Cronbach’s Alpha

Coefficient commonly used to report internal consistency of items in an index.

Ranges 0 to 1.0

Values of 0.8 and higher indicate good reliability

Measurement Error - Noise

There is no consistent direction to the error

Measurements sometimes overshoot or undershoot the true value

Noise reduces a measure’s reliability but does not make it invalid. This is because the overshot and undershot values can cancel each other out (e.g ± 2) to get the true value

Measurement Error - Bias

A patterned error, consistently overestimating or underestimating a true value

Unlike noise, bias does