Methods in Context

Experiments evaluation

Experiments evaluation

→ Good for issues which can be examined in small-scale contexts w/ clear phys/social boundaries

• teacher expectations, classroom interactions, labelling, pupil self-concepts

→ Bad for large scale topics hard to replicate in a lab/find suitable situation

• gender/achievement, education policy, selection/segregation

Applying experiments in the classroom

→ May use to study parts of classroom life

→ Classroom has clear boundaries r.e. space + time, easier to achieve a degree of control + have effective experiment

Reliability and experiments

→ Often quite simple, so easy to repeat

→ Experiments may not be exactly replicable, but schools are broadly similar

• e.g. ==Pygmalion in the Classroom ==

Ethical issues with experiments

→ Some experiments use pupils in real learning situations; could affect education

→ Young people vulnerable

• Less able to understand events, especially young pupils

• Less able to give informed consent

Limiting application if experiments

→ Small scale, can only examine single aspects of behaviour

→ HArd to examine larger issues as cant replicate in a lab/find suitable field experiment opportunities

Controlling variables in experiments

→ Experiments require researchers to control variables in the situation

→ Schools are large, complex - many variables e.g. streaming, type of school, class size

→ Impossible to control/identify all variables

Questionnaires evaluation

→ Good for large-scale topics to get lots of responses quick/cheap

• Class/achievement, parental attitudes, subject choice, material deprivation

→ Issues involving intense social interactions where asking Qs of those involved is unlikely to produce meaningful data

• Labelling, gender/classroom behaviour, classroom interaction

Practical issues with questionnaires

→ Good to gather large amts of basic info quick/cheap

• Large numbers of pupils/teachers/ed establishments

→ Researchers can use these to correlate factors

• e.g. achievement/attendance/behaviour, with school size/class size/number of staff

Sampling frames and questionnaires

→ School is a good source of ready-made sampling frames

→ Also ready-made opportunity samples eg class lists

Response rates with questionnaires

→ Often low, but can be higher when done in school

• Head teacher puts authority behind research, pressure to cooperate

→ Pupils/teachers/parents also accustomed to completing school questionnaires

Researching pupils with questionnaires

→ Shorter attention spans = more effective to use short questionnaires

• LIMITS info you can gather

→ Pupils w/ poor literacy will struggle to complete

• e.g. disabled children?

Operationalising concepts for questionnaires

→ Turning abstract ideas e.g. deferred gratification may be difficult

• Young people less likely to understand researcher’s questions

  • May need to explain more, could affect data

Validity and questionnaires

→ Life experiences of children are narrower; may not know answers to Qs, questionnaires may be of little value

→ ESPECIALLY PRIMARY AGE KIDS

Sampling and questionnaires

→ Schools might not keep lists reflecting researchers interests

• e.g. schools may not have lists of pupils sorted by ethnicity

Evaluation of structured interviews

→ Good for large-scale topics to get lots of responses quick/cheap

• Class/achievement, parental attitudes, subject choice, material deprivation

→ Worse for topics requiring direct observation/examining formal documentation

• Official curriculum

• Classroom interaction

Response rate and structured interviews

→ Less disruptive to school activities; usually take less time than unstruc interviews

• More liekly to gain access

• Support from head teacher = may increase response rate

Reliability of structured interviews

→ Easy to replicate

→ Can identify large-scale patterns e.g. gender + subject choice

Validity of structured interviews

→ Younger people tend to have better verbal than literacy skills

• More valid than questionnaires?

→ Formal = pupils wont feel at ease; may be less forthcoming

• Similar conditions to lessons/exams/other controlled situations

Question designs in structured interviews

→ Hard to make questions for young people; less linguistic/intellectual skills, may not understand complex concepts

→ Limited answers = more limited vocab, use words incorrectly

• May need clarification, not possible in struc interviews

Ethical issues and structured interviews

→ Parental permission - may not be given for various reasons

• Sensitive topics?

Power/status differences and structured interviews

→ Pupils/teachers not equal in power, affects their behaviour

• Alter responses to seek adult approval

• Untrue but socially acceptable answers

→ See adults as authority figures; researcher may come across as a teacher in disguise

• Reduce validity

Evaluation of unstructured interviews

→ Useful to find meanings/attitudes people hold; open-ended Qs

• racialised expectations

• parent attitudes

• pupil subcultures

• how school policies are actually implemented in practice

→ Less useful for large-scale topics/topics requiring detailed recording of acc events

• Patterns of achievement

• speech codes in the classroom

Power/status inequality and unstruc interviews

→ Informal so helps establish a rapport

→ Labov: encourages interviewees to open up/respond more fully

• Useful for sensitive topics

Practical issues with unstruc interviews

→ Gives pupils time/space/encouragement to work out responses without rushing them

→ Shorter attention span = may find long interviews too demanding

Validity and unstruc interviews

→ Children struggle to keep to the point; many give contradictory/irrelevant responses

→ Overall suitable for young people; interviewer can explain meanings

Reliability and unstruc interviews

→ Some interviewers try to maintain relaxed atmosphere

• Nodding, smiling, maintain eye contact

→ Hard to standardised = less reliable, interviewers may get diff results

Social desirability and unstruc interviews

→ Pupils defer to adults; may answer what they think the interviewer wants

→ Teachers want to protect professional self-image; may try to presen themselves in most positive light

• Unstruc interviews let researchers probe behind this image

Interviewer training for unstruc interviews

→ Need more training

• Don’t interrupt

• TOlerate long pauses

• Don’t repeat Qs

Evaluation of structured observation

→ Good for issues to be examined in small-scale contexts with clear physical/social boundaries e.g. classrooms

• Classroom itneractions, racialised expectations, labelling, gender + classroom behaviour

→ Wrose for large scale/difficult to observe topics

• Class/achievement, education policy, material deprivation

Practical issues in structured observation

→ Classroom suited to structured obs; closed off physical/social environment

→ Short lessons = observer wont get fatigued, more accurate info

→ Simple = quick/cheap/less training

→ May have too many diff behaviours to categorise e.g. in playground activity

Reliability and structured observation

→ Range of classroom behaviours is limited; limited behaviour categories can be used

• Easy to replicate

→ Also generates quant. data; easy to compare

Validity and structured observation

→ Interpretivists say invalid

→ Ignores meanings pupils/teachers attach

Observer presence and structured observation

→ Presence of stranger can be off putting + hard to disguise; esp non-participants

• Affects behaviour

• Reduces validity

Evaluation of participant observation

→ Good for small-scale context with clear boundaries e.g. classrooms

• Classroom interaction, racialised expectations, the male gaze, pupil subcultures

→ Worse for large-scale topics; studying a small group produces unrep data

• Mat dep, class/achievement, education policy

Validity and participant observation

→ More likely to overcome problems wit status differences, allows researcher to build rapport/gain acceptance

→ Both teachers/pupils still skilled at altering behaviour when observed by those in authority

• Hard to know if behaviour is genuine

• do this for OFSTED etc

Practical issues with participant observation

→ Schools are complex; takes time to understand how they run

→ Classroom obs may be less disruptive than interviews, easier to get access

→ BUT restricted by timetable/holidays + gatekeepers control access

→ Not much privacy; busy

Ethical issues with participant observation

→ Pupils more vulnerable; may not be able to give informed consent

• Means it often has to be overt

→ How to protect school’s identity?

• Poor public image due to research can damage school’s rep + thus education of pupils (less ppl will want to go there so less funding)

Hawthorne effect and participant observation

→ Most obs has to be overt; few roles to adopt, as researcher stands out as older than pupisl

→ HE is unavoidable

→ Teachers may be suspicious of observer, alter normal behaviour

Representativeness of participant observation

→ Can only do small-scale

→ Over 35k schools, impossible to represent everyone

Evaluation of official stats

→ Good for topics on which govt collects national statistics

• Policy, mat dep, class/achievement

→ Worse for smaller issues which govt doesnt collect data on

• Classroom interaction, racialised expectations, labelling, gender/class behaviour

Practical issues with official stats

• govt collects stats from every school; saves time/money + can make comparisons

• allow us to examine trends through time

• govt are often interests in same education issues as sociologists

  • subject choice, racism, inequality, etc.

  • so likely to be useful to researchers

• but key definitions of concepts may differ from those sociologists use

  • e.g. govt measure achievement by 5 A-C grades at GCSE

Representativeness and official stats

• some official stats on eudcation are very representative

  • all schools have to complete school census 3x yearly

• impossible for researchers to collect this range of data themselves; covering virtually every pupil in the country

Reliability and official stats

• standard definitions/categories; replicated from year to year

  • can make direct comparisons eg of exam performance

• govts may change definitions

  • e.g. several definitions of ‘value added’ have been used to measure school performance

  • reduces reliability

Validity and official stats

• interpretivists challenge validity of educational statistics; socially constructed

  • e.g. pupil attendance stats are outcome of decisions/definitions from parents/teachers/pupils

• schools may manipulate statistical records due to pressure to present themselves positively

  • to maintain their funding/parental support

  • undermines validity of statistics

• some stats are less open to manipulation

  • e.g. pupil roll numbers, exam results

Eval of documents

• Good for issues requiring historical viewpoint/analysing texts

  • Education in the past

  • Stereotyping in school books

  • Official curriculum

• Worse for issues which require docs to have been created by those involved (this is unlikely to be the case)

  • w/c experience of schooling

  • class interaction

  • labelling

Practical issues with documents

• most education is run by state + schools compete for ‘customers’, lots of info about education is publicly available

  • school policy statements

  • local authority guidelines

  • school brochures/websites

Ethical issues with documents

• few ethical concerns with public documents; already in the public domain

• more ethical problems with personal documents e.g. school reports/pupil workbooks/teacher diaries

Reliability and documents

• many school docs are in a systematic format; can draw comparisons

  • e.g. attendance registers

• but may have accidental mistakes as done by individuals

Credibility and dcuments

• give an ‘official’ picture of what’s happening in a school/college

• schools want to present themselves in positive light in the education market

  • documents constructed with a parental audience in mind

• makes them less believable/valid

Representativeness and documents

• some docs are legally required; likely to be representativeness

• not all behaviour is recorded; reduces rep.

  • e.g. racist incidents, whatever goes unreported

• personal docs produced by pupils/teachers may be collected in an unsystematic way

Validity and documents

• can provide insight into meanings held by teachers/pupils; high in validity

• all docs open to interpretation

  • researcher cant be sure their interpretation is accurate