Wang et al

Original Paper: Social Skills Assessment in Young Children With Autism

Abstract

  • Key Concept: Impairment in reciprocal social interaction and social skills is a defining characteristic of Autism Spectrum Disorders (ASD), impacting individuals across a wide range of severity and age groups. Early identification and effective intervention are crucial due to the pervasive nature of these challenges.

  • Objective: To evaluate the usefulness and psychometric properties of two prominent behavior rating scales, the Social Skills Rating System (SSRS) and the Preschool and Kindergarten Behavior Scales (PKBS), for assessing social skills in young children diagnosed with ASD within authentic, real-world early childhood educational programs. The goal was to determine if these tools could reliably guide individualized intervention design and measure progress.

  • Findings:

    • SSRS and PKBS demonstrated adequate psychometric properties in this ASD population, including strong internal consistency (reliability), robust construct validity (measuring what they intend to measure), convergent validity (correlating with similar measures), and criterion validity (concurring with an established external measure, AEPS). This indicates they are reliable and valid for a static assessment of social skills.

    • Despite their validity, both tools proved ineffective in detecting significant social skills progress or intervention outcomes over a period, suggesting a lack of sensitivity to change in this specific population. This has critical implications for their utility in longitudinal monitoring of intervention effectiveness.

Keywords

  • Assessment, Social Skills, SSRS, PKBS, AEPS, Autism

Introduction

  • Social Skill Development: Young children typically develop a complex range of social behaviors, such as joint attention, imitation, turn-taking, sharing, and emotional regulation, that are fundamental for forming meaningful interactions with peers and adults and building healthy relationships. This development is crucial for cognitive, emotional, and academic growth.

  • Research Findings: Children with autism consistently show distinct social development differences very early in life, often manifesting as reduced eye contact, lack of social smiles, delayed babbling, and difficulties with reciprocal gaze (Dawson et al., 2000; Wimpory et al., 2000).

  • Impact of Deficits: Persistent social skills deficiencies can lead to significant immediate and long-term challenges, including poor educational adjustment, peer rejection, academic disengagement, and an increased risk of long-term issues such as delinquency, violence, and school drop-out (Lyon et al., 1996; Carpenter et al., 2008). These difficulties stem from an inability to navigate social rules and build supportive relationships within various environments.

Definition of Autism

  • Autism, or ASD, is a neurodevelopmental disorder characterized by a diverse presentation of symptoms that vary significantly among individuals. Key diagnostic criteria, as outlined in the American Psychiatric Association's Diagnostic and Statistical Manual of Mental Disorders (DSM-IV, 2000, cited as Siegel, 1996; American Psychiatric Association, 2000), include notable impairments in social interaction and communication, alongside restricted, repetitive patterns of behavior, interests, or activities. These restricted patterns might include repetitive motor mannerisms, adherence to routines, highly fixated interests, and unusual sensory sensitivities.

  • Comparative Differences: Compared to their typically developing peers, children with ASD frequently show fewer social initiations, engage in shorter and less complex social interactions, and tend to partake in solitary or parallel play more often. Qualitatively, their interactions may lack shared enjoyment, show atypical use of gestures, and demonstrate difficulties with reciprocal communication (Kennedy and Shukla, 1995; Lord and Magill-Evans, 1995; McGee et al., 1997; Sigman and Ruskin, 1999).

  • Incidence: The prevalence of ASD has been steadily rising over the past few decades, with estimates at the time of this paper placing it around 1 in 110 children (CDC, 2009). This increase is attributed to a combination of factors including heightened public awareness, broader diagnostic criteria, and improved diagnostic practices.

Importance of Assessment Tools

  • Need for Tools: The increased incidence of ASD underscores the critical need for highly effective and reliable assessment tools. Such tools are essential for accurately identifying specific social skill deficits, establishing baseline functioning, tailoring individualized educational programs (IEPs), measuring intervention efficacy, and ultimately demonstrating progress to families and stakeholders.

  • Assessment Methods: Gresham and Elliott (1984) identified six primary assessment methods for social skills:

    • Behavioral Observation: Direct observation in natural settings.

    • Behavior Rating Scales: Standardized questionnaires completed by informants.

    • Interviewing: Structured or semi-structured discussions with individuals or informants.

    • Self-Report Procedures: Individuals report on their own social behaviors.

    • Projective-Expressive Techniques: Indirect assessment through tasks like drawing or storytelling.

    • Sociometrics: Peer-rejection/acceptance measures.

  • Behavioral Observation: This method involves direct, systematic observation of a child's social behaviors in their natural environment. While offering high ecological validity and detailed qualitative data, it is often resource-intensive, requires extensive training for observers, and can be influenced by the child's reactivity to being observed.

  • Behavior Rating Scales: These scales are completed by informants (e.g., teachers, parents) and offer a user-friendly, time-efficient approach to assessment. They require less specialized training than direct observation and have generally demonstrated robust psychometric properties (Merrell, 2001), making them a practical choice for initial screening and routine monitoring.

Evaluation of Behavior Rating Scales

  • A comprehensive review highlighted the SSRS as one of the most practical and broad-ranging tools available for assessing social skills in young children (Demaray et al., 1995). The PKBS was developed with similar objectives, specifically targeting preschool-aged children. Both instruments cover a wide array of social competencies and common problem behaviors.

  • Use in ASD: Despite their widespread use in general child populations, the psychometric properties of SSRS and PKBS, particularly when applied to children specifically diagnosed with ASD, remained largely under-explored. This gap is significant because tools standardized on neurotypical populations may not accurately capture or measure the unique manifestations of social skills deficits in children with ASD.

Study Purpose

  • Objectives:

    1. To rigorously investigate the psychometric properties—including internal consistency, construct validity, convergent validity, and criterion validity—of the SSRS and PKBS when used with young children with ASD in real-world educational settings. This objective aimed to confirm the reliability and validity of these instruments for this specific population.

    2. To critically assess the measures' sensitivity to detecting meaningful changes in social behavior following a period of intervention. This was crucial for determining the scales' utility in monitoring progress longitudinally and evaluating the effectiveness of social skills interventions.

Method

Participants

  • Demographics: The study cohort comprised 22 children (20 males, 2 females), ranging in age from 36 to 76 months, with a mean age of 56.556.5 months (approximately 4 years and 8 months). This gender ratio is consistent with prevalence trends in ASD, where males are diagnosed more frequently than females.

  • Diagnoses: All participants had received a confirmed diagnosis of autism or pervasive developmental delay (PDD), established by qualified external evaluators. This diagnostic rigor, often involving standardized instruments like the Autism Diagnostic Observation Schedule (ADOS) or the Autism Diagnostic Interview-Revised (ADI-R), ensured a homogeneous and clinically relevant sample.

  • Program: Participants were actively enrolled in Project DATA, an inclusive early childhood program renowned for providing individualized instruction and comprehensive support within an integrated educational setting. Project DATA typically incorporates evidence-based practices, such as applied behavior analysis (ABA) principles and peer-mediated strategies, to promote social and developmental growth.

Instruments

  1. SSRS: The Social Skills Rating System is a 30-item teacher-report scale designed to assess various aspects of social functioning. It comprises three main domains: Social Skills (e.g., cooperation, assertion, self-control), Problem Behavior (e.g., externalizing behaviors, internalizing behaviors, hyperactivity), and Academic Competence. Normative data for the SSRS were initially established from a large representative sample of 4,170 children.

    • Psychometric Properties (General Population):

      • Internal consistency (reliability of items within a scale): r=0.82−0.94r=0.82−0.94 across subscales, indicating good to excellent reliability.

      • Test-retest reliability (consistency over time): r=0.75−0.88r=0.75−0.88, suggesting stable scores across repeated administrations.

  2. PKBS: The Preschool and Kindergarten Behavior Scales is a 76-item teacher-report scale specifically developed to examine social skills and problem behavior in children aged 3-6 years. It is divided into two primary scales: Social Skills (e.g., social cooperation, social interaction, social independence) and Problem Behavior (e.g., internalizing, externalizing, critical items).

    • Psychometric Properties (General Population):

      • Internal consistency: r=0.96r=0.96, demonstrating exceptionally high internal consistency.

      • Test-retest reliability: r=0.69−0.78r=0.69−0.78, indicating substantial stability over time for this age group.

  3. AEPS: The Assessment, Evaluation, and Programming System for Infants and Children (AEPS) is a comprehensive, criterion-referenced assessment and intervention system. It covers various developmental domains, including a dedicated social domain scale. The AEPS is particularly valued for its observational, curriculum-based approach, which makes it highly sensitive to tracking incremental progress and is widely used by practitioners for educational planning and outcome monitoring.

Procedure

  • Rating Process: Both the SSRS and PKBS were completed by the participants' primary classroom teachers. Ratings were conducted at two distinct time points: a pre-test phase (in the autumn) and a post-test phase approximately 7 months later (in the spring). This longitudinal design allowed for the assessment of change over a period sufficient for social skill interventions to potentially manifest effects, aligning with a typical academic year. Teachers were selected as informants due to their extensive daily interaction with the children.

Data Analysis

  • Methods: Statistical Package for the Social Sciences (SPSS) software was utilized to perform the data analysis. The specific statistical techniques employed included:

    • Calculation of internal consistency using Cronbach's alpha (αα) to assess the homogeneity and reliability of the scale items for the ASD population.

    • Validity tests conducted through Pearson's correlation coefficients (r) to examine construct, convergent, and criterion validity, evaluating relationships between different measures and subscales.

    • Descriptive analysis of means and standard deviations to summarize the central tendency and variability of scores at pre-test and post-test.

    • Paired t-tests were used to assess statistically significant differences in pre-test and post-test scores, allowing for within-subject comparisons of social skill changes over time, typically with a significance level set at p<0.05p<0.05.

Results

Psychometric Properties (Within ASD Population)

  • Internal Consistency: The internal consistency coefficients were found to be exceptionally high for all instruments within the ASD sample: SSRS (α=0.96α=0.96), PKBS (α=0.96α=0.96), and AEPS (α=0.98α=0.98). These values indicate excellent item homogeneity and reliability, suggesting that the items within each scale consistently measure the same underlying construct in children with ASD.

  • Construct Validity: High correlations were observed within the subscales of both SSRS and PKBS, affirming strong construct validity. This indicates that the respective subscales are indeed measuring the intended social skill and problem behavior constructs in this population.

  • Convergent Validity: Significant, strong positive correlations were found between the SSRS and PKBS, with coefficients ranging from r=0.59r=0.59 to r=0.95r=0.95. This robust relationship suggests that the two scales measure similar underlying social skill and problem behavior constructs, thereby providing convergent evidence of their joint validity in assessing children with ASD.

  • Criterion Validity: Both the SSRS (r=0.83r=0.83) and PKBS (r=0.88r=0.88) demonstrated strong positive correlations with the AEPS social domain. This indicates strong concurrent validity, meaning the teacher-reported rating scales align very well with an independent, observational-based measure (AEPS) of social functioning in children with ASD.

Descriptive Statistics

  • Pre-test and Post-test Scores:

    • SSRS scores (Social Skills domain): The mean pre-test score was 24.524.5, which showed a negligible change to a post-test mean of 24.124.1. This indicates virtually no perceived improvement on this scale.

    • PKBS scores (Social Skills domain): The mean pre-test score was 56.256.2, slightly increasing to a post-test mean of 57.357.3. This also suggests minimal perceived change by teachers on the PKBS.

    • AEPS scores (Social Domain): In contrast, the AEPS scores confirmed a progression from a pre-test mean of 56.556.5 to a post-test mean of 61.461.4. This observable increase suggests that children did, in fact, make developmental progress in social skills, as measured by a direct observational assessment.

Sensitivity to Change

  • Despite the observed progression on the AEPS, neither the SSRS nor the PKBS rating scale demonstrated a statistically significant change in scores from pre-test to post-test (i.e., paired t-tests yielded non-significant p-values). This critical finding highlights a disparity between teacher perceptions (as captured by SSRS and PKBS) and actual developmental progress (as captured by AEPS), raising serious questions about the sensitivity of SSRS and PKBS to detect meaningful social skills improvements in young children with ASD over time.

Discussion

  • Strengths of SSRS and PKBS: The study confirmed that SSRS and PKBS possess high internal consistency and strong validity for assessing social skills in young children with ASD. This makes them suitable tools for initial screening, diagnosis confirmation, and establishing baseline levels of social functioning. They provide a reliable 'snapshot' of a child's social profile at a given point.

  • Limitations: Several limitations were identified: the SSRS was originally norm-referenced predominantly for older children, which might make it inherently less sensitive to the nuanced developmental changes in younger children with ASD. Additionally, both scales rely solely on teacher assessments, thereby excluding other crucial perspectives such as parent reports, which can offer valuable insights into social behaviors in different contexts. The response options on the scales might also be too broad to capture subtle improvements.

  • Implementation Suggestions: Given their demonstrated psychometric strengths, SSRS and PKBS are best implemented as part of a comprehensive, multi-method assessment battery for young children with ASD. This approach should involve integrating data from direct behavioral observation (like AEPS), parent reports, and clinical interviews to triangulate findings, minimize informant bias, and provide a more holistic and accurate understanding of a child's social competencies and challenges.

Limitations of the Study

  • The study's conclusions are constrained by several limitations. The small sample size of 22 participants significantly limits the generalizability of the findings to the broader ASD population. The exclusive reliance on teacher reports, without incorporating other informants, introduces potential subjectivity and lack of a complete ecological picture. The absence of inter-rater reliability checks means consistency across different teachers could not be verified, potentially affecting the reliability of the scores. Furthermore, the timeframe of approximately 7 months, while aligning with an academic year, may be too short to capture statistically significant developmental changes on these particular scales for children with ASD, whose progress can often be gradual and incremental. Potential ceiling or floor effects on the scales may also have masked genuine improvements.

Future Research Directions

  • Future research should aim to explore larger and more diverse samples of children with ASD to enhance statistical power and generalizability. Incorporating parent reports, in addition to teacher assessments, is crucial for gaining a comprehensive 360-degree view of social skills across different environments (home and school). Investigating the performance of newly developed scales specifically designed for children with ASD or those that emphasize incremental change would also be beneficial. Finally, research should focus on how these assessment tools—or improved versions—can be directly utilized to meaningfully inform and guide the development of individualized intervention planning and track its effectiveness with greater precision.

References

  • Achenbach, T. M., American Psychiatric Association, and numerous studies cited throughout the paper to substantiate claims, definitions, and measures employed in the research.