Banning Mobile Phones in Schools: Evidence from Regional-Level Policies in Spain

Introduction

The debate on banning mobile phones in schools is ongoing in many countries.
Governments aim to improve academic performance and reduce bullying through these policies.
This paper analyzes the impact of mobile phone bans on academic achievement and bullying incidence using regional-level data from Spain.
The study focuses on Galicia and Castilla La Mancha (CLM) regions, which implemented mobile phone bans in 2015.
The research uses a quasi-natural experiment approach, comparing the treated regions with the rest of Spain.
One advantage of regional-level data is the comparability of units in terms of institutional and cultural traits.
Galicia and CLM are regions with wealth levels below the Spanish average, making the analysis of non-spending-based policy interventions particularly interesting.
The study uses region-level panel data and compares the treated regions with the rest of Spain before and after the intervention.
For academic outcomes, the study uses PISA scores in maths and sciences from 2006 to 2018.
- PISA (Programme for International Student Assessment) scores are internationally comparable and evaluate competencies and skills.
For bullying analysis, the study uses officially reported cases for every 10,000 school students in three age intervals (6-17 years old) from 2012-2017.
The data on bullying, by region and year, was requested from the Spanish police forces and made public by the Ministry of Education in 2018.
The study also discusses the potential positive and negative effects of mobile devices on education.
- Mobile devices can make children more involved in learning and increase collaboration, but can also lead to distraction and decreased concentration.
The paper also relates to the literature on the impact of technology on students’ outcomes, which is far from conclusive.
Recent studies provide direct evidence on the causal effects of banning mobile phones policies on academic outcomes.
Research on the relationship between mobile phones in schools and bullying is scarce, but smartphones are one of the main conduits for cyberbullying among children.
The study contributes by offering a new perspective by looking at differences between regions rather than differences across schools and students.
The study also checks the bullying effects in different age groups: under-12, 12–14 and from 15–17 years old.
The paper highlights the potential effects of a non-spending-based policy on the educational attainment of middle-school students.
The study addresses the potential effects of these policies in enhancing the school social environment.
The policy analyzed in this paper is a timely issue of primary relevance looking ahead on a future where technology will dominate the workplace.
The study finds that students’ PISA scores in Galicia improved, and bullying incidence fell in the treated regions after the mobile phone ban.

Data and methods

Spain is administratively organized in 17 regions.
The regional governments are autonomous to decide upon the regulation and administration of education.
Two Spanish Regional Governments (CLM and Galicia) passed laws to ban in all the educational centers of primary and secondary stages the use of mobile phones by students as of 2015.
In the rest of the regions, the use of mobile phones is unregulated, in most of the cases allowing each school to decide upon the use of mobile phones.
To conduct the analysis, a region-level panel is created using official sources of data for all the 17-Spanish regions before and after the mobile phones-ban, with the exceptions that we comment below.
We set the year 2015 as the first year where the intervention could have had an effect on our outcome variables.
For the analysis of academic outcomes, use the scores obtained by Spanish school students in the PISA installments from 2006 to 2018.
There are use in total five PISA assessments, corresponding to years 2006, 2009, 2012, 2015 and 2018.
The scoring of every PISA assessment is attributed to the academic achievement of students developed up to the previous year.
A yearly time series of PISA scores is constructed interpolating the scores from one PISA wave to the next.
The series of constructed scores spanning 2006–2017 is used in the analysis.
The CLM region did not participate in two out of the five PISA installments (2006 and 2012).
For the analysis of bullying, the information provided by the Spanish Ministry of Education in 2018 about officially reported cases of school bullying from 2012 to 2017 is used.
The regions of Cataluña and País Vasco did not report this information, and, for this reason, these two regions are not included in our analysis of bullying.
The cases were reported separately for four age intervals, namely, school students 6–8, 9–11, 12–14 and 15–17 years-old.
Three age groups are defined for our analysis below: primary schools students (under-12 years old), and secondary school students, distinguishing in this case the two age groups mentioned.
For each of these age intervals, construct the number of cases for every 10,000 school students of that age.
Three additional covariates are constructed, with across region and yearly variation, to be used in the SCM and DID estimation.
- The first variable is the percentage of children over 10 owning a mobile phone.
- Second, series of public real spending (excluding the financial component) on education in the primary and secondary stages of education per school-student are constructed.
- Finally, series of households’ per capita real disposable income for each region-year are also constructed.
- Nominal variables are deflated using CPI indexes at the region-year level.

Synthetic Control Method (SCM)

Used for analyzing academic outcomes, especially in Galicia due to complete data.
SCM is designed for events or policies affecting a few aggregate units (cities, regions, countries).
Measures intervention effects by comparing the treated unit's outcome variable evolution with similar untreated units.
Requires tracking the outcome variable for treated and untreated units before intervention.
Advantages: Requires only aggregate-level data and solves control unit selection biases.
Constructs a synthetic control as a weighted combination of unaffected units from a donor pool.
The counterfactual outcome $Y_{Nit}$ is estimated as the outcome of the synthetic unit.
Effect on the treated unit $t{c1t}$ is calculated as: $t{c1t} = Y{Nit} - \sum{j=2}^{J+1} v{jt}Y{jt}$
Weights (v) are non-negative and sum to one: $Y{cNjt} = \sum{j=2}^{J+1} v{jt}Y{jt}$
Requires potential predictors of pre-intervention outcome trends (past outcomes, mobile phone usage, education spending, disposable income).
Uses a weighting-matrix, V, for the relative importance of predictors in constructing the synthetic control.
Optimal weighting matrices W and V are found by minimizing the root mean squared prediction error (RMSPE) of the pre-intervention outcome.
Standardized p-values from placebo tests evaluate the significance of estimates.
Model estimated on each untreated unit, removing the actual treated unit from the donor pool.
Significant intervention effect should result in a low probability of comparable effects in other units.
SCM is not applied to CLM due to incomplete PISA data, but DID regression is used instead.

Difference-in-Differences (DID) Analysis

Applied to PISA scores of Galicia and CLM and bullying data.
DID equation: $Y{it} = a + bPostt \cdot Di + gx{it} + di + tt + u_{it}$
- $Y_{it}$ is the outcome variable (PISA scores or bullying cases).
- $Post_t$ is a dummy variable for the implementation year and subsequent years (2015–2017).
- $D_i$ is a dummy variable for the treated region.
- $X_{it}$ contains covariates (mobile phone usage, education spending, per capita income).
- $d_i$ represents region-fixed effects.
- $t_t$ represents year dummies.
- $u_{it}$ is the error term.
Parameter $b$ identifies the treatment effect.
Extended specification includes a term $pre{t-1} \cdot Di$ : $Y{it} = a + b0 pre{t-1} \cdot Di + b1 Postt \cdot Di + gx{it} + di + tt + u_{it}$
- $pre_{t-1}$ is a dummy variable for pre-intervention years.
- Used to rule out differences between treated and control regions before the ban.
- Estimate of $b_0$ should be non-significant for valid DID estimation.
Equations estimated separately for Galicia and CLM.
Bullying analysis includes three age intervals: under-12, 12-14, and 15-17 years old.

Results

Impacts on Academic Performance

Table 2 and Figure 4 display the results for the SCM applied to Galicia.
SCM constructs a synthetic Galicia for PISA results on maths as a combination of Navarra (41.2%), Canarias (21.6%), La Rioja (14.4%), Extremadura (12.8%) and Cataluña (10%);
for the PISA results in sciences, it is a combination of Castilla–León (79%), Islas Baleares (17%) Cataluña (3%) and Madrid (1%).
The SCM estimation exhibits then sparcity in the choice of regions to construct the counterfactual
Close match between the pre- and post-intervention values of the predictors and low pre-intervention prediction error.
Figure 4 visualizes the almost perfect fit between the treated unit (Galicia) and its synthetic counterpart in the pre-intervention period.
After the ban, there is a positive gap in favor of the Galicia region in both PISA indicators.
In maths, this positive gap seems to respond to a combination of increasing scores in the case of Galicia and somewhat decreasing scores in the synthetic Galicia
The estimated effects are of an order of magnitude of around 10.7 and 12.7 points on maths and sciences, respectively, in the year 2017
The p-values derived from the placebo tests in the SCM analysis indicate that for no other region, the SCM finds comparable results to the ones obtained for the treated region.
The estimated effects are equivalent to 0.6–0.8 years of learning in maths and around 0.72 to near one year of learning in sciences.
According to the OECD Report-2010 on the long-run impact of improving PISA outcomes, a modest goal of having all OECD countries boost their average PISA scores by 25 points over the next 20 years would imply an aggregate gain of OECD gross domestic product (GDP) of US$115tn over the lifetime of the generation born in 2010.
Table 3 displays the estimation results of the DID methodology both for Galicia and for CLM. For the sake of brevity, we only present, in this case, the results corresponding to specification
All columns in Table 3 include the full set of covariates also used above, namely, the share of children who have a mobile phone in the region-year, educational public spending and per capita real disposable income of the region-year.
A first observation for the Galicia region, Columns (1) and (2), is that there is no evidence of pre-treatment differences in the PISA scores.
However, on average, over the after-ban period, the academic results in maths increased by more than 6 points in maths and by more than 8 points in sciences.
The estimated magnitude of the effects, which are average estimates over the three-year period after the ban, are broadly comparable to those obtained with the SCM:
- values of around 7.01 and 8.2 for maths and sciences, respectively.
The results for the CLM region are less conclusive. The estimated effects do not render statistical significance in the case of maths, and the pre-intervention dummy turns out to be statistically significant both in maths and in sciences.
our results point out to a positive impact of educational public spending on the academic performance of students in the PISA assessments on maths.
Once all these controls are included in the regressions, no significant effects are found for the region-year levels of per capita real disposable income.

Impacts on Bullying

Tables 4 and 5 display the estimated impacts of the mobile phone ban on officially reported cases of bullying for Galicia and CLM, respectively.
For the under-12 years-old interval, we find no significant treatment effects in neither case: taken as placebo or falsification checks
For school students 12–14 and 15–17 years old the picture is different: the results point to a reduction in bullying after the mobile phones ban.
significant reductions of around 15% to 18% among 12–14 years old students for Galicia and CLM, respectively, and by around 18% to 9.5% among 15–17 years old teenagers for Galicia and CLM, respectively.
The pre-trend effects are not statistically significant in any of the age intervals

Conclusions

The paper highlights the potential effects of a regional-level non-spending-based policy on a fundamental driver of development, such as the skills in maths and sciences of middle- school students.
The study is focused on two low-wealth regions, which have more limited opportunities to rely on large and sustained levels of educational spending.
The implementation of this policy in the two mentioned regions in 2015 constitutes a quasi-natural experiment that we exploit to conduct a comparative-case analysis based on the SCM and DID regressions.
We find that, during the less than three years that the mobile phones ban was in force (from 2015 to 2017), students’ scores in Galicia improved by around 10 points in maths and 12 points in sciences as compared to a synthetic Galicia that had followed exactly the same trend in these scores before the intervention.
We additionally find that bullying incidence fell by around 15% to 18% among 12–14 years old students for Galicia and CLM, respectively, and by around 18% to 9.5% among 15–17 years old teenagers for Galicia and CLM, respectively.
The policy analyzed in this paper is a timely issue of primary relevance looking ahead on a future where technology will dominate the workplace and everything will be connected and data-driven.
there are some limitations in the paper that advise us to interpret our findings as suggestive evidence:
- The first refers to the aggregate nature of the data and, in part as a consequence of this, the modest size of the data samples used in estimation.
- The second derives from the fact that the regulation is not a categorical prohibition of mobile phones in schools but, instead, it gives flexibility to the institutions that want to use mobile phones as a learning tool only.