“If you torture the data long enough, it will confess to anything.”
-Ronald H. Coase
A study published in Psychological Science proposed that the large negative effect sizes of smartphone use on school performance seen in previous research may have been overestimated.
The study was conducted by Andreas Bjerre-Nielsen, Asger Anderson, Kelton Minor, and David Lassen from the University of Copenhagen.
From newsstands to Netflix documentaries, when it comes to the topic of technology use, the consensus among popular media outlets seems to be that it does more psychological harm than good. However, in the academic world, there has been some recent pushback from researchers who have found that when analyses are conducted more rigorously, the picture is not so clear.
One such study was conducted in response to a body of literature suggesting that adolescent technology use negatively impacts psychological well-being. Using a specification curve analysis on data from over 350,000 adolescents, the study found that although there was a statistically significant negative effect of tech use on well-being, it was far too small to be practically meaningful (roughly as small as the negative effect they found of eating potatoes on well-being, and smaller than the negative effect of wearing glasses on well-being).
A similar trend has occurred in research on the relationship between smartphone use and academic success. Previous studies have found that the amount of time students spend on their phones negatively predicts academic performance, with the effect being almost twice as strong when phones are being used during class time, specifically. However, much of this work has been cross-sectional in nature, using GPAs from a single course or an average across courses as the dependent variable.
The problem with relying on cross-sectional data, according to Dr. Bjerre-Nielsen and colleagues, is that it fails to account for any unmeasured individual student or course characteristics that may be confounding factors in the relationship between a person’s smartphone use and their success in a given course.
To account for these factors in the current study, the researchers decided they would have to track the same group of students’ smartphone use and academic performance across multiple courses over time. The resulting panel-structured data would allow them to make inferences about specific students across classes, or about specific classes across students.
Like previous work in this area, the researchers hypothesized that they would find a moderate to large negative effect of electronic device use during class on academic performance when their panel data was modeled both cross-sectionally (like in previous research) and with fixed-effects (accounting for unmeasured characteristics).
To test their hypothesis, the researchers recruited 470 university students who volunteered to receive smartphones that constantly measured their phone activity over the 2-year period for which the study ran.
The smartphones participating students were given were able to track whether or not phones were being used in 15-minute increments. Attendance for any given class was assessed using the phones’ built in geo-location data. Academic performance was evaluated using the students specific course grades.
The results supported the researchers’ initial hypothesis. When the data was modeled cross-sectionally (including only one observation per participant, as has been typical of previous research), they found that more in-class smartphone use was associated with markedly worse academic performance.
However, when the researchers took advantage of the more detailed panel data (giving them the ability to control for each student’s and every course’s unmeasured stable characteristics), they found that the magnitude of the association decreased significantly. Though still negative, the effects were much less pronounced (i.e., a 6.1% increase in smartphone use accounted for as little as a 0.045 decrease in standardized GPA).
This research adds to the growing body of academic literature suggesting that the negative effects of technology are not as clear-cut as they tend to be presented in popular media that, ironically, seem to dominate the discourse in social media circles.