AUDIE CORNISH, HOST:
So far, investigators have not commented on the online presence of the 17-year-old charged in the massacre Friday at a Texas high school. They haven't spoken publicly about whether there were threats in his posts, but they're obviously being scrutinized. Across the country, hundreds of schools from elementary to college are using technology to detect threats made against them on social media. School leaders who use the software say it can save lives, though it also has critics. From member station WBUR in Boston, Lisa Mullins has the story.
LISA MULLINS, BYLINE: For students and teachers at Shawsheen Tech in Billerica, Mass., this security system is invisible. It's software that scans social media looking for threats against the school.
TIM BROADRICK: It's receiving and filtering and then gives us alerts when certain kinds of public communication are detected.
MULLINS: That's Superintendent Tim Broadrick. His school buys the service for about $10,000 a year from a Vermont-based company called Social Sentinel.
GARY MARGOLIS: We are a home alarm system, so to speak, for social media.
MULLINS: Gary Margolis is Social Sentinel's founder and CEO. He says the software scans about 1 billion public social media posts every day looking for so-called language of harm. The company has built a library of 450,000 such words and phrases. Margolis won't say what they are.
MARGOLIS: We went back, for example, and looked at the language that school shooters have used in the past in various manifestos or that they've shared on social media. And we went to understand similarities and patterns.
MULLINS: The technology isn't foolproof, though. In February, a Shawsheen Tech student was arrested for allegedly distributing a snapchat video in which he threatened a school shooting. Social Sentinel didn't catch it because it isn't allowed to scan Snapchat and doesn't scan videos. But Superintendent Broadrick says the service works. It even detects posts from students who are suicidal.
BROADRICK: One of the alerts that they sent us indicated that one of our students' mental health condition was deteriorating, maybe to the point that that student's life could have been in danger.
MULLINS: It turned out the student had an imminent suicide plan. School counselors got the teenager help. Nearby in Arlington, police get up to 10 Social Sentinel alerts a day by email or by text. Each alert links to a social media post. In one case, there was a student who sounded suicidal and police intervened. Chief Fred Ryan says most of the alerts are benign.
POLICE CHIEF FRED RYAN: Let me just show you. So here's a post.
MULLINS: This tweet from a local man includes the term gun violence. But...
RYAN: This person is obviously advocating for anti-gun policy. It's not a credible threat. We move on.
MULLINS: Social Sentinel no longer sells to police. But Chief Ryan is still a client.
RYAN: This service enables us to be aware without compromising people's expectation of privacy, civil rights and sort of government lurking around social media accounts. That's not what we do.
MULLINS: Kade Crockford challenges that. She directs a technology project at the American Civil Liberties Union of Massachusetts.
KADE CROCKFORD: This is an expansion of the school's ability to police what students are doing inside of school or on campus to their outside of school conduct.
MULLINS: For example, a student tweets about smoking pot. If the post contains one of Social Sentinel's hot-button words, school officials will receive an alert, even if there's no threat. A criminologist who studies mass shootings has another concern with the technology. Northeastern University Professor James Alan Fox says it's extremely difficult to predict who will attack a school. It's still a rare event and thousands of people share their fascination with guns, violent video games and dark song lyrics on social media but most would never turn violent.
JAMES ALAN FOX: We're going to identify lots of people who aren't dangerous, for whom it's they want to be a big shot but they're not going to take a shot. And what's going to happen to them? That suspicion itself can turn their lives in the wrong direction.
MULLINS: Social Sentinel says the company cannot yet show with data how effective the software is but says clients routinely report the technology has helped prevent someone from getting hurt. For NPR News, I'm Lisa Mullins in Boston. Transcript provided by NPR, Copyright NPR.