Leslie Matisse knows her daughter said it was wrong. However, she didn’t expect a 13-year-old girl to be arrested for that.
The teenage girl makes an offensive joke while chatting with her classmates online, School Surveillance Software.
Before the morning was over, the Tennessee eighth grader had been arrested. She was questioned, strip searched, and spent the night in prison cell, her mother says.
Earlier that day, her friends teased the teen about her tanned complexion, calling her “Mexican” despite her not. When a friend asked what she was planning on Thursday, she wrote: “I’ll kill everything in Mexico on Thursday.”
Mattis said the comments were “wrong” and “silly”, but the context showed that they were not a threat.
“It made me feel, is this America we live in?” Mattis said of her daughter’s arrest. “And it was this stupid, stupid technique that didn’t pick up random words and look at the context.”
American school surveillance systems are increasingly monitoring everything students write on school accounts and devices. Thousands of school districts We use software such as Gaggle and Lightspeed Alert across the country to track children’s online activities and look for signs that could hurt them and others. With the help of artificial intelligencetechnology can immerse yourself in online conversations and immediately notify both school officials and law enforcement agencies.
Educators say the technology saved lives. However, critics warn that careless words can criminalize children.
“It’s a routine access and presence to law enforcement in students’ lives, including at home,” says Elizabeth Laird, director of the Center for Democracy Technology.
Schools increase vigilance due to threats
In countries that are tired of school shootings, several states hold more difficult boundaries about school threats. Among them is Tennessee. Tennessee has passed the Zero Tolerance Act of 2023, requiring law enforcement to immediately report the threat of mass violence against schools.
The 13-year-old girl, arrested in August 2023, was texting her friend about the chat feature tied to school emails at Fairview Middle School, which uses Gaggle to monitor student accounts. (The Associated Press withheld the girl’s name to protect her privacy. The district did not respond to requests for comment.)
Taken to prison, the teen was questioned and stripped searched, and her parents were not allowed to talk to her until the next day. Litigation They opposed the school system. She didn’t know why her parents weren’t there.
“She then said, ‘I thought you hated me.’ said Mattis, the girl’s mother.
The court ordered eight weeks of house arrest, psychological assessments and 20 days at an alternative school for girls.
Gaggle CEO Jeff Patterson said in an interview that the school system did not use Gaggle in the way it intended. The goal is to find early warning signs and intervene before the issue escalates to law enforcement, he said.
“I would like to see it treated as a teaching moment, not a law enforcement moment,” Patterson said.
Private student chat faces unexpected scrutiny
Shahar Pasch, a Florida education lawyer, said students who think they are chatting with friends personally often don’t realize they are under constant surveillance.
The teenage girl she represented joked about the school shooting in her private Snapchat story. Snapchat’s auto-detection software picked up the comments, the company warned the FBI, and the girl was arrested on the school grounds within hours.
16-year-old Alexa Manganiotis said she was surprised at how quickly software monitoring works. Last year, she was piloted by Drey Foods School of the Arts in West Palm Beach, where she attends. Light Speed Alertmonitoring program. Interviewing a teacher at her school’s newspaper, Alexa discovers that two students once entered something they were threatening about the teacher on a school computer and deleted it. Lightspeed picked it up and “They were taken away five minutes later,” Alexa said.
Teens face more sudden consequences than adults for what they write online, Alexa said.
“If an adult makes a superracist joke that threatens on a computer, they can be removed and they won’t be arrested,” she said.
Amy Bennett, Chief of Staff at Lightspeed Systems, said the software will help understaffed schools “become aggressive rather than punitive” by identifying early warning signs of bullying, self-harm, violence or abuse.
The technology can also involve law enforcement in responding to a mental health crisis. At Polk County Schools in Florida, a district of over 100,000 students, the school’s safety program received nearly 500 gagle alerts over four years, officials said at a meeting of the Public School Board. This led to 72 unwilling hospitalization cases under the Baker Act. This is a state law that allows authorities to request people’s mental health assessments against their will if they pose a risk to themselves or others.
“So many kids who go through unwilling exams remember it as a truly traumatic and harmful experience. It’s not useful for their mental health care,” said Sam Boyd, an attorney at the Southern Poverty Law Center. The Pork and West Palm Beach School Districts did not provide comment.
Analysis shows a high rate of false alarms
Information that could allow schools to assess the effectiveness of the software, such as the rate of false alerts, is not publicly available unless technology companies are closely held and tracked the school itself.
Gaggle has warned the Lawrence, Kansas school district of over 1,200 incidents in the last 10 months. However, almost two-thirds of these alerts were considered non-problem by school staff, including over 200 false alarms from civil servant homework.
One student Photo Class Gaggle was called to the principal’s office for concerns that he detected nude. The photo was automatically deleted from the student’s Google Drive, but students with a backup of the flagged image on their device showed that it was an incorrect alarm. District officials later said they adjusted the software settings to reduce false alerts.
Natasha Torkaban, a 2024 graduate, said that because of the term “mental health,” he was flagged for compiling a friend’s university essay.
“Ideally, I don’t think we’ll stick to AI’s new, shiny solution to the deep issues of teenage mental health and suicide rates in America, but that’s where we’re at,” Torkzaban says. She was among a group of student journalists and artists at Lawrence High School. Litigation Gaggle claimed they were subjected to unconstitutional surveillance over the school system last week.
School officials say they are seriously concerned about Gagle, but say the technology has detected dozens of imminent suicides and threats of violence.
“Sometimes we need to look at trade for greater profits,” said Anne Costello, a member of the school board at the July 2024 board meeting.
Two years after the trial, Mattis says her daughter is getting better, but she is still “terrified” to come across one of the school officials who arrested her. According to her, one bright spot was the compassion of a teacher at her daughter’s alternative school. Every day, they took time to let their children share their feelings and frustration without judging them.
“I don’t just want my kids to be these little soldiers, they don’t,” Mathis said. “They’re just humans.”
___
The report is a dataset created by researchers Tyler Simko, Mirya Holman and Rebecca Johnson courtesy of DistrictView, a school board meeting posted on YouTube.
___
Associated Press Education Compensation receives financial support from several private foundations. AP is solely responsible for all content. Find the AP standard For charity, list of ap.org supporters and funded compensation areas.
