Probe of school surveillance software finds privacy abuses, inaccurate results

An investigation into the educational technology company GoGuardian revealed the surveillance software used by schools across the country has routinely invaded students’ privacy and incorrectly flagged non-explicit content as harmful, according to an investigation by the Electronic Frontier Foundation.

GoGuardian is used to surveil about 27 million students in 11,500 schools nationwide, according to EFF, which says the software consistently flags innocuous content such as college websites, counseling and therapy sites, and pages with information about drug abuse, LGBTQ+ issues and sexual health.

The company is part of a much larger student surveillance ecosystem that includes companies like Bark and Gaggle. They all provide schools with a vast quantity of student data while incorrectly identifying content, according to EFF, a nonprofit civil liberties group.

EFF designed a system it dubbed the Red Flag Machine to document GoGuardian’s error-prone indexing, relying on real data from the company and presenting users with typically flagged content before asking them which keywords they believed might have caused the flag.

The investigation, released Monday, found that those false positives flagged students for “viewing content that is not only benign, but often educational or informative,” EFF said in a press release. The students were allowed to click through and view the sites, but school administrators were made aware of their search histories.

That content included an array of harmless topics, according to EFF Director of Investigations Dave Maass, including pages about Black authors, the Holocaust and the LGBTQ+ rights movement. Maass added that EFF even detected students flagged for searching “Shark Tank” cast bios and fitness guides for the Marine Corps.

A spokesperson for GoGuardian defended the company’s product, saying in an emailed statement that “schools have a responsibility to keep students safe in digital spaces and on school-issued devices.”

“GoGuardian provides educators with the ability to protect students from harmful or explicit content as required by federal law, while giving access to legitimate education tools,” the spokesperson added, pointing to a company website.

According to EFF, GoGuardian is effectively a “mass surveillance system” that allows school officials to monitor search histories in order to identify “at risk” students. EFF asserts that the company even collects students’ location data and live views of screens, allowing school administrators to create “comprehensive” profiles of students and creating what EFF calls a “stunning invasion of privacy.”

Just last month the ACLU published a similar report assessing how educational technology invades student privacy, asserting that what it called the “EdTech Surveillance industry” Is valued at $3.1 billion a year with a projected 8% annual growth rate.

The ACLU report says school districts’ worries about school shootings, student self-harm and bullying have led to the booming industry’s success.

“The EdTech Surveillance industry deliberately whips up fear around tragic, albeit uncommon events such as school shootings and suicides in order to drive demand for their products,” the ACLU report said.

Student surveillance technologies harm all students, but disproportionately impact students of color; students with disabilities; LGBTQ+ and nonbinary students; undocumented students; and low income students, the ACLU report said.