The effectiveness of NYC's AI gun detection technology is in question. Evolv scanners were implemented at a Bronx hospital in 2022, but their significant false positive rate raises concerns.
NEW YORK, NEW YORK - NOVEMBER 09: People walk along the train platform at the 42nd Street subway station on November 09, 2021, in New York City. On Friday, Congress passed the Infrastructure Investment and Jobs Act, a $1.2 trillion infrastructure package, as part of President Joe Biden's economic agenda. (Photo: Michael M. Santiago/Getty Images)
NYC's AI Gun Detection Systems Face Criticism
New York City's AI gun detection systems prove ineffective, raising concerns over public safety. Initially installed at a Bronx hospital in 2022, Evolv scanners exhibited a significant false positive rate.
Mayor Eric Adams' recent announcement regarding the testing of AI gun detectors at subway stations received praise for its purported effectiveness. However, skepticism from critics, including the Legal Aid Society, has been validated, highlighting potential flaws in the pilot program.
During a recent press briefing, Mayor Adams discussed plans to evaluate the performance of the city's new technology based on various metrics such as hit ratio and false alarms.
Despite positive feedback from stakeholders, Hell Gate reported that concerns arise from existing data revealing high false positive rates from Evolv scanners installed at Jacobi Medical Center in the Bronx in 2022.
During the seven-month trial period at Jacobi Medical Center, approximately 194,000 individuals underwent scanning procedures. Shockingly, the scanners raised alarms in over 50,000 instances, indicating a concerning incident rate of around 26%.
Astonishingly, more than 85% of these alarms, totaling around 43,800, turned out to be false positives. Furthermore, about 14% of the alarms, totaling 7,027, were triggered by law enforcement officers, presumably carrying their service weapons.
Surprisingly, only 295 alarms, comprising a mere 0.57%, were linked to non-law enforcement individuals carrying items such as knives, guns, or unidentified threats labeled as "other."
Concerns Arise Over Accuracy, Effectiveness
Throughout the pilot, the accuracy of Evolv's scanners remained consistently low, with no discernible improvement over time. Surprisingly, not a single month saw the alarm-to-visitor ratio drop below 25%.
In September 2022, the concluding month of the trial, approximately 27,900 visitors underwent scanning, resulting in nearly 7,000 alarms. Shockingly, out of these alarms, only 345 potential threats were identified, a staggering false positive rate of 95%.
Moreover, a mere 0.45% of alerts were associated with non-law enforcement threats. Throughout the entire trial period, a total of 24 guns, 139 knives, and 132 other potential threats were discovered, out of the 50,000 alarms triggered.
Daniel Schwarz, a senior privacy and technology strategist for the New York Civil Liberties Union, described the findings as merely coincidental, suggesting that the discovery of weapons was largely accidental amidst the extensive scanning procedures.
Evolv's AI-powered scanners are marketed as utilizing "safe, ultra-low frequency, electromagnetic fields, and advanced sensors" to uncover hidden weapons. The company's CEO has boasted that these machines can detect a wide array of weapons, including firearms, explosives, and large tactical knives.
However, findings from the 2022 pilot, coupled with industry evaluations, and investigations conducted by regulatory authorities such as the Federal Trade Commission and the Securities and Exchange Commission, as well as a shareholder-initiated class-action lawsuit, paint a less rosy picture of the scanners' effectiveness than Evolv suggests.
