WhatsApp is rolling out an optional new tool to help flag potential scams while safeguarding the native privacy of the service. Scam Alert is an AI-powered feature that uses an on-device machine learning model to analyze new messages from unknown contacts and identify behaviors typical of scams. The service is designed to function entirely on the device, preventing any message data from leaving the user’s phone for classification, although nothing will be automatically reported to WhatsApp, Meta, or any third parties. The feature is in a test phase for now. Once turned on, the model is downloaded onto the device and passively scans the messages in the background.
If it finds that a conversation seems very likely to be a scam, the user will see a warning message within the chat window that is not viewable by the other person. The user then can take one of several actions: block the sender, report the chat as spam, ignore and move on or trust the contact so that they won’t be flagged again. Trusting a contact, by the way, is also the only way the user can optionally send the last couple of messages in the chat directly to WhatsApp to help improve the model. Meta has presented Scam Alert as a tool that complements WhatsApp’s end-to-end encryption, rather than undermines it. The model resides completely on the device, analysis does not reveal message contents to the server, it cannot share information without user input.
For over a decade, these characteristics have appealed to privacy and safety advocates frustrated by the persistent occurrence of scams on messaging platforms. Messaging scams have always been a global occurrence, with fraudsters engaged in everything from social engineering schemes, to fake investment opportunities, to romance scams, and coercion into sending money through messaging threads from unknown numbers. By training the model against N contacts and known scams, WhatsApp intends to identify many of these before they are shared. The technical implementation embodies the wider trend in the tech industry toward increasing AI capability on the device. On device running enables that capacity to be used without network access for each classification and to keep the conversation data on the phone.
Meta has also released some early technical specifications and opened its security bug bounty program to allow researchers’ scrutiny of the system ahead of wider rollout, as a means of reassuring users that it is set out and does not introduce additional privacy risks. To general users, its practical usefulness hinges on the accuracy of the alerts. False positives may be frustrating if valid messages from new contacts are meanwhile classified as scams; false negatives might reduce confidence in this new application.
Allowing chats to be trusted and giving the option of disabling Scam Alert altogether provide users with moderating controls which should allow the feature to be tailored to consumer usage patterns. As the model learns from existing scam report data, its accuracy should increase over time.
