Exploiting Meta's AI Support Bot
Over the weekend, the Instagram accounts for the Obama White House and the Chief Master Sergeant of the U.S. Space Force were briefly defaced with pro-Iranian images and messages. This was made possible by a vulnerability in Meta's AI support bot, which was tricked into resetting account passwords.
A screenshot from a video released on Telegram claimed to show how Meta's AI customer support bot could be tricked into resetting a target's password. The video demonstrated a remarkably simple exploit that involved using a VPN connection with an IP address that is in or near the target's usual hometown, requesting a password reset for the account, and then choosing to chat with Meta's AI support assistant.
How the Exploit Worked
From there, the video shows the attacker told the bot to link the account in question to a new email address, after which the bot dutifully sent that address a one-time code that allowed a password reset. The Telegram account that posted the video also linked to screenshots of pro-Iran images, videos, and messages that defaced the hacked Instagram accounts.
The hackers claimed to have used the exploit to hijack a number of valuable Instagram account names that allegedly have a resale value of more than a half million dollars. Meta has not responded to requests for comment on the video's claims, but Meta's Andy Stone said on Twitter/X that the issue had been resolved and that they were securing impacted accounts.
Response and Aftermath
The security blog thecybersecguru.com reports that Meta pushed an emergency patch over the weekend, and clarified that no back end database was breached. According to thecybersecguru.com, Instagram has notoriously poor human support infrastructure, and recovering a locked account can take weeks of back-and-forth with an automated ticketing system.
Meta's solution was to deploy a conversational AI layer to handle common recovery workflows, such as relinking a lost email address, triggering a password reset, and verifying account ownership. The assistant, presumably, was supposed to reduce friction for legitimate users stuck in account-access hell.
New Security Territory
Ian Goldin, a threat researcher at Lumen's Black Lotus Labs, said we're entering uncharted security territory as more large online platforms start allowing AI chatbots to handle sensitive account recovery requests. Just like human customer support employees can be social engineered into providing unauthorized access to someone's account, AI bots are equally eager to help and vulnerable to persuasion and trickery.
Goldin said,
AI chatbots create interesting new attack surface, and we're likely going to see a lot more of these kinds of attacks.Securing your various online accounts means taking full advantage of the most secure form of multi-factor authentication (MFA) offered, such as a passkey or security key.
In this case, even using the least robust form of MFA that Instagram offers — a one-time code sent via SMS — likely would have blocked the exploit. The hackers who released the video on Telegram said their exploit failed to work against any accounts that had MFA enabled.
- Use the most secure form of multi-factor authentication (MFA) offered, such as a passkey or security key.
- Be cautious when using AI-powered customer support bots, as they can be vulnerable to social engineering attacks.
- Keep your online accounts secure by regularly reviewing and updating your account settings and security measures.
Source: Krebs on Security