Meta employees are concerned about security issues with its personal AI agent Muse, which the company is betting on in hopes of generating billions in AI revenue, Reuters reports. The AI agent has been downloaded more than 10 million times since September. In internal messages, Meta employees complain that the bot, designed to make users’ lives easier, accessed encrypted private conversations without permission, made up information about purchases, and sent messages apparently intended for other people, the news agency reported.

What problems did Meta employees uncover

Reuters says it reviewed more than 20 internal complaints about unauthorized actions by Muse. Many were posted in a feedback group where Meta employees report identified problems to the Superintelligence team developing the AI agent.

  • According to the complaints, Muse sent users messages about a childcare request they had never made, career advice they had never asked for, product orders they had never placed, and bank account balances that did not belong to them.

  • For example, in one case, the AI agent inexplicably added this comment to a conversation about the weather and fishing: “By the way, two bank account updates: your balance at Capital One is $2,722.58, and at Charles Schwab it’s $5,899.57.”

  • In another case, a Meta employee discovered that Muse on her personal phone had reminded her to reply to a message from her sister in the encrypted messaging app Signal. “This seems like a serious privacy violation. I definitely didn’t give it access to other apps, and I don’t remember being asked,” the employee wrote in an internal post. A colleague replied that they had also encountered unwanted messages from Muse related to third-party apps, and discovered that the AI agent gets access to push notifications by default on Android phones.

  • Employees also reported other glitches: Muse informed users about nonexistent transactions, overstated the prices of items they had actually bought (causing users to lose real money), and, in one case, booked nonrefundable accommodation on Airbnb without permission.

  • In yet another case, the AI agent unexpectedly offered to “message the seller” about a coffee table the employee wasn’t interested in. When the employee asked Muse what it was talking about, the agent replied that it had “received some information about picking up a table that apparently doesn’t belong to you.” The employee concluded that Muse had pulled in context belonging to another user and expressed concern about a possible information leak between accounts.

  • A member of the Superintelligence team developing Muse said in a feedback group in early November that the agent appeared to be inventing messages in response to built-in prompts to engage with users between sessions. “Unfortunately, these hallucinations most often involve the most unpleasant topics, such as finances/security/protection, which is awful,” he noted.

How Meta responded

Meta spokesperson Daniel Roberts called the examples cited by Reuters “isolated incidents.” He said the company prioritizes safety, conducts extensive internal testing, and continually updates its products to improve protection.

According to him, many of the disputed messages were “hallucinations”—that is, cases where an AI model makes up information—and the issue has already been fixed.

Reuters was unable to determine whether the Muse AI agent had provided users with real data belonging to other people, such as bank account balances, or had simply made up the information.

Context

The incidents involving Muse illustrate the risks inherent in a new category of consumer AI agents entering the market and being touted as the next major wave in the development of artificial intelligence technology, Reuters writes.

Meta is currently ahead of its competitors in this market. The Muse app became one of the fastest-growing apps in history: it was downloaded more than 700,000 times in its first 24 hours and around 900,000 times in its first week. By comparison, ChatGPT’s debut week on iOS in 2023 saw 500,000 downloads. The app is being heavily promoted to the 3.6 billion users of Facebook, Instagram, and WhatsApp. Muse also has a high retention rate: according to Meta, about a quarter of users who create accounts in the app return to it every day.

AI agents need broad access to smartphones and computers to work effectively. However, they are also prone to making mistakes, exploiting obscure vulnerabilities, and pursuing their goals in ways that can sometimes cause harm. For example, leading AI developers OpenAI and Anthropic have reported cases in which their AI agents unexpectedly hacked into other companies’ systems and independently weakened systems designed to monitor their activity. Some banks, retailers, and other businesses have responded to these risks by blocking AI agents from accessing their systems. ㅤ

#HighTechNews