Your attacker has an AI assistant too — what a security demonstration this month means for the way you approve payments

The same AI assistants that are making finance teams faster are also making fraud faster, creating a dangerous gap where money can disappear. A recent security demonstration highlighted how attackers can use AI to execute sophisticated scams.

During Black Hat week in Las Vegas, security firm Barracuda showcased a step-by-step attack sequence. The demonstration involved compromising an employee's email account and then using the account's own AI assistant to automate malicious actions. These included hiding security alerts, identifying key targets, and drafting convincing phishing emails in the employee's natural writing style.

The attack culminated in a wire transfer fraud. After gaining access to an executive's mailbox, the AI assistant was used to find a pending wire transfer of $247,500. The attackers then had the AI draft an email to the finance team requesting a change in the recipient's banking details. Additional AI-driven rules were implemented to divert confirmation emails and delete incriminating messages, effectively covering the attackers' tracks.

Barracuda emphasized that the demonstration, which used Microsoft Copilot, applies to other widely available AI assistants. Their primary concern is not that these tools create new privileges, but that they "dramatically increase the speed, scale" of attacks, exploiting existing vulnerabilities and human trust. The incident serves as a stark warning to businesses about the evolving threat landscape in the age of AI.

“

the primary risk is not that these tools create new privileges, but that they “dramatically increase the speed, scale