Corporate AI systems harbor hidden vulnerabilities, experts warn of data leaks

As businesses increasingly adopt artificial intelligence, the security of these AI applications is becoming a critical blind spot, according to cybersecurity experts in Taiwan. While AI tools like chatbots and virtual assistants are implemented to cut costs and boost efficiency, their inherent security vulnerabilities are often overlooked.

“

Based on actual red team testing experience, some AI chat systems used by companies have exploitable vulnerabilities.

DEVCORE CEO Weng Hao-cheng highlighted that many companies focus on an AI system's ability to provide correct answers rather than scrutinizing the developer's security testing, access controls, data storage, and encryption mechanisms. "Based on actual red team testing experience, some AI chat systems used by companies have exploitable vulnerabilities," Weng stated. Successful breaches could grant attackers access to all user chat logs, with some unencrypted conversations potentially exposing highly sensitive employee information.

Employees often use these corporate AI tools for routine inquiries about leave policies or overtime, but also to access sensitive data like colleagues' salaries. In one tested case, the AI system actually returned salary information, revealing significant gaps in access control. Weng described AI applications as a "potential time bomb within companies that cannot be ignored," especially since chat logs, necessary for search and historical queries, may not be fully end-to-end encrypted.

“

AI applications have become a 'potential time bomb within companies that cannot be ignored.'

Experts urge companies to broaden their security risk assessments beyond traditional network and application vulnerabilities. Assessments must now encompass large language models (LLMs), APIs, AI applications, and AI agents. Companies need to verify what data AI can access, which internal systems it connects to, its account permissions, and whether AI-generated commands could impact operations. Regular security drills should also incorporate AI attack scenarios, such as data breaches via AI customer service or the impact of malicious commands on AI agents. Given the rapid evolution of AI models and applications, annual security checks are insufficient; continuous monitoring is essential to address emerging risks.

“

Companies should reassess their security risk assessment scope. If a company has already introduced AI, it should simultaneously include large language models (LLMs), APIs, AI applications, AI agents, and their workflows in the inventory and evaluation.