AI "Hallucinations" at Romania's Tax Authority Lead to Rejection of Appeals with Non-Existent Laws

The Romanian National Agency for Fiscal Administration (ANAF) is embracing artificial intelligence in its operations, particularly in resolving tax disputes. However, this technological advancement is marred by a significant issue: AI "hallucinations." These are instances where the AI generates fabricated legal arguments, citing non-existent laws, court rulings, or legal precedents. This is deeply concerning because these fabricated justifications are then used by ANAF to reject appeals filed by taxpayers.

“

The use by ANAF of artificial intelligence systems that generate unreal legal 'arguments' - a phenomenon technical known as AI hallucination - is not a simple technological imperfection. It is a systemic vulnerability, capable of vitiating administrative-fiscal decisions to the point of nullity, with direct consequences for taxpayers and the credibility of the state as a tax administrator.

Legal professionals are sounding the alarm, warning that such errors pose a grave threat to taxpayers' fundamental right to a fair defense. The reliance on AI-generated, inaccurate information not only complicates the already intricate fiscal processes but also risks invalidating administrative decisions. As lawyer Elena Grecu stated, "The use by ANAF of artificial intelligence systems that generate unreal legal 'arguments' - a technical phenomenon known as AI hallucination - is not a simple technological imperfection. It is a systemic vulnerability, capable of vitiating administrative-fiscal decisions to the point of nullity, with direct consequences for taxpayers and the credibility of the state as a tax administrator."

“

such errors can seriously affect taxpayers' right to a fair defense and unnecessarily complicate fiscal processes, especially when automatically generated arguments are used to reject taxpayer appeals.

This situation raises serious questions about ANAF's implementation of AI and its compliance with regulations like the EU's AI Act (Regulation (EU) 2024/1689). The AI Act classifies systems used by public authorities that significantly impact fundamental rights as high-risk. ANAF's use of AI for individual tax situations, risk assessment, and decision-making falls squarely into this category. This classification entails precise obligations for the deployer, including ensuring accuracy, robustness, effective human oversight, and transparency. The current practice of using AI-generated, unverifiable legal arguments directly violates these requirements, turning the promised benefits of AI into a source of systemic risk and undermining trust in the fiscal administration.

“

the AI Act classifies systems used by public authorities with a significant impact on fundamental rights as high-risk.