AI Startup Challenges Government's National AI Team Selection

South Korean AI startup Motif Technologies has lodged a formal protest with the Ministry of Science and ICT, questioning the evaluation process for the government's "National AI Foundation Model" project. Despite its AI model, "Motif 3," achieving the top score of 47 points on the Artificial Analysis Intelligence Index (AAII) – a comprehensive global performance metric – the company was eliminated in the second stage of the "Independent AI Foundation Model" initiative.

The AAII score placed Motif 3 ahead of competitors like Upstage (37 points), SK Telecom (35 points), and LG AI Research (31 points). However, Motif was ultimately ranked last in the project's overall assessment, which combined benchmark, expert, and user evaluations. Motif argues that the significant gap in the AAII score should have translated differently in the overall assessment, suggesting the scoring methodology may not adequately reflect performance differences.

Motif has requested the Ministry to disclose the detailed evaluation results, the basis for benchmark scoring, specific criteria and outcomes of expert evaluations, and the methodology for user assessments. The company also raised concerns about potential bias in the expert evaluation, noting a question about foreign capital involvement in major shareholders. They also questioned the user evaluation process, stating their request for a blind assessment was not met, potentially allowing brand recognition to influence the results.

While the Ministry cited Motif's lower scores in usability and applicability as reasons for its elimination, Motif countered that the value of a foundation model lies in its underlying performance, which can be extended to various domains, rather than just chatbot usability. Despite its dissatisfaction with the evaluation, Motif stated it is not seeking to be selected for the "National AI" project and will not participate in the third stage. Instead, the company aims to prove its capability to develop a leading AI model independently and compete globally.

“

We will prove that we are a team that can create the best frontier model in Korea with our own strength and win in global competition.