AI firm Motif challenges exclusion from national AI project, demands score transparency

Motif Technologies has formally appealed its exclusion from the second phase of South Korea's national AI foundation model project, known as "Dokpamo." The company argues that its disqualification is difficult to comprehend, especially after its "Motif3" model achieved the highest score in the global "Intelligence Index" benchmark by Artificial Analysis (AA), outperforming competitors like Upstage, SK Telecom, and LG AI Research.

Despite scoring 47 points on the AAII benchmark, significantly higher than LG AI Research's 31 points, Motif was eliminated in the comprehensive evaluation phase. The company claims this outcome is inconsistent with its strong performance in objective, global benchmarks. Motif is demanding that the Ministry of Science and ICT release detailed scores for each evaluation item, the basis for assigning weight to global benchmarks, the specific criteria for expert evaluations, and the methodology used for user and blind assessments.

“

We formally filed an appeal regarding the selection results of the second phase of the independent AI foundation model project (Dokpamo). We respectfully request the detailed disclosure of evaluation scores and a review.

Motif's appeal highlights concerns about the evaluation methodology, particularly the conversion of benchmark scores into final weighted points. While Motif3 scored 16 points higher than LG AI Research's model in the raw benchmark, the final weighted score difference was reduced to just 4 points. Motif argues that this compression significantly diminishes the impact of superior benchmark performance.

"This appeal is not to argue that we must be selected for Dokpamo," Motif stated in a press release. "We hope this becomes a process to build consensus on what Dokpamo is for, and to find directions for improvement that both we and other competing companies can agree on."

“

This appeal is not to argue that we must be selected for Dokpamo. We hope this becomes a process to build consensus on what Dokpamo is for, and to find directions for improvement that both we and other competing companies can agree on.