Tiny Software 'Colibrì' Lifts Giant AI Brain on Laptops

A tiny bird is trying to lift an elephant. This is the impression left by Colibrì, a software project that allows a "cricket-class" computer to run the "giant AI brain" GLM-5.2. The GLM-5.2 AI model boasts around 744 billion parameters, requiring about 370 GB even after compression.

Remarkably, Colibrì can run this massive AI on a 12-core laptop with only 25 GB of RAM. It achieves this feat by not loading all parameters into memory at once. Instead, it retrieves only the currently needed parts, keeping the rest stored on SSD or NVMe storage, calling them up as required. This technique is particularly effective for AI models like GLM-5.2 that use a "Mixture-of-Experts" (MoE) approach, similar to a large company only calling in employees with the specific skills needed for a task.

Traditionally, running such large AI models demands computers with immense memory or multiple expensive high-end graphics cards. Colibrì takes a different path, intelligently managing resources between RAM, VRAM, and storage. The project's name, "Colibrì" (Italian for hummingbird), serves as a fitting metaphor. Hummingbirds, despite their small size, possess extraordinary flight capabilities, achieving agility through unique movement and constant energy intake, rather than sheer size.

Colibrì mirrors this by not confronting giant AI models with equally giant hardware. It navigates between different computer components, fetching what's needed and leaving the rest, moving with intelligence rather than brute force. This experiment proves that limitations can be overcome not always by increasing power, but often by finding new ways through creativity and ingenuity. It offers a broader lesson: limitations don't always require more strength; they can compel us to discover novel solutions.