Hand-crank powered AI: CrankGPT runs large language model on Raspberry Pi

The conventional image of artificial intelligence involves massive energy consumption and reliance on cloud computing. However, a project named CrankGPT is challenging this notion by demonstrating that a large language model can run locally on a Raspberry Pi 5, powered entirely by a manual hand crank.

The CrankGPT device, developed by Squeez Labs, aims to address the significant energy demands of modern AI. While future advancements may lead to more efficient algorithms, this project offers a practical, albeit unconventional, solution. The team successfully powered a Raspberry Pi 5, equipped with 8GB of RAM and a cooling fan, using only the hand crank. This allowed it to function as an AI voice assistant.

Building the system involved overcoming the challenge of maintaining a stable power supply from the generator. The Raspberry Pi, especially when running speech recognition and language models, can draw significant power, leading to voltage drops and triggering protective circuits. To counteract this, Squeez Labs designed a custom capacitor board. This board acts as an energy reservoir, smoothing out voltage fluctuations and providing enough stored power to keep the system running during brief interruptions in cranking.

All components of the CrankGPT system operate locally on the Raspberry Pi, eliminating the need for cloud services. Software optimizations, including a minimal DietPi installation and custom-written low-latency code, further enhance its efficiency. Speech recognition is handled by Moonshine ASR, voice activity detection by Silero VAD, and language model responses are generated by smaller models running within llama.cpp. Piper is used for text-to-speech synthesis, chosen for its ability to produce synthesized speech efficiently.

This project highlights a novel approach to AI accessibility, proving that even resource-intensive AI models can be operated off-grid. The CrankGPT system, by using a hand crank as its sole power source, opens up possibilities for AI applications in remote areas or situations where traditional power sources are unavailable.