Andrew Rossignol has done what I was sure was impossible:
I have been diving into the world of large language models (LLMs), and a question began to gnaw at me: could I bring the cutting-edge of AI to the nostalgic glow of my trusty 2005 PowerBook G4? Armed with a 1.5GHz processor, a full gigabyte of RAM, and a limiting 32-bit address space, I embarked on an experiment that actually yielded results. I have successfully managed to achieve LLM inference on this classic piece of Apple history, proving that even yesteryear’s hardware can have a taste of tomorrow’s AI.
Andrew has the TinyStories 110M Llama2 LLM up and running inference on a 12-inch PowerBook. As hot as those things got running AppleWorks, I am in awe that he got this working. I’ve got a couple of 12-inch PowerBooks… maybe I should get one off the shelf…