A storage appliance that shipped before AVX2 existed now runs Google's Gemma 4 26B in my basement, no GPU. Here's how we got a modern MoE model onto a CPU older than the instructions its inference engine assumed were there.
I am playing with olmo-3:7b and I found that my PC gives me 9 tokens per second.
When I looked up a graphics card that could handle it, I found out that it one would only cost me like 0.4 Monero, which I thought was going to be a lot worse.
I am playing with olmo-3:7b and I found that my PC gives me 9 tokens per second.
When I looked up a graphics card that could handle it, I found out that it one would only cost me like 0.4 Monero, which I thought was going to be a lot worse.