Based on my research, due to the fact that your other Gemma models work is actually the key clue.
Gemma 4 ships in several sizes (E2B, E4B, 12B, 26B, 31B), but the 12B is the "unified" one, it uses a brand new encoder free architecture Google tags as gemma4_unified.
The other sizes use architectures your current oMLX already supports, which is why they load fine. The 12B unified is the only one that needs the newer backend.
It fails on v0.3.12, oMLX v0.3.12 bundles an older mlx-vlm that doesn't know gemma4_unified yet, so the model won't load. The "Internal server error" is just a generic 500, the real error will be in the oMLX log reads:
-Model type gemma4_unified not supported
-No module named 'mlx_vlm...gemma4_unified'
It's not a memory/bf16 issue, the 4-bit and 8-bit versions of the 12B hit the exact same error (other people have confirmed this), so it's not about bf16 being too big. it's purely the architecture.
try this fix that's worked for others:
Update oMLX to v0.4.2 or newer (v0.4.3 is best) from the oMLX releases page on GitHub. That release updates mlx-vlm to 0.6.1, which adds Gemma 4 Unified (12B) support.
Reload gemma-4-12B-it-bf16, it should load and run. (The oMLX author and the mlx maintainers both confirmed this)
Skip the v0.4.2.dev1 prerelease specifically, it loads the model but had a chat glitch that spat gibberish. Just go to the latest stable.
** whenever you see a generic "Internal server error," check the oMLX console/log window. the real Python traceback prints there even though the API only returns 500.
**bf16 is the unquantized version (~24 GB on disk, more in RAM). It'll be fine after the update if you have the memory.
if it feels tight, the 4-bit or 8-bit 12B is lighter and runs the same once you're on 0.4.2+.
let me know if this works!
Stay Frosty,