Muse Spark Open just dropped - 816B total params with 42B active (MoE architecture), 1M context window, native multimodal support (text/image/video/audio). Optimized for Apple Metal, which means local inference on M-series chips without cloud dependency.
The interesting part: it's designed specifically for long-horizon tasks and multi-step reasoning, not just chat. Early tests show it handles complex software engineering workflows better than expected for an open-weight model.
Architecture details are sparse, but 816B total with 42B active suggests a sparse mixture-of-experts setup similar to Mixtral's approach but at much larger scale. The 1M context is table stakes now, but combined with Metal optimization means you can actually use that context locally without melting your GPU.
This is the first serious US-based open-weight competitor in the 100B+ param range that's optimized for edge deployment. If the benchmarks hold up, this changes the local LLM game significantly.
The interesting part: it's designed specifically for long-horizon tasks and multi-step reasoning, not just chat. Early tests show it handles complex software engineering workflows better than expected for an open-weight model.
Architecture details are sparse, but 816B total with 42B active suggests a sparse mixture-of-experts setup similar to Mixtral's approach but at much larger scale. The 1M context is table stakes now, but combined with Metal optimization means you can actually use that context locally without melting your GPU.
This is the first serious US-based open-weight competitor in the 100B+ param range that's optimized for edge deployment. If the benchmarks hold up, this changes the local LLM game significantly.