Open Athena completes main pretraining of Marin 535B-A23B and releases Mochi model

ME News · 2026-11-06T04:35:58.000Z

ME AI reports that Open Athena held its first community meetup at Fulcrum Science in New York, where Will Held presented progress on Marin pretraining. The team has officially completed the main pretraining phase for Marin 535B-A23B, a mixture-of-experts model named Mochi. The article argues that Marin’s core value lies in reviving traditional scientific methods: publishing not only model weights, but also reproducible data, preregistered experiments, and transparent decision-making processes. The team livestreamed its largest-scale pretraining run, showing real-time loss curves and distinguishing between the forecast line and the tuning-correction line. The process focused mainly on optimizing kernel speed and fine-tuning optimizer dynamics, with no loss spikes throughout. The team positions itself as transparently scaling up the most fundamental methods, creating research questions and artifacts for the academic community. Next, the team will move on to post-training and plans to hold an in-person event in San Francisco. (Source: ME)

ME AI 消息，Open Athena 在纽约 Fulcrum Science 举办首次社区聚会，Will Held 在会上介绍了 Marin 预训练进展。团队已正式完成 Marin 535B-A23B 的主预训练阶段，该模型名为 Mochi，属于混合专家模型。 文中观点认为，Marin 的核心价值在于恢复传统科学方法：不仅发布权重，还提供可复现数据、预注册实验和透明的决策过程。团队对最大规模预训练过程进行了直播，展示实时损失曲线，并区分预测线与调优修正线；过程中主要优化内核速度并微调优化器动态，全程未出现损失尖峰。团队将自身定位为透明地放大最基础的方法，为学术界创造可研究的问题和工件。下一步将进入后训练阶段，并计划在旧金山举办线下活动。（来源：ME）