Mochi 535B completed its main pretraining phase with 18 trillion tokens. The team plans long-context work this weekend, SFT next week, and checkpoint release after EU paperwork.
Key points
Main pretraining phase completed
18 trillion tokens processed
Sources
The Ultimate MoE Training Handbook released, 800k GPU hours
The Ultimate MoE Training Handbook is now free and open, created after 800k GPU hours of experiments. It contains ~70k words, 55 interactive figures, and code examples.
Key points
800k GPU hours spent running MoE experiments.
~70k words, 55 interactive figures, and runnable code included.
Available on Hugging Face Spaces.
Sources
- Original post
- huggingface.co - Linked in the post
Anthropic valuation comparison with Google IPO valuation
Anthropic is valued at $1.5 trillion, about 20× its $80 billion ARR, while Google IPO was $23 billion at roughly 7× forward sales. A backer claims the market is missing the next big AI opportunity.
Key points
Anthropic valuation: $1.5 trillion, ~20× $80 billion ARR.
Google IPO valuation: $23 billion, ~7× forward sales.
Sources
Muse Realtime Avatar launches in US Muse app
Muse Realtime Avatar is now available in the US Muse app, providing real-time embodiment for personal agents. It was previewed at Connect and a research blog details the technology.
Key points
Launch: rolled out in the US Muse app.
Preview: shown during Connect.
Sources
- Original post
- go.meta.me - Linked in the post