Noam Shazeer
Transformer co-author who scaled mixture-of-experts to enormous models
A co-author of “Attention Is All You Need”, co-founder of Character.AI and now a lead on Google’s Gemini. His 2017 work revived mixture-of-experts at very large scale.
In the library
Nothing of theirs is filed in the hub yet. Browse the full library.
Recommended next
Hand-picked from the hub based on what Noam Shazeer covers.
The Ultimate MoE Training Handbook
Hugging Face's free interactive technical handbook covers mixture-of-experts training, hardware setup, expert routing, load balancing, distributed training and serving. Intended for readers with machine-learning engineering experience.
Why this: Also about mixture of experts
Introducing Recursive Flint
Recursive introduces Flint, a mixture-of-experts coding model with 40 billion total parameters and 2.5 billion active per token, built to run on local hardware. Benchmark results are the company's own claims.
Why this: Also about mixture of experts