Bigger AI models aren't always better: how a 70B model beat a 280B one by reading more

For a few years, the way to build a better AI model was simple. Make it bigger. Then in 2022...