"Once scaled with enough data and compute, Transformer language models demonstrated substantial performance improvements."
Kaplan and colleagues documented empirical relationships between language-model performance, model parameters, dataset size, and training compute. Their research supports the argument that scaling became a central engine of progress in neural language modelling.
View Source