"Artificial neural network parameter counts have expanded into the trillions, including sparsely activated models developed by Google researchers."
The researchers demonstrated the training of sparsely activated models containing up to one trillion parameters. Their work establishes the extraordinary growth of artificial neural network scale while highlighting that parameter count and computational activity are distinct quantities, an important distinction when interpreting biological brain comparisons.
View Source