Ravid Shwartz-Ziv compared Karpathy's autonomous research approach with established Bayesian optimization and found that the traditional optimizer achieved a slightly better result faster in his experiment.

Format Web Page

Location PART 3 - THE THIRD AXIS CHAPTER 15: DEPLOY NOW, PRAY LATER How Code Diverges From Prose

"Ravid Shwartz-Ziv compared Karpathy’s autonomous research approach with established Bayesian optimization and found that the traditional optimizer achieved a slightly better result faster in his experiment."

Shwartz-Ziv reported an 85-trial comparison using Optuna's Tree-structured Parzen Estimator on selected nanochat training hyperparameters. His eight-parameter search improved validation bits-per-byte more quickly and reached a slightly better result in the reported comparison. The experiment demonstrates the value of comparing autonomous agent workflows with established optimization techniques on clearly measurable tasks.

View Source

Source title Karpathy's LLM agent vs Bayesian optimization

Source authors Ravid Shwartz Ziv

Source year 2026

Confidence mid

Importance mid