"Code came out more repetitive than English, with entropy sinking under two bits per token in places where English models were still hovering close to eight."
Figure 1 reports Java cross-entropy below two bits per token in some experiments, while the corresponding English measurements remained above seven bits. The comparison shows why statistical prediction can work particularly well for source code and supports the section's opening distinction between programming and natural-language writing.
View Source