Evidence

Showing from:

All locations
Filter

Alternate views

Google Sheet CSV JSON
Format Online paper / PDF

"Language models learn unevenly from common and rare linguistic patterns, with infrequently encountered tokens receiving less effective generalization."
View Source
Location PART 3 - THE THIRD AXIS CHAPTER 14: AGI & THE SYNTHESIZER Squeezing a Balloon
Source title Mitigating Frequency Bias and Anisotropy in Language Model Pre-Training with Syntactic Smoothing
Source authors Richard Diehl Martinez; Zébulon Goriely; Andrew Caines; Paula Buttery; Lisa Beinborn
Source year 2024
Confidence high
Importance high
Format Online paper / PDF

"Modern large language models are far more powerful than Word2vec, but they still convert written language into numerical representations and learn patterns from text."
View Source
Location PART 3 - THE THIRD AXIS CHAPTER 14: AGI & THE SYNTHESIZER Squeezing a Balloon
Source title Attention Is All You Need
Source authors Ashish Vaswani; Noam Shazeer; Niki Parmar; Jakob Uszkoreit; Llion Jones; Aidan N. Gomez; Lukasz Kaiser; Illia Polosukhin
Source year 2017
Confidence high
Importance high
Format Online paper / PDF

"Researchers found that the same word-vector geometry revealed biases in the corpus, including the association of men with computer programmers and women with homemakers."
View Source
Location PART 3 - THE THIRD AXIS CHAPTER 14: AGI & THE SYNTHESIZER Squeezing a Balloon
Source title Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Source authors Tolga Bolukbasi; Kai-Wei Chang; James Zou; Venkatesh Saligrama; Adam Kalai
Source year 2016
Confidence high
Importance high

Filter