What is a word vector?Īt one level, it’s simply a vector of weights.

The last two papers give a more detailed explanation of some of the very concisely expressed ideas in the Milokov papers.Ĭheck out the word2vec implementation on Google Code. The third paper (‘Linguistic Regularities…’) describes vector-oriented reasoning based on word vectors and introduces the famous “King – Man + Woman = Queen” example. From the second paper we get more illustrations of the power of word vectors, some additional information on optimisations for the skip-gram model (hierarchical softmax and negative sampling), and a discussion of applying word vectors to phrases.

word2vec Explained: Deriving Mikolov et al’s Negative Sampling Word-Embedding Method – Goldberg and Levy 2014įrom the first of these papers (‘Efficient estimation…’) we get a description of the Continuous Bag-of-Words and Continuous Skip-gram models for learning word vectors (we’ll talk about what a word vector is in a moment…).

word2vec Parameter Learning Explained – Rong 2014.

Linguistic Regularities in Continuous Space Word Representations – Mikolov et al.

Distributed Representations of Words and Phrases and their Compositionality – Mikolov et al.

Efficient Estimation of Word Representations in Vector Space – Mikolov et al.

at Google on efficient vector representations of words (and what you can do with them). For today’s post, I’ve drawn material not just from one paper, but from five! The subject matter is ‘word2vec’ – the work of Mikolov et al.

What is another word for amazing