This page contains the sparse word representations used for the experiments of the TACL paper entitled Sparse Coding of Neural Word Embeddings for Multilingual Sequence Labeling.

Update (11/08/2017): The source code used to run the experiments for the paper can be accessed from github

During the experiments we determined sparse word representations based on the dense distributed word representations of the polyglot project using the objective function

where $D$ belongs to the convex set of matrices comprising of unit vectors, $\mathbf{x}_i$ refers to a dense polyglot word representation and $\boldsymbol{\alpha}_i$ corresponds to its sparse counterpart. The files below contain a scipy.sparse.csr_matrix object storing an $\alpha \in \mathbb{R}^{\lvert V \rvert \times 1024}$ sparse matrix for each language (with $\lvert V \rvert$ denoting the size of the vocabulary).

The vocabulary for each language is tied to that of the original polyglot representations, so in order to figure out which row of $\alpha$ corresponds to which vocabulary unit, one should obtain the original polyglot vocabularies. The vocabularies can be downloaded from the project website (together with the dense embeddings themselves). Loading the vocabulary and both the sparse and dense embeddings thus can be performed via:

import pickle
vocabulary, dense_polyglot_embeddings = pickle.load(open(path_to_dense_polyglot_embeddings, 'rb'))
sparse_embeddings = pickle.load(open(path_to_alphas_file, 'rb'))

We also created word representations using the objective functions introduced by Faruqui et al. (2015). This approach comes in two flavors (using an unconstrained and a non-negativity constrained objective function). For more details please refer to the paper.

Brown clusters were also trained for the 25 languages to be found below: