Johannes E. M. Mosig

Machine Learning Researcher @ Rasa

3 posts

Why Rasa uses Sparse Layers in Transformers

By Johannes Mosig and Vladimir Vlasov. Feed forward neural network layers are typically fully connected, or dense. But do we actually need to connect every input…

Johannes E. M. Mosig

Exploring Semantic Map Embeddings / Part II

In this second part of our series on semantic maps, we show how to create them and see how they perform as featurizers for DIET.…

Johannes E. M. Mosig

Exploring Semantic Map Embeddings / Part I

We explore a new sparse text embedding that has some interesting properties.…

Johannes E. M. Mosig