Language Models and Text Prediction
Text predicting tools are very helpful when paired with our common sense. They help us type faster and with fewer mistakes, and some can even suggest emojis or words to better suit the tone of the text!
Essentially, language models have to know the grammar of the language used. To understand the context of the sentence the model also needs to have sufficient general knowledge to successfully predict words. With elaborate neural networks and an enormous amount of training data, they can learn how to anticipate a sentence. Through self-supervised learning, the models can learn about a language in general and transfer the knowledge for another purpose.
