RNNs, LSTMs, GRUs, and CNNs for text
So far, we have discussed how to represent text in a way that is digestible for the model; in this section, we will discuss how to analyze the text once a representation has been obtained. Traditionally, once we obtained a representation of the text, it was fed to models such as naïve Bayes or even algorithms such as logistic regression. The success of neural networks has made these machine learning algorithms outdated. In this section, we will discuss deep learning models that can be used for various tasks.
RNNs
The problem with classical neural networks is that they have no memory. This is especially problematic for time series and text inputs. In a sequence of words t, the word w at time t depends on the w at time t-1. In fact, in a sentence, the last word is often dependent on several words in the sentence. Therefore, we want an NN model that maintains a memory of previous inputs. An RNN maintains an internal state that maintains...
