Analyzing Text Data with Deep Learning
Language is one of the most amazing abilities of human beings; it evolves during the individual’s lifetime and is capable of conveying a message with complex meaning. Language in its natural form is not understandable to machines, and it is extremely challenging to develop an algorithm that can pick up the different nuances. Therefore, in this chapter, we will discuss how to represent text in a form that is digestible by machines.
In natural form, text cannot be directly fed to a deep learning model. In this chapter, we will discuss how text can be represented in a form that can be used by machine learning models. Starting with natural text, we will transform the text into numerical vectors that are increasingly sophisticated (one-hot encoding, bag of words (BoW), term frequency-inverse document frequency (TF-IDF)) until we create vectors of real numbers that represent the meaning of a word (or document) and allow us to conduct operations...
