Recurrent neural networks (RNNs) are a type of neural network that can handle sequential data by saving the output of each layer and feeding it back as input. RNNs were created to address issues with feed-forward neural networks, which cannot handle sequential data, only consider the current input, and cannot memorize previous inputs. RNNs have applications in areas like image captioning, time series prediction, natural language processing, and machine translation.