Recurrent Neural Networks (RNNs) are gaining a lot of attention in recent years because it has shown great promise in many natural language processing tasks.
The following table summarizes the different recurrent neural network structures and the problems they are good at solving. The structure diagrams are pretty intuitive for understanding what kind of network structure is good for your problem.
For deep learning beginners, choosing the right tool can help learning. There are many different tools available and it is confusing to select one at the beginning. The following diagram shows the popular tools.
Deep learning has gained a lot of attention because it is particularly good at some type of learning which is very useful for real-world applications. Running some simple examples is a good way to start learning this technique. Setting up a development environment is the first step.
I have spent a few days hand-rolling neural networks such as CNN and RNN. This post shows my notes of neural network backpropagation derivation. The derivation of Backpropagation is one of the most complicated algorithms in machine learning. There are many resources for understanding how to compute gradients using backpropagation. But in my opinion, most of them lack a simple example to demonstrate the problem and walk through the algorithm.