Keras implementation of Nested LSTMs from the paper Nested LSTMs
From the paper:
Nested LSTMs add depth to LSTMs via nesting as opposed to stacking. The value of a memory cell in an NLSTM is computed by an LSTM cell, which has its own inner memory cell. Nested LSTMs outperform both stacked and single-layer LSTMs with similar numbers of parameters in our experiments on various character-level language modeling tasks, and the inner memories of an LSTM learn longer term dependencies compared with the higher-level units of a stacked LSTM
Via Cells
from nested_lstm import NestedLSTMCell
from keras.layers import RNN
ip = Input(shape=(nb_timesteps, input_dim))
x = RNN(NestedLSTMCell(units=64, depth=2))(ip)
...Via Layer
from nested_lstm import NestedLSTM
ip = Input(shape=(nb_timesteps, input_dim))
x = NestedLSTM(units=64, depth=2)(ip)
...
Keras code heavily derived from the Tensorflow implementation - https://github.com/hannw/nlstm