Training Safe LSTMs with Input-to-State Stability Guarantees

Recurrent neural networks (RNN) have excellent performance with time-series inputs due to their ability to model latent state dynamics, which makes RNNs useful for system identification of nonlinear dynamic systems. Although neural networks are powerful and have a wide range of applications, their lack of rigorous safety guarantees i.e. stability and robustness, makes them rarely used in safety-relevant fields such as medical devices and autonomous driving. Recasting RNNs as a class of nonlinear dynamical systems and viewing RNNs from the perspective of system theory allows us to analyse their stability. Recent work has established a sufficient condition on the network weights to guarantee Input-to-State Stability. The current approach to enforce this condition is computationally infeasible for large networks and datasets, as it requires solving a constrained non-linear optimization problem. To make training of large input-to-state stable LSTMs computationally feasible, we propose a training method and network architecture to ensure input-to-state stability of LSTMs, which retains the computational advantages of backpropagation-based training.