Conventionally, deep forecasting models only provide point forecasts. In safety-critical applications such as autonomous vehicles, supply chain planning, and medical planning, understanding uncertainty is essential for decision-making. Therefore, incorporating model uncertainty becomes paramount. Scarcity of data usually impedes the training of complex parametric models that can accurately capture forecast uncertainty without overfitting. Traditional approaches for capturing model uncertainty with non-parametric models (e.g. Gaussian processes) typically do not scale well to high feature dimensions. Recent advancements in Bayesian deep learning alleviate these issues by allowing us to simultaneously capture deep model parameter uncertainty while predicting distributions over outputs.
In this project, we examine the effectiveness of Bayesian deep learning versus traditional autoregressive models for probabilistic, low-data, time series forecasting. Specifically, we investigate driver behavior modeling and how Bayesian autoregressive models can be used to forecast probabilistic driver behavior.
We first baseline fixed-parameter feed-forward and recurrent autoregressive neural networks for predicting future highway driver trajectory distributions given past trajectory and headway information. We then relax the fixed-parameter assumption and determine distributions over recurrent neural network parameter weights that when sampled from, yield an ensemble of networks that can be used to form a distribution over future trajectories. We apply variational Bayesian methods (namely Bayes-by-backprop and Bayes-by-backprop with posterior sharpening) that learn posterior weight distributions by minimizing the variational free energy (the latter introducing latent variable dependencies). Using Monte Carlo estimation, we observe that Bayesian deep learning significantly reduces the trajectory root-weighted-squared-error (RWSE), though all models produce qualitatively accurate trajectory predictions.
