This article proposes an alternative framework for modeling the stochastic dynamics of mortality rates. A simple age basis combined with two stochastic period factors is used to explain the key mortality drivers, while the remaining structure is modeled via a multivariate autoregressive residuals model. The latter captures the stationary mortality dynamics and introduces dependencies between adjacent age-period cells of the mortality matrix that, among other things, can be structured to capture cohort effects in a transparent manner and incorporate across ages correlations in a natural way. Our approach is compared with models with and without a univariate cohort process. The age- and period-related latent states of the mortality basis are more robust when the residuals surface is modeled via the multivariate time-series model, implying that the process indeed acts independently of the assumed mortality basis. Under the Bayesian paradigm, the posterior distribution of the models is considered to explore coherently the extent of parameter uncertainty. Samples from the posterior predictive distribution are used to project mortality, and an in-depth sensitivity analysis is conducted. The methodology is easily extendable in multiple ways that give a different form and degree of significance to the different components of mortality dynamics.