We consider parametric models of partially-observed bivariate Markov chains. If the model is well-specified, we show under quite general conditions that the limiting normalized log-likelihood is maximized only by parameters for which the stationary distribution is the same as the one of the true parameter. This is a key feature for obtaining the consistency of the Maximum Likelihood Estimators (MLE), in cases where the parameter may not be identifiable. The specific cases of Hidden Markov Models and Observation-driven time series are investigated. In contrast with previous approaches, this result is established by relying on the unicity of the invariant distribution of the Markov chain associated to the complete data, regardless its rate of convergence to the equilibrium.