Solar energy is currently a popular renewable resource, yet limited daily. In green cellular networks, multiple constraints optimization (MCO) problems arise naturally. For example, a typical objective is to control the power transmission of hybrid base stations (connected to both solar panels and electrical grid) in order to maximize user's average throughput, under the constraints of consumed grid energy and user's blocking rate. However, such problems have been generally proved to be NP-hard. In this paper, we formulate this generic MCO problem as a quantized Markovian cost-reward model, with no assumption on input data. We then propose a novel algorithm, namely Cost-constrained Viterbi Algorithm, which recursively returns the optimal policy with linear computational complexity for this model. As an application, we provide engineering rules for the design of hybrid base stations through extensive simulations. In comparison with brute force method for a simple scenario, we find that our algorithm does achieve the constrained optimal policy.