| This dissertation is concerned with heavy traffic and Markov modulated diffusion models that are applied to resource allocation problems in wireless communication system and the numerical analysis for their associated continuous time stochastic control problems. To be specific, the heavy traffic model is a two-dimensional stochastic differential equation with reflection (SDER), and the other model is a second-order Markov modulated diffusion process.;Under the so called "heavy traffic" assumption, SDER has been developed as an approximation model for a multi-buffer and various channel state wireless communication system. We study how the reflection process can affect the solution of the SDER and the resource (reserve power) allocation theoretically and numerically. We have shown that Multi-Completely S is a necessary condition for the existence and uniqueness for the SDER instead of the well known Completely S in the wireline system [70]. Using Markov Chain Approximation (MCA) method [52], various effects of factors, especially the reflection processes (nominal power reallocation) are studied via numerical experiments. After optimal control policies are obtained via MCA method under an appropriate grid size setting, Monte Carlo and real time simulation experiments are done using heavy traffic policies v.s. heuristic wedge control policies.;Secondly, we propose a queue-based Markov Modulated power control model in the thesis. The new model has at lease two advantages compared with previous work. First of all, channel gain is modeled by a finite-state continuous-time Markov chain, see [83,90]; secondly, the whole mobile system is modeled as a stochastic control problem, whose constraints are described by queue dynamics, a second order Markov Modulated (channel gain) diffusion process. Furthermore, we model the affect of the SINR on the queue dynamics through a bit error rate (BER) function, thus our model combines two essential parts: SINR and real transmission rate. Similarly, the corresponding HJB equation is proved through Dynkin's formula [26], then we derive the transition probabilities for a Markov chain approximating the dynamics based on HJB equation and show the convergence of transition probability matrix. Finally, the framework of numerical method for the stochastic control problem is provided using Markov Chain Approximation approach in [52]. |
