|
A distributed virtual environment (DVE) consists of multiple network nodes (servers), each of which can host many users that consume CPU resources on that node and communicate with users on other nodes. Users can be dynamically migrated between the nodes, and the ultimate goal for the migration policy is to minimize the average system response time perceived by the users. In order to achieve this, the user migration policy should minimize network communication while balancing the load among the nodes so CPU resources of the individual nodes are not overwhelmed. This paper considers a multi-player online game as an example of a DVE and presents an adaptive distributed user migration policy, which uses Reinforcement Learning to tune itself and thus minimize the average system response time perceived by the users. Performance of the self-tuning policy was compared on a simulator with the standard benchmark non-adaptive migration policy and with the optimal static user allocation policy in a variety of scenarios, and the self-tuning policy was shown to greatly outperform both benchmark policies, with performance difference increasing as the network became more overloaded. These results provide yet another demonstration of the power and generality of the methodology for designing adaptive distributed and scalable migration policies, which has already been applied successfully to several other domains [1, 2].
[1] D. Vengerov, “A Reinforcement Learning Approach to Dynamic Resource Allocation.” Engineering
Applications of Artificial Intelligence. Volume 20, Issue 3, pp. 383-390, April 2007.
[2] D. Vengerov, “A Reinforcement Learning Framework for Online Data Migration in Hierarchical
Storage Systems.” Journal of Supercomputing, Vol. 43, No. 1, pp. 1-19, January, 2008.
|
