Dynamic Tuning of Online Data Migration Policies in
Hierarchical Storage Systems using Reinforcement Learning
by David Vengerov
June 19, 2006 - Multi-tier storage systems are becoming more and more widespread in the
industry. In order to minimize the request response time in such systems, the
most frequently accessed ("hot") files should be located in the fastest storage
tiers (which are usually smaller and more expensive than the other tiers).
Unfortunately, it is impossible to know ahead of time which files are going to
be "hot", especially because the file access patterns change over time. This
report presents a solution approach to this problem, where each tier uses
Reinforcement Learning (RL) to learn its own cost function that predicts its
future request response time, and the files are then migrated between the tiers
so as to decrease the sum of costs of the tiers involved.
The presented solution framework is general enough to apply to many other
problems, where decisions need to be made dynamically based on observable
information with the goal of maximizing the sum of future performance
observations. In particular, this report demonstrates how a set of shared
resources (e.g., data, CPUs, memory, bandwidth, etc.) can be dynamically
partitioned among entities (agents) so as to maximize the long-term benefit
received by all agents.
The presented approach for dynamically learning and using utility (or cost)
functions brings Sun closer toward achieving the Utility Computing vision, where
the systems dynamically optimize themselves in response to the changing
environment, so as to continue providing a high utility to their users. This
technology has been prototyped for use in several Sun products, and more studies
are under way.
Read Technical Report now »
