A statistical framework of optimal workload consolidation with application to capacity planning for on-demand computing

• 成果类型: Article
• 署名作者: Li, Ta-Hsin
• 署名单位: International Business Machines (IBM); IBM USA
• 刊物名称: JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
• ISSN/ISSBN: 0162-1459
• DOI: 10.1198/016214506000000744
• 发表日期: 2007
• 页码: 841-855
• 关键词: EXPECTED SHORTFALL

摘要

In on-demand computing services, the customer pays based on actual usage, and the service provider is free to allocate unused capacity to other customers. Capacity planning becomes an important issue in such a shared environment. It is desirable that a portfolio effect occurs so that less total capacity is required in a shared system than in a dedicated system. In this article a quantile-based statistical approach is proposed to quantify the portfolio effect and the associated risks. It is shown that the portfolio effect may or may not exist for a given set of workloads, depending crucially on their joint distribution and the assumed risk levels. For Gaussian workloads, the portfolio effect almost always exists regardless of the statistical correlation and risk level. For non-Gaussian workloads, the portfolio effect is not guaranteed and may even be negative, even if the workloads are negatively correlated or statistically independent. However, when simultaneously recorded workload history is available, the portfolio effect can be estimated directly from the data. Based on the data-driven approach, an optimization problem is formulated for capacity planning with the aim of maximizing the portfolio effect for a given set of workloads. This problem calls for consolidation of the workloads into one or more portfolios so that each portfolio can be served satisfactorily by a dedicated system and the total capacity requirement is minimized. Iterative algorithms are proposed to solve the optimization problem numerically. The method is applied to a server consolidation problem with real workload data.