Efficient resource allocation is a key challenge in modern cloud computing. Over-provisioning leads to unnecessary costs, while under-provisioning risks performance degradation and SLA violations. This work presents an artificial intelligence approach to predict resource utilization in big data pipelines using Random Forest regression. We preprocess the Google Borg cluster traces to clean, transform, and extract relevant features (CPU, memory, usage distributions). The model achieves high predictive accuracy (R Square = 0.99, MAE = 0.0048, RMSE = 0.137), capturing non-linear relationships between workload characteristics and resource utilization. Error analysis reveals impressive performance on small-to-medium jobs, with higher variance in rare large-scale jobs. These results demonstrate the potential of AI-driven prediction for cost-aware autoscaling in cloud environments, reducing unnecessary provisioning while safeguarding service quality.

翻译：高效资源分配是现代云计算面临的关键挑战。资源过度配置会导致不必要的成本，而资源配置不足则存在性能下降和违反服务等级协议的风险。本研究提出一种基于随机森林回归的人工智能方法，用于预测大数据流水线中的资源利用率。我们对谷歌Borg集群追踪数据进行了预处理，包括清洗、转换和提取相关特征（CPU、内存、使用分布）。该模型实现了较高的预测精度（R平方=0.99，平均绝对误差=0.0048，均方根误差=0.137），能够捕捉工作负载特征与资源利用率之间的非线性关系。误差分析表明，模型在中小型任务上表现出色，而在罕见的大规模任务上则具有较高的方差。这些结果证明了人工智能驱动的预测在云环境中实现成本感知自动扩展的潜力，既能减少不必要的资源配置，又能保障服务质量。