Due to complexity and dynamics of construction work, resource, and cash flows, poor management of them usually leads to time and cost overruns, bankruptcy, even project failure. Existing approaches in construction failed to achieve optimal control of resource flow in a dynamic environment with uncertainty. Therefore, this paper introducess a model and method to adaptive control the resource flows to optimize the work and cash flows of construction projects. First, a mathematical model based on a partially observable Markov decision process is established to formulate the complex interactions of construction work, resource, and cash flows as well as uncertainty and variability of diverse influence factors. Meanwhile, to efficiently find the optimal solutions, a deep reinforcement learning (DRL) based method is introduced to realize the continuous adaptive optimal control of labor and material flows, thereby optimizing the work and cash flows. To assist the training process of DRL, a simulator based on discrete event simulation is also developed to mimic the dynamic features and external environments of a project. Experiments in simulated scenarios illustrate that our method outperforms the vanilla empirical method and genetic algorithm, possesses remarkable capability in diverse projects and external environments, and a hybrid agent of DRL and empirical method leads to the best result. This paper contributes to adaptive control and optimization of coupled work, resource, and cash flows, and may serve as a step stone for adopting DRL technology in construction project management.
翻译:由于施工工作、资源和现金流的复杂性与动态性,管理不善通常会导致工期延误、成本超支、企业破产甚至项目失败。现有施工管理方法难以在具有不确定性的动态环境中实现资源流的最优控制。为此,本文提出了一种自适应控制资源流以优化施工项目工作与现金流的模型与方法。首先,基于部分可观测马尔可夫决策过程建立数学模型,以刻画施工工作、资源与现金流之间的复杂交互作用,以及多种影响因素的随机性与变异性。同时,为高效寻找最优解,引入基于深度强化学习的算法,实现对劳动力和材料流的连续自适应最优控制,进而优化工作与现金流。为辅助深度强化学习的训练过程,还开发了基于离散事件仿真的模拟器,以模拟项目的动态特征及外部环境。模拟场景实验表明,本方法优于传统经验方法与遗传算法,在不同项目与外部环境中展现出卓越的适应性,而深度强化学习与经验方法的混合智能体取得了最佳效果。本文为耦合的工作流、资源流与现金流的自适应控制与优化提供了理论贡献,并可作为深度强化学习技术在施工项目管理中应用的基石。