Recent advancements in language models have demonstrated remarkable improvements in various natural language processing (NLP) tasks such as web navigation. Supervised learning (SL) approaches have achieved impressive performance while utilizing significantly less training data compared to previous methods. However, these SL-based models fall short when compared to reinforcement learning (RL) approaches, which have shown superior results. In this paper, we propose a novel approach that combines SL and RL techniques over the MiniWoB benchmark to leverage the strengths of both methods. We also address a critical limitation in previous models' understanding of HTML content, revealing a tendency to memorize target elements rather than comprehend the underlying structure. To rectify this, we propose methods to enhance true understanding and present a new baseline of results. Our experiments demonstrate that our approach outperforms previous SL methods on certain tasks using less data and narrows the performance gap with RL models, achieving 43.58\% average accuracy in SL and 36.69\% when combined with a multimodal RL approach. This study sets a new direction for future web navigation and offers insights into the limitations and potential of language modeling for computer tasks.

翻译：近年来，语言模型在网络导航等各类自然语言处理任务中取得了显著进展。与以往需要大量训练数据的方法相比，监督学习方法能够在显著减少数据量的同时实现令人瞩目的性能。然而，这些基于监督学习的模型在效果上仍不及表现更优的强化学习方案。本文提出一种融合监督学习与强化学习技术的新方法，基于MiniWoB基准测试发挥两种方法的优势。此外，我们揭示了以往模型在理解HTML内容方面存在关键局限——模型倾向于记忆目标元素而非理解底层结构。针对这一问题，我们提出了增强真实理解能力的改进方法，并建立了新的基线结果。实验表明，我们的方法在特定任务上以更少数据超越了先前的监督学习方法，并缩小了与强化学习模型的性能差距：监督学习模式下平均准确率达43.58%，结合多模态强化学习方法后达36.69%。本研究为未来网络导航开辟了新方向，并揭示了语言模型在计算机任务中的局限性与潜力。