Language workbenches are tools that enable the definition, reuse, and composition of programming languages and their ecosystems, aiming to streamline language development. To facilitate their adoption by language designers, the comprehensibility of the language used to define other languages is an important aspect to evaluate. Moreover, considering that language workbenches are relatively new tools, user acceptance emerges as a crucial factor to be accounted for during their assessment. Current literature often neglects user-centred aspects like comprehensibility and acceptance in the assessment of this breed of tools. This paper addresses this gap through a family of experiments assessing Neverlang, a modular language workbench. The study adopts a tailored version of the Method Evaluation Model (MEM) to evaluate the comprehensibility of Neverlang's meta-language and programs, as well as user acceptance in terms of perceived ease of use, perceived usefulness, and intention to use. It also investigates the relationships among these dimensions. The experiments were conducted in three iterations involving participants from academia. The results reveal that users demonstrate sufficient comprehension of Neverlang's meta-language, particularly concerning its syntax, express a favourable perception of its usefulness, and indicate their intention to use it. However, the results also indicate that Neverlang's ease of use remains a challenge. Additionally, variations in the perceived ease of use and perceived usefulness, whether low or high, influence the users' intention to use the tool. Surprisingly, no significant correlation is found between comprehensibility and user acceptance. Notably, higher comprehensibility of the meta-language does not necessarily translate into greater acceptance, underscoring the complex interplay between comprehension and adoption.
翻译:语言工作台是一种支持编程语言及其生态系统的定义、复用与组合的工具,旨在简化语言开发流程。为促进语言设计者采纳此类工具,用于定义其他语言的元语言的可理解性成为评估中的重要方面。此外,鉴于语言工作台是相对新兴的工具,用户接受度在其评估过程中亦成为需要考量的关键因素。现有文献在评估此类工具时,常忽视以用户为中心的可理解性与接受度等维度。本文通过针对模块化语言工作台Neverlang的系列实验填补了这一空白。研究采用定制版的"方法评估模型"(MEM),评估Neverlang元语言及其程序的可理解性,以及用户在感知易用性、感知有用性和使用意愿维度的接受度,并探究这些维度间的关联性。实验分三次迭代进行,参与者来自学术界。结果表明:用户对Neverlang元语言(特别是其语法)展现出足够的理解能力,对其有用性持积极态度,并表达使用意愿。然而,结果也显示Neverlang的易用性仍是待解决的挑战。此外,无论感知易用性与感知有用性处于低水平或高水平,其变化均会影响用户的使用意愿。值得注意的是,元语言更高的可理解性并不必然转化为更高的接受度,这揭示了理解与采纳之间复杂的相互作用关系。