Model-based reasoning is becoming increasingly common in software engineering. The process of building and analyzing models helps stakeholders to understand the ramifications of their software decisions. But complex models can confuse and overwhelm stakeholders when these models have too many candidate solutions. We argue here that a technique based on partial orderings lets humans find acceptable solutions via a binary chop needing $O(log(N))$ queries (or less). This paper checks the value of this approach via the iSNEAK partial ordering tool. Pre-experimentally, we were concerned that (a)~our automated methods might produce models that were unacceptable to humans; and that (b)~our human-in-the-loop methods might actual overlooking significant optimizations. Hence, we checked the acceptability of the solutions found by iSNEAK via a human-in-the-loop double-blind evaluation study of 20 Brazilian programmers. We also checked if iSNEAK misses significant optimizations (in a corpus of 16 SE models of size ranging up to 1000 attributes by comparing it against two rival technologies (the genetic algorithms preferred by the interactive search-based SE community; and the sequential model optimizers developed by the SE configuration community~\citep{flash_vivek}). iSNEAK 's solutions were found to be human acceptable (and those solutions took far less time to generate, with far fewer questions to any stakeholder). Significantly, our methods work well even for multi-objective models with competing goals (in this work we explore models with four to five goals). These results motivate more work on partial ordering for many-goal model-based problems.

翻译：模型推理在软件工程中日益普及。构建和分析模型的过程有助于利益相关者理解其软件决策的影响。但当模型包含过多候选解时，复杂模型可能使利益相关者感到困惑和不知所措。本文论证，基于偏序的技术可使人类通过二分搜索在$O(log(N))$次查询（甚至更少）中找到可接受的解。本文通过iSNEAK偏序工具评估该方法的有效性。实验前我们曾担忧：(a)自动化方法可能生成人类无法接受的模型；(b)人在环路的方法可能遗漏重要优化。为此，我们通过包含20名巴西程序员的人在环路双盲评估研究，检验了iSNEAK所求解的可接受性。同时，我们通过将iSNEAK与两项竞争技术（交互式搜索SE社区偏好的遗传算法，以及SE配置社区开发的序列模型优化器~\citep{flash_vivek}）进行对比（在包含16个属性规模高达1000的SE模型语料库上），检验其是否遗漏重要优化。结果表明：iSNEAK的求解具有人类可接受性（且生成这些解所需时间显著减少，向利益相关者提出的问题数量大幅降低）。值得注意的是，即使面对存在竞争目标的（本文探索四至五个目标）多目标模型，该方法仍表现良好。这些结果激励了偏序在多目标模型问题上的进一步研究。