We introduce a novel positional encoding strategy for Transformer-style models, addressing the shortcomings of existing, often ad hoc, approaches. Our framework provides a flexible mapping from the algebraic specification of a domain to an interpretation as orthogonal operators. This design preserves the algebraic characteristics of the source domain, ensuring that the model upholds the desired structural properties. Our scheme can accommodate various structures, including sequences, grids and trees, as well as their compositions. We conduct a series of experiments to demonstrate the practical applicability of our approach. Results suggest performance on par with or surpassing the current state-of-the-art, without hyperparameter optimizations or ``task search'' of any kind. Code will be made available at \url{github.com/konstantinosKokos/UnitaryPE}.

翻译：我们提出了一种针对Transformer风格模型的新型位置编码策略，旨在克服现有方法中常有的临时性缺陷。该框架提供了一种灵活的映射机制，能将领域的代数规范解释为正交算子。这种设计保留了源域的代数特性，确保模型维持所需的结构属性。本方案可适配多种数据结构，包括序列、网格、树结构及其组合形式。我们通过一系列实验验证了该方法的实际应用性。结果表明，在无需任何超参数优化或"任务搜索"的情况下，本方法性能可与当前最先进技术持平或更优。相关代码将开放于\url{github.com/konstantinosKokos/UnitaryPE}。