We propose succinctness as a measure of the expressive power of a transformer in describing a concept. To this end, we prove that transformers are highly expressive in that they can represent formal languages substantially more succinctly than standard representations of formal languages like finite automata and Linear Temporal Logic (LTL) formulas. As a by-product of this expressivity, we show that verifying properties of transformers is provably intractable (i.e. EXPSPACE-complete).

翻译：我们提出简洁性作为衡量Transformer在描述概念时表达能力的一种度量。为此，我们证明Transformer具有高度的表达能力，因为它们能够比有限自动机和线性时序逻辑（LTL）公式等标准形式语言表示方法更简洁地表示形式语言。作为这种表达能力的一个副产品，我们证明验证Transformer的性质在计算上是难解的（即EXPSPACE完全问题）。