We formulate a stochastic process, FiLex, as a mathematical model of lexicon entropy in deep learning-based emergent language systems. Defining a model mathematically allows it to generate clear predictions which can be directly and decisively tested. We empirically verify across four different environments that FiLex predicts the correct correlation between hyperparameters (training steps, lexicon size, learning rate, rollout buffer size, and Gumbel-Softmax temperature) and the emergent language's entropy in 20 out of 20 environment-hyperparameter combinations. Furthermore, our experiments reveal that different environments show diverse relationships between their hyperparameters and entropy which demonstrates the need for a model which can make well-defined predictions at a precise level of granularity.

翻译：我们构建了一个随机过程FiLex，作为基于深度学习的突现语言系统中词库熵的数学模型。通过建立数学定义，该模型能够生成可直接进行决定性验证的清晰预测。我们在四个不同环境中进行实证验证，结果表明FiLex在20个环境-超参数组合中，正确预测了超参数（训练步数、词库规模、学习率、rollout缓冲区大小及Gumbel-Softmax温度）与突现语言熵之间的相关性。此外，实验揭示不同环境中超参数与熵之间呈现多样化关系，这凸显了构建能够在精确粒度层面上做出明确定义预测的模型的必要性。