We provide new estimates of an asymptotic upper bound on the entropy of English using the large language model LLaMA-7B as a predictor for the next token given a window of past tokens. This estimate is significantly smaller than currently available estimates in \cite{cover1978convergent}, \cite{lutati2023focus}. A natural byproduct is an algorithm for lossless compression of English text which combines the prediction from the large language model with a lossless compression scheme. Preliminary results from limited experiments suggest that our scheme outperforms state-of-the-art text compression schemes such as BSC, ZPAQ, and paq8h.

翻译：我们利用大语言模型LLaMA-7B作为基于历史词元窗口的下一个词元预测器，提出了英语熵渐近上界的新估计值。该估计值显著小于现有文献\cite{cover1978convergent}和\cite{lutati2023focus}中可获得的估计值。其自然衍生成果是一种英语文本无损压缩算法，该算法将大语言模型的预测与无损压缩方案相结合。有限实验的初步结果表明，我们的方法在性能上优于BSC、ZPAQ和paq8h等最先进的文本压缩方案。