Training capable language models typically relies on web-scale pretraining followed by multi-stage post-training, creating substantial computational and data-governance barriers for settings in which training data must satisfy explicit permissibility constraints. We introduce Mimir, a 1.78B-parameter language model based on the Hierarchical Reasoning Model Text (HRM-Text) architecture, a recurrent-depth transformer whose layers are split into a low-level (L) and a high-level (H) stack that are applied repeatedly in an interleaved schedule. Mimir is trained from random initialization directly on a curated mixture of instruction, reasoning, tool-use, multilingual, and synthetic data: 161 datasets and 70.5B tokens per epoch for Mimir v1, extended to 103B tokens per epoch for its continued-training successor Mimir v1.5, with synthetic replacements for sources that do not satisfy permissibility criteria. Across 20 benchmarks in English, Danish (a low-resource language with ~6M speakers), and Math & Code, Mimir v1.5 attains the highest average score of all sub-2B models in every domain, outperforming both HRM-Text 1B and the looped transformers Ouro 1.4B and Ouro 2.6B, and Mimir matches or exceeds the performance of models of up to 9B parameters, such as Qwen 3.5 9B and Gemma 4 E4B, on the English and Danish averages. Two independent memorisation audits indicate low memorisation risk despite multi-epoch training. Inference-time ablations show that the number of L-stack applications can be changed with graceful degradation, whereas changing the number of H-stack applications breaks the model. Our results provide evidence that hierarchical recurrent language models can learn effectively from post-training-oriented, permissible data mixtures without conventional web-scale pretraining.

翻译：暂无翻译