Spiking neural networks (SNNs) are promising for energy-efficient inference, and time-to-first-spike (TTFS) coding is especially attractive because each neuron fires at most once. In practice, however, this benefit is often reduced by the cost of computing a temporal decay term and multiplying it by the synaptic weight. We address this issue by turning a physical hardware "bug," the natural signal decay in optoelectronic devices, into the main computation of TTFS, named Otters++. Specifically, we use the measured decay of a custom In$_2$O$_3$ optoelectronic synapse to directly realize the TTFS temporal term, removing the need for explicit digital decay computation. To scale this idea to Transformer models, we establish a layer-wise functional equivalence between the Otters++ and a quantized neural network (QNN), and develop a hybrid training method that uses device-faithful SNN computation in the forward pass and QNN straight-through gradients through the equivalent QNN path in the backward pass, together with model distillation. This avoids differentiation through discrete first-spike events and reduces the over-sparsity problem in direct TTFS-SNN training. We further make training aware of measured device noise by sampling run-to-run variation, and refine the system-level energy model by accounting for device sharing and multi-hop communication. On GLUE dataset, Otters++ improves the average score to 84.17\% while maintaining a clear energy advantage over prior spiking Transformer baselines. These results show that physically grounded TTFS computing can be efficient, trainable, and robust under realistic hardware effects.

翻译：脉冲神经网络（SNN）因能效推理具有潜力，而首次脉冲时间（TTFS）编码尤为吸引人，因为每个神经元最多只放电一次。然而在实践中，这一优势常因计算时间衰减项并乘以突触权重的开销而削弱。我们通过将物理硬件的“缺陷”——光电设备中的自然信号衰减——转化为TTFS的主要计算（称为Otters++）来解决此问题。具体地，利用定制In$_2$O$_3$光电突触的实测衰减直接实现TTFS时间项，从而省去显式数字衰减计算。为将该思想扩展至Transformer模型，我们建立了Otters++与量化神经网络（QNN）之间的逐层功能等价性，并开发了一种混合训练方法：前向传播中使用设备保真的SNN计算，反向传播中通过等价QNN路径使用QNN直通梯度，并结合模型蒸馏。这避免了离散首次脉冲事件的微分，并减轻了直接TTFS-SNN训练中的过度稀疏性问题。通过采样逐次运行差异使训练感知实测设备噪声，并通过考虑设备共享与多跳通信精化系统级能效模型。在GLUE数据集上，Otters++将平均分数提升至84.17%，同时相比此前脉冲Transformer基线保持明显能效优势。这些结果表明，基于物理实现的TTFS计算在实际硬件效应下可实现高效、可训练且鲁棒的推理。