A natural and informal approach to verifiable (or zero-knowledge) ML inference over floating-point data is: ``prove that each layer was computed correctly up to tolerance $δ$; therefore the final output is a reasonable inference result''. This short note gives a simple counterexample showing that this inference is false in general: for any neural network, we can construct a functionally equivalent network for which adversarially chosen approximation-magnitude errors in individual layer computations suffice to steer the final output arbitrarily (within a prescribed bounded range).
翻译:一种自然且非正式的针对浮点数据的可验证(或零知识)机器学习推理方法是:“证明每一层的计算在容差 $δ$ 范围内是正确的;因此最终输出是一个合理的推理结果”。本短文给出了一个简单的反例,表明这种推理在一般情况下并不成立:对于任何神经网络,我们都可以构造一个功能等效的网络,其中各层计算中由对手选择的、在近似幅度范围内的误差,足以使最终输出任意偏离(在预设的有界范围内)。