3D shape reconstruction typically requires identifying object features or textures in multiple images of a subject. This approach is not viable when the subject is semi-transparent and moving in and out of focus. Here we overcome these challenges by rendering a candidate shape with adaptive blurring and transparency for comparison with the images. We use the microscopic nematode Caenorhabditis elegans as a case study as it freely explores a 3D complex fluid with constantly changing optical properties. We model the slender worm as a 3D curve using an intrinsic parametrisation that naturally admits biologically-informed constraints and regularisation. To account for the changing optics we develop a novel differentiable renderer to construct images from 2D projections and compare against raw images to generate a pixel-wise error to jointly update the curve, camera and renderer parameters using gradient descent. The method is robust to interference such as bubbles and dirt trapped in the fluid, stays consistent through complex sequences of postures, recovers reliable estimates from blurry images and provides a significant improvement on previous attempts to track C. elegans in 3D. Our results demonstrate the potential of direct approaches to shape estimation in complex physical environments in the absence of ground-truth data.

翻译：三维形状重建通常需要通过多张图像识别物体的特征或纹理。当物体呈现半透明状态且在焦平面内外移动时，该方法难以适用。本文通过引入自适应模糊与透明度处理对候选形状进行渲染，并将其与原始图像进行对比，以克服上述挑战。我们以自由探索三维复杂流体（其光学特性持续变化）的显微镜下秀丽隐杆线虫作为案例研究对象。采用内在参数化方法将细长蠕虫建模为三维曲线，该模型自然兼容了生物学约束与正则化条件。为应对光学特性的动态变化，我们开发了一种新型可微分渲染器，通过二维投影构建图像并与原始图像逐像素对比产生误差，利用梯度下降联合优化曲线、相机及渲染器参数。该方法对流体中的气泡、污物等干扰具有鲁棒性，能在复杂姿态序列中保持一致性，从模糊图像中恢复可靠估计，相较于此前三维线虫追踪方法实现显著提升。研究结果表明，在缺乏真实数据的情况下，直接形状估计方法在复杂物理环境中具有巨大潜力。