The goal of the challenge is to develop a test-time adaptation (TTA) method, which could adapt the model to gradually changing domains in video sequences for semantic segmentation task. It is based on a synthetic driving video dataset - SHIFT. The source model is trained on images taken during daytime in clear weather. Domain changes at test-time are mainly caused by varying weather conditions and times of day. The TTA methods are evaluated in each image sequence (video) separately, meaning the model is reset to the source model state before the next sequence. Images come one by one and a prediction has to be made at the arrival of each frame. Each sequence is composed of 401 images and starts with the source domain, then gradually drifts to a different one (changing weather or time of day) until the middle of the sequence. In the second half of the sequence, the domain gradually shifts back to the source one. Ground truth data is available only for the validation split of the SHIFT dataset, in which there are only six sequences that start and end with the source domain. We conduct an analysis specifically on those sequences. Ground truth data for test split, on which the developed TTA methods are evaluated for leader board ranking, are not publicly available. The proposed solution secured a 3rd place in a challenge and received an innovation award. Contrary to the solutions that scored better, we did not use any external pretrained models or specialized data augmentations, to keep the solutions as general as possible. We have focused on analyzing the distributional shift and developing a method that could adapt to changing data dynamics and generalize across different scenarios.
翻译:本挑战赛的目标是开发一种测试时自适应(TTA)方法,使模型能够适应视频序列中语义分割任务的渐变域。该方法基于合成驾驶视频数据集SHIFT。源模型在晴朗天气日间拍摄的图像上训练。测试时的域变化主要由天气条件和一天中时间的改变引起。TTA方法在每个图像序列(视频)上独立评估,即在处理下一个序列前,模型会重置为源模型状态。图像逐帧到达,每帧到达时必须作出预测。每个序列包含401张图像,从源域开始,逐渐偏移至不同域(天气或时间变化),直至序列中间。序列后半部分,域逐渐回移至源域。仅SHIFT数据集的验证集提供真实数据,其中仅有6个序列以源域开始和结束。我们专门针对这些序列进行了分析。用于排行榜排名的测试集(评估所开发TTA方法)的真实数据未公开。提出的方案在挑战赛中获第三名并获得创新奖。与排名更优的方案不同,我们未使用任何外部预训练模型或专门的数据增强,以尽可能保持方案通用性。我们专注于分析分布偏移,并开发能够适应动态数据变化且泛化至不同场景的方法。