Both CNN-based and Transformer-based object detection with bounding box representation have been extensively studied in computer vision and medical image analysis, but circular object detection in medical images is still underexplored. Inspired by the recent anchor free CNN-based circular object detection method (CircleNet) for ball-shape glomeruli detection in renal pathology, in this paper, we present CircleFormer, a Transformer-based circular medical object detection with dynamic anchor circles. Specifically, queries with circle representation in Transformer decoder iteratively refine the circular object detection results, and a circle cross attention module is introduced to compute the similarity between circular queries and image features. A generalized circle IoU (gCIoU) is proposed to serve as a new regression loss of circular object detection as well. Moreover, our approach is easy to generalize to the segmentation task by adding a simple segmentation branch to CircleFormer. We evaluate our method in circular nuclei detection and segmentation on the public MoNuSeg dataset, and the experimental results show that our method achieves promising performance compared with the state-of-the-art approaches. The effectiveness of each component is validated via ablation studies as well. Our code is released at https://github.com/zhanghx-iim-ahu/CircleFormer.

翻译：基于边界框表示的CNN与Transformer目标检测方法在计算机视觉和医学图像分析领域已得到广泛研究，但医学图像中的圆形目标检测仍待深入探索。受近期基于无锚框CNN的肾病理球状肾小球圆形检测方法（CircleNet）启发，本文提出CircleFormer——一种基于动态锚定圆形的Transformer圆形医学目标检测方法。具体而言，Transformer解码器中采用圆形表示的查询向量迭代优化圆形目标检测结果，并引入圆形交叉注意力模块计算圆形查询与图像特征之间的相似度。同时提出广义圆形交并比（gCIoU）作为圆形目标检测的新型回归损失函数。此外，通过为CircleFormer添加简易分割分支，该方法可轻松泛化至分割任务。我们在公开MoNuSeg数据集上对圆形细胞核检测与分割进行了评估，实验结果表明，与现有最优方法相比，本方法取得了具有竞争力的性能。消融研究进一步验证了各组件的有效性。代码已开源至https://github.com/zhanghx-iim-ahu/CircleFormer。