SimpSON: Simplifying Photo Cleanup With Single-Click Distracting Object Segmentation Network

1University of Maryland, College Park 2Adobe Research

SimpSON selects all similar distractors with user's single click before any inpainting framework (e.g., Content-aware Filling) cleans them.


We introduce a framework that can quickly select all similar objects/ distractors in image with one single-click.

In photo editing, it is common practice to remove visual distractions to improve the overall image quality and highlight the primary subject. However, manually selecting and removing these small and dense distracting regions can be a laborious and time-consuming task. In this paper, we propose an interactive distractor selection method that is optimized to achieve the task with just a single click. Our method surpasses the precision and recall achieved by the traditional method of running panoptic segmentation and then selecting the segments containing the clicks. We also showcase how a transformer-based module can be used to identify more distracting regions similar to the user's click position. Our experiments demonstrate that the model can effectively and accurately segment unknown distracting objects interactively and in groups. By significantly simplifying the photo cleaning and retouching process, our proposed model provides inspiration for exploring rare object segmentation and group selection with a single click.


Overview of the proposed SimpSON framework.
Analytics of categories and object sizes in our Distractor-20K dataset.


Slide left to view the segmentation results!


      author    = {Huynh, Chuong and Zhou, Yuqian and Lin, Zhe and Barnes, Connelly and Shechtman, Eli and Amirghodsi, Sohrab and Shrivastava, Abhinav},
      title     = {SimpSON: Simplifying Photo Cleanup With Single-Click Distracting Object Segmentation Network},
      booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      month     = {June},
      year      = {2023},
      pages     = {14518-14527}