By leveraging intricate data generation pipelines, Segment Anything Model (SAM) excels in interactive segmentation. However, SAM has shown weaknesses in specific scenarios, primarily due to the ambiguity of single point prompts. For example, prompting SAM to segment a human clicking on his torso, the model can produce a mask of the whole human, but also of the individual parts e.g. his upper body or shirt. To mitigate this issue in an interactive segmentation scenario, we allow SAM to use information from a few example (image, mask) pairs without updating its weights i.e. few-shot prompting.
Presented at the Streiflicht 2023 at Ulm University with a custom Annotation Tool.
