My Vision, Computer Vision

Overview본 논문에서는 CLIP 모델을 REF(Referring Expression Segmentation) Task에 사용한다.동시에 Vision-Language Decoder 및 Text-to-pixel 대조 학습을 제안한다.Problem Statement(당시) Multi-modal 분야에서 CLIP 모델이 성공적인 결과를 보여준 바 있다. 따라서 본 논문에서는 CLIP을 RES에 도입한다.하지만 위 사진에서처럼, CLIP을 Naive하게 사용하는 방법으로는 최적의 성능을 내지 못하는데, 이유는 Pixel-level 예측 태스크인 RES와 다르게, CLIP은 Image-level(Contrastive)로 훈련되었기 때문이다.따라서 시각적 특징을 세부적으로 학습해야하는 목표와 맞지 않게, CLI..

GSVA: Generalized Segmentation via Multimodal Large Language ModelsGeneralized Referring Expression Segmentation (GRES) extends the scope of classic RES to refer to multiple objects in one expression or identify the empty targets absent in the image. GRES poses challenges in modeling the complex spatial relationships of tarxiv.orgAuthor: Xia, Zhuofan, et al.Journal: CVPR 20204Published Date: 202..

Bring Adaptive Binding Prototypes to Generalized Referring Expression SegmentationReferring Expression Segmentation (RES) has attracted rising attention, aiming to identify and segment objects based on natural language expressions. While substantial progress has been made in RES, the emergence of Generalized Referring Expression Segmentarxiv.orgAuthor : Li, Weize, et al.Journal : IEEE Transactio..

Vision Transformers for Dense PredictionWe introduce dense vision transformers, an architecture that leverages vision transformers in place of convolutional networks as a backbone for dense prediction tasks. We assemble tokens from various stages of the vision transformer into image-like represearxiv.org Author : Ranftl, R., Bochkovskiy, A., & Koltun, V.Journal : ICCV 2021Keyword : DPRPublished ..

ALBERT: A Lite BERT for Self-supervised Learning of Language RepresentationsIncreasing model size when pretraining natural language representations often results in improved performance on downstream tasks. However, at some point further model increases become harder due to GPU/TPU memory limitations and longer training times. Toarxiv.orgAuthor : Lan, Zhenzhong, et al.Journal : ICLR 2020Keyword ..

Talking to DINO: Bridging Self-Supervised Vision Backbones with Language for Open-Vocabulary SegmentationOpen-Vocabulary Segmentation (OVS) aims at segmenting images from free-form textual concepts without predefined training classes. While existing vision-language models such as CLIP can generate segmentation masks by leveraging coarse spatial information frarxiv.orgAuthor : Barsellotti, Luca, ..