02 / CLIP · Few-shot vision-language learning
Learning with less.
How far can a handful of labelled images take CLIP?
Read the diagram
- Image + text inputs
- Pretrained CLIP
- Few-shot adaptation
- Classification evaluation
01 / Context
The problem
CLIP already knows a lot about images and language before it sees a single example from your task. The question here is what you can do with it when you only have a few labelled images to adapt it with.
02 / Contribution
What I did
I wrote the experiments in PyTorch: the CLIP setup, each adaptation method, and the classification runs that tie them together.
03 / Decisions
The approach
Each method comes at the problem from a different side. Tip-Adapter++ builds a cache from the few examples you have. CoOp and CoCoOp learn the text prompt instead of relying on a hand-written one. PEFT trains a small set of parameters and leaves the rest of CLIP frozen.
- Image + text inputs
- Pretrained CLIP
- Few-shot adaptation
- Classification evaluation
How the main pieces fit together.
04 / Evidence
What I can show
The code is public: configs, experiment scripts, source and tests. I’m not quoting accuracy numbers on this page, because the finished results report isn’t in the repository.
05 / Reflection
Limits & lessons
Running several methods doesn’t tell you which one is best. A fair comparison needs the dataset, splits, seeds and evaluation protocol sitting right next to the numbers, so I’m not ranking the methods here.