28
Domain Adaptation CSC2539 - Visual Recognition with Text Lluís Castrejón University of Toronto
Domain Adaptation
CSC2539 - Visual Recognition with TextLluís Castrejón
University of Toronto
What is this?
Game: Caption the following images using one short sentence.
2 / 46
What is this?
3 / 46
What is this?
4 / 46
What is this?
5 / 46
Domain Adaptation
Use the same model with different data distributions in trainingand test
P (X) ̸= P (′X);P (Y |X) ≈ P (Y ′|X ′)
Credit: KristenGrauman8 / 46
Learning Aligned Cross-Modal Representations from Weakly Aligned Data
Lluis Castrejon, Yusuf Aytar, Carl Vondrick, Hamed Pirsiavash, Antonio Torralba
Motivation
Motivation
Motivation
??
Motivation
?
Cross-Modal Scene Understanding
CMPlaces
Dataset of 205 scene categories
CMPlaces
Dataset of 205 scene categories
CMPlaces
Dataset of 205 scene categories
Strong vs weak alignment
Strong vs weak alignment
Not scalable!
Cross-modal Networks
Cross-modal Networks
Problem: Parts of the network specialize to certain domains
Cross-modal Networks
Solution: Use regularization to enforce alignments
Cross-modal Networks
Chair
BedBedroom
Cross-modal Networks
T-SNE
Visualizing Activations
Cross-Modal Retrieval
Cross-Modal Retrieval
Inverting the representation
Thanks!
http://cmplaces.csail.mit.edu/
![First-Person Actions in Egocentric Videos Making Third ... · [2]Hamed Pirsiavash and Deva Ramanan, \Detecting activities of daily living in first-person camera views," in CVPR, 2012.](https://static.documents.pub/doc/80x56/5f71e1009bc1006bd5032d23/first-person-actions-in-egocentric-videos-making-third-2hamed-pirsiavash-and.jpg)
