![Page 1: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/1.jpg)
Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation
Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim
05:00 -- 07:00 PM @ East Exhibition Hall B + C #31
![Page 2: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/2.jpg)
![Page 3: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/3.jpg)
Data SampleGround Truth
![Page 4: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/4.jpg)
Data SampleGround Truth
![Page 5: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/5.jpg)
Data SampleGround Truth
![Page 6: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/6.jpg)
θData SampleGround TruthMAML
![Page 7: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/7.jpg)
Sinusoid
AbsoluteValue
Tanh
Data SampleGround Truth
![Page 8: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/8.jpg)
Tanh
Data SampleGround Truth
Sinusoid
AbsoluteValue
![Page 9: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/9.jpg)
Tanh
θData SampleGround TruthMAML (1-mode)
Sinusoid
AbsoluteValue
![Page 10: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/10.jpg)
θ3θ2
θ1
Tanh
Data SampleGround TruthMAML (1-mode)MAML (3-mode)
Sinusoid
AbsoluteValue
![Page 11: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/11.jpg)
Task Network
x
yy
![Page 12: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/12.jpg)
Task Network
x
yy
x
y
( (Samples
× K
Modulation Network
![Page 13: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/13.jpg)
Evaluations: Regression
![Page 14: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/14.jpg)
Evaluations: Regression
![Page 15: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/15.jpg)
ProMP
Evaluations: Reinforcement Learning
Ours
![Page 16: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/16.jpg)
Evaluations: Classification(a) Omniglot (e) Aircraft(c) FC100(b) Mini-ImageNet (d) CUB
![Page 17: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/17.jpg)
Evaluations: Classification(a) Omniglot (e) Aircraft(c) FC100(b) Mini-ImageNet (d) CUB
![Page 18: Multimodal Model Agnostic Meta-Learning via Task-Aware ... · Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim](https://reader034.vdocuments.pub/reader034/viewer/2022051408/6007cc3854dcb6174901896e/html5/thumbnails/18.jpg)
Multimodal Model Agnostic Meta-Learning via Task-Aware Modulation
Risto Vuorio* Shao-Hua Sun* Hexiang Hu Joseph J. Lim
05:00 -- 07:00 PM @ East Exhibition Hall B + C #31