[DL Hacks]Meta-Learning LT

Meta-Learning LT M2

• Meta-Learning • Meta-Learning • LT Few-Shot Meta-Learning 3 • • https://lilianweng.github.io/lil-log/2018/11/30/meta-learning.html • •

• https://lilianweng.github.io/lil-log/2018/11/30/meta-learning.html

Meta-Learning is • ↑ Few-Shot Learning : = (ex. “okapi”) ( ) (fine-tuning ~ )

1 “cats”, “birds” ? “cats” 4 “birds” 2 2 “flowers”, “bikes” ? “flowers” 4 “bikes”

Few-Shot Learning Few-Shot Learning 2 • 1 • 1 • ( ) ( ( ) )

Few-Shot Learning

Meta-Learning • • • ( ) → “Meta” Learning

Meta-Learning

Meta-Learning

Meta-Learning Meta-Learning • Metric-based • Model-based • Optimization-based 3

Metric-based • Meta-Learning • End-to-End ( ) Matching Networks [Vinyals et al., 2016] [2] • Few-Shot !" , $" kNN

Model-Based • !" ($|&) • Memory-Augmented Neural Networks [Santoro et al., 2016] [3] • Neural Turing Machine • Few-Shot • [4] …

Optimization-based Few-Shot Fine-tuning • • Learner Meta-Leaner • Model-Agnostic Meta-Learning [Finn, et al., 2017] [5] • • Meta-Learner

• Meta-Learning • Few-Shot • • • • Meta-Learning 3 • Metric-based, Model-based, Optimization-based

[1] Meta-Learning: Learning to Learn Fast, https://lilianweng.github.io/lillog/2018/11/30/meta-learning.html [2] Oriol Vinyals, et al. “Matching networks for one shot learning.” NIPS. 2016 [3] Adam Santoro, et al. “Meta-learning with memory-augmented neural networks.” ICML. 2016. [4] [Slide Share] Meta-Learning with Memory Augmented Neural Network, https://www.slideshare.net/YusukeWatanabe3/metalearning-with-memory-augmentedneural-network [5] Chelsea Finn, Pieter Abbeel, and Sergey Levine. “Model-agnostic meta-learning for fast adaptation of deep networks.” ICML 2017.

• https://lilianweng.github.io/lil-log/2018/11/30/meta-learning.html
• http://papers.nips.cc/paper/6385-matching-networks-for-one-shot-learning.pdf
• http://proceedings.mlr.press/v48/santoro16.pdf
• https://www.slideshare.net/YusukeWatanabe3/metalearning-with-memory-augmented-neural-network
• https://arxiv.org/abs/1703.03400