【Unite Tokyo 2018】Unity for ディープ・ラーニング:ツールキット『ML-Agents』のご紹介

>100 Views

May 08, 18

スライド概要

講演者:Mike Geig(Unity Technologies)

こんな人におすすめ
・機械学習や深層学習に興味をお持ちの開発者

受講者が得られる知見
・ 『ML-Agents』ツールキットに含まれる最新の学習メソッド(カリキュラム学習、模倣学習など)
・ それらを使用してUnityでエージェントをトレーニングする方法

profile-image

リアルタイム3Dコンテンツを制作・運用するための世界的にリードするプラットフォームである「Unity」の日本国内における販売、サポート、コミュニティ活動、研究開発、教育支援を行っています。ゲーム開発者からアーティスト、建築家、自動車デザイナー、映画製作者など、さまざまなクリエイターがUnityを使い想像力を発揮しています。

シェア

またはPlayer版

埋め込む »CMSなどでJSが使えない場合

関連スライド

各ページのテキスト
2.

Unity for Deep Learning: ML-Agents Explained

3.

Mike Geig Head of Global Evangelism Content

4.

Let’s start with one important question...

5.

Let’s start with one important question... Why program system to complete a specific task when you can design it to learn?

8.

ML Training Environment Requirements Visual Complexity Physical Complexity Cognitive Complexity

9.

The Unity Ecosystem

10.

ML-Agents v0.3 ML-Agents v0.2 ML-Agents v0.1 Components ● Additional environments (two new continuous control Components ● Learning Environments environments, plus two ● Flexible training scenarios (single platforming environments) agent, simultaneous single agent, ● Curriculum Learning adversarial self-play, cooperative ● Broadcasting multi-agent, competitive multi- ● Flexible monitor agent, ecosystem ● Monitoring agent’s decision making ● Complex Visual observations Components ● Imitation Learning ● Multi-Brain training ● On-demand decision-making ● Memory-enhanced agents

11.

How does it work?

12.

Unity ML-Agents Workflow Create Environment Train Agents Embed Agents

13.

Create Environment (Unity) Observe & Act Decide Coordinate

14.

Unity ML-Agents Workflow Create Environment Train Agents Embed Agents

15.

Training Methods Reinforcement Learning Imitation Learning ● Learn through rewards ● Learn through demonstrations ● Trial-and-error ● No rewards necessary ● Super-speed simulation ● Real-time interaction ● Agent becomes “optimal” at task ● Agent becomes “human-like” at task

16.

Unity ML-Agents Workflow Create Environment Train Agents Embed Agents

17.

Embed Agents (Unity) ● Simply import a .bytes file (trained brain) into Unity project ● Set corresponding brain component to “Internal” mode. ● Support for Mac, Windows, Linux, iOS, and Android.

18.

Let’s see it in action!

19.

Learning Scenarios

20.

Twelve Agents, One Brain, Independent Rewards Goal Balance ball as long as possible Observations Actions Platform rotation, ball position and rotation Platform rotation (in x and z) Rewards Bonus for keeping ball up

22.

Two Agents, One Brain, Cooperative Rewards Goal Keep ball up as long as possible Observations Positions and velocities of racket and ball Actions Forward, backward, and upward movement Rewards +0.1 when sent over net by agent -0.1 when ball falls because of agent

24.

Four Agents, Multi-Brain, Competitive Rewards Striker Goal Get the ball into the opponents goal Goalie Goal Defend own goal from opponents Observations Local ray-cast perception on nearby objects Actions Movement and rotation in x, z plane Striker Rewards +1 when its team scores goal -0.1 when opponent scores goal Goalie Rewards -1 when opponent scores goal +0.1 when its team scores goal

25.

Multi-Stage Soccer Training Offense Train one brain with positive reward for ball entering opponents goal Defense Train one brain with negative reward for ball entering their goal Combined Train both brains together to play against opponent team

27.

Learning Methods

28.

Curriculum Learning

29.

Easy Curriculum Learning ● Bootstrap learning of difficult task with simpler task ● Utilize custom reset parameters ● Change environment task based on reward or fixed progress Difficult

31.

Imitation Learning

32.

Imitation Learning Collect demonstrations from a teacher Learn policy via imitation

33.

ML-Agents v0.3 ML-Agents v0.2 ML-Agents v0.1 Components ● Additional environments (two new continuous control Components ● Learning Environments environments, plus two ● Flexible training scenarios (single platforming environments) agent, simultaneous single agent, ● Curriculum Learning adversarial self-play, cooperative ● Broadcasting multi-agent, competitive multi- ● Flexible monitor agent, ecosystem ● Monitoring agent’s decision making ● Complex Visual observations Components ● Imitation Learning ● Multi-Brain training ● On-demand decision-making ● Memory-enhanced agents

34.

We are hiring!

35.

Get it Now github.com/Unity-Technologies/ml-agents Contact us https://unity3d.ai [email protected]

36.

Thank you! Mike Geig [email protected] @MikeGeig