628 Views
March 16, 22
スライド概要
Yusuke Tamura, Shiro Yano, Hisashi Osumi, "Visual Attention Model for Manipulating Human Attention by a Robot," Proceedings of the 2014 IEEE International Conference on Robotics and Automation, pp.5307-5312, 2014.
東北大学大学院工学研究科ロボティクス専攻 田村研究室
Visual Attention Model for Manipulating Human Attention by a Robot Yusuke TAMURA1, Shiro YANO2, Hisashi OSUMI1 1 Chuo University, JAPAN 2 Ritsumeikan University, JAPAN 2014 IEEE International Conference on Robotics & Automation, Hong Kong, June 4, 2014
MOTIVATION Smooth interaction between humans... 2. Estimation/Prediction 1. Observation 3. Decision 2
MOTIVATION Smooth interaction between humans... 2. Estimation/Prediction 0. Elicitation 1. Observation 3. Decision 2
MOTIVATION Smooth interaction between humans... 2. Estimation/Prediction 0. Elicitation 1. Observation 3. Decision Manipulation of other’s attention is important for smooth interaction 2
MOTIVATION Smooth interaction between robot and human... 2. Estimation/Prediction 0. Elicitation 1. Observation 3. Decision 3
MOTIVATION Smooth interaction between robot and human... 2. Estimation/Prediction 0. Elicitation 1. Observation 3. Decision A robot has to understand how its behaviors affect human’s attention. 3
OBJECTIVE 4
OBJECTIVE 4
OBJECTIVE Development of a model of human attention for manipulating human attention by a robot (robot’s) behavior attention area environment 4
RELATED WORKS Computational visual attention model - Saliency map (Itti et al., 1998) - Saliency map + Face position (Cerf et al., 2008) - Saliency map + Face position/orientation (Ozeki et al., 2011) Input image Intensity Colors Orientation Motion Center-surround differences & normalization Across-scale combinations & normalization Saliency map 5
MANIPULATION OF ATTENTION Magicians are experts of manipulating human attention. 6
MANIPULATION OF ATTENTION Magicians are experts of manipulating human attention. Magic requires a method to achieve an effect. How the trick works What the spectator perceives 6
MANIPULATION OF ATTENTION Magicians are experts of manipulating human attention. Magic requires a method to achieve an effect. How the trick works What the spectator perceives Magicians manipulate spectators’ attention in order to prevent them from uncovering the methods by various skills. Eye gaze (Kuhn et al., 2009) Hand movement (Otero-Millan et al., 2011) Face, hands, and gaze (Akashi et al., 2013) 6
MODEL OF VISUAL ATTENTION Hypothesis: Human visual attention is influenced by visual saliency and others’ face, hands and gaze. 7
MODEL OF VISUAL ATTENTION Hypothesis: Human visual attention is influenced by visual saliency and others’ face, hands and gaze. Saliency map 7
MODEL OF VISUAL ATTENTION Hypothesis: Human visual attention is influenced by visual saliency and others’ face, hands and gaze. Saliency map Manipulation map 7
MODEL OF VISUAL ATTENTION Hypothesis: Human visual attention is influenced by visual saliency and others’ face, hands and gaze. Saliency map Manipulation map Attention map 7
SALIENCY MAP Visual saliency of input images Input image Intensity Colors (x2) Orientation (x4) Motion Center-surround differences & normalization Across-scale combinations & normalization Saliency map 8
MANIPULATION MAP Attention area (Akashi et al., 2013) Concentration Distraction Manipulation map describes - Position of face - Position of left/right hand - Direction of gaze 9
FACE & HAND xR Face-Hand map xF F (x) = X i 1 p exp 2⇡ |Si | ⇢ 1 (x 2 xi )T Si 1 (x xi ) xL 10
GAZE x d g xF Gaze map 1 G(x) = p 2⇡ exp d ✓ d(x) 2 d2 2 ◆ 11
MANIPULATION MAP + Face-hand map Gaze map Manipulation map 12
ATTENTION MAP Input&image Face1Hand&map Gaze&map Saliency&map Manipulation&map Attention map Attention&map 13
MAGIC VIDEO A magician performs “vanishing a ball” trick. method: a false transfer of the ball effect: vanish of the ball 1920 x 1080 14
SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 15
SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 15
SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 15
SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 15
SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 15
SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 15
SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 15
SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 16
SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 16
SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 16
SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 16
RESULT: SALIENCY MAP 17
RESULT: SALIENCY MAP 17
RESULT: MANIPULATION MAP 18
RESULT: MANIPULATION MAP 18
RESULT: PROPOSED MODEL 19
RESULT: PROPOSED MODEL 19
MEASUREMENT OF GAZE Measuring participants’ gaze points while watching the magic video 23-‐inch screen Participants: 5 healthy volunteers (21-24 yrs) Measurement: faceLAB4.2 (60Hz) cameras (FCB-‐EX480B, SONY) 20
EVALUATION INDEX gaze%point:%xg attention%value:% Attention mapA(xg ) evaluation%index:% A(xg ) A(x) saliency%value:% Saliency mapS(xg ) evaluation%index:% Attention'map attention value: Saliency'map saliency value: evaluation index: evaluation index: S(xg ) S(x) 21
RESULTS Saliency map Proposed model Gaze point Scene 1 2 3 4 22
RESULTS Saliency map Proposed model Gaze point Scene 1 2 3 4 22
RESULTS Saliency map Proposed model Gaze point Scene 1 2 3 4 22
RESULTS Saliency map Proposed model Gaze point Scene 5 6 7 8 23
RESULTS Saliency map Proposed model Gaze point Scene 5 6 7 8 23
RESULTS Saliency map Proposed model Gaze point Scene 5 6 7 8 23
Ratio to overall average EVALUATION 5 ** 4 3 2 1 0 Proposed Saliency ** p<0.01 The proposed model can better explain human visual attention than the original saliency map. 24
SUMMARY A model of human attention considering visual saliency and human face, hands and gaze was developed. The model can well explain human attention. FUTURE WORKS Developing an inverse model to calculate appropriate behaviors for any intended attention area. Implementing the model into a small humanoid robot. 25