mesh2gazebo: Building a simulator environment using mesh data of 3D scan application

Gazebo community meeting mesh2gazebo: Building a simulator environment using mesh data of 3D scan application CyberAgent, Inc. AI Lab Activity Understanding Team Yasuhiro Yoshimura

About me Yasuhiro Yoshimura CyberAgent, Inc. AI Lab Activity Understanding Team Research Engineer I research robotics and the sensing to understand environment. I wrote book for developer of OpenCV. 2

Motivation ● I hope to quickly build a simulator environment to develop software for an autonomous robot. ● And, I hope to build a simulator environment by simply steps with my smartphone. Real world Gazebo World 3

Problem of previous work ● Previous work(pointcloud2gazebo) has the following problem.😣 ○ ○ ○ The quality of mesh data is NOT good. Mesh data does NOT have color information. This tool supports only Gazebo Classic. Real world 3D Point Cloud Gazebo World convert 4

• https://www.docswell.com/s/dandelion1124/ZENDP8-2024-02-13-094148

mesh2gazebo ● The latest work(=mesh2gazebo) solves these problems!😆 ○ ○ ○ The quality of mesh data is good. Mesh data has color information. This tool supports Gazebo Ignition. Real World 3D Mesh data Gazebo World convert 5

mesh2gazebo I proposed mesh2gazebo to build a simulator environment using mesh data of 3D scan application. 3D Scan Application mesh2gazebo Mesh data (obj data, texture) Gazebo Gazebo World 6

Real world https://www.youtube.com/watch?v=4dIjLJ6BXzM 7

• https://www.youtube.com/watch?v=4dIjLJ6BXzM

demo movie: launch Gazebo world https://www.youtube.com/watch?v=z63dIfPx9iE 8

• https://www.youtube.com/watch?v=z63dIfPx9iE

demo movie: navigation https://www.youtube.com/watch?v=x92R9y_1isM 9

• https://www.youtube.com/watch?v=x92R9y_1isM

Visualization of mesh data in RViz2 scan RGB camera Mobile robot costmap I visualized scan, costmap and mesh data. 10

System Information This time, I checked on the following environment. ● ROS 2 Jazzy ● Gazebo Harmonic v8.7.0 ● Scaniverse ○ 3D Scanner application for iOS 11

• https://scaniverse.com/

Technical briefing ● 3D Scanning ○ ○ I used Scaniverse for the 3D Scanning. ■ I scan the floor surface for post processing. ■ I export OBJ format via Scaniverse. I convert from OBJ format to DAE format by MeshLab for Gazebo. Exported mesh data Texture data 12

• https://scaniverse.com/
• https://www.meshlab.net/

Technical briefing ● Post Processing(1/2) ○ ○ If there is the difference between Gazebo and mesh, I need to rotate. I used MeshLab to rotate mesh data. Y Z X 13

• https://www.meshlab.net/

Technical briefing ● Post Processing(2/2) ○ ○ If the floor surface is NOT flat, it is difficult to run for mobile robot. So, I used MeshLab to make floor surface flat. before after Z Z 14

• https://www.meshlab.net/

Technical briefing ● Making of Gazebo World（1/2） ○ I specified the file path of mesh data in <collision> tag in SDF file. <model name="mesh2gazebo"> <static>true</static> <link name="link"> <collision name="collision"> <geometry> <mesh> <scale>1 1 1</scale> <uri>file:///home/jazzy/Scaniverse_2024_11_18_162729.dae</uri> </mesh> </geometry> </collision> Filepath of ... mesh data </link> </model> 15

Technical briefing ● Making of Gazebo World（2/2） ○ I specified the file path of mesh data in <visual> tag in SDF file. <model name="mesh2gazebo"> <static>true</static> <link name="link"> ... <visual name="visual"> <geometry> <mesh> <scale>1 1 1</scale> <uri>file:///home/jazzy/Scaniverse_2024_11_18_162729.dae</uri> </mesh> </geometry> Filepath of </visual> mesh data </link> </model> 16

Future works ● I’ll publish the repository of mesh2gazebo. ○ This repository has the template of Gazebo World using mesh data. 17

Appendix：Tips for Gazebo ● Decrease of CPU usage ○ ○ I disabled cast shadows to decrease CPU usage. So, I set cast_shadows to 0. <light type="directional" name="sun"> <cast_shadows>0</cast_shadows> <pose>0 0 10 0 0 0</pose> <diffuse>0.8 0.8 0.8 1</diffuse> <specular>0.2 0.2 0.2 1</specular> <attenuation> <range>1000</range> <constant>0.9</constant> <linear>0.01</linear> <quadratic>0.001</quadratic> </attenuation> <direction>-0.5 0.1 -0.9</direction> </light> 18

• http://sdformat.org/spec?ver=1.11&amp;elem=light#light_cast_shadows

Acknowledgment The in-store image was collected at EZOHUB TOKYO with the cooperation of SATUDORA HOLDINGS CO.,LTD. 19

• https://ezohub.jp/office/tokyo/