Mobile Delivery Robots: Mixed Reality-Based Simulation Relying on ROS and Unity 3D

Yuzhou Liu; Georg Novotny; Nikita Smirnov; Walter Morales Alvarez; Cristina Olaverri-Monreal

doi:10.1109/IV47402.2020.9304701

Mobile Delivery Robots: Mixed Reality-Based Simulation Relying on ROS and Unity 3D

Yuzhou Liu, Georg Novotny, Nikita Smirnov, Walter Morales Alvarez, Cristina Olaverri-Monreal

Department Intelligent Transport Systems

Research output: Chapter in Book/Report/Conference proceeding › Conference proceedings › peer-review

Abstract

In the context of Intelligent Transportation Systems and the delivery of goods, new technology approaches need to be developed in order to cope with certain challenges that last mile delivery entails, such as navigation in an urban environment. Autonomous delivery robots can help overcome these challenges. We propose a method for performing mixed reality (MR) simulation with ROS-based robots using Unity, which synchronizes the real and virtual environment, and simultaneously uses the sensor information of the real robots to locate themselves and project them into the virtual environment, so that they can use their virtual doppelganger to perceive the virtual world. Using this method, real and virtual robots can perceive each other and the environment in which the other party is located, thereby enabling the exchange of information between virtual and real objects. Through this approach a more realistic and reliable simulation can be obtained. Results of the demonstrated use-cases verified the feasibility and efficiency as well as the stability of implementing MR using Unity for Robot Operating System (ROS)-based robots.

Original language	English
Title of host publication	Proceedings IEEE Intelligent Vehicles Symposium 2020
Place of Publication	Las Vegas
Publisher	IEEE
Pages	15-20
Number of pages	6
DOIs	https://doi.org/10.1109/IV47402.2020.9304701
Publication status	Published - Oct 2020

Fields of science

303 Health Sciences
303008 Ergonomics
201306 Traffic telematics
202031 Network engineering
202036 Sensor systems
202038 Telecommunications
202040 Transmission technology
203 Mechanical Engineering
211908 Energy research
211911 Sustainable technologies
102 Computer Sciences
102001 Artificial intelligence
102002 Augmented reality
102003 Image processing
102013 Human-computer interaction
102015 Information systems
102019 Machine learning
102021 Pervasive computing
102024 Usability research
102026 Virtual reality
102029 Practical computer science
102034 Cyber-physical systems
501026 Psychology of perception
501 Psychology
501025 Traffic psychology
201305 Traffic engineering
202 Electrical Engineering, Electronics, Information Engineering
202003 Automation
202030 Communication engineering
202034 Control engineering
202035 Robotics
202037 Signal processing
202041 Computer engineering
203004 Automotive technology
211902 Assistive technologies
211909 Energy technology
211917 Technology assessment
501030 Cognitive science

JKU Focus areas

Digital Transformation
Sustainable Development: Responsible Technologies and Management

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1109/IV47402.2020.9304701

Cite this

@inproceedings{da768d9a0e874e5cb5db001204646af9,

title = "Mobile Delivery Robots: Mixed Reality-Based Simulation Relying on ROS and Unity 3D",

abstract = "In the context of Intelligent Transportation Systems and the delivery of goods, new technology approaches need to be developed in order to cope with certain challenges that last mile delivery entails, such as navigation in an urban environment. Autonomous delivery robots can help overcome these challenges. We propose a method for performing mixed reality (MR) simulation with ROS-based robots using Unity, which synchronizes the real and virtual environment, and simultaneously uses the sensor information of the real robots to locate themselves and project them into the virtual environment, so that they can use their virtual doppelganger to perceive the virtual world. Using this method, real and virtual robots can perceive each other and the environment in which the other party is located, thereby enabling the exchange of information between virtual and real objects. Through this approach a more realistic and reliable simulation can be obtained. Results of the demonstrated use-cases verified the feasibility and efficiency as well as the stability of implementing MR using Unity for Robot Operating System (ROS)-based robots.",

author = "Yuzhou Liu and Georg Novotny and Nikita Smirnov and {Morales Alvarez}, Walter and Cristina Olaverri-Monreal",

year = "2020",

month = oct,

doi = "10.1109/IV47402.2020.9304701",

language = "English",

pages = "15--20",

booktitle = "Proceedings IEEE Intelligent Vehicles Symposium 2020",

publisher = "IEEE",

}

TY - GEN

T1 - Mobile Delivery Robots: Mixed Reality-Based Simulation Relying on ROS and Unity 3D

AU - Liu, Yuzhou

AU - Novotny, Georg

AU - Smirnov, Nikita

AU - Morales Alvarez, Walter

AU - Olaverri-Monreal, Cristina

PY - 2020/10

Y1 - 2020/10

N2 - In the context of Intelligent Transportation Systems and the delivery of goods, new technology approaches need to be developed in order to cope with certain challenges that last mile delivery entails, such as navigation in an urban environment. Autonomous delivery robots can help overcome these challenges. We propose a method for performing mixed reality (MR) simulation with ROS-based robots using Unity, which synchronizes the real and virtual environment, and simultaneously uses the sensor information of the real robots to locate themselves and project them into the virtual environment, so that they can use their virtual doppelganger to perceive the virtual world. Using this method, real and virtual robots can perceive each other and the environment in which the other party is located, thereby enabling the exchange of information between virtual and real objects. Through this approach a more realistic and reliable simulation can be obtained. Results of the demonstrated use-cases verified the feasibility and efficiency as well as the stability of implementing MR using Unity for Robot Operating System (ROS)-based robots.

AB - In the context of Intelligent Transportation Systems and the delivery of goods, new technology approaches need to be developed in order to cope with certain challenges that last mile delivery entails, such as navigation in an urban environment. Autonomous delivery robots can help overcome these challenges. We propose a method for performing mixed reality (MR) simulation with ROS-based robots using Unity, which synchronizes the real and virtual environment, and simultaneously uses the sensor information of the real robots to locate themselves and project them into the virtual environment, so that they can use their virtual doppelganger to perceive the virtual world. Using this method, real and virtual robots can perceive each other and the environment in which the other party is located, thereby enabling the exchange of information between virtual and real objects. Through this approach a more realistic and reliable simulation can be obtained. Results of the demonstrated use-cases verified the feasibility and efficiency as well as the stability of implementing MR using Unity for Robot Operating System (ROS)-based robots.

U2 - 10.1109/IV47402.2020.9304701

DO - 10.1109/IV47402.2020.9304701

M3 - Conference proceedings

SP - 15

EP - 20

BT - Proceedings IEEE Intelligent Vehicles Symposium 2020

PB - IEEE

CY - Las Vegas

ER -