Robots are a part of our world – and some of them are really smart. Think of Sophia, the AI-powered robot who knows jokes, opinions and can express like and dislike for various things. Of course, Sophia is a pinnacle of technology, a one of a kind example of how advanced robotics can be. But there are countless other less sophisticated robots, tirelessly working in factories to make everyday items from pieces of furniture to cars. What if we these robots could be immersed in a holographic world created by AR and VR technologies?
Pairing Robots with AR and VR Technologies: a Match Made in Heaven
For human beings, the AR and VR environment is exciting and immersive, allowing interactions with holographic items which could not be possible (or would be very difficult) in the real world. For robots, it is a deep immersion in their native environment: one controlled by computers, created, programmed and with instructions given in computer language.
Simply put, using AR and VR technologies to train and monitor robots means talking to them in their native language. This will certainly speed up the learning process and improve the way people and robots work together.
Here are some current and potential applications of AR and VR in robotics:
1. Using Augmented Reality to Improve Human-Robot Communication
Traditionally, people programmed robots and then left them alone to do their job. But since robots evolved beyond the phase of mindless laborers that perform simple and repetitive tasks, it became necessary for people to interact with robots during various work tasks.
Having robots as coworkers in the near future is a valid possibility. In order to get to that point, humans and robots have to find a way to communicate with each other in real time. A team of robotic specialists at the University of Boulder, Colorado, have come up with an innovative idea: using augmented reality to predict what a drone will do next.
In the experiment, the human operator wore Microsoft HoloLens mixed reality headset in order to see the field of vision of the drone in the AR window and predict its trajectory according to its movements, existing obstacles and other factors.
2. Virtual Reality Trains Robots for Object Identification
The concept of machine learning is simple: an AI-powered robot is exposed to a series of data and learns how to group it in similar categories, discriminate between different items and recognize new items which look similar to those it has already learned.
In real life, these operations are time consuming and complex. However, a team of researchers at the University of California, Berkeley, managed to train a robot to pick objects indicated by the researchers after being trained to recognize various items in a virtual reality environment.
Using VR has the advantage that robots can be taught anything with minimum cost and effort – the human trainer only needs a 3D VR model of the object, not the object itself. Thus, robots can be taught details of the finest biological structures in the human body, or large scale territories for reconnoiter and exploration purposes.
3. Medical Robots Equipped with AR Can Reshape the Future of Disabled Persons
Robots are already used in healthcare. Advanced robotic arms perform delicate surgeries and other robots are present in drug manufacturing facilities. But robots are also capable of providing direct assistance to patients.
A group of students at the Monterrey Institute of Technology and Higher Education, in Mexico, developed an exoskeleton which can assist people with mobility problems to stand, move, sit and balance. Taking one step further, this robotic outer body was equipped with augmented reality capacities, allowing the human operator to view each separate part and choose which one to move.
4. The Future of the Justice System: Virtual Reality Crime Scene Robots
Jurors have a difficult task: they need to give a guilty or not guilty verdict based on spoken, written and photographic evidence. Their decision will determine another person’s future: freedom or imprisonment. Thus, it is of the utmost importance that each member of a jury has a clear and unbiased image of what happened in the respective case.
In the future, they could be assisted in this task by a robot equipped with virtual reality technology that will allow jurors to explore the crime scene as many times as they need, from every angle, and to focus on every minute detail. At least, this is how a PhD researcher in forensic science and criminal investigations, who is working on such a robot, inspired by the Mars Curiosity rover created by NASA, believes robots can be employed in serving justice.
5. Programming by Demonstration Taken to the Next Level
Programming by demonstration is similar to employee training: the human operator demonstrates a task to a robot until it is able to replicate it. For simple tasks, there is little need for advanced technology. Things are different when robots need to be trained in very complex or dangerous tasks.
This is where AR and VR technologies step in: the entire demonstration can be created in a virtual environment, or superimposed over the real life environment using augmented reality. For example, OpenAI robots developed by Elon Musk’s company are trained by example in VR using a concept called “Vision Network”. This training concept is also effective for swarm robotics: training a large number of robots that will perform the same tasks.
