Computer scientists developed a deep learning method to create realistic objects for virtual environments that can be used to train robots. The researchers used TACC’s Maverick2 supercomputer to train the generative adversarial network. The network is the first that can produce colored point clouds with fine details at multiple resolutions.
To navigate built environments, robots must be able to sense and make decisions about how to interact with their locale. Researchers at the company were interested in using machine and deep learning to train their robots to learn about objects, but doing so requires a large dataset of images. While there are millions of photos and videos of rooms, none were shot from the vantage point of a robotic vacuum. Efforts to train using images with human-centric perspectives failed.
Beksi’s research focuses on robotics, computer vision, and cyber-physical systems. «In particular, I’m interested in developing algorithms that enable machines to learn from their interactions with the physical world and autonomously acquire skills necessary to execute high-level tasks,» he said.
Years later, now with a research group including six PhD computer science students, Beksi recalled the Roomba training problem and begin exploring solutions. A manual approach, used by some, involves using an expensive 360 degree camera to capture environments (including rented Airbnb houses) and custom software to stitch the images back into a whole. But Beksi believed the manual capture method would be too slow to succeed.
Instead, he looked to a form of deep learning known as generative adversarial networks, or GANs, where two neural networks contest with each other in a game until the ‘generator’ of new data can fool a ‘discriminator.’ Once trained, such a network would enable the creation of an infinite number of possible rooms or outdoor environments, with different kinds of chairs or tables or vehicles with slightly different forms, but still — to a person and a robot — identifiable objects with recognizable dimensions and characteristics.
«You can perturb these objects, move them into new positions, use different lights, color and texture, and then render them into a training image that could be used in dataset,» he explained. «This approach would potentially provide limitless data to train a robot on.»
«Manually designing these objects would take a huge amount of resources and hours of human labor while, if trained properly, the generative networks can make them in seconds,» said Mohammad Samiul Arshad, a graduate student in Beksi’s group involved in the research.
Story Source: Materials provided by University of Texas at Austin, Texas Advanced Computing Center. Original written by Aaron Dubrow. Note: Content may be edited for style and length.