Generative AI models can swiftly create images based on prompts and have been used for various purposes, including identifying inherent biases and preserving memories. Researchers from Stephen James’s Robot Learning Lab in London are now using image-generating AI models to create training data for robots. They have developed a system called Genima, which refines the image-generating AI model Stable Diffusion to draw robots’ movements, aiding them in simulations and the real world. The research is set to be presented at the Conference on Robot Learning (CoRL) soon. The system could simplify the training of various types of robots, from mechanical arms to humanoid robots and driverless cars. It could also enhance the performance of AI web agents, which are capable of executing complex tasks with minimal supervision, by improving their scrolling and clicking abilities. The approach of Genima is distinct because it uses images as both input and output, making it easier for machines to learn from, according to Ivan Kapelyukh, a PhD student at Imperial College London. This not only benefits users by making robot movements more interpretable, but also allows them to anticipate potential issues before deployment.
Genima leverages Stable Diffusion’s pattern recognition capability to convert the model into a decision-making system. The researchers fine-tuned Stable Diffusion to overlay data from robot sensors onto images captured by its cameras. The desired action, such as opening a box, is rendered into a series of colored spheres on top of the image, which directs the robot’s joint movements. The second part of the process involves converting these spheres into actions using another neural network called ACT, and then conducting simulations and real-world tasks using a robot arm, resulting in average success rates of 50% and 64%, respectively. Although the success rates are not particularly high, the researchers are hopeful about improving the robot’s speed and accuracy, particularly by applying Genima to video-generation AI models to predict sequences of future actions. This research has the potential to be useful for training home robots to perform various domestic tasks, and its generalized approach means it can benefit all types of robots, according to Zoey Chen, a PhD student at the University of Washington.
The ainewsarticles.com article you just read is a brief synopsis; the original article can be found here: Read the Full Article…
