A robotic vision system that mimics key visual functions of the human brain promises to let robots manoeuvre quickly and safely through cluttered environments, and to help guide the visually impaired.

It’s something any toddler can do – cross a cluttered room to find a toy.

It's also one of those seemingly trivial skills that have proved to be extremely hard for computers to master. Analysing shifting and often-ambiguous visual data to detect objects and separate their movement from one’s own has turned out to be an intensely challenging artificial intelligence problem.

Three years ago, researchers at the European-funded research consortium Decisions in Motion (http://www.decisionsinmotion.org/) decided to look to nature for insights into this challenge.

In a rare collaboration, neuro- and cognitive scientists studied how the visual systems of advanced mammals, primates and people work, while computer scientists and roboticists incorporated their findings into neural networks and mobile robots.

The approach paid off. Decisions in Motion has already built and demonstrated a robot that can zip across a crowded room guided only by what it “sees” through its twin video cameras, and are hard at work on a head-mounted system to help visually impaired people get around.

“Until now, the algorithms that have been used are quite slow and their decisions are not reliable enough to be useful,” says project coordinator Mark Greenlee. “Our approach allowed us to build algorithms that can do this on the fly, that can make all these decisions within a few milliseconds using conventional hardware.”

How do we see movement?

The Decisions in Motion researchers used a wide variety of techniques to learn more about how the brain processes visual information, especially information about movement.

These included recording individual neurons and groups of neurons firing in response to movement signals, functional magnetic resonance imaging to track the moment-by-moment interactions between different brain areas as people performed visual tasks, and neuropsychological studies of people with visual processing problems.

The researchers hoped to learn more about how the visual system scans the environment, detects objects, discerns movement, distinguishes between the independent movement of objects and the organism’s own movements, and plans and controls motion towards a goal.

One of their most interesting discoveries was that the primate brain does not just detect and track a moving object; it actually predicts where the object will go.

“When an object moves through a scene, you get a wave of activity as the brain anticipates its trajectory,” says Greenlee. “It’s like feedback signals flowing from the higher areas in the visual cortex back to neurons in the primary visual cortex to give them a sense of what’s coming.”

Greenlee compares what an individual visual neuron sees to looking at the world through a peephole. Researchers have known for a long time that high-level processing is needed to build a coherent picture out of a myriad of those tiny glimpses. What's new is the importance of strong anticipatory feedback for perceiving and processing motion.

“This proved to be quite critical for the Decisions in Motion project,” Greenlee says. “It solves what is called the ‘aperture problem’, the problem of the neurons in the primary visual cortex looking through those little peepholes.”

Building a better robotic brain

Armed with a better understanding of how the human brain deals with movement, the project’s computer scientists and roboticists went to work. Using off-the-shelf hardware, they built a neural network with three levels mimicking the brain’s primary, mid-level, and higher-level visual subsystems.

They used what they had learned about the flow of information between brain regions to control the flow of information within the robotic “brain”.

“It’s basically a neural network with certain biological characteristics,” says Greenlee. “The connectivity is dictated by the numbers we have from our physiological studies.”

The computerised brain controls the behaviour of a wheeled robotic platform supporting a moveable head and eyes, in real time. It directs the head and eyes where to look, tracks its own movement, identifies objects, determines if they are moving independently, and directs the platform to speed up, slow down and turn left or right.

Greenlee and his colleagues were intrigued when the robot found its way to its first target – a teddy bear – just like a person would, speeding by objects that were at a safe distance, but passing nearby obstacles at a slower pace.

”That was very exciting,” Greenlee says. “We didn’t program it in – it popped out of the algorithm.”

In addition to improved guidance systems for robots, the consortium envisions a lightweight system that could be worn like eyeglasses by visually or cognitively impaired people to boost their mobility. One of the consortium partners, Cambridge Research Systems, is developing a commercial version of this, called VisGuide.

Decisions in Motion received funding from the ICT strand of the EU’s Sixth Framework Programme for research. The project’s work was featured in a video by the New Scientist in February this year.

(July 6, 2009) — The cooperative soccer playing robots of the Universität Stuttgart are world champions in the middle size league of robot soccer. After one of the most interesting competitions in the history of Robocup from 29th June to 5th July, 2009, in Graz, the 1. RFC Stuttgart on the last day of the competition succeeded in winning the world championship 2009 in an exciting game against the team of Tech United from Eindhoven (The Netherlands) with the final result of 4:1.

During the competition Stuttgart's robots had to make their way against 13 other teams from eight countries, among them the current world champion Cambada (Portugal). Besides the teams from Germany, Italy, The Netherlands, Portugal, and Austria, teams from China, Japan, and Iran competed against each other.

The 1.RFC Stuttgart includes staff of two Institutes, namely the department of Image Understanding (Head: Prof. Levi) of the Institute of Parallel and Distributed Systems and the Institute of Technical Optics (Head: Prof. Osten), achieved also the 2nd place at the so-called "technical challenge" and a further 1st place at the "scientific challenge".

After the final match of the competition, the middle-size league robots of the 1. RFC Stuttgart - the new world champion - had to play against the human officials of the RoboCup federation. It turned out, that hereby the robots were the inferior team. Clearly the RoboCup community has still to bridge a vast distance to reach their final goal to let a humanoid robot team play against the human world champion by the year 2050.

The success tells its own tale but one might wonder which scientific interest is behind the RoboCup competitions. Preconditions for the successful participation at these competitions are extensive efforts in current research topics of computer science such as real-time image processing and architectures, cooperative robotics and distributed planning. Possible application scenarios of these research activities reach from autonomous vehicles, cooperative manufacturing robotics, service robotics to the point of planetary or deep-sea exploration by autonomous robotic systems. In this context autonomous means that no or only a limited human intervention is necessary.

(July 10, 2009) — Four percent of all children suffer from amblyopia, better known as "lazy eye syndrome." Traditional treatment for the condition requires the use of an eye patch, often for months at a time, before the eye is corrected. This can lead to social stigma during a formative part of childhood, and worse, it's not 100% effective.

Now Tel Aviv University's eye and brain specialist Dr. Uri Polat of the Goldschleger Eye Research Institute has developed a computer therapy that could spare kids from the ugly eye patch, letting them enjoy themselves during therapy. The treatment, currently available for adults only, corrects the activity of the neurons in the brain, the main operator of eye function.

A leading expert in lazy eye syndrome recently assessed Dr. Polat's invention and found that twenty hours in front of Dr. Polat's computer treatment had the same effect as about 500 hours of wearing an eye patch. The review was published recently in Vision Research. Dr. Polat's research group has also reported the new treatment's efficacy in a number of scientific publications, including the Proceedings of the National Academy of Science (PNAS).

Not just any video game will work

In his carefully designed treatment, special and random objects appear, keeping the patient constantly alert and expecting the unexpected. A version of the therapy as a game is now in under development for children.

"As far as I know this is really a one-of-a-kind, non-invasive and effective way to treat lazy eye, without the use of an embarrassing eye patch," says Dr. Polat. "This is probably the first treatment that attempts to correct lazy eyes in adults, something that doctors had previously given up on. Doctors don't suggest intervention after the age of nine, because it usually doesn't work."

Making eye therapy fun

Taking it from the lab bench to a commercial product, Dr. Polat wants to make sure that the treatment will be as stimulating a regular video game. The existing game-like therapy he developed for the computer was "a bit boring," he admits, making it hard for some kids to sit through an entire session of treatment, which can be administered by a parent or therapist at home or at school.

That's why he's now collaborating with researchers at Rochester University in New York, where gaming specialists plan to add more entertainment value to the new therapy while keeping all of its therapeutic power.

"You see these poor kids in kindergarten wearing the patch. Everyone hates it, especially the parents who know what it's doing to their kid's self-esteem," says Dr. Polat. "My aim is to not only treat adults, but to treat kids using a computer two or three times a week, one hour each time, without the need for them having to wear a patch."

Dr. Polat's solution currently has the U.S. Food and Drug Administration seal of approval, Dr. Polat adds.

(July 11, 2009) — A hyper-realistic Einstein robot at the University of California, San Diego has learned to smile and make facial expressions through a process of self-guided learning. The UC San Diego researchers used machine learning to “empower” their robot to learn to make realistic facial expressions.

“As far as we know, no other research group has used machine learning to teach a robot to make realistic facial expressions,” said Tingfan Wu, the computer science Ph.D. student from the UC San Diego Jacobs School of Engineering who presented this advance on June 6 at the IEEE International Conference on Development and Learning.

The faces of robots are increasingly realistic and the number of artificial muscles that controls them is rising. In light of this trend, UC San Diego researchers from the Machine Perception Laboratory are studying the face and head of their robotic Einstein in order to find ways to automate the process of teaching robots to make lifelike facial expressions.

This Einstein robot head has about 30 facial muscles, each moved by a tiny servo motor connected to the muscle by a string. Today, a highly trained person must manually set up these kinds of realistic robots so that the servos pull in the right combinations to make specific face expressions. In order to begin to automate this process, the UCSD researchers looked to both developmental psychology and machine learning.

Developmental psychologists speculate that infants learn to control their bodies through systematic exploratory movements, including babbling to learn to speak. Initially, these movements appear to be executed in a random manner as infants learn to control their bodies and reach for objects.

“We applied this same idea to the problem of a robot learning to make realistic facial expressions,” said Javier Movellan, the senior author on the paper presented at ICDL 2009 and the director of UCSD’s Machine Perception Laboratory, housed in Calit2, the California Institute for Telecommunications and Information Technology.

Although their preliminary results are promising, the researchers note that some of the learned facial expressions are still awkward. One potential explanation is that their model may be too simple to describe the coupled interactions between facial muscles and skin.

To begin the learning process, the UC San Diego researchers directed the Einstein robot head (Hanson Robotics’ Einstein Head) to twist and turn its face in all directions, a process called “body babbling.” During this period the robot could see itself on a mirror and analyze its own expression using facial expression detection software created at UC San Diego called CERT (Computer Expression Recognition Toolbox). This provided the data necessary for machine learning algorithms to learn a mapping between facial expressions and the movements of the muscle motors.

Once the robot learned the relationship between facial expressions and the muscle movements required to make them, the robot learned to make facial expressions it had never encountered.

For example, the robot learned eyebrow narrowing, which requires the inner eyebrows to move together and the upper eyelids to close a bit to narrow the eye aperture.

“During the experiment, one of the servos burned out due to misconfiguration. We therefore ran the experiment without that servo. We discovered that the model learned to automatically compensate for the missing servo by activating a combination of nearby servos,” the authors wrote in the paper presented at the 2009 IEEE International Conference on Development and Learning.

“Currently, we are working on a more accurate facial expression generation model as well as systematic way to explore the model space efficiently,” said Wu, the computer science PhD student. Wu also noted that the “body babbling” approach he and his colleagues described in their paper may not be the most efficient way to explore the model of the face.

While the primary goal of this work was to solve the engineering problem of how to approximate the appearance of human facial muscle movements with motors, the researchers say this kind of work could also lead to insights into how humans learn and develop facial expressions.

“Learning to Make Facial Expressions,” by Tingfan Wu, Nicholas J. Butko, Paul Ruvulo, Marian S. Bartlett, Javier R. Movellan from Machine Perception Laboratory, University of California San Diego. Presented on June 6 at the 2009 IEEE 8^th International Conference On Development And Learning.