Carnegie Mellon researchers, taking part in DARPA's Mind's Eye program, have created visually intelligent software to recognize human activities in video and then predict what might happen next.
Mind's Eye surveillance to watch, identify and predict human behavior from video Carnegie Mellon researchers, taking part in DARPA's Mind's Eye program, have created visually intelligent software to recognize human activities in video and then predict what might happen next.
If a person holding a gun were to walk up to you, what might you think would happen next? Researchers from Carnegie Mellon have created intelligent software that will identify human activities in videos and then predict what might happen next. It should come as little surprise that the spookily named 'Mind's Eye' program is sponsored by DARPA's Information Innovation Office.
"A truly 'smart' camera would be able to describe with words everything it sees and reason about what it cannot see," said DARPA. Visually intelligent technology previously 'thought' in terms of nouns to describe a scene, but Carnegie Mellon researchers have made smart software that can also think in terms of action verbs. "A video shows a woman carrying a box into a building. Later, it shows her leaving the building without it. What was she doing?" asked Carnegie Mellon University.
The Mind's Eye software "will compare the video motion to actions it's already been trained to recognize (such as walk, jump, and stand) and identify patterns of actions (such as pick up and carry). The software examines these patterns to infer what the person in the video is doing. It also makes predictions about what is likely to happen next and can guess at activities that might be obscured or occur off-camera."
Carnegie Mellon University's National Robotics Engineering Center explained the image below as: "The Mind's Eye program will automate video analysis - recognizing current behavior, interpolating actions that occur off-camera, and predicting future behavior..."