A computer science team from the University of Texas have developed something which for the longest time has been considered as a purely human trait – the ability to take a brief glimpse at a scene and fill in the blanks.
According to the scientists on the team, this artificial intelligence can generate its own interpretation of an environment after taking a short glimpse around  – filling in all of the blanks it did not have time to “see”. This advancement was made possible through the use of deep learning – machine learning based on the human brains neural network. After being fed thousands of existing environments in learning trials, the AI is now capable of generating it’s own interpretation of its surroundings when presented with glimpses of completely random environments. After taking a short glance at a new environment, it can then fall back on its past knowledge of other surroundings it has seen to help fill in the blanks.
Whats more, the AI is then able to think of where to look next if given the opportunity. For example, when presented with a glimpse of a kitchen, the system will be able to predict where it should look to get the most information to refine its filling of the blanks – such as looking in a certain area where it thinks it will find a fridge in that particular area of the environment.
Since developing this method of training, the computer science team have begun testing a method of reinforcement learning that could speed up the training of such an agent. The method involves two operating agents – the primary agent and a ’sidekick’ agent which helps with the processing of information.
“Just as you bring in prior information about the regularities that exist in previously experienced environments – like all the grocery stores you have ever been to – this agent searches in a non-exhaustive way, it learns to make intelligent guesses about where to gather visual information to succeed in perception tasks.” Said professor Kristen Grauman, who has been leading the project.
The team says its likely the AI will be useful in search and rescue operations, as tasks such as location people, avoiding hazards and searching for logical quick routes through buildings or possible escapes from a house can be thought up by the system as the rescue is taking place.
The challenge facing the scientists is expanding the abilities of the system in order for it to be useful in the proposed search and rescue operations. Currently the system can process information from a single point within the environment it is getting glimpses at, however it cannot move around. Allowing it to learn and predict its environment as it moves in 3D space – as a human does in new environments – is the next goal for the team at the University of Texas.