The high powered computers used by Google are normally busy with image recognition which requires them to make associations between images to learn what objects, places and people look like. But Google has allowed the androids to identify and curate subtleties, which has resulted in strange images like a knight made from dog faces, strange hallucinatory cloud sheep and merging landscapes of fountains and bridges.
Google is working on artificial neural networks. These use algorithms to try to replicate the way in which humans think, but by using the algorithms in other ways the image recognition computers create weird images and compositions.
The androids need to be trained in order to work well. This is done by exposing them to millions of pictures and then changing settings to show the computer what it has got wrong and what it should do to be correct. An image is passed into a first layer and fed through up to 30 other layers which can allow the computer to identify the things in the image.
The first layer of recognition could pick out the edges of objects in a vague way, creating something that looked like an artist’s stylised sketch of the original image. But by filtering the image through the other layers the computer can recognise what objects those edges resemble. The last layer can put everything together and create a complete image.
In order to experiment with the computers, Google fed images into the other end. The computers were shown what they should see in the final layer and were made to find objects in an unrelated image. Researchers from Google found that the neural networks were able to generate images as well as analyse them and could create patterns and objects in images that weren’t there to begin with. In this way, the networks can take an image of abstract noise and then create objects and features within it that have come from its own imagination.
Google told the machines to over-interpret the images, so that when they thought they saw something in an image they were made to make it look more like their interpretation. For example if a robot saw something that looked like a pig in a cloud, it would amplify the image by making it look more like a pig. Feedback loops would then be used to recognise and emphasise the objects again and again, creating a picture which can barely be recognised from the first.
Neural networks form an important of part of machine learning, in which computer software is refined to understand whether it has got things right or wrong rather than just being programmed to do a task. But that can make it hard for the software to know what it has overlooked and what it should be looking for. Google has shown that when the networks were asked to find dumbbells in pictures of noise, the computers revealed that they believed dumbbells should always have arms holding them. In order to overcome this, the researchers could show the computers more pictures of dumbbells on the ground or stacked up.
The researchers at Google plan to use the findings to understand how algorithms work. To do this they may be able to find out how much the network has learned by exploring the level of abstraction they are using to think about images. The findings may also be used to understand how humans think too.
Google software engineers Alexander Modvinstev, Christopher Olah and Mike Tyka wrote that the techniques help them to understand and visualise how neural networks are able to “carry out difficult classification tasks, improve network architecture, and check what the network has learned during training”. In a post about the techniques, they also said that it makes them wonder if neural networks could become “a tool for artists- a new way to remix visual concepts- or perhaps even shed a little light on the roots of the creative process in general”.
The image recognition software is already being used in Google products- the newly announced Google Photos allows users to search through images by entering text such as the names of animals or objects. The resulting pictures have been categorised by computers. Human input from a large number of users may help to improve this technology for the future by marking correct and incorrect categorisations.