2016.04.28 Neural Doodle and synthetic images

Today we discussed Neural Doodle: a program to convert simple digital paint "sketches" into complex paintings in the style of famous artists. This same idea was used to upconvert the textures from minecraft.

Steve went over the paper Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis which describes how these types of synthetic images are made in general. Basically you start with a random noise image. Then you run this through a network pre-trained on a large corpus (e.g. ImageNet). You then update the random noise image to match different neuron layers to a style image (earlier neuron layer) and content image (later or output neuron layer). After several iterations the random noise eventually becomes a synthetic image with the "content" or structure of the content image and the style of the style image.