An experiment to leverage the power of cycle-GANs to generate Studio Ghibli like anime landscapes from live images. Cycle Generative Adversarial Networks, often abbreviated as Cycle GANs, are a powerful class of deep learning models designed for image-to-image translation tasks. They were introduced to the machine learning community by researchers at UC Berkeley in 2017. Cycle GANs are particularly remarkable for their ability to learn mappings between two different domains of images, such as transforming photographs into paintings, making them the perfect tool for this use case.

Data

Japanese landscapes are a mesmerizing fusion of natural beauty and cultural significance, from the iconic Mount Fuji to serene Zen gardens, captivating with their timeless charm and deep spiritual resonance. I used a selenium script to scrape data off of an Instagram page RawJapan. This page contains beautiful pictures of Japanese landscapes and proved to be the best source.

This is what that data looked like:

I used a python script to extract frames from different Ghibli movies. This resulted in a large dataset with tons of duplicate images. I used VisiPics to identify and remove similar images from the dataset and we are left with a diverse collection of Ghibli images.

After training the GANs in cyclic fashion for about 3 epochs, we start getting wonderful results.

You can find details of implementation in the notebook here.