Project Idea
Ivan Bilibin devised what a Russian fairy tale looked like in a book: patterned borders, clear black outlines, flat watercolor washes, fir trees, towers, and sarafans. Many remember his illustrations for «The Frog Princess» from their childhood.
I became curious whether this style could be used to depict an ordinary day. I fine-tuned Stable Diffusion XL on Bilibin’s illustrations and created a series titled «Fairy Tale of the Mundane»: Ivan Tsarevich delivering orders on a scooter, Vasilisa working late on a laptop, bogatyrs waiting for a bus in the rain, and the Frog Princess taking a selfie by a water lily.
Examples of images used to train the generative model
The notebook is based on the DreamBooth LoRA tutorial and the official Diffusers script.— Course Tutorial Notebook
— Official SDXL DreamBooth LoRA Script
— Diffusers DreamBooth Documentation
Examples of images used to train the generative model
Bilibin never drew scooters, laptops, or bus stops. That’s why these scenes clearly show whether the model has learned the style on her own. If the courier on the scooter looks like an illustration from a book, it means it succeeded.
Dataset
For my study, I selected 28 illustrations by Bilibin from the folktales «Tsarevna-Lyagushka, ” „Marya Morevna, ” „Finnist Yasna Sokol’s Feather, ” „The White Duck, ” „Sister Alyonushka and Brother Ivanushka, ” „The Tale of the Golden Rooster, ” „The Tale of the Fisherman and the Fish, ” and „Ivan Tsarevich, the Firebird, and the Gray Wolf.“Bilibin passed away in 1942, and his works are in the public domain. I downloaded all files from Wikimedia Commons; the link and license for each are recorded in the dataset_sources.csv table.
Training sample: 54 frames from 28 Bilibin illustrations
From each scan, I cropped the margins and divided the sheet into two squares: a tall sheet for the top and bottom halves, and a wide sheet for the left and right halves. This preserves more of the composition than cutting in the center, and doubles the number of frames. I cropped the nearly square sheets in the center. From 28 sheets, I produced 54 frames of 768×768.
Cropping Diagram
The model BLIP created the captions for the frames. At the beginning of each caption, I added the trigger word bilibinstyle, and at the end, I added a description of the style. Example: «bilibinstyle, an illustration of a man riding a horse through a forest, Russian fairy tale book illustration, ornamental border, crisp black ink outlines, flat watercolor colors.» The style description is repeated in all captions, so the model associates it with the word bilibinstyle.
Entire selection
Description of the Learning Process
Model: Stable Diffusion XL base 1.0 + LoRA (DreamBooth method), official script train_dreambooth_lora_sdxl.py from the Diffusers library, Google Colab, Tesla T4 GPU
Parameters: size 768 px, batch 1, 600 steps, learning rate 1e-4, fp16, 8-bit Adam, and gradient checkpointing to fit within the memory of a free GPU. Training took 24 minutes, and the loss dropped to 0.01 by the end.
What Went Wrong. The first run crashed: the old torchao library in Colab conflicted with the fresh peft version, so it had to be deleted. Wikimedia wasn’t returning images on frequent requests, so I added pauses and retries. I was making the initial generations at 1024 px, and the model would repeat the plot multiple times in the frame, turning the image into a pattern. Since the model was trained on 768 px, I generated the series at that resolution.
Resulting Images and Their Analysis
LoRA Strength: 0, 0.5, 0.9
First, I tested what the training yields. The prompt, featuring a courier on a scooter in a birch forest, and the seed remained the same—only the LoRA strength changed. At 0, it renders a standard SDXL image, which resembles a comic book. At 0.5, the linework becomes more graphic. At 0.9, the image is closest to Bilibino: muted ochre and greens, a rhythm in the trunks, and graphic grass. The difference isn’t immediately obvious because the prompt’s reference to a book illustration already styles the base model. The signature linework and palette only emerge with the LoRA. For this series, I opted for a strength of 0.9.
In the «Tale of Daily Life» series, there are eight scenes. The characters are taken from the same fairy tales as those in the dataset, but they are engaged in modern-day activities. Ivan Tsarevich delivers orders via electric scooter through a pine forest. Vasilisa the Wise works late at night on a laptop. Baba Yaga’s hut has become a package pickup point. The Firebird glows with a neon sign above a night kiosk. Princess Frog takes a selfie. The Golden Cockerel sits atop a cell phone tower. Three bogatyrs wait for the bus in the rain. The Gray Wolf transports Ivan and the princess through the night city.
All prompts are structured the same way: the trigger word bilibinstyle, the plot, and the same phrase regarding style as in the dataset captions. In the negative prompt, I included photos, 3D, and blur to ensure the image remained illustrative.
What Worked. The style is evident in every frame: a black outline, flat color fills, muted ochre with green tones, rhythmic tree trunks, and patterned fabrics. The scenes featuring modern objects that are large and simple in form turned out best—Vasilyisa’s laptop, the courier’s scooter, the Frog Princess’s smartphone. These objects are drawn with the same outline and colors as the sundresses, fitting seamlessly into the scene.
What Didn’t Work. The model pulls the narrative back into the fairy tale realm. The parcel pickup point morphed into a village fair by the hut, a neon sign became a lantern, and the cell tower became a wooden tower. In one frame, the grey wolf resembles a horse; the dataset has many riders, and the model defaulted to the familiar. The most noticeable issue is with anatomy: hands, fingers, and faces become distorted, especially when there are many people in the frame. None of the frames feature the ornamental border, a key element of the Bilibin style. This is likely due to the cropping: the sheet is divided into two squares, the border is cut, and the model barely sees it in its entirety.
Conclusion. The LoRA learned the color, line, and texture. It failed to learn the composition of a book page or the border. Next time, I would leave the sheets whole and describe the border in the captions using separate keywords.
Description of Generative Model Applications
Stable Diffusion XL base 1.0 (Stability AI) — the base model I fine-tuned using DreamBooth LoRA and used to generate the series. huggingface.co/stabilityai/stable-diffusion-xl-base-1.0SDXL VAE fp16 fix — a decoder for running SDXL in fp16 on a free GPU. huggingface.co/madebyollin/sdxl-vae-fp16-fixBLIP (Salesforce, blip-image-captioning-large) — automated captions for the dataset frames. huggingface.co/Salesforce/blip-image-captioning-largeClaude (Anthropic) — assistance with notebook code, error correction, dataset collection from Wikimedia Commons, and drafting the long-form article structure and text. claude.aiThe concept selection, scene and image curation for the series, and the final conclusions are mine.