Обучение генеративной сети Stable Diffusion иллюстративной системе для визуальной коммуникации бегового клуба

I have been running for a long time, but I have never joined a running club. For me, running is not so much about results, long distances, or strict pace—it’s more about a pleasant urban ritual that I can practice almost every day: whether I’m in a good mood or tired, after studying or working—simply as a way to recharge and be alone with myself or with other people.The main goal of this project is to create a recognizable visual language for run announcements that can be repeated week after week, by changing the illustration themes and the relevant information. Instead of standard sports posters with aggressive dynamism, sharp lines, and imagery of overcoming challenges, I chose a different mood: calm characters, fluid silhouettes, gentle movement, friendliness, and visual lightness. These illustrations should invite people to run at their own pace—regardless of speed, fitness level, or athletic experience.

I trained Stable Diffusion on a series of illustrations in a unified authorial style. The resulting model generates characters and simple narrative scenes, which are then used to assemble editable posters. These posters allow users to independently change the date, time, distance, and meeting location.

The main idea is not about a single beautiful poster, but about a repeatable communication system and characters that allows for the creation of new run announcements without generating each illustration from scratch.

For the training, I prepared my own illustrations: 17 square PNG images with unified descriptions. The scenarios included solo running, running in pairs, post-run socializing, coffee breaks, warm-ups, resting, and running with a dog. This variety of situations helped the model internalize not only the specific character forms but also the general principles of the style.

What was important to preserve: the coral outline, minimalism, the unified color palette, the rounded plasticity, athletic wear, simple facial representation, and the light background.

I utilized the base generative model Stable Diffusion XL 1.0 combined with the DreamBooth LoRA approach. Instead of fully fine-tuning the large diffusion model, LoRA allows for the retention of discrete, compact trained weights that capture the style of the illustration set. Training was conducted in Google Colab with access to a T4 GPU accelerator.

First, I connected the video card and Google Drive, where I stored the source images, intermediate files, and the training results. The resolution for training was determined by the amount of video memory available: 512 × 512 pixels for GPUs with less than 20 GB. This allowed me to work with SDXL in an accessible environment.

For the assignment, I used 17 source illustrations sharing a unified style: coral outline graphics, simple forms, sporty characters, and a lighthearted mood. The images were saved as 1024 × 1024 pixel PNG files and compiled into a ZIP archive. In Colab, the archive was unpacked, after which the code verified the number of images and the presence of captions for each file.

Next, I prepared the environment for the DreamBooth LoRA: I installed the necessary versions of Diffusers, Transformers, Accelerate, PEFT, and Datasets and connected the SDXL training script. The ImageFolder format was used for the local image collection. A separate metadata.jsonl file contained the captions; the code verified that all 17 examples were loaded and confirmed the presence of the prompt column. These captions helped the model associate visual characteristics with textual descriptions.

I selected Stable Diffusion XL 1.0 as the base model and utilized a DreamBooth LoRA to adapt the style. Key parameters include 500 training steps, LoRA rank 16, batch size 1, gradient accumulation 2, a learning rate of 0.0001, and fp16 precision. The training process uses the keyword runpeachstyle, which ties the prompts to my visual aesthetic. LoRA trains additional parameters, eliminating the need for a full retraining of SDXL.