Training for a generic neuronet to fit the style of the artist Victor Sire

Victor Sire is a French painter who works at the intersection of textiles, painting, and cultural archaeology. Its work consists of complex embroidery that binds the visual codes of pop culture, architectural motives, cartoon images, video games, old television. His inspiration ranges from series and B-movies to aesthetics of the «atomic age» and suburban welfare. Attention to postmodern architecture and decorative structures makes his creativity both ironic, disturbing and deeply personal. It works slowly, manually, for many hours a day, creating what it calls a «fragmented landscape», a visual puzzle of cultural backslides.

A LoRA model based on his work is an attempt to capture the very sense of a built world in which the image is born at the junction of nostalgia, fragment and craft. The artist says he starts with «sniffs of sketches from notebooks,» collecting landscapes where skyscrapers, cartoon silhouettes, ornaments, pieces of TV shows, and architectural details are adjacent. All of this gets into the fabric of the image as fragments of cultural memory — familiar and strange at the same time, as if they came from a dream recorded on an old film.

In this series, I deliberately withdrew from Victor Sire’s usual visual codes to explore the territory of my own associations and alarms, not through quotations, but through the atmosphere. Although the model was trained in material soaked in pop culture, it was important for me to use this lexicon differently, not as entertainment, but as a space of decay and internal tension. The starting point was the motive of lightning — as a symbol of a sudden breakdown in a stable, bound-up reality. In the art textile, lightning sounds particularly inappropriate: it’s like a mistake in the tent, like a visual intervention in «they.» That was the starting point: the desire to create a series where the fabric of the world is soaked with current, where normality is about to crack.

I wanted to build my own labyrinth of exclusion, where the American dream’s place is a restless void inhabited by a cross. It’s like the City of Zero wasn’t filmed in Kaluga, but on the highway between Palm Springs and Las Vegas, under the music of a synthesizer and with an impending storm.

To learn the neuronet model, I have collected a dataset that includes 45 works by Victor Sire, his portfolio currently contains about 60 pieces. The work was selected manually to cover as fully as possible the main features of Sire’s visual language: composites, architectural motives, colour balance features, a characteristic textile invoice, and sometimes large scene plans were added.

After sampling the images were manually presented to the same format. All the works were converted into a square side ratio of 1:1, with a resolution of 512×512 pixels, which ensured that the visual details were of sufficient quality while maintaining a reasonable amount of data for learning.

After processing the images, I used the BLIP model to generate automatic signatures. It gave each image a short textual annotation that reflected its visual content. The maximum length of the description was limited to 50 currents. The results were collected in .jsonl format, where each line matched the image and its text signature. This file was later used as an annotated dataset for the LoRA model.

I went on to learn LoRA on the Stable Diffusion XL (SDXL) base. This used a script train_dreambooth_lora_sdxl.py with a set of optimized parameters. The base model was selected as stableai/stable-discussion-xl-base-1.0, with the VAE model madebyollin/sdxl-vae-fp16-fix. The training was carried out on the «victorsiret» dataset, annotated through a prompt column containing previously generated signatures. The image resolution was set at 1024×1024 pixel, which allowed for the preservation of small textile and architectural parts.

After the first iteration of the training, it seemed to me that the model had begun to capture the main features of the style, ranging from interior geometry to a typical bound invoice. Look, for example, at this cute ghost cat at the fireplace. This first version can be seen on Hugging Face. However, visually I felt that the number of stitches, the invoicing, and the density of the drawing remained insufficient. The visual landscape was like a draft — stylistically accurate but technically flat. This led me to think of trying a more recent, powerful model architecture that can convey greater depth and microdynamics of textiles.

In the second iteration, I decided to move to the «Stable Diffusion 1.5» model (runwayml/stable-discussion-v1-5), because it gives a more rich and «tactile» picture closer to the sense of textiles (it was originally desirable to try FLUX, but even GPU P100 could not pull it). The image resolution was reduced to 512×512 to match the original embroidery scale and to maintain the clearness of the stitch. I also changed the learning parameters by reducing the range to 5e-5, using cosine sceduler with warmer (lr_warmup_steps=100) and increasing the number of steps to 2000 to achieve more sustainable convergence. All memory optimizations — fp16, gradiant checkpointing, 8bit Adam — were saved. These changes have resulted in greater density, depth and expression in the output of the generation.