Patchwork—also known as scrap quilting—is a creative style characterized by sewing fabric scraps and/or pieces of diverse materials into a cohesive mosaic.
The main feature of patchwork is the uniqueness of each individual piece that forms the larger mosaic. In this project, I aimed to train a model capable of generating such unique fragments. The resulting generations could, in theory, be used as «sketches» for actual quilt patterns or knitting diagrams.
Model Training
For reference, photos of common patchwork fragment types, as well as examples of so-called granny squares—simple knitted squares often used in place of direct fabric scraps in quilting—were utilized. A dataset of 40 images was used for training.
Learning References (Patchwork)
Learning References (Patchwork)
Learning References (Granny Square)
Learning References (Granny Square)
The project was assembled on the Google Colab platform, utilizing libraries provided in the source code. Stable Diffusion served as the primary model, fine-tuned using LoRa and DreamBooth.
The image descriptions in the dataset were automatically generated using the BLIP image captioning model, as well as predefined prefixes and suffixes.
Before starting the course, all dataset images were converted into a uniform, compact format using a dedicated function to simplify the process.
Generations
In conclusion, every generated image proved to be truly unique, consistent with the specified style, and feasible to produce.
However, the model has certain characteristics: when requesting a fragment featuring an animal or object, the model performs significantly better when the «patchwork» parameter is specified. Conversely, when generating colors, the model prefers to use the granny square technique, and patchwork design variations turn out mediocre. Furthermore, the dataset used to train the model contained examples of both requests across both techniques.
The exception to this rule is roses. A request to create a design featuring a rose yields a consistently good result regardless of the specified technique.
