Olive Around the World

I love to travel, but I can’t take my beloved cat, Oliva, with me. So, I wanted to create an alternative reality where Oliva can travel with me.For this project, I gathered a dataset of photos of Oliva and trained my own LoRA model based on Stable Diffusion XL. My goal was to teach the neural network to recognize the appearance of my specific cat and then generate photos of her in different locations around the world.

In the first phase, a dataset of 16 personal photographs, cropped to a square format, was collected.

In Google Colab, I prepared the following environment:— the execution environment was switched to GPU mode;
— the following libraries were installed: diffusers, transformers, accelerate, peft, bitsandbytes;
— the DreamBooth and LoRA training scripts were uploaded.Afterward, the images were uploaded to Colab and checked for correctness.

To improve the model’s learning, I utilized the BLIP model, which automatically generated text descriptions for each photograph of Olivy. I appended the general descriptor «a photo of TOK cat» to every description, using TOK cat as Olivy’s unique identifier. This helped the model associate the photographs with a specific character, remember my cat’s characteristic features, and subsequently reproduce her in new images and settings.

For training my model, I utilized DreamBooth and LoRA. Stable Diffusion XL (SDXL) was chosen as the base model. I prepared photographs of my cat, Olivka, at a resolution of 1800×1800 px for training. To reduce training time and GPU resource consumption, the number of steps was limited to 500. Upon completion of the training, the resulting LoRA model was saved and uploaded to Hugging Face.After training, I connected the LoRA to the Stable Diffusion XL base model and began generating new images using prompts. As part of the «Olivka Around the World» project, I created photos of Olivka in various countries and cities, utilizing recognizable landmarks: the Eiffel Tower in Paris, the Great Wall of China, Senso-ji Temple in Tokyo, Big Ben in London, the Burj Khalifa in Dubai, and other locations. The main objective was to preserve Olivka’s recognizable appearance while placing her in new settings and creating the impression of a genuine journey.

After training the model, I was able to generate a series of images featuring Olivka in different countries and cities. In most generations, the model maintained her key physical features: the color of her fur, the shape of her muzzle, and her overall appearance. Thanks to this, Olivka remained a recognizable character throughout the entire series.

During the generation process, I noticed that the result heavily depends on the prompt’s phrasing. If I specified only the city or country, the setting often turned out neutral, making the location difficult to identify. More successful results were achieved by specifying particular landmarks, such as the Eiffel Tower, the Great Wall of China, or Big Ben.

However, the model did not always accurately maintain Olive’s appearance. In some generations, facial features, body proportions, or color details changed. The more complex the prompt was, and the more supplementary elements needed to be created, the more noticeable these alterations could become.

As a result of the experiment, I concluded that the most effective prompts are short and specific, structured as «a photo of TOK cat» and specifying one recognizable location. This approach allowed me to generate a more cohesive series while maintaining a balance between Olivy’s recognizability and the diversity of the locations on her virtual journey.

For me, this project became a way to merge my personal history, love for travel, and the capabilities of generative artificial intelligence. Even though the model doesn’t always perfectly replicate Olivka’s appearance, the final series conveys the initial idea—to create an imagined journey where I can bring my pet along.