The main character of the project is a frog that my best friend gave me. I wanted to see how she (the frog, not my friend) would look in different places that I won’t be in myself anytime soon. To achieve this, I used the generative neural network Stable Diffusion XL.
The project aims to determine if a neural network can reproduce an image of a knitted toy and generate a corresponding background scaled to its size.
Source photographs
First of all, I wanted to see the Frog on the beach.
It is evident that the neural network has learned the general characteristics of the Frog: a knitted body, plastic eyes, and a volumetric line on its face.
prompt = «a photo of JABBA toy on a beach.»
In the early generations, there is a common characteristic: the sand looks like yarn or fabric. I attribute this to the fact that among the original photos of the Frog I used to train the model, there is a separate series against a background with a knitted texture, which the model learned along with the frog.→Original Photo
Despite the decent rendering of the Frog «on the beach, ” I was not satisfied with the generations due to the monotonous sand. Moving forward, I decided to strive for a more richly detailed environment. To achieve this, I added specific refinements to the next prompt.
prompt = «a photo of JABBA toy in the garden among colorful flowers»
I liked the result much more, so I reached out to Deepseek («the task was to write an English prompt to generate an image, starting with 'a photo of JABBA toy, ' and then including an interesting location where a knitted toy could be placed (for example, a garden with flowers, a Gothic cathedral, a fair, a museum, a store, an aquarium, a pond, a hotel in the Alps, a flag spire, an Antarctic glacier). Write about 20 variations, each with 2–3 sentences using epithets») to see how the neural network would render the prompt featuring the «garden.»
It is noticeable that in most generations, the neural network covers the paws with the environment or draws them incorrectly. To solve this problem, the source photo album was expanded. Seven images of the Frog in a single pose on a white background, where the paws are clearly visible, have been added.
prompt = «a photo of a JABBA toy sitting on a giant mushroom in an enchanted fairy garden, surrounded by tiny glowing spirits. Luminescent flowers and sparkling dew drops create a dreamlike, whimsical atmosphere.»
Expanding the album and detailing the prompt improved the generations, so no further edits were necessary. All subsequent generations are presented from most to least successful.
prompt = «a photo of a JABBA toy sitting proudly on a dusty wooden shelf in a cozy vintage toy shop, surrounded by antique porcelain dolls and tin robots. Warm amber lamplight highlights its grotesque yet endearing features.»
prompt = «a photo of JABBA toy perched on a frost-covered pumpkin in an autumn pumpkin patch at dusk. Golden fallen leaves swirl around it while a carved jack-o'-lantern grins menacingly nearby.»
The frog turned out best in the «store shelf» setting, the «fairy garden, ” and on the pumpkin. The body proportions are roughly consistent with the original, the little feet look natural—on some, there are four toes instead of three, but this doesn’t detract from the overall impression. The frog looks great in the image, and the surrounding objects interact realistically with the toy.
prompt = «a photo of a JABBA toy floating in a swirling cosmic nebula among glittering stars and distant galaxies. Radiant purple and gold stardust swirls around its serene, otherworldly expression.»
It’s interesting how the neural network depicted a frog in space. Instead of placing the toy in realistic cosmic nebulae, it drew a «hand-made» galaxy, even though that wasn’t in the prompt.
prompt = «a photo of a JABBA toy sitting in a lush lavender field in Provence at golden hour. Endless purple rows stretch to the horizon as gentle bees hum around its contented face.»
The frog «in the lavender field» also looks natural. The neural network correctly «placed» the toy among the live plants. The left image has artifacts around the lower limbs, but they are not critical.
prompt = «a photo of JABBA toy perched on a moss-covered rock in a misty ancient forest, surrounded by towering ferns and glowing fireflies. Soft golden sunbeams filter through the emerald canopy, casting a magical glow on its wrinkled green skin.»
«Frog in the Forest,» when contrasted with «Frog in the Lavender Field,» appears less convincing. The placement of the toy and its proportion relative to the surrounding leaves give the impression of a photo collage rather than a photograph.
prompt = «A photo of a JABBA toy nestled in a woven basket on a rustic French bakery counter, surrounded by warm golden croissants and dusty flour. Soft morning light streams through a lace-curtained window.»
prompt = «a photo of a JABBA toy floating lazily in a bubbling hot spring nestled among snow-capped mountains. Thick white steam curls around its leathery body as frost sparkles on the surrounding pine trees.»
A similar issue occurs with «Frog in the Mountains» and «Frog in the Bakery.» The surrounding objects are definitely smaller than they should be.The left image of «in the mountains» is the only one where the neural network «forgot» the knitted texture.
prompt = «a photo of a JABBA toy sprawled across a neon-lit Tokyo street corner at midnight, surrounded by glowing vending machines and rain-slicked asphalt. Vibrant pink and blue reflections dance across its rubbery skin.»
The «Frog in Tokyo» turned out differently. The left one is pretty good, with a minor artifact on its leg, while the right one ended up being a giant frog.
prompt = «a photo of a JABBA toy placed on the snowy windowsill of a cozy Alpine cabin, surrounded by flickering candles. Soft snowfall drifts outside while warm firelight glows from within.»
prompt = «a photo of JABBA toy resting on a velvet throne inside an abandoned Gothic cathedral, with shattered stained-glass windows casting colorful shards of light. Dusty cobwebs drape dramatically over its slimy form.»
Less successful generations were produced with the prompts «in a Gothic cathedral» and «in the Alps.» The paws were noticeably distorted, and the background did not suit the toy. Individual items were not scaled correctly.
