Stable Diffusion tutorial: Stable Diffusion Image Variations using lambda diffusers

Thursday, November 10, 2022 by omarsesa930
Stable Diffusion tutorial: Stable Diffusion Image Variations using lambda diffusers

๐Ÿง Short introduction to Stable Diffusion

Stable Diffusion is a text-to-image latent diffusion model created by the researchers and engineers from CompVis, Stability AI and LAION. It's trained on 512x512 images from a subset of the LAION-5B database.

About lambda diffusers

This version of Stable Diffusion has been fine tuned from CompVis/stable-diffusion-v1-3-original to accept CLIP image embedding rather than text embeddings. This allows the creation of "image variations" similar to DALLE-2 using Stable Diffusion. This version of the weights has been ported to huggingface Diffusers, to use this with the Diffusers library requires the Lambda Diffusers repo.

Let's get started and learn how to use Stable Diffusion Image Variations using lambda diffusers!

To run this tutorial we will use Google Colab and Google Drive

โš™๏ธ Preparing dependencies

Download the necessary files:

%cd /content
!git clone

%cd /content/lambda-diffusers
!git checkout 4d4386e5d2ae667d18987b1a505a47efc643e9c9

install the required libs:

%cd /content/lambda-diffusers

# for lambda-diffusers
!pip install -r requirements.txt

Import the required libs

%cd /content/lambda-diffusers

from pathlib import Path
from lambda_diffusers import StableDiffusionImageEmbedPipeline
from PIL import Image
import torch
from torch import autocast

from diffusers import StableDiffusionPipeline
from google.colab.patches import cv2_imshow
import cv2

device = "cuda" if torch.cuda.is_available() else "cpu"

๐ŸŽฅ Image to Image

Load the pipeline.

pipe = StableDiffusionImageEmbedPipeline.from_pretrained("lambdalabs/sd-image-variations-diffusers")
pipe =

download the intial image:

!rm -rf outputs
!rm -rf inputs
!mkdir outputs inputs

!wget -c \
      -O inputs/test_01.jpg
Tutorial accompaniment image

Generate the images

input_img_path = "inputs/test_01.jpg"

im =
num_samples = 2
image = pipe(num_samples*[im], guidance_scale=3.0)
image = image["sample"]

base_path = Path("outputs")
base_path.mkdir(exist_ok=True, parents=True)
for idx, im in enumerate(image):"{idx:06}.jpg")
  1. we load the image
  2. run the model
  3. save the output images

Show the image

Here we resize the images then concat them horizontally and show them

def hconcat_resize_min(im_list, interpolation=cv2.INTER_CUBIC):
  h_min = min(im.shape[0] for im in im_list)
  im_list_resize = [cv2.resize(im, (int(im.shape[1] * h_min / im.shape[0]), h_min), interpolation=interpolation) for im in im_list]
  return cv2.hconcat(im_list_resize)
kinput_cv_im = cv2.imread(input_img_path)

im_h = None
for i in range(num_samples):
  cv_im = cv2.imread(f"outputs/{i:06}.jpg")
  if i == 0:
    im_h = hconcat_resize_min([input_cv_im, cv_im])
    im_h = cv2.hconcat([im_h, cv_im])


As you can see here we have different variations of the intial image

Tutorial accompaniment image

Open in Colab

A big thank you to Hassen Shair for helping me write the tutorial!

Thanks for reading! Wait for next tutorials!

Discover tutorials with similar technologies

Upcoming AI Hackathons and Events