Stable Diffusion tutorial: Stable Diffusion Image Variations using lambda diffusers

by Omar Sesa on NOV 10, 2022

🧐 Short introduction to Stable Diffusion

Stable Diffusion is a text-to-image latent diffusion model created by the researchers and engineers from CompVis, Stability AI and LAION. It's trained on 512x512 images from a subset of the LAION-5B database.

About lambda diffusers

This version of Stable Diffusion has been fine tuned from CompVis/stable-diffusion-v1-3-original to accept CLIP image embedding rather than text embeddings. This allows the creation of "image variations" similar to DALLE-2 using Stable Diffusion. This version of the weights has been ported to huggingface Diffusers, to use this with the Diffusers library requires the Lambda Diffusers repo.

Let's get started and learn how to use Stable Diffusion Image Variations using lambda diffusers!

To run this tutorial we will use Google Colab and Google Drive

βš™οΈ Preparing dependencies

Download the necessary files:

%cd /content
!git clone https://github.com/LambdaLabsML/lambda-diffusers.git

%cd /content/lambda-diffusers
!git checkout 4d4386e5d2ae667d18987b1a505a47efc643e9c9

install the required libs:

%cd /content/lambda-diffusers


# for lambda-diffusers
!pip install -r requirements.txt

Import the required libs

%cd /content/lambda-diffusers

from pathlib import Path
from lambda_diffusers import StableDiffusionImageEmbedPipeline
from PIL import Image
import torch
from torch import autocast

from diffusers import StableDiffusionPipeline
from google.colab.patches import cv2_imshow
import cv2



device = "cuda" if torch.cuda.is_available() else "cpu"

πŸŽ₯ Image to Image

Load the pipeline.

pipe = StableDiffusionImageEmbedPipeline.from_pretrained("lambdalabs/sd-image-variations-diffusers")
pipe = pipe.to(device)

download the intial image:

!rm -rf outputs
!rm -rf inputs
!mkdir outputs inputs

!wget -c https://raw.githubusercontent.com/Omarsesa/Lablab/main/R.jpg \
      -O inputs/test_01.jpg
cv2_imshow(cv2.imread('inputs/test_01.jpg'))

Generate the images

input_img_path = "inputs/test_01.jpg"

im = Image.open(input_img_path)
num_samples = 2
image = pipe(num_samples*[im], guidance_scale=3.0)
image = image["sample"]

base_path = Path("outputs")
base_path.mkdir(exist_ok=True, parents=True)
for idx, im in enumerate(image):
  im.save(base_path/f"{idx:06}.jpg")
  1. we load the image
  2. run the model
  3. save the output images

Show the image

Here we resize the images then concat them horizontally and show them

def hconcat_resize_min(im_list, interpolation=cv2.INTER_CUBIC):
  h_min = min(im.shape[0] for im in im_list)
  im_list_resize = [cv2.resize(im, (int(im.shape[1] * h_min / im.shape[0]), h_min), interpolation=interpolation) for im in im_list]
  
  return cv2.hconcat(im_list_resize)
kinput_cv_im = cv2.imread(input_img_path)

im_h = None
for i in range(num_samples):
  cv_im = cv2.imread(f"outputs/{i:06}.jpg")
  
  if i == 0:
    im_h = hconcat_resize_min([input_cv_im, cv_im])
  else:
    im_h = cv2.hconcat([im_h, cv_im])
    #pass

cv2_imshow(im_h)

As you can see here we have different variations of the intial image

Open in Colab

A big thank you to Hassen Shair for helping me write the tutorial!

Thanks for reading! Wait for next tutorials!

TECHNOLOGIES

profile image

Omar Sesa

I am a professionally qualified computer engineer with 7 years of practical and teaching experience. I received the B.Sc. degree from the Communications and Computers Engineering Department, MET. I'm currently pursuing an M.Sc. degree at the Computers Engineering and Control Systems Department, Faculty of Engineering, Mansoura University, Egypt. I'm also a Teaching Assistant with the Communications and Computers Engineering Department, Misr Higher Institute of Engineering and Technology in Mansoura. My major research interests include applying AI in machine learning, neural networks, modern control theory, embedded systems, bioinformatics, and robotics.

Discover more tutorials with similar technologies