0

Given the unet part of the stable diffusion model: https://huggingface.co/stabilityai/stable-diffusion-2-inpainting/tree/main/unet how can I convert it to model.onnx and weights.bin?

I tried the following but it doesn't create any model:

import gc
from diffusers import StableDiffusionInpaintPipeline, DPMSolverMultistepScheduler
model_id_inpaint = "stabilityai/stable-diffusion-2-inpainting"

pipe_inpaint = StableDiffusionInpaintPipeline.from_pretrained(model_id_inpaint)
scheduler_inpaint = DPMSolverMultistepScheduler.from_config(pipe_inpaint.scheduler.config)

text_encoder_inpaint = pipe_inpaint.text_encoder
text_encoder_inpaint.eval()
unet_inpaint = pipe_inpaint.unet
unet_inpaint.eval()
vae_inpaint = pipe_inpaint.vae
vae_inpaint.eval()

encoder_hidden_state = torch.ones((2, 77, 1024))
latents = torch.randn((2, 9, 64, 64))
t = torch.from_numpy(np.array(1, dtype=np.float32))

with torch.no_grad():
    torch.onnx._export(
    unet_inpaint, 
    (latents, t, encoder_hidden_state), 
    "model.onnx",
     input_names=['latent_model_input', 't', 'encoder_hidden_states'],
     output_names=['out_sample'],
     onnx_shape_inference=False
     )

Or can I somehow manually download the three files from https://huggingface.co/stabilityai/stable-diffusion-2-inpainting/tree/main/unet and convert it to .onnx in another way?

Christoph Rackwitz
  • 11,317
  • 4
  • 27
  • 36
Martin
  • 219
  • 1
  • 10
  • What do you mean by "doesn't create any model"? The script should generate a file named `model.onnx`, or raise an exception on failure - expand on what happens when you run the code. – simeonovich May 23 '23 at 12:42

0 Answers0