Stable Diffusion 3 Medium Fine-tuning Tutorial
Interested in fine-tuning your own image models with Stable Diffusion 3 Medium?
In this tutorial, we’ll walk you through the steps to fine-tune Stable Diffusion 3 Medium (SD3M) to generate high-quality, customized images. If you’re familiar with SD1.5 or SDXL, this guide will highlight the key differences in fine-tuning with SD3M and provide insights into how you can get started.
What You’ll Learn:
How to perform full fine-tuning and LoRA training on SD3M.
The differences between fine-tuning SD3 Medium and previous models like SD1.5 and SDXL.
Quick-start configurations for getting better results with SD3 Medium.
Sneak peek of our upcoming image model.
Why Fine-Tune with SD3 Medium?
Stable Diffusion 3 Medium offers improved model architecture and more flexibility for creative control. Fine-tuning SD3M allows you to tailor the model to your specific image generation needs, whether you're focused on style, composition, or unique subject matter. If you’ve already worked with SD1.5 or SDXL, you’ll notice SD3M provides enhanced output quality and faster training times.
Who Is This For?
This guide is designed for engineers or technical users with experience or interest in fine-tuning machine learning models. If you're already familiar with fine-tuning SD1.5 or SDXL, this guide will help you transition to SD3M.
Meet the Author
This guide has been written by Yeo Wang, a Generative Media Solutions Engineer at Stability AI. In this tutorial, he shares insights from his own experience fine-tuning SD3 Medium and provides quick-start configurations for both full fine-tuning and LoRA training. You might have seen some of his videos on YouTube or know about him through the community (Github).
Base model
Fine-tuned model
Prompt: A three fourth perspective portrait view of a young woman with messy blonde hair and light purple eyes, looking at viewer with a closed mouth smile, slightly visible right pointy fantasy ear, wearing a black feather hair tie on right side of hair, wearing a pink feather above right ear, wearing silver earrings, wearing baggy white collared shirt with a black cloak wrapped around shoulders, bright yellow rim light hitting left side of face, cropped, a faded pink simple background during golden hour.
Base model
Fine-tuned model
Prompt: A person stands in the foreground with their back turned to the camera, appearing to be about to enter a doorway. They have short hair and are dressed in casual clothing. The background features a misty, dimly-lit street lined with cars, old building facades, and a brightly lit gas station sign that reads "iperoil" with prices 1.775 and 1.699 visible. The style conveys a gritty, realistic urban environment, highlighted by the vintage design of the gas station sign. The scene appears to be set late at night or early dawn, with moody, greenish lighting shrouded in fog, giving a sense of quiet solitude and contemplation.
Base model
Fine-tuned model
Prompt: A front wide view of a small cyberpunk city with futuristic skyscrapers with gold rooftops situated on the side of a cliff overlooking an ocean, day time view with green tones, some boats floating in the foreground on top of reflective orange water, large mechanical robot structure reaching high above the clouds in the far background, atmospheric perspective, teal sky.
