Use LoRA Models with Automatic1111’s Stable Diffusion Web UI

LoRA (Low-Rank Adaptation of Large Linguistic Models) techniques have come to be the norm for expanding the Stable Diffusion models. In this piece, I’ll demonstrate how to utilize LoRA techniques with Automatic1111’s Stable Diffusion Web UI.

Activating the Expansion

• Head to the Extensions section and select Install from URL.
• Type in https://github.com/kohya-ss/sd-webui-additional-networks.git as the URL and hit Install.

Select Installed, then Apply and restart UI.

• A Lora folder already exists in the webui, but it isn’t the default folder for this extension.
• To use this folder instead, select Settings -> Additional Networks.
• Enter the folder path in the first text box.
• Be sure to adjust the path according to your own computer’s specifications.
• This folder can be found under models/Lora within your webui folder.

Use a LoRA Model

• Go to https://civitai.com/models/5200/2b-nierautomata-lora-yorha-edition Click Download Latest to obtain the LoRA model.
• Transfer the downloaded model to the folder you chose in the settings.
• Proceed to https://huggingface.co/andite/anything-v4.0/tree/main and download the anything-v4.5 model if you don’t have it yet.
• Restart the webui.
• Enter the following into the Prompt:

yorha no. 2 type b, 1girl, bangs, black blindfold, black dress, black gloves, black hairband, blindfold, blindfold removed, breasts, cleavage cutout, clothing cutout, commentary request, dress, gloves, hairband, half-closed eyes, hand up, highres, io (sinking=carousel), juliet sleeves, long sleeves, looking at viewer, medium breasts, mole, mole under mouth, nier (series), nier automata, no blindfold, parted lips, puffy sleeves, short hair, solo, thighhighs, turtleneck, upper body, white hair, bokeh
<lora:2bNierAutomataLora_v2b:0.5>

• Add Negative prompt

Blur image , to many fingers ,bad face ,(worst quality, low quality:1.3)

• Enter these parameters – Steps: 20, Sampler: DPM++ SDE Karras, CFG scale: 7, Seed: 945719311, Size: 512×512 or try your own thank you !!
• Now click on Generate to generate the image.

In order to utilize the LoRA model, two key pieces of information are necessary, which are highlighted in red in the prompt. The first is the trigger words, “yorha no. 2 type b,” which are typically included in the model description. The second is lora:2bNierAutomataLora_v2b:0.5, which activates the LoRA model.

The format for this is lora:lora_model_file_name:weight, although the file name extension is not required in the prompt. For instance, the file name for this model is 2bNierAutomataLora_v2b.safetensors, but only 2bNierAutomataLora_v2b is used in the expression.

The weight determines the impact of the LoRA model on the original model, and it is suggested by most authors to be set between 0.4 and 0.6. Additionally, some models may require token descriptions for the character, such as hair color, hair style, and eye color. These minimum details should always be included.

You can find more LoRA and other models here and here.