Downgrade Gradio to 4.38.1

ImageEditor color switching is broken on newer versions.

.gitignore

@@ -2,6 +2,7 @@ __pycache__
 *.ckpt
 *.safetensors
 *.pth
+.DS_Store
 /ESRGAN/*
 /SwinIR/*
 /repositories
@@ -40,3 +41,4 @@ notification.mp3
 /test/test_outputs
 /cache
 trace.json
+/sysinfo-????-??-??-??-??.json

CHANGELOG.md

@@ -1,8 +1,14 @@
+## 1.10.1
+
+### Bug Fixes:
+* fix image upscale on cpu ([#16275](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16275))
+
+
 ## 1.10.0
 
 ### Features:
 * A lot of performance improvements (see below in Performance section)
-* Stable Diffusion 3 support ([#16030](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16030))
+* Stable Diffusion 3 support ([#16030](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16030), [#16164](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16164), [#16212](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16212))
   * Recommended Euler sampler; DDIM and other timestamp samplers currently not supported
   * T5 text model is disabled by default, enable it in settings
 * New schedulers:
@@ -11,6 +17,7 @@
   * Normal ([#16149](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16149))
   * DDIM ([#16149](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16149))
   * Simple ([#16142](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16142))
+  * Beta ([#16235](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16235))
 * New sampler: DDIM CFG++ ([#16035](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16035))
 
 ### Minor:
@@ -25,6 +32,8 @@
 * Add option to enable clip skip for clip L on SDXL ([#15992](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15992))
 * Option to prevent screen sleep during generation ([#16001](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16001))
 * ToggleLivePriview button in image viewer ([#16065](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16065))
+* Remove ui flashing on reloading and fast scrollong ([#16153](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16153))
+* option to disable save button log.csv ([#16242](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16242))
 
 ### Extensions and API:
 * Add process_before_every_sampling hook ([#15984](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15984))
@@ -73,6 +82,10 @@
 * Fix SD2 loading ([#16078](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16078), [#16079](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16079))
 * fix infotext Lora hashes for hires fix different lora ([#16062](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16062))
 * Fix sampler scheduler autocorrection warning ([#16054](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16054))
+* fix ui flashing on reloading and fast scrollong ([#16153](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16153))
+* fix upscale logic ([#16239](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16239))
+* [bug] do not break progressbar on non-job actions (add wrap_gradio_call_no_job) ([#16202](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16202))
+* fix OSError: cannot write mode P as JPEG ([#16194](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16194))
 
 ### Other:
 * fix changelog #15883 -> #15882 ([#15907](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15907))
@@ -89,10 +102,17 @@
 * Bump spandrel to 0.3.4 ([#16144](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16144))
 * Defunct --max-batch-count ([#16119](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16119))
 * docs: update bug_report.yml ([#16102](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16102))
-* Maintaining Project Compatibility for Python 3.9 Users Without Upgrade Requirements. ([#16088](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16088))
+* Maintaining Project Compatibility for Python 3.9 Users Without Upgrade Requirements. ([#16088](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16088), [#16169](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16169), [#16192](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16192))
 * Update torch for ARM Macs to 2.3.1 ([#16059](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16059))
 * remove deprecated setting dont_fix_second_order_samplers_schedule ([#16061](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16061))
 * chore: fix typos ([#16060](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16060))
+* shlex.join launch args in console log ([#16170](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16170))
+* activate venv .bat ([#16231](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16231))
+* add ids to the resize tabs in img2img ([#16218](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16218))
+* update installation guide linux ([#16178](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16178))
+* Robust sysinfo ([#16173](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16173))
+* do not send image size on paste inpaint ([#16180](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16180))
+* Fix noisy DS_Store files for MacOS ([#16166](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16166))
 
 
 ## 1.9.4

README.md

@@ -78,7 +78,7 @@ A web interface for Stable Diffusion, implemented using Gradio library.
 - Clip skip
 - Hypernetworks
 - Loras (same as Hypernetworks but more pretty)
-- A separate UI where you can choose, with preview, which embeddings, hypernetworks or Loras to add to your prompt 
+- A separate UI where you can choose, with preview, which embeddings, hypernetworks or Loras to add to your prompt
 - Can select to load a different VAE from settings screen
 - Estimated completion time in progress bar
 - API
@@ -122,16 +122,38 @@ Alternatively, use online services (like Google Colab):
 # Debian-based:
 sudo apt install wget git python3 python3-venv libgl1 libglib2.0-0
 # Red Hat-based:
-sudo dnf install wget git python3 gperftools-libs libglvnd-glx 
+sudo dnf install wget git python3 gperftools-libs libglvnd-glx
 # openSUSE-based:
 sudo zypper install wget git python3 libtcmalloc4 libglvnd
 # Arch-based:
 sudo pacman -S wget git python3
 ```
+If your system is very new, you need to install python3.11 or python3.10:
+```bash
+# Ubuntu 24.04
+sudo add-apt-repository ppa:deadsnakes/ppa
+sudo apt update
+sudo apt install python3.11
+
+# Manjaro/Arch
+sudo pacman -S yay
+yay -S python311 # do not confuse with python3.11 package
+
+# Only for 3.11
+# Then set up env variable in launch script
+export python_cmd="python3.11"
+# or in webui-user.sh
+python_cmd="python3.11"
+```
 2. Navigate to the directory you would like the webui to be installed and execute the following command:
 ```bash
 wget -q https://raw.githubusercontent.com/AUTOMATIC1111/stable-diffusion-webui/master/webui.sh
 ```
+Or just clone the repo wherever you want:
+```bash
+git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui
+```
+
 3. Run `webui.sh`.
 4. Check `webui-user.sh` for options.
 ### Installation on Apple Silicon

extensions-builtin/Lora/network.py

@@ -7,6 +7,7 @@ import torch.nn as nn
 import torch.nn.functional as F
 
 from modules import sd_models, cache, errors, hashes, shared
+import modules.models.sd3.mmdit
 
 NetworkWeights = namedtuple('NetworkWeights', ['network_key', 'sd_key', 'w', 'sd_module'])
 
@@ -114,7 +115,10 @@ class NetworkModule:
         self.sd_key = weights.sd_key
         self.sd_module = weights.sd_module
 
-        if hasattr(self.sd_module, 'weight'):
+        if isinstance(self.sd_module, modules.models.sd3.mmdit.QkvLinear):
+            s = self.sd_module.weight.shape
+            self.shape = (s[0] // 3, s[1])
+        elif hasattr(self.sd_module, 'weight'):
             self.shape = self.sd_module.weight.shape
         elif isinstance(self.sd_module, nn.MultiheadAttention):
             # For now, only self-attn use Pytorch's MHA

extensions-builtin/Lora/network_lora.py

@@ -1,6 +1,7 @@
 import torch
 
 import lyco_helpers
+import modules.models.sd3.mmdit
 import network
 from modules import devices
 
@@ -10,6 +11,13 @@ class ModuleTypeLora(network.ModuleType):
         if all(x in weights.w for x in ["lora_up.weight", "lora_down.weight"]):
             return NetworkModuleLora(net, weights)
 
+        if all(x in weights.w for x in ["lora_A.weight", "lora_B.weight"]):
+            w = weights.w.copy()
+            weights.w.clear()
+            weights.w.update({"lora_up.weight": w["lora_B.weight"], "lora_down.weight": w["lora_A.weight"]})
+
+            return NetworkModuleLora(net, weights)
+
         return None
 
 
@@ -29,7 +37,7 @@ class NetworkModuleLora(network.NetworkModule):
         if weight is None and none_ok:
             return None
 
-        is_linear = type(self.sd_module) in [torch.nn.Linear, torch.nn.modules.linear.NonDynamicallyQuantizableLinear, torch.nn.MultiheadAttention]
+        is_linear = type(self.sd_module) in [torch.nn.Linear, torch.nn.modules.linear.NonDynamicallyQuantizableLinear, torch.nn.MultiheadAttention, modules.models.sd3.mmdit.QkvLinear]
         is_conv = type(self.sd_module) in [torch.nn.Conv2d]
 
         if is_linear:

extensions-builtin/Lora/networks.py

@@ -1,3 +1,4 @@
+from __future__ import annotations
 import gradio as gr
 import logging
 import os
@@ -19,6 +20,7 @@ from typing import Union
 
 from modules import shared, devices, sd_models, errors, scripts, sd_hijack
 import modules.textual_inversion.textual_inversion as textual_inversion
+import modules.models.sd3.mmdit
 
 from lora_logger import logger
 
@@ -165,12 +167,26 @@ def load_network(name, network_on_disk):
 
     keys_failed_to_match = {}
     is_sd2 = 'model_transformer_resblocks' in shared.sd_model.network_layer_mapping
+    if hasattr(shared.sd_model, 'diffusers_weight_map'):
+        diffusers_weight_map = shared.sd_model.diffusers_weight_map
+    elif hasattr(shared.sd_model, 'diffusers_weight_mapping'):
+        diffusers_weight_map = {}
+        for k, v in shared.sd_model.diffusers_weight_mapping():
+            diffusers_weight_map[k] = v
+        shared.sd_model.diffusers_weight_map = diffusers_weight_map
+    else:
+        diffusers_weight_map = None
 
     matched_networks = {}
     bundle_embeddings = {}
 
     for key_network, weight in sd.items():
-        key_network_without_network_parts, _, network_part = key_network.partition(".")
+
+        if diffusers_weight_map:
+            key_network_without_network_parts, network_name, network_weight = key_network.rsplit(".", 2)
+            network_part = network_name + '.' + network_weight
+        else:
+            key_network_without_network_parts, _, network_part = key_network.partition(".")
 
         if key_network_without_network_parts == "bundle_emb":
             emb_name, vec_name = network_part.split(".", 1)
@@ -182,7 +198,11 @@ def load_network(name, network_on_disk):
                 emb_dict[vec_name] = weight
             bundle_embeddings[emb_name] = emb_dict
 
-        key = convert_diffusers_name_to_compvis(key_network_without_network_parts, is_sd2)
+        if diffusers_weight_map:
+            key = diffusers_weight_map.get(key_network_without_network_parts, key_network_without_network_parts)
+        else:
+            key = convert_diffusers_name_to_compvis(key_network_without_network_parts, is_sd2)
+
         sd_module = shared.sd_model.network_layer_mapping.get(key, None)
 
         if sd_module is None:
@@ -346,6 +366,28 @@ def load_networks(names, te_multipliers=None, unet_multipliers=None, dyn_dims=No
     purge_networks_from_memory()
 
 
+def allowed_layer_without_weight(layer):
+    if isinstance(layer, torch.nn.LayerNorm) and not layer.elementwise_affine:
+        return True
+
+    return False
+
+
+def store_weights_backup(weight):
+    if weight is None:
+        return None
+
+    return weight.to(devices.cpu, copy=True)
+
+
+def restore_weights_backup(obj, field, weight):
+    if weight is None:
+        setattr(obj, field, None)
+        return
+
+    getattr(obj, field).copy_(weight)
+
+
 def network_restore_weights_from_backup(self: Union[torch.nn.Conv2d, torch.nn.Linear, torch.nn.GroupNorm, torch.nn.LayerNorm, torch.nn.MultiheadAttention]):
     weights_backup = getattr(self, "network_weights_backup", None)
     bias_backup = getattr(self, "network_bias_backup", None)
@@ -355,21 +397,15 @@ def network_restore_weights_from_backup(self: Union[torch.nn.Conv2d, torch.nn.Li
 
     if weights_backup is not None:
         if isinstance(self, torch.nn.MultiheadAttention):
-            self.in_proj_weight.copy_(weights_backup[0])
-            self.out_proj.weight.copy_(weights_backup[1])
+            restore_weights_backup(self, 'in_proj_weight', weights_backup[0])
+            restore_weights_backup(self.out_proj, 'weight', weights_backup[1])
         else:
-            self.weight.copy_(weights_backup)
+            restore_weights_backup(self, 'weight', weights_backup)
 
-    if bias_backup is not None:
-        if isinstance(self, torch.nn.MultiheadAttention):
-            self.out_proj.bias.copy_(bias_backup)
-        else:
-            self.bias.copy_(bias_backup)
+    if isinstance(self, torch.nn.MultiheadAttention):
+        restore_weights_backup(self.out_proj, 'bias', bias_backup)
     else:
-        if isinstance(self, torch.nn.MultiheadAttention):
-            self.out_proj.bias = None
-        else:
-            self.bias = None
+        restore_weights_backup(self, 'bias', bias_backup)
 
 
 def network_apply_weights(self: Union[torch.nn.Conv2d, torch.nn.Linear, torch.nn.GroupNorm, torch.nn.LayerNorm, torch.nn.MultiheadAttention]):
@@ -388,22 +424,22 @@ def network_apply_weights(self: Union[torch.nn.Conv2d, torch.nn.Linear, torch.nn
 
     weights_backup = getattr(self, "network_weights_backup", None)
     if weights_backup is None and wanted_names != ():
-        if current_names != ():
-            raise RuntimeError("no backup weights found and current weights are not unchanged")
+        if current_names != () and not allowed_layer_without_weight(self):
+            raise RuntimeError(f"{network_layer_name} - no backup weights found and current weights are not unchanged")
 
         if isinstance(self, torch.nn.MultiheadAttention):
-            weights_backup = (self.in_proj_weight.to(devices.cpu, copy=True), self.out_proj.weight.to(devices.cpu, copy=True))
+            weights_backup = (store_weights_backup(self.in_proj_weight), store_weights_backup(self.out_proj.weight))
         else:
-            weights_backup = self.weight.to(devices.cpu, copy=True)
+            weights_backup = store_weights_backup(self.weight)
 
         self.network_weights_backup = weights_backup
 
     bias_backup = getattr(self, "network_bias_backup", None)
     if bias_backup is None and wanted_names != ():
         if isinstance(self, torch.nn.MultiheadAttention) and self.out_proj.bias is not None:
-            bias_backup = self.out_proj.bias.to(devices.cpu, copy=True)
+            bias_backup = store_weights_backup(self.out_proj.bias)
         elif getattr(self, 'bias', None) is not None:
-            bias_backup = self.bias.to(devices.cpu, copy=True)
+            bias_backup = store_weights_backup(self.bias)
         else:
             bias_backup = None
 
@@ -411,6 +447,7 @@ def network_apply_weights(self: Union[torch.nn.Conv2d, torch.nn.Linear, torch.nn
         # Only report if bias is not None and current bias are not unchanged.
         if bias_backup is not None and current_names != ():
             raise RuntimeError("no backup bias found and current bias are not unchanged")
+
         self.network_bias_backup = bias_backup
 
     if current_names != wanted_names:
@@ -418,7 +455,7 @@ def network_apply_weights(self: Union[torch.nn.Conv2d, torch.nn.Linear, torch.nn
 
         for net in loaded_networks:
             module = net.modules.get(network_layer_name, None)
-            if module is not None and hasattr(self, 'weight'):
+            if module is not None and hasattr(self, 'weight') and not isinstance(module, modules.models.sd3.mmdit.QkvLinear):
                 try:
                     with torch.no_grad():
                         if getattr(self, 'fp16_weight', None) is None:
@@ -478,6 +515,24 @@ def network_apply_weights(self: Union[torch.nn.Conv2d, torch.nn.Linear, torch.nn
 
                 continue
 
+            if isinstance(self, modules.models.sd3.mmdit.QkvLinear) and module_q and module_k and module_v:
+                try:
+                    with torch.no_grad():
+                        # Send "real" orig_weight into MHA's lora module
+                        qw, kw, vw = self.weight.chunk(3, 0)
+                        updown_q, _ = module_q.calc_updown(qw)
+                        updown_k, _ = module_k.calc_updown(kw)
+                        updown_v, _ = module_v.calc_updown(vw)
+                        del qw, kw, vw
+                        updown_qkv = torch.vstack([updown_q, updown_k, updown_v])
+                        self.weight += updown_qkv
+
+                except RuntimeError as e:
+                    logging.debug(f"Network {net.name} layer {network_layer_name}: {e}")
+                    extra_network_lora.errors[net.name] = extra_network_lora.errors.get(net.name, 0) + 1
+
+                continue
+
             if module is None:
                 continue
 

extensions-builtin/canvas-zoom-and-pan/javascript/zoom.js

@@ -16,6 +16,20 @@ onUiLoaded(async() => {
     // Helper functions
     // Get active tab
 
+    function debounce(func, wait) {
+        let timeout;
+
+        return function executedFunction(...args) {
+            const later = () => {
+                clearTimeout(timeout);
+                func(...args);
+            };
+
+            clearTimeout(timeout);
+            timeout = setTimeout(later, wait);
+        };
+    }
+
     /**
      * Waits for an element to be present in the DOM.
      */
@@ -58,6 +72,30 @@ onUiLoaded(async() => {
         }
     }
 
+    // // Hack to make the cursor always be the same size
+    function fixCursorSize() {
+        window.scrollBy(0, 1);
+    }
+
+    function copySpecificStyles(sourceElement, targetElement, zoomLevel = 1) {
+        const stylesToCopy = ['top', 'left', 'width', 'height'];
+
+        stylesToCopy.forEach(styleName => {
+            if (sourceElement.style[styleName]) {
+                // Convert style value to number and multiply by zoomLevel.
+                let adjustedStyleValue = parseFloat(sourceElement.style[styleName]) / zoomLevel;
+
+                // Set the adjusted style value back to target element's style.
+                // Important: this will work fine for top and left styles as they are usually in px.
+                // But be careful with other units like em or % that might need different handling.
+                targetElement.style[styleName] = `${adjustedStyleValue}px`;
+            }
+        });
+
+        targetElement.style["opacity"] = sourceElement.style["opacity"];
+    }
+
+
     // Detect whether the element has a horizontal scroll bar
     function hasHorizontalScrollbar(element) {
         return element.scrollWidth > element.clientWidth;
@@ -167,48 +205,6 @@ onUiLoaded(async() => {
         return config;
     }
 
-    /**
-     * The restoreImgRedMask function displays a red mask around an image to indicate the aspect ratio.
-     * If the image display property is set to 'none', the mask breaks. To fix this, the function
-     * temporarily sets the display property to 'block' and then hides the mask again after 300 milliseconds
-     * to avoid breaking the canvas. Additionally, the function adjusts the mask to work correctly on
-     * very long images.
-     */
-    function restoreImgRedMask(elements) {
-        const mainTabId = getTabId(elements);
-
-        if (!mainTabId) return;
-
-        const mainTab = gradioApp().querySelector(mainTabId);
-        const img = mainTab.querySelector("img");
-        const imageARPreview = gradioApp().querySelector("#imageARPreview");
-
-        if (!img || !imageARPreview) return;
-
-        imageARPreview.style.transform = "";
-        if (parseFloat(mainTab.style.width) > 865) {
-            const transformString = mainTab.style.transform;
-            const scaleMatch = transformString.match(
-                /scale\(([-+]?[0-9]*\.?[0-9]+)\)/
-            );
-            let zoom = 1; // default zoom
-
-            if (scaleMatch && scaleMatch[1]) {
-                zoom = Number(scaleMatch[1]);
-            }
-
-            imageARPreview.style.transformOrigin = "0 0";
-            imageARPreview.style.transform = `scale(${zoom})`;
-        }
-
-        if (img.style.display !== "none") return;
-
-        img.style.display = "block";
-
-        setTimeout(() => {
-            img.style.display = "none";
-        }, 400);
-    }
 
     const hotkeysConfigOpts = await waitForOpts();
 
@@ -224,7 +220,6 @@ onUiLoaded(async() => {
         canvas_hotkey_grow_brush: "KeyW",
         canvas_disabled_functions: [],
         canvas_show_tooltip: true,
-        canvas_auto_expand: true,
         canvas_blur_prompt: false,
     };
 
@@ -264,18 +259,6 @@ onUiLoaded(async() => {
     );
     const elemData = {};
 
-    // Apply functionality to the range inputs. Restore redmask and correct for long images.
-    const rangeInputs = elements.rangeGroup ?
-        Array.from(elements.rangeGroup.querySelectorAll("input")) :
-        [
-            gradioApp().querySelector("#img2img_width input[type='range']"),
-            gradioApp().querySelector("#img2img_height input[type='range']")
-        ];
-
-    for (const input of rangeInputs) {
-        input?.addEventListener("input", () => restoreImgRedMask(elements));
-    }
-
     function applyZoomAndPan(elemId, isExtension = true) {
         const targetElement = gradioApp().querySelector(elemId);
 
@@ -289,14 +272,118 @@ onUiLoaded(async() => {
         elemData[elemId] = {
             zoom: 1,
             panX: 0,
-            panY: 0
+            panY: 0,
         };
+
         let fullScreenMode = false;
 
+
+        // Cursor manipulation script for a painting application.
+        // The purpose of this code is to create custom cursors (for painting and erasing)
+        // that can change depending on which button the user presses.
+        // When the mouse moves over the canvas, the appropriate custom cursor also moves,
+        // replicating its appearance dynamically based on various CSS properties.
+
+        // This is done because the original cursor is tied to the size of the kanvas, it can not be changed, so I came up with a hack that creates an exact copy that works properly
+
+        const eraseButton = targetElement.querySelector(`button[aria-label='Erase button']`);
+        const paintButton = targetElement.querySelector(`button[aria-label='Draw button']`);
+
+        const canvasCursors = targetElement.querySelectorAll("span.svelte-btgkrd");
+        const paintCursorCopy = canvasCursors[0].cloneNode(true);
+        const eraserCursorCopy = canvasCursors[1].cloneNode(true);
+
+        canvasCursors.forEach(cursor => cursor.style.display = "none");
+
+        canvasCursors[0].parentNode.insertBefore(paintCursorCopy, canvasCursors[0].nextSibling);
+        canvasCursors[1].parentNode.insertBefore(eraserCursorCopy, canvasCursors[1].nextSibling);
+
+
+        // targetElement.appendChild(paintCursorCopy);
+        // paintCursorCopy.style.display = "none";
+
+        // targetElement.appendChild(eraserCursorCopy);
+        // eraserCursorCopy.style.display = "none";
+
+        let activeCursor;
+
+        paintButton.addEventListener('click', () => {
+            activateTool(paintButton, eraseButton, paintCursorCopy);
+        });
+
+        eraseButton.addEventListener('click', () => {
+            activateTool(eraseButton, paintButton, eraserCursorCopy);
+        });
+
+        function activateTool(activeButton, inactiveButton, activeCursorCopy) {
+            activeButton.classList.add("active");
+            inactiveButton.classList.remove("active");
+
+            // canvasCursors.forEach(cursor => cursor.style.display = "none");
+
+            if (activeCursor) {
+                activeCursor.style.display = "none";
+            }
+
+            activeCursor = activeCursorCopy;
+            // activeCursor.style.display = "none";
+            activeCursor.style.position = "absolute";
+        }
+
+        const canvasAreaEventsHandler = e => {
+
+            canvasCursors.forEach(cursor => cursor.style.display = "none");
+
+            if (!activeCursor) return;
+
+            const cursorNum = eraseButton.classList.contains("active") ? 1 : 0;
+
+            if (elemData[elemId].zoomLevel != 1) {
+                copySpecificStyles(canvasCursors[cursorNum], activeCursor, elemData[elemId].zoomLevel);
+            } else {
+                // Update the styles of the currently active cursor
+                copySpecificStyles(canvasCursors[cursorNum], activeCursor);
+            }
+
+            let offsetXAdjusted = e.offsetX;
+            let offsetYAdjusted = e.offsetY;
+
+            // Position the cursor based on the current mouse coordinates within target element.
+            activeCursor.style.transform =
+           `translate(${offsetXAdjusted}px, ${offsetYAdjusted}px)`;
+        };
+
+        const canvasAreaLeaveHandler = () => {
+            if (activeCursor) {
+                // activeCursor.style.opacity = 0
+                activeCursor.style.display = "none";
+            }
+        };
+
+        const canvasAreaEnterHandler = () => {
+            if (activeCursor) {
+                // activeCursor.style.opacity = 1
+                activeCursor.style.display = "block";
+            }
+        };
+
+        const canvasArea = targetElement.querySelector("canvas");
+
+        // Attach event listeners to the target element and canvas area
+        targetElement.addEventListener("mousemove", canvasAreaEventsHandler);
+        canvasArea.addEventListener("mouseout", canvasAreaLeaveHandler);
+        canvasArea.addEventListener("mouseenter", canvasAreaEnterHandler);
+
+        // Additional listener for handling zoom or other transformations which might affect visual representation
+        targetElement.addEventListener("wheel", canvasAreaEventsHandler);
+
+        // Remove border, cause bags
+        const canvasBorder = targetElement.querySelector(".border");
+        canvasBorder.style.display = "none";
+
         // Create tooltip
         function createTooltip() {
-            const toolTipElement =
-                targetElement.querySelector(".image-container");
+            const toolTipElement = targetElement.querySelector(".image-container");
             const tooltip = document.createElement("div");
             tooltip.className = "canvas-tooltip";
 
@@ -359,25 +446,15 @@ onUiLoaded(async() => {
 
             // Add a hint element to the target element
             toolTipElement.appendChild(tooltip);
+
+            return tooltip;
         }
 
         //Show tool tip if setting enable
-        if (hotkeysConfig.canvas_show_tooltip) {
-            createTooltip();
-        }
+        const canvasTooltip = createTooltip();
 
-        // In the course of research, it was found that the tag img is very harmful when zooming and creates white canvases. This hack allows you to almost never think about this problem, it has no effect on webui.
-        function fixCanvas() {
-            const activeTab = getActiveTab(elements)?.textContent.trim();
-
-            if (activeTab && activeTab !== "img2img") {
-                const img = targetElement.querySelector(`${elemId} img`);
-
-                if (img && img.style.display !== "none") {
-                    img.style.display = "none";
-                    img.style.visibility = "hidden";
-                }
-            }
+        if (!hotkeysConfig.canvas_show_tooltip) {
+            canvasTooltip.style.display = "none";
         }
 
         // Reset the zoom level and pan position of the target element to their initial values
@@ -385,7 +462,7 @@ onUiLoaded(async() => {
             elemData[elemId] = {
                 zoomLevel: 1,
                 panX: 0,
-                panY: 0
+                panY: 0,
             };
 
             if (isExtension) {
@@ -394,45 +471,22 @@ onUiLoaded(async() => {
 
             targetElement.isZoomed = false;
 
-            fixCanvas();
             targetElement.style.transform = `scale(${elemData[elemId].zoomLevel}) translate(${elemData[elemId].panX}px, ${elemData[elemId].panY}px)`;
 
             const canvas = gradioApp().querySelector(
-                `${elemId} canvas[key="interface"]`
+                `${elemId} canvas`
             );
 
             toggleOverlap("off");
             fullScreenMode = false;
 
-            const closeBtn = targetElement.querySelector("button[aria-label='Remove Image']");
+            const closeBtn = targetElement.querySelector("button[aria-label='Clear canvas']");
             if (closeBtn) {
                 closeBtn.addEventListener("click", resetZoom);
             }
 
-            if (canvas && isExtension) {
-                const parentElement = targetElement.closest('[id^="component-"]');
-                if (
-                    canvas &&
-                    parseFloat(canvas.style.width) > parentElement.offsetWidth &&
-                    parseFloat(targetElement.style.width) > parentElement.offsetWidth
-                ) {
-                    fitToElement();
-                    return;
-                }
-
-            }
-
-            if (
-                canvas &&
-                !isExtension &&
-                parseFloat(canvas.style.width) > 865 &&
-                parseFloat(targetElement.style.width) > 865
-            ) {
-                fitToElement();
-                return;
-            }
-
             targetElement.style.width = "";
+            fixCursorSize();
         }
 
         // Toggle the zIndex of the target element between two values, allowing it to overlap or be overlapped by other elements
@@ -459,10 +513,10 @@ onUiLoaded(async() => {
         ) {
             const input =
                 gradioApp().querySelector(
-                    `${elemId} input[aria-label='Brush radius']`
+                    `${elemId} input[type='range']`
                 ) ||
                 gradioApp().querySelector(
-                    `${elemId} button[aria-label="Use brush"]`
+                    `${elemId} button[aria-label="Size button"]`
                 );
 
             if (input) {
@@ -482,10 +536,15 @@ onUiLoaded(async() => {
 
         // Reset zoom when uploading a new image
         const fileInput = gradioApp().querySelector(
-            `${elemId} input[type="file"][accept="image/*"].svelte-116rqfv`
+            `${elemId} .upload-container input[type="file"][accept="image/*"]`
         );
+
         fileInput.addEventListener("click", resetZoom);
 
+        // Create clickble area
+        const inputCanvas = targetElement.querySelector("canvas");
+
+
         // Update the zoom level and pan position of the target element based on the values of the zoomLevel, panX and panY variables
         function updateZoom(newZoomLevel, mouseX, mouseY) {
             newZoomLevel = Math.max(0.1, Math.min(newZoomLevel, 15));
@@ -503,6 +562,9 @@ onUiLoaded(async() => {
                 targetElement.style.overflow = "visible";
             }
 
+            // Hack to make the cursor always be the same size
+            fixCursorSize();
+
             return newZoomLevel;
         }
 
@@ -538,67 +600,6 @@ onUiLoaded(async() => {
             }
         }
 
-        /**
-         * This function fits the target element to the screen by calculating
-         * the required scale and offsets. It also updates the global variables
-         * zoomLevel, panX, and panY to reflect the new state.
-         */
-
-        function fitToElement() {
-            //Reset Zoom
-            targetElement.style.transform = `translate(${0}px, ${0}px) scale(${1})`;
-
-            let parentElement;
-
-            if (isExtension) {
-                parentElement = targetElement.closest('[id^="component-"]');
-            } else {
-                parentElement = targetElement.parentElement;
-            }
-
-
-            // Get element and screen dimensions
-            const elementWidth = targetElement.offsetWidth;
-            const elementHeight = targetElement.offsetHeight;
-
-            const screenWidth = parentElement.clientWidth;
-            const screenHeight = parentElement.clientHeight;
-
-            // Get element's coordinates relative to the parent element
-            const elementRect = targetElement.getBoundingClientRect();
-            const parentRect = parentElement.getBoundingClientRect();
-            const elementX = elementRect.x - parentRect.x;
-
-            // Calculate scale and offsets
-            const scaleX = screenWidth / elementWidth;
-            const scaleY = screenHeight / elementHeight;
-            const scale = Math.min(scaleX, scaleY);
-
-            const transformOrigin =
-                window.getComputedStyle(targetElement).transformOrigin;
-            const [originX, originY] = transformOrigin.split(" ");
-            const originXValue = parseFloat(originX);
-            const originYValue = parseFloat(originY);
-
-            const offsetX =
-                (screenWidth - elementWidth * scale) / 2 -
-                originXValue * (1 - scale);
-            const offsetY =
-                (screenHeight - elementHeight * scale) / 2.5 -
-                originYValue * (1 - scale);
-
-            // Apply scale and offsets to the element
-            targetElement.style.transform = `translate(${offsetX}px, ${offsetY}px) scale(${scale})`;
-
-            // Update global variables
-            elemData[elemId].zoomLevel = scale;
-            elemData[elemId].panX = offsetX;
-            elemData[elemId].panY = offsetY;
-
-            fullScreenMode = false;
-            toggleOverlap("off");
-        }
-
         /**
          * This function fits the target element to the screen by calculating
          * the required scale and offsets. It also updates the global variables
@@ -608,9 +609,11 @@ onUiLoaded(async() => {
         // Fullscreen mode
         function fitToScreen() {
             const canvas = gradioApp().querySelector(
-                `${elemId} canvas[key="interface"]`
+                `${elemId} canvas`
             );
 
+            // print(canvas)
+
             if (!canvas) return;
 
             if (canvas.offsetWidth > 862 || isExtension) {
@@ -621,6 +624,7 @@ onUiLoaded(async() => {
                 targetElement.style.overflow = "visible";
             }
 
+            fixCursorSize();
             if (fullScreenMode) {
                 resetZoom();
                 fullScreenMode = false;
@@ -728,7 +732,7 @@ onUiLoaded(async() => {
 
         targetElement.isExpanded = false;
         function autoExpand() {
-            const canvas = document.querySelector(`${elemId} canvas[key="interface"]`);
+            const canvas = document.querySelector(`${elemId} canvas`);
             if (canvas) {
                 if (hasHorizontalScrollbar(targetElement) && targetElement.isExpanded === false) {
                     targetElement.style.visibility = "hidden";
@@ -744,26 +748,6 @@ onUiLoaded(async() => {
 
         targetElement.addEventListener("mousemove", getMousePosition);
 
-        //observers
-        // Creating an observer with a callback function to handle DOM changes
-        const observer = new MutationObserver((mutationsList, observer) => {
-            for (let mutation of mutationsList) {
-                // If the style attribute of the canvas has changed, by observation it happens only when the picture changes
-                if (mutation.type === 'attributes' && mutation.attributeName === 'style' &&
-                    mutation.target.tagName.toLowerCase() === 'canvas') {
-                    targetElement.isExpanded = false;
-                    setTimeout(resetZoom, 10);
-                }
-            }
-        });
-
-        // Apply auto expand if enabled
-        if (hotkeysConfig.canvas_auto_expand) {
-            targetElement.addEventListener("mousemove", autoExpand);
-            // Set up an observer to track attribute changes
-            observer.observe(targetElement, {attributes: true, childList: true, subtree: true});
-        }
-
         // Handle events only inside the targetElement
         let isKeyDownHandlerAttached = false;
 
@@ -790,15 +774,7 @@ onUiLoaded(async() => {
         targetElement.addEventListener("mouseleave", handleMouseLeave);
 
         // Reset zoom when click on another tab
-        if (elements.img2imgTabs) {
-            elements.img2imgTabs.addEventListener("click", resetZoom);
-            elements.img2imgTabs.addEventListener("click", () => {
-                // targetElement.style.width = "";
-                if (parseInt(targetElement.style.width) > 865) {
-                    setTimeout(fitToElement, 0);
-                }
-            });
-        }
+        elements.img2imgTabs.addEventListener("click", resetZoom);
 
         targetElement.addEventListener("wheel", e => {
             // change zoom level
@@ -878,6 +854,7 @@ onUiLoaded(async() => {
             elemData[elemId].panY += movementY * panSpeed;
 
             // Delayed redraw of an element
+            const canvas = targetElement.querySelector("canvas");
             requestAnimationFrame(() => {
                 targetElement.style.transform = `translate(${elemData[elemId].panX}px, ${elemData[elemId].panY}px) scale(${elemData[elemId].zoomLevel})`;
                 toggleOverlap("on");
@@ -936,7 +913,6 @@ onUiLoaded(async() => {
 
         gradioApp().addEventListener("mousemove", handleMoveByKey);
 
-
     }
 
     applyZoomAndPan(elementIDs.sketch, false);
@@ -966,9 +942,30 @@ onUiLoaded(async() => {
     };
 
     window.applyZoomAndPan = applyZoomAndPan; // Only 1 elements, argument elementID, for example applyZoomAndPan("#txt2img_controlnet_ControlNet_input_image")
-
     window.applyZoomAndPanIntegration = applyZoomAndPanIntegration; // for any extension
 
+
+    // Return zoom functionality when send img via buttons
+    const img2imgArea = document.querySelector("#img2img_settings");
+    const checkForTooltip = (e) => {
+        const tabId = getTabId(elements); // Make sure that the item is passed correctly to determine the tabId
+
+        if (tabId === "#img2img_sketch" || tabId === "#inpaint_sketch" || tabId === "#img2maskimg") {
+            const zoomTooltip = document.querySelector(`${tabId} .canvas-tooltip`);
+
+            if (!zoomTooltip) {
+                applyZoomAndPan(tabId, false);
+                // resetZoom()
+            }
+        }
+    };
+
+    // Wrapping your function through debounce to reduce the number of calls
+    const debouncedCheckForTooltip = debounce(checkForTooltip, 20);
+
+    // Assigning an event handler
+    img2imgArea.addEventListener("mousemove", debouncedCheckForTooltip);
+
     /*
         The function `applyZoomAndPanIntegration` takes two arguments:
 

extensions-builtin/canvas-zoom-and-pan/scripts/hotkey_config.py

@@ -11,7 +11,6 @@ shared.options_templates.update(shared.options_section(('canvas_hotkey', "Canvas
     "canvas_hotkey_reset": shared.OptionInfo("R", "Reset zoom and canvas position"),
     "canvas_hotkey_overlap": shared.OptionInfo("O", "Toggle overlap").info("Technical button, needed for testing"),
     "canvas_show_tooltip": shared.OptionInfo(True, "Enable tooltip on the canvas"),
-    "canvas_auto_expand": shared.OptionInfo(True, "Automatically expands an image that does not fit completely in the canvas area, similar to manually pressing the S and R buttons"),
     "canvas_blur_prompt": shared.OptionInfo(False, "Take the focus off the prompt when working with a canvas"),
     "canvas_disabled_functions": shared.OptionInfo(["Overlap"], "Disable function that you don't use", gr.CheckboxGroup, {"choices": ["Zoom","Adjust brush size","Hotkey enlarge brush","Hotkey shrink brush","Moving canvas","Fullscreen","Reset Zoom","Overlap"]}),
 }))

javascript/aspectRatioOverlay.js

@@ -1,10 +1,8 @@
-
-let currentWidth = null;
-let currentHeight = null;
-let arFrameTimeout = setTimeout(function() {}, 0);
+let currentWidth;
+let currentHeight;
+let arFrameTimeout;
 
 function dimensionChange(e, is_width, is_height) {
-
     if (is_width) {
         currentWidth = e.target.value * 1.0;
     }
@@ -22,18 +20,18 @@ function dimensionChange(e, is_width, is_height) {
 
     var tabIndex = get_tab_index('mode_img2img');
     if (tabIndex == 0) { // img2img
-        targetElement = gradioApp().querySelector('#img2img_image div[data-testid=image] img');
+        targetElement = gradioApp().querySelector('#img2img_image div[data-testid=image] canvas');
     } else if (tabIndex == 1) { //Sketch
-        targetElement = gradioApp().querySelector('#img2img_sketch div[data-testid=image] img');
+        targetElement = gradioApp().querySelector('#img2img_sketch div[data-testid=image] canvas');
     } else if (tabIndex == 2) { // Inpaint
-        targetElement = gradioApp().querySelector('#img2maskimg div[data-testid=image] img');
+        targetElement = gradioApp().querySelector('#img2maskimg div[data-testid=image] canvas');
     } else if (tabIndex == 3) { // Inpaint sketch
-        targetElement = gradioApp().querySelector('#inpaint_sketch div[data-testid=image] img');
+        targetElement = gradioApp().querySelector('#inpaint_sketch div[data-testid=image] canvas');
+    } else if (tabIndex == 4) { // Inpaint upload
+        targetElement = gradioApp().querySelector('#img_inpaint_base div[data-testid=image] img');
     }
 
-
     if (targetElement) {
-
         var arPreviewRect = gradioApp().querySelector('#imageARPreview');
         if (!arPreviewRect) {
             arPreviewRect = document.createElement('div');
@@ -41,14 +39,11 @@ function dimensionChange(e, is_width, is_height) {
             gradioApp().appendChild(arPreviewRect);
         }
 
-
-
         var viewportOffset = targetElement.getBoundingClientRect();
+        var viewportscale = Math.min(targetElement.clientWidth / targetElement.width, targetElement.clientHeight / targetElement.height);
 
-        var viewportscale = Math.min(targetElement.clientWidth / targetElement.naturalWidth, targetElement.clientHeight / targetElement.naturalHeight);
-
-        var scaledx = targetElement.naturalWidth * viewportscale;
-        var scaledy = targetElement.naturalHeight * viewportscale;
+        var scaledx = targetElement.width * viewportscale;
+        var scaledy = targetElement.height * viewportscale;
 
         var clientRectTop = (viewportOffset.top + window.scrollY);
         var clientRectLeft = (viewportOffset.left + window.scrollX);
@@ -75,21 +70,18 @@ function dimensionChange(e, is_width, is_height) {
         }, 2000);
 
         arPreviewRect.style.display = 'block';
-
     }
-
 }
 
-
 onAfterUiUpdate(function() {
     var arPreviewRect = gradioApp().querySelector('#imageARPreview');
     if (arPreviewRect) {
         arPreviewRect.style.display = 'none';
     }
+
     var tabImg2img = gradioApp().querySelector("#tab_img2img");
     if (tabImg2img) {
-        var inImg2img = tabImg2img.style.display == "block";
-        if (inImg2img) {
+        if (tabImg2img.style.display == "block") {
             let inputs = gradioApp().querySelectorAll('input');
             inputs.forEach(function(e) {
                 var is_width = e.parentElement.id == "img2img_width";

javascript/gradio.js

@@ -0,0 +1,7 @@
+
+// added to fix a weird error in gradio 4.19 at page load
+Object.defineProperty(Array.prototype, 'toLowerCase', {
+    value: function() {
+        return this;
+    }
+});

javascript/imageviewer.js

@@ -177,7 +177,7 @@ function modalTileImageToggle(event) {
 }
 
 onAfterUiUpdate(function() {
-    var fullImg_preview = gradioApp().querySelectorAll('.gradio-gallery > div > img');
+    var fullImg_preview = gradioApp().querySelectorAll('.gradio-gallery > button > button > img');
     if (fullImg_preview != null) {
         fullImg_preview.forEach(setupImageForLightbox);
     }

javascript/ui.js

@@ -38,9 +38,6 @@ function extract_image_from_gallery(gallery) {
     if (gallery.length == 0) {
         return [null];
     }
-    if (gallery.length == 1) {
-        return [gallery[0]];
-    }
 
     var index = selected_gallery_index();
 
@@ -49,7 +46,7 @@ function extract_image_from_gallery(gallery) {
         index = 0;
     }
 
-    return [gallery[index]];
+    return [[gallery[index]]];
 }
 
 window.args_to_array = Array.from; // Compatibility with e.g. extensions that may expect this to be around
@@ -116,14 +113,6 @@ function get_img2img_tab_index() {
 function create_submit_args(args) {
     var res = Array.from(args);
 
-    // As it is currently, txt2img and img2img send back the previous output args (txt2img_gallery, generation_info, html_info) whenever you generate a new image.
-    // This can lead to uploading a huge gallery of previously generated images, which leads to an unnecessary delay between submitting and beginning to generate.
-    // I don't know why gradio is sending outputs along with inputs, but we can prevent sending the image gallery here, which seems to be an issue for some.
-    // If gradio at some point stops sending outputs, this may break something
-    if (Array.isArray(res[res.length - 3])) {
-        res[res.length - 3] = null;
-    }
-
     return res;
 }
 
@@ -189,7 +178,6 @@ function submit_img2img() {
     var res = create_submit_args(arguments);
 
     res[0] = id;
-    res[1] = get_tab_index('mode_img2img');
 
     return res;
 }
@@ -207,7 +195,6 @@ function submit_extras() {
 
     res[0] = id;
 
-    console.log(res);
     return res;
 }
 
@@ -376,9 +363,9 @@ function selectCheckpoint(name) {
     gradioApp().getElementById('change_checkpoint').click();
 }
 
-function currentImg2imgSourceResolution(w, h, scaleBy) {
-    var img = gradioApp().querySelector('#mode_img2img > div[style="display: block;"] img');
-    return img ? [img.naturalWidth, img.naturalHeight, scaleBy] : [0, 0, scaleBy];
+function currentImg2imgSourceResolution(w, h, r) {
+    var img = gradioApp().querySelector('#mode_img2img > div[style="display: block;"] :is(img, canvas)');
+    return img ? [img.naturalWidth || img.width, img.naturalHeight || img.height, r] : [0, 0, r];
 }
 
 function updateImg2imgResizeToTextAfterChangingImage() {

javascript/ui_settings_hints.js

@@ -14,10 +14,16 @@ onOptionsChanged(function() {
         if (!commentBefore && !commentAfter) return;
 
         var span = null;
-        if (div.classList.contains('gradio-checkbox')) span = div.querySelector('label span');
-        else if (div.classList.contains('gradio-checkboxgroup')) span = div.querySelector('span').firstChild;
-        else if (div.classList.contains('gradio-radio')) span = div.querySelector('span').firstChild;
-        else span = div.querySelector('label span').firstChild;
+        if (div.classList.contains('gradio-checkbox')) {
+            span = div.querySelector('label span');
+        } else if (div.classList.contains('gradio-checkboxgroup')) {
+            span = div.querySelector('span').firstChild;
+        } else if (div.classList.contains('gradio-radio')) {
+            span = div.querySelector('span').firstChild;
+        } else {
+            var elem = div.querySelector('label span');
+            if (elem) span = elem.firstChild;
+        }
 
         if (!span) return;
 

modules/api/api.py

@@ -113,7 +113,7 @@ def encode_pil_to_base64(image):
             image.save(output_bytes, format="PNG", pnginfo=(metadata if use_metadata else None), quality=opts.jpeg_quality)
 
         elif opts.samples_format.lower() in ("jpg", "jpeg", "webp"):
-            if image.mode == "RGBA":
+            if image.mode in ("RGBA", "P"):
                 image = image.convert("RGB")
             parameters = image.info.get('parameters', None)
             exif_bytes = piexif.dump({
@@ -207,7 +207,7 @@ class Api:
         self.router = APIRouter()
         self.app = app
         self.queue_lock = queue_lock
-        api_middleware(self.app)
+        #api_middleware(self.app)  # XXX this will have to be fixed
         self.add_api_route("/sdapi/v1/txt2img", self.text2imgapi, methods=["POST"], response_model=models.TextToImageResponse)
         self.add_api_route("/sdapi/v1/img2img", self.img2imgapi, methods=["POST"], response_model=models.ImageToImageResponse)
         self.add_api_route("/sdapi/v1/extra-single-image", self.extras_single_image_api, methods=["POST"], response_model=models.ExtrasSingleImageResponse)

modules/api/models.py

@@ -1,6 +1,6 @@
 import inspect
 
-from pydantic import BaseModel, Field, create_model
+from pydantic import BaseModel, Field, create_model, ConfigDict
 from typing import Any, Optional, Literal
 from inflection import underscore
 from modules.processing import StableDiffusionProcessingTxt2Img, StableDiffusionProcessingImg2Img
@@ -92,9 +92,7 @@ class PydanticModelGenerator:
         fields = {
             d.field: (d.field_type, Field(default=d.field_value, alias=d.field_alias, exclude=d.field_exclude)) for d in self._model_def
         }
-        DynamicModel = create_model(self._model_name, **fields)
-        DynamicModel.__config__.allow_population_by_field_name = True
-        DynamicModel.__config__.allow_mutation = True
+        DynamicModel = create_model(self._model_name, __config__=ConfigDict(populate_by_name=True, frozen=False), **fields)
         return DynamicModel
 
 StableDiffusionTxt2ImgProcessingAPI = PydanticModelGenerator(
@@ -102,13 +100,13 @@ StableDiffusionTxt2ImgProcessingAPI = PydanticModelGenerator(
     StableDiffusionProcessingTxt2Img,
     [
         {"key": "sampler_index", "type": str, "default": "Euler"},
-        {"key": "script_name", "type": str, "default": None},
+        {"key": "script_name", "type": str | None, "default": None},
         {"key": "script_args", "type": list, "default": []},
         {"key": "send_images", "type": bool, "default": True},
         {"key": "save_images", "type": bool, "default": False},
         {"key": "alwayson_scripts", "type": dict, "default": {}},
-        {"key": "force_task_id", "type": str, "default": None},
-        {"key": "infotext", "type": str, "default": None},
+        {"key": "force_task_id", "type": str | None, "default": None},
+        {"key": "infotext", "type": str | None, "default": None},
     ]
 ).generate_model()
 
@@ -117,27 +115,27 @@ StableDiffusionImg2ImgProcessingAPI = PydanticModelGenerator(
     StableDiffusionProcessingImg2Img,
     [
         {"key": "sampler_index", "type": str, "default": "Euler"},
-        {"key": "init_images", "type": list, "default": None},
+        {"key": "init_images", "type": list | None, "default": None},
         {"key": "denoising_strength", "type": float, "default": 0.75},
-        {"key": "mask", "type": str, "default": None},
+        {"key": "mask", "type": str | None, "default": None},
         {"key": "include_init_images", "type": bool, "default": False, "exclude" : True},
-        {"key": "script_name", "type": str, "default": None},
+        {"key": "script_name", "type": str | None, "default": None},
         {"key": "script_args", "type": list, "default": []},
         {"key": "send_images", "type": bool, "default": True},
         {"key": "save_images", "type": bool, "default": False},
         {"key": "alwayson_scripts", "type": dict, "default": {}},
-        {"key": "force_task_id", "type": str, "default": None},
-        {"key": "infotext", "type": str, "default": None},
+        {"key": "force_task_id", "type": str | None, "default": None},
+        {"key": "infotext", "type": str | None, "default": None},
     ]
 ).generate_model()
 
 class TextToImageResponse(BaseModel):
-    images: list[str] = Field(default=None, title="Image", description="The generated image in base64 format.")
+    images: list[str] | None = Field(default=None, title="Image", description="The generated image in base64 format.")
     parameters: dict
     info: str
 
 class ImageToImageResponse(BaseModel):
-    images: list[str] = Field(default=None, title="Image", description="The generated image in base64 format.")
+    images: list[str] | None = Field(default=None, title="Image", description="The generated image in base64 format.")
     parameters: dict
     info: str
 
@@ -163,7 +161,7 @@ class ExtrasSingleImageRequest(ExtrasBaseRequest):
     image: str = Field(default="", title="Image", description="Image to work on, must be a Base64 string containing the image's data.")
 
 class ExtrasSingleImageResponse(ExtraBaseResponse):
-    image: str = Field(default=None, title="Image", description="The generated image in base64 format.")
+    image: str | None = Field(default=None, title="Image", description="The generated image in base64 format.")
 
 class FileData(BaseModel):
     data: str = Field(title="File data", description="Base64 representation of the file")
@@ -190,15 +188,15 @@ class ProgressResponse(BaseModel):
     progress: float = Field(title="Progress", description="The progress with a range of 0 to 1")
     eta_relative: float = Field(title="ETA in secs")
     state: dict = Field(title="State", description="The current state snapshot")
-    current_image: str = Field(default=None, title="Current image", description="The current image in base64 format. opts.show_progress_every_n_steps is required for this to work.")
-    textinfo: str = Field(default=None, title="Info text", description="Info text used by WebUI.")
+    current_image: str | None = Field(default=None, title="Current image", description="The current image in base64 format. opts.show_progress_every_n_steps is required for this to work.")
+    textinfo: str | None = Field(default=None, title="Info text", description="Info text used by WebUI.")
 
 class InterrogateRequest(BaseModel):
     image: str = Field(default="", title="Image", description="Image to work on, must be a Base64 string containing the image's data.")
     model: str = Field(default="clip", title="Model", description="The interrogate model used.")
 
 class InterrogateResponse(BaseModel):
-    caption: str = Field(default=None, title="Caption", description="The generated caption for the image.")
+    caption: str | None = Field(default=None, title="Caption", description="The generated caption for the image.")
 
 class TrainResponse(BaseModel):
     info: str = Field(title="Train info", description="Response string from train embedding or hypernetwork task.")
@@ -223,7 +221,7 @@ _options = vars(parser)['_option_string_actions']
 for key in _options:
     if(_options[key].dest != 'help'):
         flag = _options[key]
-        _type = str
+        _type = str | None
         if _options[key].default is not None:
             _type = type(_options[key].default)
         flags.update({flag.dest: (_type, Field(default=flag.default, description=flag.help))})
@@ -233,7 +231,7 @@ FlagsModel = create_model("Flags", **flags)
 class SamplerItem(BaseModel):
     name: str = Field(title="Name")
     aliases: list[str] = Field(title="Aliases")
-    options: dict[str, str] = Field(title="Options")
+    options: dict[str, Any] = Field(title="Options")
 
 class SchedulerItem(BaseModel):
     name: str = Field(title="Name")
@@ -243,6 +241,9 @@ class SchedulerItem(BaseModel):
     need_inner_model: Optional[bool] = Field(title="Needs Inner Model")
 
 class UpscalerItem(BaseModel):
+    class Config:
+        protected_namespaces = ()
+
     name: str = Field(title="Name")
     model_name: Optional[str] = Field(title="Model Name")
     model_path: Optional[str] = Field(title="Path")
@@ -253,6 +254,9 @@ class LatentUpscalerModeItem(BaseModel):
     name: str = Field(title="Name")
 
 class SDModelItem(BaseModel):
+    class Config:
+        protected_namespaces = ()
+
     title: str = Field(title="Title")
     model_name: str = Field(title="Model Name")
     hash: Optional[str] = Field(title="Short hash")
@@ -261,6 +265,9 @@ class SDModelItem(BaseModel):
     config: Optional[str] = Field(title="Config file")
 
 class SDVaeItem(BaseModel):
+    class Config:
+        protected_namespaces = ()
+
     model_name: str = Field(title="Model Name")
     filename: str = Field(title="Filename")
 
@@ -300,12 +307,12 @@ class MemoryResponse(BaseModel):
 
 
 class ScriptsList(BaseModel):
-    txt2img: list = Field(default=None, title="Txt2img", description="Titles of scripts (txt2img)")
-    img2img: list = Field(default=None, title="Img2img", description="Titles of scripts (img2img)")
+    txt2img: list | None = Field(default=None, title="Txt2img", description="Titles of scripts (txt2img)")
+    img2img: list | None = Field(default=None, title="Img2img", description="Titles of scripts (img2img)")
 
 
 class ScriptArg(BaseModel):
-    label: str = Field(default=None, title="Label", description="Name of the argument in UI")
+    label: str | None = Field(default=None, title="Label", description="Name of the argument in UI")
     value: Optional[Any] = Field(default=None, title="Value", description="Default value of the argument")
     minimum: Optional[Any] = Field(default=None, title="Minimum", description="Minimum allowed value for the argumentin UI")
     maximum: Optional[Any] = Field(default=None, title="Minimum", description="Maximum allowed value for the argumentin UI")
@@ -314,9 +321,9 @@ class ScriptArg(BaseModel):
 
 
 class ScriptInfo(BaseModel):
-    name: str = Field(default=None, title="Name", description="Script name")
-    is_alwayson: bool = Field(default=None, title="IsAlwayson", description="Flag specifying whether this script is an alwayson script")
-    is_img2img: bool = Field(default=None, title="IsImg2img", description="Flag specifying whether this script is an img2img script")
+    name: str | None = Field(default=None, title="Name", description="Script name")
+    is_alwayson: bool | None = Field(default=None, title="IsAlwayson", description="Flag specifying whether this script is an alwayson script")
+    is_img2img: bool | None = Field(default=None, title="IsImg2img", description="Flag specifying whether this script is an img2img script")
     args: list[ScriptArg] = Field(title="Arguments", description="List of script's arguments")
 
 class ExtensionItem(BaseModel):

modules/call_queue.py

@@ -47,6 +47,22 @@ def wrap_gradio_gpu_call(func, extra_outputs=None):
 
 
 def wrap_gradio_call(func, extra_outputs=None, add_stats=False):
+    @wraps(func)
+    def f(*args, **kwargs):
+        try:
+            res = func(*args, **kwargs)
+        finally:
+            shared.state.skipped = False
+            shared.state.interrupted = False
+            shared.state.stopping_generation = False
+            shared.state.job_count = 0
+            shared.state.job = ""
+        return res
+
+    return wrap_gradio_call_no_job(f, extra_outputs, add_stats)
+
+
+def wrap_gradio_call_no_job(func, extra_outputs=None, add_stats=False):
     @wraps(func)
     def f(*args, extra_outputs_array=extra_outputs, **kwargs):
         run_memmon = shared.opts.memmon_poll_rate > 0 and not shared.mem_mon.disabled and add_stats
@@ -66,9 +82,6 @@ def wrap_gradio_call(func, extra_outputs=None, add_stats=False):
                 arg_str += f" (Argument list truncated at {max_debug_str_len}/{len(arg_str)} characters)"
             errors.report(f"{message}\n{arg_str}", exc_info=True)
 
-            shared.state.job = ""
-            shared.state.job_count = 0
-
             if extra_outputs_array is None:
                 extra_outputs_array = [None, '']
 
@@ -77,11 +90,6 @@ def wrap_gradio_call(func, extra_outputs=None, add_stats=False):
 
         devices.torch_gc()
 
-        shared.state.skipped = False
-        shared.state.interrupted = False
-        shared.state.stopping_generation = False
-        shared.state.job_count = 0
-
         if not add_stats:
             return tuple(res)
 
@@ -123,3 +131,4 @@ def wrap_gradio_call(func, extra_outputs=None, add_stats=False):
         return tuple(res)
 
     return f
+

modules/errors.py

@@ -109,7 +109,7 @@ def check_versions():
 
     expected_torch_version = "2.1.2"
     expected_xformers_version = "0.0.23.post1"
-    expected_gradio_version = "3.41.2"
+    expected_gradio_version = "4.38.1"
 
     if version.parse(torch.__version__) < version.parse(expected_torch_version):
         print_error_explanation(f"""

modules/gradio_extensions.py

@@ -0,0 +1,166 @@
+import inspect
+import warnings
+from functools import wraps
+
+import gradio as gr
+import gradio.component_meta
+
+
+from modules import scripts, ui_tempdir, patches
+
+
+class GradioDeprecationWarning(DeprecationWarning):
+    pass
+
+
+def add_classes_to_gradio_component(comp):
+    """
+    this adds gradio-* to the component for css styling (ie gradio-button to gr.Button), as well as some others
+    """
+
+    comp.elem_classes = [f"gradio-{comp.get_block_name()}", *(getattr(comp, 'elem_classes', None) or [])]
+
+    if getattr(comp, 'multiselect', False):
+        comp.elem_classes.append('multiselect')
+
+
+def IOComponent_init(self, *args, **kwargs):
+    self.webui_tooltip = kwargs.pop('tooltip', None)
+
+    if scripts.scripts_current is not None:
+        scripts.scripts_current.before_component(self, **kwargs)
+
+    scripts.script_callbacks.before_component_callback(self, **kwargs)
+
+    res = original_IOComponent_init(self, *args, **kwargs)
+
+    add_classes_to_gradio_component(self)
+
+    scripts.script_callbacks.after_component_callback(self, **kwargs)
+
+    if scripts.scripts_current is not None:
+        scripts.scripts_current.after_component(self, **kwargs)
+
+    return res
+
+
+def Block_get_config(self):
+    config = original_Block_get_config(self)
+
+    webui_tooltip = getattr(self, 'webui_tooltip', None)
+    if webui_tooltip:
+        config["webui_tooltip"] = webui_tooltip
+
+    config.pop('example_inputs', None)
+
+    return config
+
+
+def BlockContext_init(self, *args, **kwargs):
+    if scripts.scripts_current is not None:
+        scripts.scripts_current.before_component(self, **kwargs)
+
+    scripts.script_callbacks.before_component_callback(self, **kwargs)
+
+    res = original_BlockContext_init(self, *args, **kwargs)
+
+    add_classes_to_gradio_component(self)
+
+    scripts.script_callbacks.after_component_callback(self, **kwargs)
+
+    if scripts.scripts_current is not None:
+        scripts.scripts_current.after_component(self, **kwargs)
+
+    return res
+
+
+def Blocks_get_config_file(self, *args, **kwargs):
+    config = original_Blocks_get_config_file(self, *args, **kwargs)
+
+    for comp_config in config["components"]:
+        if "example_inputs" in comp_config:
+            comp_config["example_inputs"] = {"serialized": []}
+
+    return config
+
+
+original_IOComponent_init = patches.patch(__name__, obj=gr.components.Component, field="__init__", replacement=IOComponent_init)
+original_Block_get_config = patches.patch(__name__, obj=gr.blocks.Block, field="get_config", replacement=Block_get_config)
+original_BlockContext_init = patches.patch(__name__, obj=gr.blocks.BlockContext, field="__init__", replacement=BlockContext_init)
+original_Blocks_get_config_file = patches.patch(__name__, obj=gr.blocks.Blocks, field="get_config_file", replacement=Blocks_get_config_file)
+
+
+ui_tempdir.install_ui_tempdir_override()
+
+
+def gradio_component_meta_create_or_modify_pyi(component_class, class_name, events):
+    if hasattr(component_class, 'webui_do_not_create_gradio_pyi_thank_you'):
+        return
+
+    gradio_component_meta_create_or_modify_pyi_original(component_class, class_name, events)
+
+
+# this prevents creation of .pyi files in webui dir
+gradio_component_meta_create_or_modify_pyi_original = patches.patch(__file__, gradio.component_meta, 'create_or_modify_pyi', gradio_component_meta_create_or_modify_pyi)
+
+# this function is broken and does not seem to do anything useful
+gradio.component_meta.updateable = lambda x: x
+
+def repair(grclass):
+    if not getattr(grclass, 'EVENTS', None):
+        return
+
+    @wraps(grclass.__init__)
+    def __repaired_init__(self, *args, tooltip=None, source=None, original=grclass.__init__, **kwargs):
+        if source:
+            kwargs["sources"] = [source]
+
+        allowed_kwargs = inspect.signature(original).parameters
+        fixed_kwargs = {}
+        for k, v in kwargs.items():
+            if k in allowed_kwargs:
+                fixed_kwargs[k] = v
+            else:
+                warnings.warn(f"unexpected argument for {grclass.__name__}: {k}", GradioDeprecationWarning, stacklevel=2)
+
+        original(self, *args, **fixed_kwargs)
+
+        self.webui_tooltip = tooltip
+
+        for event in self.EVENTS:
+            replaced_event = getattr(self, str(event))
+
+            def fun(*xargs, _js=None, replaced_event=replaced_event, **xkwargs):
+                if _js:
+                    xkwargs['js'] = _js
+
+                return replaced_event(*xargs, **xkwargs)
+
+            setattr(self, str(event), fun)
+
+    grclass.__init__ = __repaired_init__
+    grclass.update = gr.update
+
+
+for component in set(gr.components.__all__ + gr.layouts.__all__):
+    repair(getattr(gr, component, None))
+
+
+class Dependency(gr.events.Dependency):
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+
+        def then(*xargs, _js=None, **xkwargs):
+            if _js:
+                xkwargs['js'] = _js
+
+            return original_then(*xargs, **xkwargs)
+
+        original_then = self.then
+        self.then = then
+
+
+gr.events.Dependency = Dependency
+
+gr.Box = gr.Group
+

modules/gradio_extensons.py

@@ -1,83 +0,0 @@
-import gradio as gr
-
-from modules import scripts, ui_tempdir, patches
-
-
-def add_classes_to_gradio_component(comp):
-    """
-    this adds gradio-* to the component for css styling (ie gradio-button to gr.Button), as well as some others
-    """
-
-    comp.elem_classes = [f"gradio-{comp.get_block_name()}", *(comp.elem_classes or [])]
-
-    if getattr(comp, 'multiselect', False):
-        comp.elem_classes.append('multiselect')
-
-
-def IOComponent_init(self, *args, **kwargs):
-    self.webui_tooltip = kwargs.pop('tooltip', None)
-
-    if scripts.scripts_current is not None:
-        scripts.scripts_current.before_component(self, **kwargs)
-
-    scripts.script_callbacks.before_component_callback(self, **kwargs)
-
-    res = original_IOComponent_init(self, *args, **kwargs)
-
-    add_classes_to_gradio_component(self)
-
-    scripts.script_callbacks.after_component_callback(self, **kwargs)
-
-    if scripts.scripts_current is not None:
-        scripts.scripts_current.after_component(self, **kwargs)
-
-    return res
-
-
-def Block_get_config(self):
-    config = original_Block_get_config(self)
-
-    webui_tooltip = getattr(self, 'webui_tooltip', None)
-    if webui_tooltip:
-        config["webui_tooltip"] = webui_tooltip
-
-    config.pop('example_inputs', None)
-
-    return config
-
-
-def BlockContext_init(self, *args, **kwargs):
-    if scripts.scripts_current is not None:
-        scripts.scripts_current.before_component(self, **kwargs)
-
-    scripts.script_callbacks.before_component_callback(self, **kwargs)
-
-    res = original_BlockContext_init(self, *args, **kwargs)
-
-    add_classes_to_gradio_component(self)
-
-    scripts.script_callbacks.after_component_callback(self, **kwargs)
-
-    if scripts.scripts_current is not None:
-        scripts.scripts_current.after_component(self, **kwargs)
-
-    return res
-
-
-def Blocks_get_config_file(self, *args, **kwargs):
-    config = original_Blocks_get_config_file(self, *args, **kwargs)
-
-    for comp_config in config["components"]:
-        if "example_inputs" in comp_config:
-            comp_config["example_inputs"] = {"serialized": []}
-
-    return config
-
-
-original_IOComponent_init = patches.patch(__name__, obj=gr.components.IOComponent, field="__init__", replacement=IOComponent_init)
-original_Block_get_config = patches.patch(__name__, obj=gr.blocks.Block, field="get_config", replacement=Block_get_config)
-original_BlockContext_init = patches.patch(__name__, obj=gr.blocks.BlockContext, field="__init__", replacement=BlockContext_init)
-original_Blocks_get_config_file = patches.patch(__name__, obj=gr.blocks.Blocks, field="get_config_file", replacement=Blocks_get_config_file)
-
-
-ui_tempdir.install_ui_tempdir_override()

modules/img2img.py

@@ -2,7 +2,6 @@ import os
 from contextlib import closing
 from pathlib import Path
 
-import numpy as np
 from PIL import Image, ImageOps, ImageFilter, ImageEnhance, UnidentifiedImageError
 import gradio as gr
 
@@ -149,25 +148,24 @@ def process_batch(p, input, output_dir, inpaint_mask_dir, args, to_scale=False,
     return batch_results
 
 
-def img2img(id_task: str, request: gr.Request, mode: int, prompt: str, negative_prompt: str, prompt_styles, init_img, sketch, init_img_with_mask, inpaint_color_sketch, inpaint_color_sketch_orig, init_img_inpaint, init_mask_inpaint, mask_blur: int, mask_alpha: float, inpainting_fill: int, n_iter: int, batch_size: int, cfg_scale: float, image_cfg_scale: float, denoising_strength: float, selected_scale_tab: int, height: int, width: int, scale_by: float, resize_mode: int, inpaint_full_res: bool, inpaint_full_res_padding: int, inpainting_mask_invert: int, img2img_batch_input_dir: str, img2img_batch_output_dir: str, img2img_batch_inpaint_mask_dir: str, override_settings_texts, img2img_batch_use_png_info: bool, img2img_batch_png_info_props: list, img2img_batch_png_info_dir: str, img2img_batch_source_type: str, img2img_batch_upload: list, *args):
+def img2img(id_task: str, request: gr.Request, mode: int, prompt: str, negative_prompt: str, prompt_styles, init_img, sketch, init_img_with_mask, inpaint_color_sketch, init_img_inpaint, init_mask_inpaint, mask_blur: int, mask_alpha: float, inpainting_fill: int, n_iter: int, batch_size: int, cfg_scale: float, image_cfg_scale: float, denoising_strength: float, selected_scale_tab: int, height: int, width: int, scale_by: float, resize_mode: int, inpaint_full_res: bool, inpaint_full_res_padding: int, inpainting_mask_invert: int, img2img_batch_input_dir: str, img2img_batch_output_dir: str, img2img_batch_inpaint_mask_dir: str, override_settings_texts, img2img_batch_use_png_info: bool, img2img_batch_png_info_props: list, img2img_batch_png_info_dir: str, img2img_batch_source_type: str, img2img_batch_upload: list, *args):
     override_settings = create_override_settings_dict(override_settings_texts)
 
     is_batch = mode == 5
 
     if mode == 0:  # img2img
-        image = init_img
+        image = init_img["composite"]
         mask = None
     elif mode == 1:  # img2img sketch
-        image = sketch
+        image = sketch["composite"]
         mask = None
     elif mode == 2:  # inpaint
-        image, mask = init_img_with_mask["image"], init_img_with_mask["mask"]
+        image, mask = init_img_with_mask["background"], init_img_with_mask["layers"][0]
         mask = processing.create_binary_mask(mask)
     elif mode == 3:  # inpaint sketch
-        image = inpaint_color_sketch
-        orig = inpaint_color_sketch_orig or inpaint_color_sketch
-        pred = np.any(np.array(image) != np.array(orig), axis=-1)
-        mask = Image.fromarray(pred.astype(np.uint8) * 255, "L")
+        image = inpaint_color_sketch["composite"]
+        orig = inpaint_color_sketch["background"]
+        mask = inpaint_color_sketch["layers"][0].getchannel("A")
         mask = ImageEnhance.Brightness(mask).enhance(1 - mask_alpha / 100)
         blur = ImageFilter.GaussianBlur(mask_blur)
         image = Image.composite(image.filter(blur), orig, mask.filter(blur))

modules/infotext_utils.py

@@ -74,29 +74,38 @@ def image_from_url_text(filedata):
     if filedata is None:
         return None
 
-    if type(filedata) == list and filedata and type(filedata[0]) == dict and filedata[0].get("is_file", False):
+    if isinstance(filedata, list):
+        if len(filedata) == 0:
+            return None
+
         filedata = filedata[0]
 
+    if isinstance(filedata, dict) and filedata.get("is_file", False):
+        filedata = filedata
+
+    filename = None
     if type(filedata) == dict and filedata.get("is_file", False):
         filename = filedata["name"]
+
+    elif isinstance(filedata, tuple) and len(filedata) == 2:  # gradio 4.16 sends images from gallery as a list of tuples
+        return filedata[0]
+
+    if filename:
         is_in_right_dir = ui_tempdir.check_tmp_file(shared.demo, filename)
         assert is_in_right_dir, 'trying to open image file outside of allowed directories'
 
         filename = filename.rsplit('?', 1)[0]
         return images.read(filename)
 
-    if type(filedata) == list:
-        if len(filedata) == 0:
-            return None
+    if isinstance(filedata, str):
+        if filedata.startswith("data:image/png;base64,"):
+            filedata = filedata[len("data:image/png;base64,"):]
 
-        filedata = filedata[0]
+        filedata = base64.decodebytes(filedata.encode('utf-8'))
+        image = images.read(io.BytesIO(filedata))
+        return image
 
-    if filedata.startswith("data:image/png;base64,"):
-        filedata = filedata[len("data:image/png;base64,"):]
-
-    filedata = base64.decodebytes(filedata.encode('utf-8'))
-    image = images.read(io.BytesIO(filedata))
-    return image
+    return None
 
 
 def add_paste_fields(tabname, init_img, fields, override_settings_component=None):
@@ -146,18 +155,19 @@ def connect_paste_params_buttons():
         destination_height_component = next(iter([field for field, name in fields if name == "Size-2"] if fields else []), None)
 
         if binding.source_image_component and destination_image_component:
+            need_send_dementions = destination_width_component and binding.tabname != 'inpaint'
             if isinstance(binding.source_image_component, gr.Gallery):
-                func = send_image_and_dimensions if destination_width_component else image_from_url_text
+                func = send_image_and_dimensions if need_send_dementions else image_from_url_text
                 jsfunc = "extract_image_from_gallery"
             else:
-                func = send_image_and_dimensions if destination_width_component else lambda x: x
+                func = send_image_and_dimensions if need_send_dementions else lambda x: x
                 jsfunc = None
 
             binding.paste_button.click(
                 fn=func,
                 _js=jsfunc,
                 inputs=[binding.source_image_component],
-                outputs=[destination_image_component, destination_width_component, destination_height_component] if destination_width_component else [destination_image_component],
+                outputs=[destination_image_component, destination_width_component, destination_height_component] if need_send_dementions else [destination_image_component],
                 show_progress=False,
             )
 
@@ -185,6 +195,8 @@ def connect_paste_params_buttons():
 def send_image_and_dimensions(x):
     if isinstance(x, Image.Image):
         img = x
+    elif isinstance(x, list) and isinstance(x[0], tuple):
+        img = x[0][0]
     else:
         img = image_from_url_text(x)
 
@@ -412,6 +424,9 @@ def create_override_settings_dict(text_pairs):
 
     res = {}
 
+    if not text_pairs:
+        return res
+
     params = {}
     for pair in text_pairs:
         k, v = pair.split(":", maxsplit=1)

modules/initialize.py

@@ -36,7 +36,7 @@ def imports():
     shared_init.initialize()
     startup_timer.record("initialize shared")
 
-    from modules import processing, gradio_extensons, ui  # noqa: F401
+    from modules import processing, gradio_extensions, ui  # noqa: F401
     startup_timer.record("other imports")
 
 

modules/initialize_util.py

@@ -4,6 +4,8 @@ import signal
 import sys
 import re
 
+import starlette
+
 from modules.timer import startup_timer
 
 
@@ -192,8 +194,7 @@ def configure_opts_onchange():
 def setup_middleware(app):
     from starlette.middleware.gzip import GZipMiddleware
 
-    app.middleware_stack = None  # reset current middleware to allow modifying user provided list
-    app.add_middleware(GZipMiddleware, minimum_size=1000)
+    app.user_middleware.insert(0, starlette.middleware.Middleware(GZipMiddleware, minimum_size=1000))
     configure_cors_middleware(app)
     app.build_middleware_stack()  # rebuild middleware stack on-the-fly
 
@@ -211,5 +212,6 @@ def configure_cors_middleware(app):
         cors_options["allow_origins"] = cmd_opts.cors_allow_origins.split(',')
     if cmd_opts.cors_allow_origins_regex:
         cors_options["allow_origin_regex"] = cmd_opts.cors_allow_origins_regex
-    app.add_middleware(CORSMiddleware, **cors_options)
+
+    app.user_middleware.insert(0, starlette.middleware.Middleware(CORSMiddleware, **cors_options))
 

modules/launch_utils.py

@@ -9,6 +9,7 @@ import importlib.util
 import importlib.metadata
 import platform
 import json
+import shlex
 from functools import lru_cache
 
 from modules import cmd_args, errors
@@ -445,7 +446,6 @@ def prepare_environment():
         exit(0)
 
 
-
 def configure_for_tests():
     if "--api" not in sys.argv:
         sys.argv.append("--api")
@@ -461,7 +461,7 @@ def configure_for_tests():
 
 
 def start():
-    print(f"Launching {'API server' if '--nowebui' in sys.argv else 'Web UI'} with arguments: {' '.join(sys.argv[1:])}")
+    print(f"Launching {'API server' if '--nowebui' in sys.argv else 'Web UI'} with arguments: {shlex.join(sys.argv[1:])}")
     import webui
     if '--nowebui' in sys.argv:
         webui.api_only()

modules/models/sd3/mmdit.py

@@ -175,6 +175,9 @@ class VectorEmbedder(nn.Module):
 #################################################################################
 
 
+class QkvLinear(torch.nn.Linear):
+    pass
+
 def split_qkv(qkv, head_dim):
     qkv = qkv.reshape(qkv.shape[0], qkv.shape[1], 3, -1, head_dim).movedim(2, 0)
     return qkv[0], qkv[1], qkv[2]
@@ -202,7 +205,7 @@ class SelfAttention(nn.Module):
         self.num_heads = num_heads
         self.head_dim = dim // num_heads
 
-        self.qkv = nn.Linear(dim, dim * 3, bias=qkv_bias, dtype=dtype, device=device)
+        self.qkv = QkvLinear(dim, dim * 3, bias=qkv_bias, dtype=dtype, device=device)
         if not pre_only:
             self.proj = nn.Linear(dim, dim, dtype=dtype, device=device)
         assert attn_mode in self.ATTENTION_MODES

modules/models/sd3/other_impls.py

@@ -5,6 +5,8 @@ import math
 from torch import nn
 from transformers import CLIPTokenizer, T5TokenizerFast
 
+from modules import sd_hijack
+
 
 #################################################################################################
 ### Core/Utility
@@ -110,9 +112,9 @@ class CLIPEncoder(torch.nn.Module):
 
 
 class CLIPEmbeddings(torch.nn.Module):
-    def __init__(self, embed_dim, vocab_size=49408, num_positions=77, dtype=None, device=None):
+    def __init__(self, embed_dim, vocab_size=49408, num_positions=77, dtype=None, device=None, textual_inversion_key="clip_l"):
         super().__init__()
-        self.token_embedding = torch.nn.Embedding(vocab_size, embed_dim, dtype=dtype, device=device)
+        self.token_embedding = sd_hijack.TextualInversionEmbeddings(vocab_size, embed_dim, dtype=dtype, device=device, textual_inversion_key=textual_inversion_key)
         self.position_embedding = torch.nn.Embedding(num_positions, embed_dim, dtype=dtype, device=device)
 
     def forward(self, input_tokens):
@@ -127,7 +129,7 @@ class CLIPTextModel_(torch.nn.Module):
         intermediate_size = config_dict["intermediate_size"]
         intermediate_activation = config_dict["hidden_act"]
         super().__init__()
-        self.embeddings = CLIPEmbeddings(embed_dim, dtype=torch.float32, device=device)
+        self.embeddings = CLIPEmbeddings(embed_dim, dtype=torch.float32, device=device, textual_inversion_key=config_dict.get('textual_inversion_key', 'clip_l'))
         self.encoder = CLIPEncoder(num_layers, embed_dim, heads, intermediate_size, intermediate_activation, dtype, device)
         self.final_layer_norm = nn.LayerNorm(embed_dim, dtype=dtype, device=device)
 

modules/models/sd3/sd3_cond.py

@@ -40,6 +40,7 @@ CLIPG_CONFIG = {
     "intermediate_size": 5120,
     "num_attention_heads": 20,
     "num_hidden_layers": 32,
+    "textual_inversion_key": "clip_g",
 }
 
 T5_URL = "https://huggingface.co/AUTOMATIC/stable-diffusion-3-medium-text-encoders/resolve/main/t5xxl_fp16.safetensors"
@@ -204,7 +205,10 @@ class SD3Cond(torch.nn.Module):
                 self.t5xxl.transformer.load_state_dict(SafetensorsMapping(file), strict=False)
 
     def encode_embedding_init_text(self, init_text, nvpt):
-        return torch.tensor([[0]], device=devices.device) # XXX
+        return self.model_lg.encode_embedding_init_text(init_text, nvpt)
+
+    def tokenize(self, texts):
+        return self.model_lg.tokenize(texts)
 
     def medvram_modules(self):
         return [self.clip_g, self.clip_l, self.t5xxl]

modules/models/sd3/sd3_impls.py

@@ -67,6 +67,7 @@ class BaseModel(torch.nn.Module):
         }
         self.diffusion_model = MMDiT(input_size=None, pos_embed_scaling_factor=None, pos_embed_offset=None, pos_embed_max_size=pos_embed_max_size, patch_size=patch_size, in_channels=16, depth=depth, num_patches=num_patches, adm_in_channels=adm_in_channels, context_embedder_config=context_embedder_config, device=device, dtype=dtype)
         self.model_sampling = ModelSamplingDiscreteFlow(shift=shift)
+        self.depth = depth
 
     def apply_model(self, x, sigma, c_crossattn=None, y=None):
         dtype = self.get_dtype()

modules/models/sd3/sd3_model.py

@@ -82,3 +82,15 @@ class SD3Inferencer(torch.nn.Module):
 
     def fix_dimensions(self, width, height):
         return width // 16 * 16, height // 16 * 16
+
+    def diffusers_weight_mapping(self):
+        for i in range(self.model.depth):
+            yield f"transformer.transformer_blocks.{i}.attn.to_q", f"diffusion_model_joint_blocks_{i}_x_block_attn_qkv_q_proj"
+            yield f"transformer.transformer_blocks.{i}.attn.to_k", f"diffusion_model_joint_blocks_{i}_x_block_attn_qkv_k_proj"
+            yield f"transformer.transformer_blocks.{i}.attn.to_v", f"diffusion_model_joint_blocks_{i}_x_block_attn_qkv_v_proj"
+            yield f"transformer.transformer_blocks.{i}.attn.to_out.0", f"diffusion_model_joint_blocks_{i}_x_block_attn_proj"
+
+            yield f"transformer.transformer_blocks.{i}.attn.add_q_proj", f"diffusion_model_joint_blocks_{i}_context_block.attn_qkv_q_proj"
+            yield f"transformer.transformer_blocks.{i}.attn.add_k_proj", f"diffusion_model_joint_blocks_{i}_context_block.attn_qkv_k_proj"
+            yield f"transformer.transformer_blocks.{i}.attn.add_v_proj", f"diffusion_model_joint_blocks_{i}_context_block.attn_qkv_v_proj"
+            yield f"transformer.transformer_blocks.{i}.attn.add_out_proj.0", f"diffusion_model_joint_blocks_{i}_context_block_attn_proj"

modules/postprocessing.py

@@ -13,6 +13,9 @@ def run_postprocessing(extras_mode, image, image_folder, input_dir, output_dir,
 
     outputs = []
 
+    if isinstance(image, dict):
+        image = image["composite"]
+
     def get_images(extras_mode, image, image_folder, input_dir):
         if extras_mode == 1:
             for img in image_folder:

modules/processing_scripts/refiner.py

@@ -22,7 +22,7 @@ class ScriptRefiner(scripts.ScriptBuiltinUI):
     def ui(self, is_img2img):
         with InputAccordion(False, label="Refiner", elem_id=self.elem_id("enable")) as enable_refiner:
             with gr.Row():
-                refiner_checkpoint = gr.Dropdown(label='Checkpoint', elem_id=self.elem_id("checkpoint"), choices=sd_models.checkpoint_tiles(), value='', tooltip="switch to another model in the middle of generation")
+                refiner_checkpoint = gr.Dropdown(label='Checkpoint', elem_id=self.elem_id("checkpoint"), choices=["", *sd_models.checkpoint_tiles()], value='', tooltip="switch to another model in the middle of generation")
                 create_refresh_button(refiner_checkpoint, sd_models.list_models, lambda: {"choices": sd_models.checkpoint_tiles()}, self.elem_id("checkpoint_refresh"))
 
                 refiner_switch_at = gr.Slider(value=0.8, label="Switch at", minimum=0.01, maximum=1.0, step=0.01, elem_id=self.elem_id("switch_at"), tooltip="fraction of sampling steps when the switch to refiner model should happen; 1=never, 0.5=switch in the middle of generation")

modules/processing_scripts/seed.py

@@ -34,7 +34,7 @@ class ScriptSeed(scripts.ScriptBuiltinUI):
             random_seed = ToolButton(ui.random_symbol, elem_id=self.elem_id("random_seed"), tooltip="Set seed to -1, which will cause a new random number to be used every time")
             reuse_seed = ToolButton(ui.reuse_symbol, elem_id=self.elem_id("reuse_seed"), tooltip="Reuse seed from last generation, mostly useful if it was randomized")
 
-            seed_checkbox = gr.Checkbox(label='Extra', elem_id=self.elem_id("subseed_show"), value=False)
+            seed_checkbox = gr.Checkbox(label='Extra', elem_id=self.elem_id("subseed_show"), value=False, scale=0, min_width=60)
 
         with gr.Group(visible=False, elem_id=self.elem_id("seed_extras")) as seed_extras:
             with gr.Row(elem_id=self.elem_id("subseed_row")):

modules/progress.py

@@ -1,3 +1,4 @@
+from __future__ import annotations
 import base64
 import io
 import time
@@ -66,11 +67,11 @@ class ProgressResponse(BaseModel):
     active: bool = Field(title="Whether the task is being worked on right now")
     queued: bool = Field(title="Whether the task is in queue")
     completed: bool = Field(title="Whether the task has already finished")
-    progress: float = Field(default=None, title="Progress", description="The progress with a range of 0 to 1")
-    eta: float = Field(default=None, title="ETA in secs")
-    live_preview: str = Field(default=None, title="Live preview image", description="Current live preview; a data: uri")
-    id_live_preview: int = Field(default=None, title="Live preview image ID", description="Send this together with next request to prevent receiving same image")
-    textinfo: str = Field(default=None, title="Info text", description="Info text used by WebUI.")
+    progress: float | None = Field(default=None, title="Progress", description="The progress with a range of 0 to 1")
+    eta: float | None = Field(default=None, title="ETA in secs")
+    live_preview: str | None = Field(default=None, title="Live preview image", description="Current live preview; a data: uri")
+    id_live_preview: int | None = Field(default=None, title="Live preview image ID", description="Send this together with next request to prevent receiving same image")
+    textinfo: str | None = Field(default=None, title="Info text", description="Info text used by WebUI.")
 
 
 def setup_progress_api(app):

modules/sd_hijack.py

@@ -359,13 +359,28 @@ class EmbeddingsWithFixes(torch.nn.Module):
                 vec = embedding.vec[self.textual_inversion_key] if isinstance(embedding.vec, dict) else embedding.vec
                 emb = devices.cond_cast_unet(vec)
                 emb_len = min(tensor.shape[0] - offset - 1, emb.shape[0])
-                tensor = torch.cat([tensor[0:offset + 1], emb[0:emb_len], tensor[offset + 1 + emb_len:]])
+                tensor = torch.cat([tensor[0:offset + 1], emb[0:emb_len], tensor[offset + 1 + emb_len:]]).to(dtype=inputs_embeds.dtype)
 
             vecs.append(tensor)
 
         return torch.stack(vecs)
 
 
+class TextualInversionEmbeddings(torch.nn.Embedding):
+    def __init__(self, num_embeddings: int, embedding_dim: int, textual_inversion_key='clip_l', **kwargs):
+        super().__init__(num_embeddings, embedding_dim, **kwargs)
+
+        self.embeddings = model_hijack
+        self.textual_inversion_key = textual_inversion_key
+
+    @property
+    def wrapped(self):
+        return super().forward
+
+    def forward(self, input_ids):
+        return EmbeddingsWithFixes.forward(self, input_ids)
+
+
 def add_circular_option_to_conv_2d():
     conv2d_constructor = torch.nn.Conv2d.__init__
 

modules/sd_samplers_kdiffusion.py

@@ -120,6 +120,10 @@ class KDiffusionSampler(sd_samplers_common.Sampler):
             if scheduler.need_inner_model:
                 sigmas_kwargs['inner_model'] = self.model_wrap
 
+            if scheduler.label == 'Beta':
+                p.extra_generation_params["Beta schedule alpha"] = opts.beta_dist_alpha
+                p.extra_generation_params["Beta schedule beta"] = opts.beta_dist_beta
+
             sigmas = scheduler.function(n=steps, **sigmas_kwargs, device=devices.cpu)
 
         if discard_next_to_last_sigma:

modules/sd_schedulers.py

@@ -2,6 +2,7 @@ import dataclasses
 import torch
 import k_diffusion
 import numpy as np
+from scipy import stats
 
 from modules import shared
 
@@ -115,6 +116,17 @@ def ddim_scheduler(n, sigma_min, sigma_max, inner_model, device):
     return torch.FloatTensor(sigs).to(device)
 
 
+def beta_scheduler(n, sigma_min, sigma_max, inner_model, device):
+    # From "Beta Sampling is All You Need" [arXiv:2407.12173] (Lee et. al, 2024) """
+    alpha = shared.opts.beta_dist_alpha
+    beta = shared.opts.beta_dist_beta
+    timesteps = 1 - np.linspace(0, 1, n)
+    timesteps = [stats.beta.ppf(x, alpha, beta) for x in timesteps]
+    sigmas = [sigma_min + (x * (sigma_max-sigma_min)) for x in timesteps]
+    sigmas += [0.0]
+    return torch.FloatTensor(sigmas).to(device)
+
+
 schedulers = [
     Scheduler('automatic', 'Automatic', None),
     Scheduler('uniform', 'Uniform', uniform, need_inner_model=True),
@@ -127,6 +139,7 @@ schedulers = [
     Scheduler('simple', 'Simple', simple_scheduler, need_inner_model=True),
     Scheduler('normal', 'Normal', normal_scheduler, need_inner_model=True),
     Scheduler('ddim', 'DDIM', ddim_scheduler, need_inner_model=True),
+    Scheduler('beta', 'Beta', beta_scheduler, need_inner_model=True),
 ]
 
 schedulers_map = {**{x.name: x for x in schedulers}, **{x.label: x for x in schedulers}}

modules/shared_gradio_themes.py

@@ -69,3 +69,44 @@ def reload_gradio_theme(theme_name=None):
     # append additional values gradio_theme
     shared.gradio_theme.sd_webui_modal_lightbox_toolbar_opacity = shared.opts.sd_webui_modal_lightbox_toolbar_opacity
     shared.gradio_theme.sd_webui_modal_lightbox_icon_opacity = shared.opts.sd_webui_modal_lightbox_icon_opacity
+
+
+def resolve_var(name: str, gradio_theme=None, history=None):
+    """
+    Attempt to resolve a theme variable name to its value
+
+    Parameters:
+        name (str): The name of the theme variable
+            ie "background_fill_primary", "background_fill_primary_dark"
+            spaces and asterisk (*) prefix is removed from name before lookup
+        gradio_theme (gradio.themes.ThemeClass): The theme object to resolve the variable from
+            blank to use the webui default shared.gradio_theme
+        history (list): A list of previously resolved variables to prevent circular references
+            for regular use leave blank
+    Returns:
+        str: The resolved value
+
+    Error handling:
+        return either #000000 or #ffffff depending on initial name ending with "_dark"
+    """
+    try:
+        if history is None:
+            history = []
+        if gradio_theme is None:
+            gradio_theme = shared.gradio_theme
+
+        name = name.strip()
+        name = name[1:] if name.startswith("*") else name
+
+        if name in history:
+            raise ValueError(f'Circular references: name "{name}" in {history}')
+
+        if value := getattr(gradio_theme, name, None):
+            return resolve_var(value, gradio_theme, history + [name])
+        else:
+            return name
+
+    except Exception:
+        name = history[0] if history else name
+        errors.report(f'resolve_color({name})', exc_info=True)
+        return '#000000' if name.endswith("_dark") else '#ffffff'

modules/shared_options.py

@@ -64,6 +64,7 @@ options_templates.update(options_section(('saving-images', "Saving images/grids"
     "use_original_name_batch": OptionInfo(True, "Use original name for output filename during batch process in extras tab"),
     "use_upscaler_name_as_suffix": OptionInfo(False, "Use upscaler name as filename suffix in the extras tab"),
     "save_selected_only": OptionInfo(True, "When using 'Save' button, only save a single selected image"),
+    "save_write_log_csv": OptionInfo(True, "Write log.csv when saving images using 'Save' button"),
     "save_init_img": OptionInfo(False, "Save init images when using img2img"),
 
     "temp_dir":  OptionInfo("", "Directory for temporary images; leave empty for default"),
@@ -218,7 +219,6 @@ options_templates.update(options_section(('img2img', "img2img", "sd"), {
     "img2img_color_correction": OptionInfo(False, "Apply color correction to img2img results to match original colors."),
     "img2img_fix_steps": OptionInfo(False, "With img2img, do exactly the amount of steps the slider specifies.").info("normally you'd do less with less denoising"),
     "img2img_background_color": OptionInfo("#ffffff", "With img2img, fill transparent parts of the input image with this color.", ui_components.FormColorPicker, {}),
-    "img2img_editor_height": OptionInfo(720, "Height of the image editor", gr.Slider, {"minimum": 80, "maximum": 1600, "step": 1}).info("in pixels").needs_reload_ui(),
     "img2img_sketch_default_brush_color": OptionInfo("#ffffff", "Sketch initial brush color", ui_components.FormColorPicker, {}).info("default brush color of img2img sketch").needs_reload_ui(),
     "img2img_inpaint_mask_brush_color": OptionInfo("#ffffff", "Inpaint mask brush color", ui_components.FormColorPicker,  {}).info("brush color of inpaint mask").needs_reload_ui(),
     "img2img_inpaint_sketch_default_brush_color": OptionInfo("#ffffff", "Inpaint sketch initial brush color", ui_components.FormColorPicker, {}).info("default brush color of img2img inpaint sketch").needs_reload_ui(),
@@ -404,6 +404,8 @@ options_templates.update(options_section(('sampler-params', "Sampler parameters"
     'uni_pc_lower_order_final': OptionInfo(True, "UniPC lower order final", infotext='UniPC lower order final'),
     'sd_noise_schedule': OptionInfo("Default", "Noise schedule for sampling", gr.Radio, {"choices": ["Default", "Zero Terminal SNR"]}, infotext="Noise Schedule").info("for use with zero terminal SNR trained models"),
     'skip_early_cond': OptionInfo(0.0, "Ignore negative prompt during early sampling", gr.Slider, {"minimum": 0.0, "maximum": 1.0, "step": 0.01}, infotext="Skip Early CFG").info("disables CFG on a proportion of steps at the beginning of generation; 0=skip none; 1=skip all; can both improve sample diversity/quality and speed up sampling"),
+    'beta_dist_alpha': OptionInfo(0.6, "Beta scheduler - alpha", gr.Slider, {"minimum": 0.01, "maximum": 1.0, "step": 0.01}, infotext='Beta scheduler alpha').info('Default = 0.6; the alpha parameter of the beta distribution used in Beta sampling'),
+    'beta_dist_beta': OptionInfo(0.6, "Beta scheduler - beta", gr.Slider, {"minimum": 0.01, "maximum": 1.0, "step": 0.01}, infotext='Beta scheduler beta').info('Default = 0.6; the beta parameter of the beta distribution used in Beta sampling'),
 }))
 
 options_templates.update(options_section(('postprocessing', "Postprocessing", "postprocessing"), {

modules/shared_state.py

@@ -162,7 +162,7 @@ class State:
             errors.record_exception()
 
     def assign_current_image(self, image):
-        if shared.opts.live_previews_image_format == 'jpeg' and image.mode == 'RGBA':
+        if shared.opts.live_previews_image_format == 'jpeg' and image.mode in ('RGBA', 'P'):
             image = image.convert('RGB')
         self.current_image = image
         self.id_live_preview += 1

modules/sysinfo.py

@@ -1,15 +1,13 @@
 import json
 import os
 import sys
-
+import subprocess
 import platform
 import hashlib
-import pkg_resources
-import psutil
 import re
+from pathlib import Path
 
-import launch
-from modules import paths_internal, timer, shared, extensions, errors
+from modules import paths_internal, timer, shared_cmd_options, errors, launch_utils
 
 checksum_token = "DontStealMyGamePlz__WINNERS_DONT_USE_DRUGS__DONT_COPY_THAT_FLOPPY"
 environment_whitelist = {
@@ -69,14 +67,46 @@ def check(x):
     return h.hexdigest() == m.group(1)
 
 
-def get_dict():
-    ram = psutil.virtual_memory()
+def get_cpu_info():
+    cpu_info = {"model": platform.processor()}
+    try:
+        import psutil
+        cpu_info["count logical"] = psutil.cpu_count(logical=True)
+        cpu_info["count physical"] = psutil.cpu_count(logical=False)
+    except Exception as e:
+        cpu_info["error"] = str(e)
+    return cpu_info
 
+
+def get_ram_info():
+    try:
+        import psutil
+        ram = psutil.virtual_memory()
+        return {x: pretty_bytes(getattr(ram, x, 0)) for x in ["total", "used", "free", "active", "inactive", "buffers", "cached", "shared"] if getattr(ram, x, 0) != 0}
+    except Exception as e:
+        return str(e)
+
+
+def get_packages():
+    try:
+        return subprocess.check_output([sys.executable, '-m', 'pip', 'freeze', '--all']).decode("utf8").splitlines()
+    except Exception as pip_error:
+        try:
+            import importlib.metadata
+            packages = importlib.metadata.distributions()
+            return sorted([f"{package.metadata['Name']}=={package.version}" for package in packages])
+        except Exception as e2:
+            return {'error pip': pip_error, 'error importlib': str(e2)}
+
+
+def get_dict():
+    config = get_config()
     res = {
         "Platform": platform.platform(),
         "Python": platform.python_version(),
-        "Version": launch.git_tag(),
-        "Commit": launch.commit_hash(),
+        "Version": launch_utils.git_tag(),
+        "Commit": launch_utils.commit_hash(),
+        "Git status": git_status(paths_internal.script_path),
         "Script path": paths_internal.script_path,
         "Data path": paths_internal.data_path,
         "Extensions dir": paths_internal.extensions_dir,
@@ -84,20 +114,14 @@ def get_dict():
         "Commandline": get_argv(),
         "Torch env info": get_torch_sysinfo(),
         "Exceptions": errors.get_exceptions(),
-        "CPU": {
-            "model": platform.processor(),
-            "count logical": psutil.cpu_count(logical=True),
-            "count physical": psutil.cpu_count(logical=False),
-        },
-        "RAM": {
-            x: pretty_bytes(getattr(ram, x, 0)) for x in ["total", "used", "free", "active", "inactive", "buffers", "cached", "shared"] if getattr(ram, x, 0) != 0
-        },
-        "Extensions": get_extensions(enabled=True),
-        "Inactive extensions": get_extensions(enabled=False),
+        "CPU": get_cpu_info(),
+        "RAM": get_ram_info(),
+        "Extensions": get_extensions(enabled=True, fallback_disabled_extensions=config.get('disabled_extensions', [])),
+        "Inactive extensions": get_extensions(enabled=False, fallback_disabled_extensions=config.get('disabled_extensions', [])),
         "Environment": get_environment(),
-        "Config": get_config(),
+        "Config": config,
         "Startup": timer.startup_record,
-        "Packages": sorted([f"{pkg.key}=={pkg.version}" for pkg in pkg_resources.working_set]),
+        "Packages": get_packages(),
     }
 
     return res
@@ -111,11 +135,11 @@ def get_argv():
     res = []
 
     for v in sys.argv:
-        if shared.cmd_opts.gradio_auth and shared.cmd_opts.gradio_auth == v:
+        if shared_cmd_options.cmd_opts.gradio_auth and shared_cmd_options.cmd_opts.gradio_auth == v:
             res.append("<hidden>")
             continue
 
-        if shared.cmd_opts.api_auth and shared.cmd_opts.api_auth == v:
+        if shared_cmd_options.cmd_opts.api_auth and shared_cmd_options.cmd_opts.api_auth == v:
             res.append("<hidden>")
             continue
 
@@ -123,6 +147,7 @@ def get_argv():
 
     return res
 
+
 re_newline = re.compile(r"\r*\n")
 
 
@@ -136,25 +161,55 @@ def get_torch_sysinfo():
         return str(e)
 
 
-def get_extensions(*, enabled):
-
+def run_git(path, *args):
     try:
-        def to_json(x: extensions.Extension):
-            return {
-                "name": x.name,
-                "path": x.path,
-                "version": x.version,
-                "branch": x.branch,
-                "remote": x.remote,
-            }
+        return subprocess.check_output([launch_utils.git, '-C', path, *args], shell=False, encoding='utf8').strip()
+    except Exception as e:
+        return str(e)
 
-        return [to_json(x) for x in extensions.extensions if not x.is_builtin and x.enabled == enabled]
+
+def git_status(path):
+    if (Path(path) / '.git').is_dir():
+        return run_git(paths_internal.script_path, 'status')
+
+
+def get_info_from_repo_path(path: Path):
+    is_repo = (path / '.git').is_dir()
+    return {
+        'name': path.name,
+        'path': str(path),
+        'commit': run_git(path, 'rev-parse', 'HEAD') if is_repo else None,
+        'branch': run_git(path, 'branch', '--show-current') if is_repo else None,
+        'remote': run_git(path, 'remote', 'get-url', 'origin') if is_repo else None,
+    }
+
+
+def get_extensions(*, enabled, fallback_disabled_extensions=None):
+    try:
+        from modules import extensions
+        if extensions.extensions:
+            def to_json(x: extensions.Extension):
+                return {
+                    "name": x.name,
+                    "path": x.path,
+                    "commit": x.commit_hash,
+                    "branch": x.branch,
+                    "remote": x.remote,
+                }
+            return [to_json(x) for x in extensions.extensions if not x.is_builtin and x.enabled == enabled]
+        else:
+            return [get_info_from_repo_path(d) for d in Path(paths_internal.extensions_dir).iterdir() if d.is_dir() and enabled != (str(d.name) in fallback_disabled_extensions)]
     except Exception as e:
         return str(e)
 
 
 def get_config():
     try:
+        from modules import shared
         return shared.opts.data
-    except Exception as e:
-        return str(e)
+    except Exception as _:
+        try:
+            with open(shared_cmd_options.cmd_opts.ui_settings_file, 'r') as f:
+                return json.load(f)
+        except Exception as e:
+            return str(e)

modules/ui.py

@@ -8,11 +8,10 @@ from contextlib import ExitStack
 
 import gradio as gr
 import gradio.utils
-import numpy as np
 from PIL import Image, PngImagePlugin  # noqa: F401
-from modules.call_queue import wrap_gradio_gpu_call, wrap_queued_call, wrap_gradio_call
+from modules.call_queue import wrap_gradio_gpu_call, wrap_queued_call, wrap_gradio_call, wrap_gradio_call_no_job # noqa: F401
 
-from modules import gradio_extensons, sd_schedulers  # noqa: F401
+from modules import gradio_extensions, sd_schedulers  # noqa: F401
 from modules import sd_hijack, sd_models, script_callbacks, ui_extensions, deepbooru, extra_networks, ui_common, ui_postprocessing, progress, ui_loadsave, shared_items, ui_settings, timer, sysinfo, ui_checkpoint_merger, scripts, sd_samplers, processing, ui_extra_networks, ui_toprow, launch_utils
 from modules.ui_components import FormRow, FormGroup, ToolButton, FormHTML, InputAccordion, ResizeHandleRow
 from modules.paths import script_path
@@ -33,7 +32,7 @@ from modules.infotext_utils import image_from_url_text, PasteField
 create_setting_component = ui_settings.create_setting_component
 
 warnings.filterwarnings("default" if opts.show_warnings else "ignore", category=UserWarning)
-warnings.filterwarnings("default" if opts.show_gradio_deprecation_warnings else "ignore", category=gr.deprecation.GradioDeprecationWarning)
+warnings.filterwarnings("default" if opts.show_gradio_deprecation_warnings else "ignore", category=gradio_extensions.GradioDeprecationWarning)
 
 # this is a fix for Windows users. Without it, javascript files will be served with text/html content-type and the browser will not show any UI
 mimetypes.init()
@@ -101,8 +100,8 @@ def calc_resolution_hires(enable, width, height, hr_scale, hr_resize_x, hr_resiz
 
 
 def resize_from_to_html(width, height, scale_by):
-    target_width = int(width * scale_by)
-    target_height = int(height * scale_by)
+    target_width = int(float(width) * scale_by)
+    target_height = int(float(height) * scale_by)
 
     if not target_width or not target_height:
         return "no image selected"
@@ -111,10 +110,11 @@ def resize_from_to_html(width, height, scale_by):
 
 
 def process_interrogate(interrogation_function, mode, ii_input_dir, ii_output_dir, *ii_singles):
-    if mode in {0, 1, 3, 4}:
-        return [interrogation_function(ii_singles[mode]), None]
+    mode = int(mode)
+    if mode in (0, 1, 3, 4):
+        return [interrogation_function(ii_singles[mode]["composite"]), None]
     elif mode == 2:
-        return [interrogation_function(ii_singles[mode]["image"]), None]
+        return [interrogation_function(ii_singles[mode]["composite"]), None]
     elif mode == 5:
         assert not shared.cmd_opts.hide_ui_dir_config, "Launched with --hide-ui-dir-config, batch img2img disabled"
         images = shared.listfiles(ii_input_dir)
@@ -267,7 +267,8 @@ def create_ui():
     with gr.Blocks(analytics_enabled=False) as txt2img_interface:
         toprow = ui_toprow.Toprow(is_img2img=False, is_compact=shared.opts.compact_prompt_box)
 
-        dummy_component = gr.Label(visible=False)
+        dummy_component = gr.Textbox(visible=False)
+        dummy_component_number = gr.Number(visible=False)
 
         extra_tabs = gr.Tabs(elem_id="txt2img_extra_tabs", elem_classes=["extra-networks"])
         extra_tabs.__enter__()
@@ -310,7 +311,7 @@ def create_ui():
                         with gr.Row(elem_id="txt2img_accordions", elem_classes="accordions"):
                             with InputAccordion(False, label="Hires. fix", elem_id="txt2img_hr") as enable_hr:
                                 with enable_hr.extra():
-                                    hr_final_resolution = FormHTML(value="", elem_id="txtimg_hr_finalres", label="Upscaled resolution", interactive=False, min_width=0)
+                                    hr_final_resolution = FormHTML(value="", elem_id="txtimg_hr_finalres", label="Upscaled resolution")
 
                                 with FormRow(elem_id="txt2img_hires_fix_row1", variant="compact"):
                                     hr_upscaler = gr.Dropdown(label="Upscaler", elem_id="txt2img_hr_upscaler", choices=[*shared.latent_upscale_modes, *[x.name for x in shared.sd_upscalers]], value=shared.latent_upscale_default_mode)
@@ -424,7 +425,7 @@ def create_ui():
             output_panel.button_upscale.click(
                 fn=wrap_gradio_gpu_call(modules.txt2img.txt2img_upscale, extra_outputs=[None, '', '']),
                 _js="submit_txt2img_upscale",
-                inputs=txt2img_inputs[0:1] + [output_panel.gallery, dummy_component, output_panel.generation_info] + txt2img_inputs[1:],
+                inputs=txt2img_inputs[0:1] + [output_panel.gallery, dummy_component_number, output_panel.generation_info] + txt2img_inputs[1:],
                 outputs=txt2img_outputs,
                 show_progress=False,
             )
@@ -538,31 +539,21 @@ def create_ui():
                             img2img_selected_tab = gr.Number(value=0, visible=False)
 
                             with gr.TabItem('img2img', id='img2img', elem_id="img2img_img2img_tab") as tab_img2img:
-                                init_img = gr.Image(label="Image for img2img", elem_id="img2img_image", show_label=False, source="upload", interactive=True, type="pil", tool="editor", image_mode="RGBA", height=opts.img2img_editor_height)
+                                init_img = gr.ImageEditor(label="Image for img2img", elem_id="img2img_image", show_label=False, interactive=True, type="pil", image_mode="RGBA")
                                 add_copy_image_controls('img2img', init_img)
 
                             with gr.TabItem('Sketch', id='img2img_sketch', elem_id="img2img_img2img_sketch_tab") as tab_sketch:
-                                sketch = gr.Image(label="Image for img2img", elem_id="img2img_sketch", show_label=False, source="upload", interactive=True, type="pil", tool="color-sketch", image_mode="RGB", height=opts.img2img_editor_height, brush_color=opts.img2img_sketch_default_brush_color)
+                                sketch = gr.ImageEditor(label="Image for img2img", elem_id="img2img_sketch", show_label=False, interactive=True, type="pil", image_mode="RGBA", brush=gr.Brush(default_color=opts.img2img_sketch_default_brush_color))
                                 add_copy_image_controls('sketch', sketch)
 
                             with gr.TabItem('Inpaint', id='inpaint', elem_id="img2img_inpaint_tab") as tab_inpaint:
-                                init_img_with_mask = gr.Image(label="Image for inpainting with mask", show_label=False, elem_id="img2maskimg", source="upload", interactive=True, type="pil", tool="sketch", image_mode="RGBA", height=opts.img2img_editor_height, brush_color=opts.img2img_inpaint_mask_brush_color)
+                                init_img_with_mask = gr.ImageEditor(label="Image for inpainting with mask", show_label=False, elem_id="img2maskimg", brush=gr.Brush(colors=[opts.img2img_inpaint_mask_brush_color], color_mode="fixed"), interactive=True, type="pil", image_mode="RGBA", layers=False)
                                 add_copy_image_controls('inpaint', init_img_with_mask)
 
                             with gr.TabItem('Inpaint sketch', id='inpaint_sketch', elem_id="img2img_inpaint_sketch_tab") as tab_inpaint_color:
-                                inpaint_color_sketch = gr.Image(label="Color sketch inpainting", show_label=False, elem_id="inpaint_sketch", source="upload", interactive=True, type="pil", tool="color-sketch", image_mode="RGB", height=opts.img2img_editor_height, brush_color=opts.img2img_inpaint_sketch_default_brush_color)
-                                inpaint_color_sketch_orig = gr.State(None)
+                                inpaint_color_sketch = gr.ImageEditor(label="Color sketch inpainting", show_label=False, elem_id="inpaint_sketch", brush=gr.Brush(default_color=opts.img2img_inpaint_sketch_default_brush_color), interactive=True, type="pil", image_mode="RGBA", layers=False)
                                 add_copy_image_controls('inpaint_sketch', inpaint_color_sketch)
 
-                                def update_orig(image, state):
-                                    if image is not None:
-                                        same_size = state is not None and state.size == image.size
-                                        has_exact_match = np.any(np.all(np.array(image) == np.array(state), axis=-1))
-                                        edited = same_size and has_exact_match
-                                        return image if not edited or state is None else state
-
-                                inpaint_color_sketch.change(update_orig, [inpaint_color_sketch, inpaint_color_sketch_orig], inpaint_color_sketch_orig)
-
                             with gr.TabItem('Inpaint upload', id='inpaint_upload', elem_id="img2img_inpaint_upload_tab") as tab_inpaint_upload:
                                 init_img_inpaint = gr.Image(label="Image for img2img", show_label=False, source="upload", interactive=True, type="pil", elem_id="img_inpaint_base")
                                 init_mask_inpaint = gr.Image(label="Mask", source="upload", interactive=True, type="pil", image_mode="RGBA", elem_id="img_inpaint_mask")
@@ -595,20 +586,14 @@ def create_ui():
                             for i, tab in enumerate(img2img_tabs):
                                 tab.select(fn=lambda tabnum=i: tabnum, inputs=[], outputs=[img2img_selected_tab])
 
-                        def copy_image(img):
-                            if isinstance(img, dict) and 'image' in img:
-                                return img['image']
-
-                            return img
-
                         for button, name, elem in copy_image_buttons:
                             button.click(
-                                fn=copy_image,
+                                fn=lambda img: img,
                                 inputs=[elem],
                                 outputs=[copy_image_destinations[name]],
                             )
                             button.click(
-                                fn=lambda: None,
+                                fn=None,
                                 _js=f"switch_to_{name.replace(' ', '_')}",
                                 inputs=[],
                                 outputs=[],
@@ -622,8 +607,8 @@ def create_ui():
                             with gr.Column(elem_id="img2img_column_size", scale=4):
                                 selected_scale_tab = gr.Number(value=0, visible=False)
 
-                                with gr.Tabs():
-                                    with gr.Tab(label="Resize to", elem_id="img2img_tab_resize_to") as tab_scale_to:
+                                with gr.Tabs(elem_id="img2img_tabs_resize"):
+                                    with gr.Tab(label="Resize to", id="to", elem_id="img2img_tab_resize_to") as tab_scale_to:
                                         with FormRow():
                                             with gr.Column(elem_id="img2img_column_size", scale=4):
                                                 width = gr.Slider(minimum=64, maximum=2048, step=8, label="Width", value=512, elem_id="img2img_width")
@@ -632,7 +617,7 @@ def create_ui():
                                                 res_switch_btn = ToolButton(value=switch_values_symbol, elem_id="img2img_res_switch_btn", tooltip="Switch width/height")
                                                 detect_image_size_btn = ToolButton(value=detect_image_size_symbol, elem_id="img2img_detect_image_size_btn", tooltip="Auto detect size from img2img")
 
-                                    with gr.Tab(label="Resize by", elem_id="img2img_tab_resize_by") as tab_scale_by:
+                                    with gr.Tab(label="Resize by", id="by", elem_id="img2img_tab_resize_by") as tab_scale_by:
                                         scale_by = gr.Slider(minimum=0.05, maximum=4.0, step=0.05, label="Scale", value=1.0, elem_id="img2img_scale")
 
                                         with FormRow():
@@ -711,12 +696,6 @@ def create_ui():
                     if category not in {"accordions"}:
                         scripts.scripts_img2img.setup_ui_for_section(category)
 
-            # the code below is meant to update the resolution label after the image in the image selection UI has changed.
-            # as it is now the event keeps firing continuously for inpaint edits, which ruins the page with constant requests.
-            # I assume this must be a gradio bug and for now we'll just do it for non-inpaint inputs.
-            for component in [init_img, sketch]:
-                component.change(fn=lambda: None, _js="updateImg2imgResizeToTextAfterChangingImage", inputs=[], outputs=[], show_progress=False)
-
             def select_img2img_tab(tab):
                 return gr.update(visible=tab in [2, 3, 4]), gr.update(visible=tab == 3),
 
@@ -734,7 +713,7 @@ def create_ui():
                 _js="submit_img2img",
                 inputs=[
                     dummy_component,
-                    dummy_component,
+                    img2img_selected_tab,
                     toprow.prompt,
                     toprow.negative_prompt,
                     toprow.ui_styles.dropdown,
@@ -742,7 +721,6 @@ def create_ui():
                     sketch,
                     init_img_with_mask,
                     inpaint_color_sketch,
-                    inpaint_color_sketch_orig,
                     init_img_inpaint,
                     init_mask_inpaint,
                     mask_blur,
@@ -801,9 +779,9 @@ def create_ui():
             res_switch_btn.click(fn=None, _js="function(){switchWidthHeight('img2img')}", inputs=None, outputs=None, show_progress=False)
 
             detect_image_size_btn.click(
-                fn=lambda w, h, _: (w or gr.update(), h or gr.update()),
+                fn=lambda w, h: (w or gr.update(), h or gr.update()),
                 _js="currentImg2imgSourceResolution",
-                inputs=[dummy_component, dummy_component, dummy_component],
+                inputs=[dummy_component, dummy_component],
                 outputs=[width, height],
                 show_progress=False,
             )
@@ -889,7 +867,7 @@ def create_ui():
                     ))
 
         image.change(
-            fn=wrap_gradio_call(modules.extras.run_pnginfo),
+            fn=wrap_gradio_call_no_job(modules.extras.run_pnginfo),
             inputs=[image],
             outputs=[html, generation_info, html2],
         )

modules/ui_common.py

@@ -3,11 +3,11 @@ import dataclasses
 import json
 import html
 import os
+from contextlib import nullcontext
 
 import gradio as gr
 
 from modules import call_queue, shared, ui_tempdir, util
-from modules.infotext_utils import image_from_url_text
 import modules.images
 from modules.ui_components import ToolButton
 import modules.infotext_utils as parameters_copypaste
@@ -103,20 +103,19 @@ def save_files(js_data, images, do_make_zip, index):
 
     # NOTE: ensure csv integrity when fields are added by
     # updating headers and padding with delimiters where needed
-    if os.path.exists(logfile_path):
+    if shared.opts.save_write_log_csv and os.path.exists(logfile_path):
         update_logfile(logfile_path, fields)
 
-    with open(logfile_path, "a", encoding="utf8", newline='') as file:
-        at_start = file.tell() == 0
-        writer = csv.writer(file)
-        if at_start:
-            writer.writerow(fields)
+    with (open(logfile_path, "a", encoding="utf8", newline='') if shared.opts.save_write_log_csv else nullcontext()) as file:
+        if file:
+            at_start = file.tell() == 0
+            writer = csv.writer(file)
+            if at_start:
+                writer.writerow(fields)
 
         for image_index, filedata in enumerate(images, start_index):
-            image = image_from_url_text(filedata)
-
+            image = filedata[0]
             is_grid = image_index < p.index_of_first_image
-
             p.batch_index = image_index-1
 
             parameters = parameters_copypaste.parse_generation_parameters(data["infotexts"][image_index], [])
@@ -130,7 +129,8 @@ def save_files(js_data, images, do_make_zip, index):
                 filenames.append(os.path.basename(txt_fullfn))
                 fullfns.append(txt_fullfn)
 
-        writer.writerow([parsed_infotexts[0]['Prompt'], parsed_infotexts[0]['Seed'], data["width"], data["height"], data["sampler_name"], data["cfg_scale"], data["steps"], filenames[0], parsed_infotexts[0]['Negative prompt'], data["sd_model_name"], data["sd_model_hash"]])
+        if file:
+            writer.writerow([parsed_infotexts[0]['Prompt'], parsed_infotexts[0]['Seed'], data["width"], data["height"], data["sampler_name"], data["cfg_scale"], data["steps"], filenames[0], parsed_infotexts[0]['Negative prompt'], data["sd_model_name"], data["sd_model_hash"]])
 
     # Make Zip
     if do_make_zip:
@@ -181,7 +181,7 @@ def create_output_panel(tabname, outdir, toprow=None):
 
         with gr.Column(variant='panel', elem_id=f"{tabname}_results_panel"):
             with gr.Group(elem_id=f"{tabname}_gallery_container"):
-                res.gallery = gr.Gallery(label='Output', show_label=False, elem_id=f"{tabname}_gallery", columns=4, preview=True, height=shared.opts.gallery_height or None)
+                res.gallery = gr.Gallery(label='Output', show_label=False, elem_id=f"{tabname}_gallery", columns=4, preview=True, height=shared.opts.gallery_height or None, interactive=False, type="pil")
 
             with gr.Row(elem_id=f"image_buttons_{tabname}", elem_classes="image-buttons"):
                 open_folder_button = ToolButton(folder_symbol, elem_id=f'{tabname}_open_folder', visible=not shared.cmd_opts.hide_ui_dir_config, tooltip="Open images output directory.")
@@ -228,7 +228,7 @@ def create_output_panel(tabname, outdir, toprow=None):
                         )
 
                     save.click(
-                        fn=call_queue.wrap_gradio_call(save_files),
+                        fn=call_queue.wrap_gradio_call_no_job(save_files),
                         _js="(x, y, z, w) => [x, y, false, selected_gallery_index()]",
                         inputs=[
                             res.generation_info,
@@ -244,7 +244,7 @@ def create_output_panel(tabname, outdir, toprow=None):
                     )
 
                     save_zip.click(
-                        fn=call_queue.wrap_gradio_call(save_files),
+                        fn=call_queue.wrap_gradio_call_no_job(save_files),
                         _js="(x, y, z, w) => [x, y, true, selected_gallery_index()]",
                         inputs=[
                             res.generation_info,

modules/ui_components.py

@@ -1,7 +1,12 @@
+from functools import wraps
+
 import gradio as gr
+from modules import gradio_extensions  # noqa: F401
 
 
 class FormComponent:
+    webui_do_not_create_gradio_pyi_thank_you = True
+
     def get_expected_parent(self):
         return gr.components.Form
 
@@ -9,12 +14,13 @@ class FormComponent:
 gr.Dropdown.get_expected_parent = FormComponent.get_expected_parent
 
 
-class ToolButton(FormComponent, gr.Button):
+class ToolButton(gr.Button, FormComponent):
     """Small button with single emoji as text, fits inside gradio forms"""
 
-    def __init__(self, *args, **kwargs):
-        classes = kwargs.pop("elem_classes", [])
-        super().__init__(*args, elem_classes=["tool", *classes], **kwargs)
+    @wraps(gr.Button.__init__)
+    def __init__(self, value="", *args, elem_classes=None, **kwargs):
+        elem_classes = elem_classes or []
+        super().__init__(*args, elem_classes=["tool", *elem_classes], value=value, **kwargs)
 
     def get_block_name(self):
         return "button"
@@ -22,7 +28,9 @@ class ToolButton(FormComponent, gr.Button):
 
 class ResizeHandleRow(gr.Row):
     """Same as gr.Row but fits inside gradio forms"""
+    webui_do_not_create_gradio_pyi_thank_you = True
 
+    @wraps(gr.Row.__init__)
     def __init__(self, **kwargs):
         super().__init__(**kwargs)
 
@@ -32,79 +40,92 @@ class ResizeHandleRow(gr.Row):
         return "row"
 
 
-class FormRow(FormComponent, gr.Row):
+class FormRow(gr.Row, FormComponent):
     """Same as gr.Row but fits inside gradio forms"""
 
     def get_block_name(self):
         return "row"
 
 
-class FormColumn(FormComponent, gr.Column):
+class FormColumn(gr.Column, FormComponent):
     """Same as gr.Column but fits inside gradio forms"""
 
     def get_block_name(self):
         return "column"
 
 
-class FormGroup(FormComponent, gr.Group):
+class FormGroup(gr.Group, FormComponent):
     """Same as gr.Group but fits inside gradio forms"""
 
     def get_block_name(self):
         return "group"
 
 
-class FormHTML(FormComponent, gr.HTML):
+class FormHTML(gr.HTML, FormComponent):
     """Same as gr.HTML but fits inside gradio forms"""
 
     def get_block_name(self):
         return "html"
 
 
-class FormColorPicker(FormComponent, gr.ColorPicker):
+class FormColorPicker(gr.ColorPicker, FormComponent):
     """Same as gr.ColorPicker but fits inside gradio forms"""
 
     def get_block_name(self):
         return "colorpicker"
 
 
-class DropdownMulti(FormComponent, gr.Dropdown):
+class DropdownMulti(gr.Dropdown, FormComponent):
     """Same as gr.Dropdown but always multiselect"""
+
+    @wraps(gr.Dropdown.__init__)
     def __init__(self, **kwargs):
-        super().__init__(multiselect=True, **kwargs)
+        kwargs['multiselect'] = True
+        super().__init__(**kwargs)
 
     def get_block_name(self):
         return "dropdown"
 
 
-class DropdownEditable(FormComponent, gr.Dropdown):
+class DropdownEditable(gr.Dropdown, FormComponent):
     """Same as gr.Dropdown but allows editing value"""
+
+    @wraps(gr.Dropdown.__init__)
     def __init__(self, **kwargs):
-        super().__init__(allow_custom_value=True, **kwargs)
+        kwargs['allow_custom_value'] = True
+        super().__init__(**kwargs)
 
     def get_block_name(self):
         return "dropdown"
 
 
-class InputAccordion(gr.Checkbox):
+class InputAccordionImpl(gr.Checkbox):
     """A gr.Accordion that can be used as an input - returns True if open, False if closed.
 
     Actually just a hidden checkbox, but creates an accordion that follows and is followed by the state of the checkbox.
     """
 
+    webui_do_not_create_gradio_pyi_thank_you = True
+
     global_index = 0
 
-    def __init__(self, value, **kwargs):
+    @wraps(gr.Checkbox.__init__)
+    def __init__(self, value=None, setup=False, **kwargs):
+        if not setup:
+            super().__init__(value=value, **kwargs)
+            return
+
         self.accordion_id = kwargs.get('elem_id')
         if self.accordion_id is None:
-            self.accordion_id = f"input-accordion-{InputAccordion.global_index}"
-            InputAccordion.global_index += 1
+            self.accordion_id = f"input-accordion-{InputAccordionImpl.global_index}"
+            InputAccordionImpl.global_index += 1
 
         kwargs_checkbox = {
             **kwargs,
             "elem_id": f"{self.accordion_id}-checkbox",
             "visible": False,
         }
-        super().__init__(value, **kwargs_checkbox)
+        super().__init__(value=value, **kwargs_checkbox)
 
         self.change(fn=None, _js='function(checked){ inputAccordionChecked("' + self.accordion_id + '", checked); }', inputs=[self])
 
@@ -115,6 +136,7 @@ class InputAccordion(gr.Checkbox):
             "elem_classes": ['input-accordion'],
             "open": value,
         }
+
         self.accordion = gr.Accordion(**kwargs_accordion)
 
     def extra(self):
@@ -143,3 +165,6 @@ class InputAccordion(gr.Checkbox):
     def get_block_name(self):
         return "checkbox"
 
+
+def InputAccordion(value=None, **kwargs):
+    return InputAccordionImpl(value=value, setup=True, **kwargs)

modules/ui_extensions.py

@@ -624,37 +624,37 @@ def create_ui():
                 )
 
                 install_extension_button.click(
-                    fn=modules.ui.wrap_gradio_call(install_extension_from_index, extra_outputs=[gr.update(), gr.update()]),
+                    fn=modules.ui.wrap_gradio_call_no_job(install_extension_from_index, extra_outputs=[gr.update(), gr.update()]),
                     inputs=[extension_to_install, selected_tags, showing_type, filtering_type, sort_column, search_extensions_text],
                     outputs=[available_extensions_table, extensions_table, install_result],
                 )
 
                 search_extensions_text.change(
-                    fn=modules.ui.wrap_gradio_call(search_extensions, extra_outputs=[gr.update()]),
+                    fn=modules.ui.wrap_gradio_call_no_job(search_extensions, extra_outputs=[gr.update()]),
                     inputs=[search_extensions_text, selected_tags, showing_type, filtering_type, sort_column],
                     outputs=[available_extensions_table, install_result],
                 )
 
                 selected_tags.change(
-                    fn=modules.ui.wrap_gradio_call(refresh_available_extensions_for_tags, extra_outputs=[gr.update()]),
+                    fn=modules.ui.wrap_gradio_call_no_job(refresh_available_extensions_for_tags, extra_outputs=[gr.update()]),
                     inputs=[selected_tags, showing_type, filtering_type, sort_column, search_extensions_text],
                     outputs=[available_extensions_table, install_result]
                 )
 
                 showing_type.change(
-                    fn=modules.ui.wrap_gradio_call(refresh_available_extensions_for_tags, extra_outputs=[gr.update()]),
+                    fn=modules.ui.wrap_gradio_call_no_job(refresh_available_extensions_for_tags, extra_outputs=[gr.update()]),
                     inputs=[selected_tags, showing_type, filtering_type, sort_column, search_extensions_text],
                     outputs=[available_extensions_table, install_result]
                 )
 
                 filtering_type.change(
-                    fn=modules.ui.wrap_gradio_call(refresh_available_extensions_for_tags, extra_outputs=[gr.update()]),
+                    fn=modules.ui.wrap_gradio_call_no_job(refresh_available_extensions_for_tags, extra_outputs=[gr.update()]),
                     inputs=[selected_tags, showing_type, filtering_type, sort_column, search_extensions_text],
                     outputs=[available_extensions_table, install_result]
                 )
 
                 sort_column.change(
-                    fn=modules.ui.wrap_gradio_call(refresh_available_extensions_for_tags, extra_outputs=[gr.update()]),
+                    fn=modules.ui.wrap_gradio_call_no_job(refresh_available_extensions_for_tags, extra_outputs=[gr.update()]),
                     inputs=[selected_tags, showing_type, filtering_type, sort_column, search_extensions_text],
                     outputs=[available_extensions_table, install_result]
                 )
@@ -667,7 +667,7 @@ def create_ui():
                 install_result = gr.HTML(elem_id="extension_install_result")
 
                 install_button.click(
-                    fn=modules.ui.wrap_gradio_call(lambda *args: [gr.update(), *install_extension_from_url(*args)], extra_outputs=[gr.update(), gr.update()]),
+                    fn=modules.ui.wrap_gradio_call_no_job(lambda *args: [gr.update(), *install_extension_from_url(*args)], extra_outputs=[gr.update(), gr.update()]),
                     inputs=[install_dirname, install_url, install_branch],
                     outputs=[install_url, extensions_table, install_result],
                 )
@@ -688,7 +688,7 @@ def create_ui():
 
                 config_save_button.click(fn=save_config_state, inputs=[config_save_name], outputs=[config_states_list, config_states_info])
 
-                dummy_component = gr.Label(visible=False)
+                dummy_component = gr.State()
                 config_restore_button.click(fn=restore_config_state, _js="config_state_confirm_restore", inputs=[dummy_component, config_states_list, config_restore_type], outputs=[config_states_info])
 
                 config_states_list.change(

modules/ui_extra_networks.py

@@ -750,9 +750,11 @@ def create_ui(interface: gr.Blocks, unrelated_tabs, tabname):
             elem_id = f"{tabname}_{page.extra_networks_tabname}_cards_html"
             page_elem = gr.HTML(page.create_html(tabname, empty=True), elem_id=elem_id)
             ui.pages.append(page_elem)
+
             editor = page.create_user_metadata_editor(ui, tabname)
             editor.create_ui()
             ui.user_metadata_editors.append(editor)
+
             related_tabs.append(tab)
 
     ui.button_save_preview = gr.Button('Save preview', elem_id=f"{tabname}_save_preview", visible=False)

modules/ui_gradio_extensions.py

@@ -41,6 +41,11 @@ def css_html():
     if os.path.exists(user_css):
         head += stylesheet(user_css)
 
+    from modules.shared_gradio_themes import resolve_var
+    light = resolve_var('background_fill_primary')
+    dark = resolve_var('background_fill_primary_dark')
+    head += f'<style>html {{ background-color: {light}; }} @media (prefers-color-scheme: dark) {{ html {{background-color:  {dark}; }} }}</style>'
+
     return head
 
 

modules/ui_postprocessing.py

@@ -5,14 +5,14 @@ from modules.ui_components import ResizeHandleRow
 
 
 def create_ui():
-    dummy_component = gr.Label(visible=False)
-    tab_index = gr.Number(value=0, visible=False)
+    dummy_component = gr.Textbox(visible=False)
+    tab_index = gr.State(0)
 
     with ResizeHandleRow(equal_height=False, variant='compact'):
         with gr.Column(variant='compact'):
             with gr.Tabs(elem_id="mode_extras"):
                 with gr.TabItem('Single Image', id="single_image", elem_id="extras_single_tab") as tab_single:
-                    extras_image = gr.Image(label="Source", source="upload", interactive=True, type="pil", elem_id="extras_image", image_mode="RGBA")
+                    extras_image = gr.ImageEditor(label="Source", interactive=True, type="pil", elem_id="extras_image", image_mode="RGBA")
 
                 with gr.TabItem('Batch Process', id="batch_process", elem_id="extras_batch_process_tab") as tab_batch:
                     image_batch = gr.Files(label="Batch Process", interactive=True, elem_id="extras_image_batch")

modules/ui_settings.py

@@ -1,7 +1,7 @@
 import gradio as gr
 
 from modules import ui_common, shared, script_callbacks, scripts, sd_models, sysinfo, timer, shared_items
-from modules.call_queue import wrap_gradio_call
+from modules.call_queue import wrap_gradio_call_no_job
 from modules.options import options_section
 from modules.shared import opts
 from modules.ui_components import FormRow
@@ -295,7 +295,7 @@ class UiSettings:
 
     def add_functionality(self, demo):
         self.submit.click(
-            fn=wrap_gradio_call(lambda *args: self.run_settings(*args), extra_outputs=[gr.update()]),
+            fn=wrap_gradio_call_no_job(lambda *args: self.run_settings(*args), extra_outputs=[gr.update()]),
             inputs=self.components,
             outputs=[self.text_settings, self.result],
         )

modules/ui_tempdir.py

@@ -4,6 +4,7 @@ from collections import namedtuple
 from pathlib import Path
 
 import gradio.components
+import gradio as gr
 
 from PIL import PngImagePlugin
 
@@ -13,25 +14,35 @@ from modules import shared
 Savedfile = namedtuple("Savedfile", ["name"])
 
 
-def register_tmp_file(gradio, filename):
-    if hasattr(gradio, 'temp_file_sets'):  # gradio 3.15
-        gradio.temp_file_sets[0] = gradio.temp_file_sets[0] | {os.path.abspath(filename)}
+def register_tmp_file(gradio_app, filename):
+    if hasattr(gradio_app, 'temp_file_sets'):  # gradio 3.15
+        if hasattr(gr.utils, 'abspath'):  # gradio 4.19
+            filename = gr.utils.abspath(filename)
+        else:
+            filename = os.path.abspath(filename)
 
-    if hasattr(gradio, 'temp_dirs'):  # gradio 3.9
-        gradio.temp_dirs = gradio.temp_dirs | {os.path.abspath(os.path.dirname(filename))}
+        gradio_app.temp_file_sets[0] = gradio_app.temp_file_sets[0] | {filename}
+
+    if hasattr(gradio_app, 'temp_dirs'):  # gradio 3.9
+        gradio_app.temp_dirs = gradio_app.temp_dirs | {os.path.abspath(os.path.dirname(filename))}
 
 
-def check_tmp_file(gradio, filename):
-    if hasattr(gradio, 'temp_file_sets'):
-        return any(filename in fileset for fileset in gradio.temp_file_sets)
+def check_tmp_file(gradio_app, filename):
+    if hasattr(gradio_app, 'temp_file_sets'):
+        if hasattr(gr.utils, 'abspath'):  # gradio 4.19
+            filename = gr.utils.abspath(filename)
+        else:
+            filename = os.path.abspath(filename)
 
-    if hasattr(gradio, 'temp_dirs'):
-        return any(Path(temp_dir).resolve() in Path(filename).resolve().parents for temp_dir in gradio.temp_dirs)
+        return any(filename in fileset for fileset in gradio_app.temp_file_sets)
+
+    if hasattr(gradio_app, 'temp_dirs'):
+        return any(Path(temp_dir).resolve() in Path(filename).resolve().parents for temp_dir in gradio_app.temp_dirs)
 
     return False
 
 
-def save_pil_to_file(self, pil_image, dir=None, format="png"):
+def save_pil_to_file(pil_image, cache_dir=None, format="png"):
     already_saved_as = getattr(pil_image, 'already_saved_as', None)
     if already_saved_as and os.path.isfile(already_saved_as):
         register_tmp_file(shared.demo, already_saved_as)
@@ -39,9 +50,10 @@ def save_pil_to_file(self, pil_image, dir=None, format="png"):
         register_tmp_file(shared.demo, filename_with_mtime)
         return filename_with_mtime
 
-    if shared.opts.temp_dir != "":
+    if shared.opts.temp_dir:
         dir = shared.opts.temp_dir
     else:
+        dir = cache_dir
         os.makedirs(dir, exist_ok=True)
 
     use_metadata = False
@@ -56,9 +68,96 @@ def save_pil_to_file(self, pil_image, dir=None, format="png"):
     return file_obj.name
 
 
+async def async_move_files_to_cache(data, block, postprocess=False, check_in_upload_folder=False, keep_in_cache=False):
+    """Move any files in `data` to cache and (optionally), adds URL prefixes (/file=...) needed to access the cached file.
+    Also handles the case where the file is on an external Gradio app (/proxy=...).
+
+    Runs after .postprocess() and before .preprocess().
+
+    Copied from gradio's processing_utils.py
+
+    Args:
+        data: The input or output data for a component. Can be a dictionary or a dataclass
+        block: The component whose data is being processed
+        postprocess: Whether its running from postprocessing
+        check_in_upload_folder: If True, instead of moving the file to cache, checks if the file is in already in cache (exception if not).
+        keep_in_cache: If True, the file will not be deleted from cache when the server is shut down.
+    """
+
+    from gradio import FileData
+    from gradio.data_classes import GradioRootModel
+    from gradio.data_classes import GradioModel
+    from gradio_client import utils as client_utils
+    from gradio.utils import get_upload_folder, is_in_or_equal, is_static_file
+
+    async def _move_to_cache(d: dict):
+        payload = FileData(**d)
+
+        # EDITED
+        payload.path = payload.path.rsplit('?', 1)[0]
+
+        # If the gradio app developer is returning a URL from
+        # postprocess, it means the component can display a URL
+        # without it being served from the gradio server
+        # This makes it so that the URL is not downloaded and speeds up event processing
+        if payload.url and postprocess and client_utils.is_http_url_like(payload.url):
+            payload.path = payload.url
+        elif is_static_file(payload):
+            pass
+        elif not block.proxy_url:
+            # EDITED
+            if check_tmp_file(shared.demo, payload.path):
+                temp_file_path = payload.path
+            else:
+                # If the file is on a remote server, do not move it to cache.
+                if check_in_upload_folder and not client_utils.is_http_url_like(
+                    payload.path
+                ):
+                    path = os.path.abspath(payload.path)
+                    if not is_in_or_equal(path, get_upload_folder()):
+                        raise ValueError(
+                            f"File {path} is not in the upload folder and cannot be accessed."
+                        )
+                if not payload.is_stream:
+                    temp_file_path = await block.async_move_resource_to_block_cache(
+                        payload.path
+                    )
+                    if temp_file_path is None:
+                        raise ValueError("Did not determine a file path for the resource.")
+                    payload.path = temp_file_path
+                    if keep_in_cache:
+                        block.keep_in_cache.add(payload.path)
+
+        url_prefix = "/stream/" if payload.is_stream else "/file="
+        if block.proxy_url:
+            proxy_url = block.proxy_url.rstrip("/")
+            url = f"/proxy={proxy_url}{url_prefix}{payload.path}"
+        elif client_utils.is_http_url_like(payload.path) or payload.path.startswith(
+            f"{url_prefix}"
+        ):
+            url = payload.path
+        else:
+            url = f"{url_prefix}{payload.path}"
+        payload.url = url
+
+        return payload.model_dump()
+
+    if isinstance(data, (GradioRootModel, GradioModel)):
+        data = data.model_dump()
+
+    return await client_utils.async_traverse(
+        data, _move_to_cache, client_utils.is_file_obj
+    )
+
+
 def install_ui_tempdir_override():
-    """override save to file function so that it also writes PNG info"""
-    gradio.components.IOComponent.pil_to_temp_file = save_pil_to_file
+    """
+    override save to file function so that it also writes PNG info.
+    override gradio4's move_files_to_cache function to prevent it from writing a copy into a temporary directory.
+    """
+
+    gradio.processing_utils.save_pil_to_cache = save_pil_to_file
+    gradio.processing_utils.async_move_files_to_cache = async_move_files_to_cache
 
 
 def on_tmpdir_changed():

modules/upscaler.py

@@ -56,8 +56,8 @@ class Upscaler:
         dest_w = int((img.width * scale) // 8 * 8)
         dest_h = int((img.height * scale) // 8 * 8)
 
-        for _ in range(3):
-            if img.width >= dest_w and img.height >= dest_h and scale != 1:
+        for i in range(3):
+            if img.width >= dest_w and img.height >= dest_h and (i > 0 or scale != 1):
                 break
 
             if shared.state.interrupted:

modules/upscaler_utils.py

@@ -41,7 +41,7 @@ def upscale_pil_patch(model, img: Image.Image) -> Image.Image:
     """
     param = torch_utils.get_param(model)
 
-    with torch.no_grad():
+    with torch.inference_mode():
         tensor = pil_image_to_torch_bgr(img).unsqueeze(0)  # add batch dimension
         tensor = tensor.to(device=param.device, dtype=param.dtype)
         with devices.without_autocast():

requirements.txt

@@ -8,7 +8,7 @@ diskcache
 einops
 facexlib
 fastapi>=0.90.1
-gradio==3.41.2
+gradio==4.38.1
 inflection
 jsonmerge
 kornia

requirements_versions.txt

@@ -7,8 +7,8 @@ clean-fid==0.1.35
 diskcache==5.6.3
 einops==0.4.1
 facexlib==0.3.0
-fastapi==0.94.0
-gradio==3.41.2
+fastapi==0.104.1
+gradio==4.38.1
 httpcore==0.15
 inflection==0.5.1
 jsonmerge==1.8.0

scripts/xyz_grid.py

@@ -118,11 +118,10 @@ def apply_size(p, x: str, xs) -> None:
 
 
 def find_vae(name: str):
-    match name := name.lower().strip():
-        case 'auto', 'automatic':
-            return 'Automatic'
-        case 'none':
-            return 'None'
+    if (name := name.strip().lower()) in ('auto', 'automatic'):
+        return 'Automatic'
+    elif name == 'none':
+        return 'None'
     return next((k for k in modules.sd_vae.vae_dict if k.lower() == name), print(f'No VAE found for {name}; using Automatic') or 'Automatic')
 
 
@@ -260,6 +259,8 @@ axis_options = [
     AxisOption("Schedule min sigma", float, apply_override("sigma_min")),
     AxisOption("Schedule max sigma", float, apply_override("sigma_max")),
     AxisOption("Schedule rho", float, apply_override("rho")),
+    AxisOption("Beta schedule alpha", float, apply_override("beta_dist_alpha")),
+    AxisOption("Beta schedule beta", float, apply_override("beta_dist_beta")),
     AxisOption("Eta", float, apply_field("eta")),
     AxisOption("Clip skip", int, apply_override('CLIP_stop_at_last_layers')),
     AxisOption("Denoising", float, apply_field("denoising_strength")),

style.css

@@ -2,14 +2,6 @@
 
 @import url('webui-assets/css/sourcesanspro.css');
 
-
-/* temporary fix to hide gradio crop tool until it's fixed https://github.com/gradio-app/gradio/issues/3810 */
-
-div.gradio-image button[aria-label="Edit"] {
-    display: none;
-}
-
-
 /* general gradio fixes */
 
 :root, .dark{
@@ -137,6 +129,10 @@ div.gradio-html.min{
     background: var(--input-background-fill);
 }
 
+.gradio-gallery > button.preview{
+    width: 100%;
+}
+
 .gradio-container .prose a, .gradio-container .prose a:visited{
     color: unset;
     text-decoration: none;
@@ -147,6 +143,15 @@ a{
     cursor: pointer;
 }
 
+.upload-container {
+    width: 100%;
+    max-width: 100%;
+}
+
+.layer-wrap > ul {
+    background: var(--background-fill-primary) !important;
+}
+
 /* gradio 3.39 puts a lot of overflow: hidden all over the place for an unknown reason. */
 div.gradio-container, .block.gradio-textbox, div.gradio-group, div.gradio-dropdown{
     overflow: visible !important;
@@ -398,7 +403,7 @@ div#extras_scale_to_tab div.form{
     z-index: 5;
 }
 
-.image-buttons > .form{
+.image-buttons{
     justify-content: center;
 }
 
@@ -1098,9 +1103,9 @@ footer {
     height:100%;
 }
 
-div.block.gradio-box.edit-user-metadata {
+.edit-user-metadata {
     width: 56em;
-    background: var(--body-background-fill);
+    background: var(--body-background-fill) !important;
     padding: 2em !important;
 }
 
@@ -1134,16 +1139,12 @@ div.block.gradio-box.edit-user-metadata {
     margin-top: 1.5em;
 }
 
-div.block.gradio-box.popup-dialog, .popup-dialog {
+.popup-dialog {
     width: 56em;
-    background: var(--body-background-fill);
+    background: var(--body-background-fill) !important;
     padding: 2em !important;
 }
 
-div.block.gradio-box.popup-dialog > div:last-child, .popup-dialog > div:last-child{
-    margin-top: 1em;
-}
-
 div.block.input-accordion{
 
 }

webui.bat

@@ -48,6 +48,7 @@ echo Warning: Failed to upgrade PIP version
 
 :activate_venv
 set PYTHON="%VENV_DIR%\Scripts\Python.exe"
+call "%VENV_DIR%\Scripts\activate.bat"
 echo venv %PYTHON%
 
 :skip_venv