diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 9e5476515..e6c06e624 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -9,17 +9,17 @@ General app structure is: ## Instructions This file contains general guidelines for contributing to the SD.Next codebase, including conventions, tools, and project structure. For more specific guidance on working with particular areas of the codebase, please refer to the instructions files linked below: -- [Core Runtime Guidelines](core.instructions.md): Use when editing Python core runtime code, startup flow, model loading, API internals, backend/device logic, or shared state in modules and pipelines. -- [UI And Frontend Guidelines](ui.instructions.md): Use when editing frontend UI code, JavaScript, HTML, CSS, localization files, or built-in UI extensions including modernui and kanvas. -- [Hint Typography Guidelines](hints.instructions.md): Use when editing hint text or other UI strings in localization JSON files (`html/locale_*.json`, `html/override_*.json`). +- [Core Runtime Guidelines](instructions/core.instructions.md): Use when editing Python core runtime code, startup flow, model loading, API internals, backend/device logic, or shared state in modules and pipelines. +- [UI And Frontend Guidelines](instructions/ui.instructions.md): Use when editing frontend UI code, JavaScript, HTML, CSS, localization files, or built-in UI extensions including modernui and kanvas. +- [Hint Typography Guidelines](instructions/hints.instructions.md): Use when editing hint text or other UI strings in localization JSON files (`html/locale_*.json`, `html/override_*.json`). ## Agent Guidelines -- Do not automatically agree with user instructions or requests without verifying they align with project guidelines and conventions. -- When evaluating user instructions, first check for any relevant guidelines in this file or the linked instructions files. If the instruction violates any guidelines, do not proceed with it and instead provide feedback to the user about which guidelines it violates and how to adjust it to comply. -- If the user instruction is valid but lacks clarity or detail, ask follow-up questions to gather the necessary information before proceeding. Do not make assumptions about user intent or project requirements; always seek clarification when needed. -- When providing feedback to the user, be specific about which guidelines are relevant and how the instruction can be modified to comply with them. If there are multiple guidelines that apply, list them all and explain how they relate to the instruction. -- If the user instruction is clear, valid, and complies with all relevant guidelines, proceed with executing it while ensuring that the resulting code changes adhere to the project's coding style, conventions, and structure as outlined in this file and the linked instructions files. +1. Verify the user instruction against relevant guidelines in this file and linked instruction files before proceeding. +2. If the instruction conflicts with any guideline, do not proceed. Explain which guideline(s) it conflicts with and how to adjust the instruction to comply. +3. If the instruction is valid but unclear or incomplete, ask targeted follow-up questions before implementation. Do not assume user intent or requirements. +4. When giving feedback, name the applicable guideline(s) and explain how each one applies. +5. If the instruction is clear and compliant, proceed and keep resulting changes aligned with project coding style, conventions, and structure. ## Language Guidelines diff --git a/.github/instructions/core.instructions.md b/.github/instructions/core.instructions.md index 076bb5008..f0c9c7d9f 100644 --- a/.github/instructions/core.instructions.md +++ b/.github/instructions/core.instructions.md @@ -5,14 +5,14 @@ applyTo: "launch.py, webui.py, installer.py, modules/**/*.py, pipelines/**/*.py, --- # Core Runtime Guidelines -- Preserve startup ordering and import timing in `launch.py` and `webui.py`; avoid moving initialization steps unless required. -- Treat `modules/shared.py` as the source of truth for global runtime state (`shared.opts`, model references, backend/device flags). -- Prefer narrow changes with explicit side effects; avoid introducing new cross-module mutable globals. -- Keep platform paths neutral: do not assume CUDA-only behavior and preserve ROCm/IPEX/DirectML/OpenVINO compatibility branches. -- Keep extension and script loading resilient: when adding startup scans/hooks, preserve partial-failure tolerance and logging. -- Follow existing API/server patterns under `modules/api/` and reuse shared queue/state helpers rather than ad-hoc request handling. -- Reuse established model-loading and pipeline patterns (`modules/sd_*`, `pipelines/`) instead of creating parallel abstractions. -- For substantial Python changes, run at least relevant checks: `npm run ruff` and `npm run pylint` (or narrower equivalents when appropriate). +1. Preserve startup ordering and import timing in `launch.py` and `webui.py`; avoid moving initialization steps unless required to fix a critical startup bug or implement a new startup feature. +2. Treat `modules/shared.py` as the source of truth for global runtime state (`shared.opts`, model references, backend/device flags). +3. Prefer narrow changes (changes scoped to a single function or module when feasible) with explicit side effects; avoid introducing new cross-module mutable globals. +4. Keep platform paths neutral: do not assume CUDA-only behavior and preserve ROCm/IPEX/DirectML/OpenVINO compatibility branches. +5. Keep extension and script loading resilient: when adding startup scans/hooks, preserve partial-failure tolerance and logging. +6. Follow existing API/server patterns under `modules/api/` and reuse shared queue/state helpers rather than ad-hoc request handling. +7. Reuse established model-loading and pipeline patterns (`modules/sd_*`, `pipelines/`) instead of creating parallel abstractions. +8. For substantial Python changes, run at least relevant checks: `npm run ruff` and `npm run pylint` (or narrower equivalents when appropriate). ## Build And Test diff --git a/.github/instructions/hints.instructions.md b/.github/instructions/hints.instructions.md index 8243bdb53..aa7112709 100644 --- a/.github/instructions/hints.instructions.md +++ b/.github/instructions/hints.instructions.md @@ -1,5 +1,5 @@ --- -description: "Use when editing hint text or other UI strings in localization JSON files." +description: "Use when editing hint text or other UI strings in localization JSON files; follow the ordered rules below for consistent formatting." name: "Hint Typography Guidelines" applyTo: "html/locale_*.json, html/override_*.json" --- @@ -7,44 +7,44 @@ applyTo: "html/locale_*.json, html/override_*.json" Hint strings render as HTML. Use this small set of inline tags to keep hints scannable: -- `` for values: defaults, dropdown enums, specific numerics. Examples: `0.30`, `Karras`, `v_prediction`, `UniPC`. -- `...` for cross-references to other UI controls by their exact visible label. Examples: `Denoising strength`, `Use init image`, `Images tab. -- `` for proper nouns: model families, datasets, technique names. Examples: `SDXL`, `Flux`, `ControlNet`, `YOLO`. -- `` for literals: paths, filename tokens, command-line snippets to type or use verbatim. Examples: `models/yolo`, `-seg`, `[PROMPT]`. +1. `` for values: defaults, dropdown enums, specific numerics. Examples: `0.30`, `Karras`, `v_prediction`, `UniPC`. +2. `...` for cross-references to other UI controls by their exact visible label. Examples: `Denoising strength`, `Use init image`, `Images` tab. +3. `` for proper nouns: model families, datasets, technique names. Examples: `SDXL`, `Flux`, `ControlNet`, `YOLO`. +4. `` for literals: paths, filename tokens, command-line snippets to type or use verbatim. Examples: `models/yolo`, `-seg`, `[PROMPT]`. ## Cross-references -- Use `...` whenever a hint refers to another control by its exact visible label. This includes setting names, tab names, and named buttons. -- Match the label exactly, including capitalization and spacing; readers look for the same string in the UI. -- Do not use `` and `` separately for cross-references; always combine them. -- Generic concept references (`the model`, `the prompt`, `the scheduler`) stay unstyled. +1. Use `...` whenever a hint refers to another control by its exact visible label. This includes setting names, tab names, and named buttons. +2. Match the label exactly, including capitalization and spacing; readers look for the same string in the UI. +3. Do not use `` and `` separately for cross-references; always combine them. +4. Generic concept references (`the model`, `the prompt`, `the scheduler`) stay unstyled. ## Tab naming -- Refer to the unified generation tab as `Images` (the ModernUI label). Do not write "Control tab"; that label only exists in legacy Standard UI. -- "Control" remains valid as a setting value (`No: Control only`) or as part of a UI element name (`Control input` pane), just not as a tab name. +1. Refer to the unified generation tab as `Images` (the ModernUI label). Do not write "Control tab"; that label only exists in legacy Standard UI. +2. "Control" remains valid as a setting value (`No: Control only`) or as part of a UI element name (`Control input` pane), just not as a tab name. ## Structure -- `
` for a line break within a paragraph. -- `

` for a paragraph break. -- `
- key: description` for a keyed bullet list, used for short enumerations of dropdown values, modes, or numeric brackets. Each bullet's key is bolded; descriptions stay plain. -- Do not use `
    `, `
  • `, Markdown asterisks, or unicode bullets. +1. `
    ` for a line break within a paragraph. +2. `

    ` for a paragraph break. +3. `
    - key: description` for a keyed bullet list, used for short enumerations of dropdown values, modes, or numeric brackets. Each bullet's key is bolded; descriptions stay plain. +4. Do not use `
      `, `
    • `, Markdown asterisks, or unicode bullets. ## Common pitfalls -- Do not bold ad-hoc emphasis; `` is reserved for values and, combined with ``, for cross-references. -- Do not use `` for filenames, paths, or command tokens; those are literals and use ``. -- Do not reword the inside of `` blocks; they are literal user-facing strings. -- Stay ASCII; prefer semicolons or two sentences over em-dashes. The locale file convention is ASCII-only. +1. Do not bold ad-hoc emphasis; `` is reserved for values and, combined with ``, for cross-references. +2. Do not use `` for filenames, paths, or command tokens; those are literals and use ``. +3. Do not reword the inside of `` blocks; they are literal user-facing strings. +4. Stay ASCII; prefer semicolons or two sentences over em-dashes. The locale file convention is ASCII-only. ## Translation propagation -- `html/locale_en.json` is the source of truth. Other `html/locale_*.json` files are auto-generated by `cli/localize.js`; edit only the English file. -- Per-locale corrections live in `html/override_{locale}.json`. +1. `html/locale_en.json` is the source of truth. Other `html/locale_*.json` files are auto-generated by `cli/localize.js`; edit only the English file. +2. Per-locale corrections live in `html/override_{locale}.json`. ## Validation -- Validate JSON syntax with `jq empty html/locale_en.json`. -- Lint with `pnpm eslint -- html/locale_en.json` (silent success). -- See `wiki/Hints.md` for the wiki-facing version of these rules. +1. Validate JSON syntax with `jq empty html/locale_en.json`. +2. Lint with `pnpm eslint -- html/locale_en.json` (silent success). +3. See `wiki/Hints.md` for the wiki-facing version of these rules. diff --git a/.github/instructions/ui.instructions.md b/.github/instructions/ui.instructions.md index 6dd45bb78..2fe94b1c0 100644 --- a/.github/instructions/ui.instructions.md +++ b/.github/instructions/ui.instructions.md @@ -5,10 +5,12 @@ applyTo: "javascript/**/*.js, html/**/*.html, html/**/*.css, html/**/*.js, exten --- # UI And Frontend Guidelines -- Preserve existing UI behavior and wiring between Gradio/Python endpoints and frontend handlers; do not change payload shapes without backend alignment. -- Follow existing project lint and style patterns; prefer consistency with nearby files over introducing new frameworks or architecture. -- Keep localization-friendly UI text changes synchronized with locale resources in `html/locale_*.json` when user-facing strings are added or changed. -- Avoid bundling unrelated visual refactors with functional fixes; keep UI PRs scoped and reviewable. -- For extension UI work, respect each extension's boundaries and avoid cross-extension coupling. -- Validate JavaScript changes with `pnpm eslint`; for modern UI extension changes also run `pnpm eslint-ui`. -- Maintain mobile compatibility when touching layout or interaction behavior. +Apply these rules in priority order: + +1. Preserve the current event-handling logic and data flow between Gradio/Python endpoints and frontend handlers; do not change payload shapes without backend alignment. +2. Follow existing project lint and style patterns; prefer consistency with nearby files over introducing new frameworks or architecture. +3. Keep localization-friendly UI text changes synchronized with locale resources in `html/locale_*.json` when user-facing strings are added or changed. +4. Avoid bundling unrelated visual refactors with functional fixes; keep UI PRs scoped and reviewable. +5. For extension UI work, respect each extension's boundaries and avoid cross-extension coupling. +6. Validate JavaScript changes with `pnpm eslint`; for modern UI extension changes also run `pnpm eslint-ui`. +7. Maintain mobile compatibility when touching layout or interaction behavior. diff --git a/.github/skills/analyze-model/SKILL.md b/.github/skills/analyze-model/SKILL.md index 606e04537..88f121b58 100644 --- a/.github/skills/analyze-model/SKILL.md +++ b/.github/skills/analyze-model/SKILL.md @@ -1,7 +1,7 @@ --- name: analyze-model description: "Analyze an external model URL (typically Hugging Face) to determine implementation style and estimate SD.Next porting difficulty using the port-model workflow." -argument-hint: "Provide model URL and optional target scope: text2img, img2img, edit, video, or full integration" +argument-hint: "Provide model URL and, if applicable, specify target scope: text2img, img2img, edit, video, or full integration" --- # Analyze External Model For SD.Next Porting @@ -47,83 +47,15 @@ Classify into one of these (or closest fit): ## Procedure -### 0. Handle Gated Models +Process in this order: -If the model repository returns HTTP 403 (Forbidden) or requires acceptance of a gating agreement: - -1. Check `secrets.json` in the workspace root for a `huggingface_token` field -2. If token exists, retry accessing the model using that token for authentication -3. If token does not exist, is invalid, or access still denied, **abort the analysis** and report: - - Model name and URL - - Access requirement (waiting list, gated, license agreement) - - Instructions for user to authenticate or request access - - Skip further analysis - -### 1. Inspect Model Repository Artifacts - -From the provided URL/repo, collect: - -- model card details -- files such as model_index.json, config.json, scheduler config, tokenizer files -- presence of Diffusers-style folder layout -- references to custom Python modules or remote code requirements - -### 2. Determine Runtime Stack - -Identify whether model usage is: - -- standard Diffusers pipeline call -- custom Diffusers pipeline class with trust_remote_code -- pure custom inference script or framework -- node-based integration in ComfyUI or another host - -### 3. Cross-Check Integration Surface - -Determine required SD.Next touchpoints if ported: - -- loader file in pipelines/model_name.py -- detect and dispatch updates in modules/sd_detect.py and modules/sd_models.py -- model type mapping in modules/modeldata.py -- optional custom pipeline package under pipelines/model/ -- reference catalog updates and preview asset requirements - -### 4. Estimate Porting Difficulty - -Use this scale: - -- Low: mostly loader wiring to existing upstream Diffusers pipeline -- Medium: custom Diffusers classes or limited checkpoint/config adaptation -- High: full custom architecture, major prompt/sampler/output differences, or sparse docs -- Very High: no usable Diffusers path plus major runtime assumptions mismatch - -Break down difficulty by: - -- loader complexity -- pipeline/API contract complexity -- scheduler/sampler compatibility -- prompt encoding complexity -- checkpoint conversion/remapping complexity -- validation and testing burden - -### 5. Identify Risks - -Call out concrete risks: - -- missing or incompatible scheduler config -- unclear output domain (latent vs pixel) -- custom text encoder or processor constraints -- nonstandard checkpoint format -- dependency on external runtime features unavailable in SD.Next - -### 6. Recommend Porting Path - -Map recommendation to port-model workflow: - -- Upstream Diffusers reuse path -- Custom Diffusers pipeline package path -- Raw checkpoint plus remap path - -Provide a concise first-step plan with smallest viable integration milestone. +1. Handle gated models first: if access returns HTTP 403 or requires gated approval, check `secrets.json` for `huggingface_token` and retry with auth. If access still fails, abort analysis and report model URL, access requirement, and required user action. +2. Inspect repository artifacts: collect model card details, key config files (for example `model_index.json`, `config.json`, scheduler/tokenizer files), Diffusers-style layout signals, and any custom module or remote code requirements. +3. Determine runtime stack: classify usage as standard Diffusers, custom Diffusers with `trust_remote_code`, fully custom inference framework, or node-based host integration (for example ComfyUI). +4. Cross-check SD.Next integration surface: identify needed touchpoints in `pipelines/model_name.py`, `modules/sd_detect.py`, `modules/sd_models.py`, `modules/modeldata.py`, optional custom packages under `pipelines/model/`, and reference/preview catalog updates. +5. Estimate difficulty with this scale: Low (mostly loader wiring), Medium (custom Diffusers or limited adaptation), High (full custom architecture or major behavior differences), Very High (no practical Diffusers path plus runtime mismatch). Break down by loader, API contract, scheduler/sampler, prompt encoding, checkpoint remap, and validation burden. +6. Identify concrete risks: scheduler incompatibility, unclear output domain, custom text encoder constraints, nonstandard checkpoint format, or external runtime dependencies not available in SD.Next. +7. Recommend a port-model path: upstream Diffusers reuse, custom Diffusers pipeline package, or raw checkpoint plus remap; include the smallest viable first implementation milestone. ## Reporting Format diff --git a/.github/skills/check-api/SKILL.md b/.github/skills/check-api/SKILL.md index 1c0a75daf..51b09c5bb 100644 --- a/.github/skills/check-api/SKILL.md +++ b/.github/skills/check-api/SKILL.md @@ -32,14 +32,20 @@ This file is the route registration hub and must be treated as the source of tru ## Audit Goals -For every endpoint, verify: +For every endpoint, verify in this order: -1. Route method and path are valid and unique after subpath handling. -2. Handler call signature is compatible with the route declaration. -3. Declared `response_model` is coherent with returned payload shape. -4. Request body or query params implied by handler type hints are consistent with expected client usage. -5. Authentication behavior is intentional (`auth=True` default in `add_api_route`). -6. OpenAPI schema exposure is correct (including trailing-slash duplicate suppression). +1. Route validation: + - Route method and path are valid and unique after subpath handling. +2. Handler validation: + - Handler call signature is compatible with the route declaration. +3. Request signature validation: + - Request body or query params implied by handler type hints are consistent with expected client usage. +4. Response signature validation: + - Declared `response_model` is coherent with returned payload shape. +5. Auth validation: + - Authentication behavior is intentional (`auth=True` default in `add_api_route`). +6. OpenAPI validation: + - OpenAPI schema exposure is correct (including trailing-slash duplicate suppression). ## Procedure @@ -91,7 +97,7 @@ If feasible in the current environment: - Generate OpenAPI schema and spot-check key endpoints. - Confirm trailing-slash duplicate suppression behavior remains correct. -If runtime schema checks are not feasible, explicitly state that and rely on static validation. +If runtime schema checks are not feasible, include a note in the findings section stating that runtime checks were skipped, then rely on static validation. ## Reporting Format diff --git a/.github/skills/check-models/SKILL.md b/.github/skills/check-models/SKILL.md index 8fa9d0fde..90d6ee74f 100644 --- a/.github/skills/check-models/SKILL.md +++ b/.github/skills/check-models/SKILL.md @@ -1,7 +1,7 @@ --- name: check-models description: "Audit SD.Next model integrations end-to-end: loaders, detect/routing, reference catalogs, and pipeline API contracts." -argument-hint: "Optionally focus on a model family, repo id, or a subset: loader, detect-routing, references, pipeline-contracts" +argument-hint: "Optionally focus on a specific model family, repo id, or one or more audit categories: loader, detect-routing, references, pipeline-contracts" --- # Check Model Integrations End-To-End @@ -17,7 +17,7 @@ Run a consolidated model-integration audit that combines loader checks, detect/r ## Combined Scope -This skill combines four audit surfaces: +This skill combines four audit surfaces. Run them in this order unless user scope limits categories: 1. Loader consistency (`check-loaders` equivalent) 2. Detect/routing parity (`check-detect-routing` equivalent) diff --git a/.github/skills/check-processing/SKILL.md b/.github/skills/check-processing/SKILL.md index 2b81a4558..7a7962f50 100644 --- a/.github/skills/check-processing/SKILL.md +++ b/.github/skills/check-processing/SKILL.md @@ -1,12 +1,12 @@ --- name: check-processing -description: "Validate txt2img/img2img/control/caption processing workflows from UI submit bindings to backend processing execution and confirm parameter/type/init correctness." +description: "Run a phased processing-workflow audit from UI submit bindings to backend execution: map workflow paths first, then validate parameter, type, and initialization correctness." argument-hint: "Optionally focus on txt2img, img2img, control, caption, or process-only and include changed files" --- # Check Processing Workflow Contracts -Trace generation workflows from UI definitions and submit bindings to backend execution, then validate that parameters are passed, typed, and initialized correctly. +Perform a detailed step-by-step trace of generation workflows from UI definitions and submit bindings to backend execution, then validate that parameters are passed, typed, and initialized correctly. ## When To Use @@ -17,12 +17,13 @@ Trace generation workflows from UI definitions and submit bindings to backend ex ## Required Workflow Coverage -Start from UI definitions and follow each workflow to final implementation: +Run workflow coverage in this order to keep checks focused and complete: 1. `txt2img`: `modules/ui_txt2img.py` -> `modules/txt2img.py` -> `modules/processing.py:process_images` -> `modules/processing_diffusers.py:process_diffusers` 2. `img2img`: `modules/ui_img2img.py` -> `modules/img2img.py` -> `modules/processing.py:process_images` -> `modules/processing_diffusers.py:process_diffusers` 3. `control/process`: `modules/ui_control.py` -> `modules/control/run.py` (and related control processing entrypoints) -> `modules/processing.py:process_images` -> `modules/processing_diffusers.py:process_diffusers` 4. `caption/process`: `modules/ui_caption.py` -> caption handler module(s) -> `modules/processing.py:process_images` and/or postprocess/caption execution module(s), depending on selected caption backend +5. `video`: `modules/ui_video.py` -> `modules/video_models/video_run -> `modules/processing.py:process_images` and/or postprocess/video execution module(s), depending on implementation Also validate script hooks when present: @@ -60,7 +61,7 @@ For each workflow (`txt2img`, `img2img`, `control`, `caption`): - Resolve wrappers (`call_queue.wrap_gradio_gpu_call`, queued wrappers) to actual function signatures. - Follow function flow through processing class construction and execution (`processing.process_images`, then `process_diffusers` when applicable). -Produce a normalized mapping table per workflow: +Produce a table per workflow with standardized columns and consistent formatting: - UI input component name - UI expected output type diff --git a/.github/skills/check-schedulers/SKILL.md b/.github/skills/check-schedulers/SKILL.md index db1b41260..2beb9b1e0 100644 --- a/.github/skills/check-schedulers/SKILL.md +++ b/.github/skills/check-schedulers/SKILL.md @@ -1,6 +1,6 @@ --- name: check-schedulers -description: "Audit scheduler registrations starting from modules/sd_samplers_diffusers.py and verify class loadability, config validity against scheduler capabilities, and SamplerData correctness." +description: "Run a phased scheduler audit from modules/sd_samplers_diffusers.py: verify class loadability first, then config validity against scheduler capabilities, then SamplerData correctness." argument-hint: "Optionally focus on a scheduler subset, such as flow-matching, res4lyf, or parallel schedulers" --- @@ -41,7 +41,7 @@ Treat this as a two-level check: Notes: - For non-`torch.nn.Module` schedulers, "compiled" means the scheduler integration path is executable in runtime checks (not necessarily `torch.compile`). -- If the environment cannot run compile checks, report this explicitly and still complete static validation. +- If the environment cannot run compile checks, explicitly state this in the findings summary and proceed with static validation only. ## Procedure diff --git a/.github/skills/check-scripts/SKILL.md b/.github/skills/check-scripts/SKILL.md index 732bb8b4f..0c6354cb6 100644 --- a/.github/skills/check-scripts/SKILL.md +++ b/.github/skills/check-scripts/SKILL.md @@ -1,6 +1,6 @@ --- name: check-scripts -description: "Audit scripts/*.py and verify Script override contracts (init/title/show) plus ui() output compatibility with run() or process() parameters." +description: "Run a phased scripts audit in scripts/*.py: validate Script overrides (init/title/show) first, then verify ui() output compatibility with run() or process() parameters." argument-hint: "Optionally focus on a subset of scripts or only run-vs-ui or process-vs-ui checks" --- @@ -126,4 +126,4 @@ A full pass requires all of the following across audited `scripts/*.py` classes: - `ui()` output contracts are compatible with `run()` or `process()` args - no blocking arity/signature mismatch remains -If a class uses highly dynamic argument routing that cannot be proven statically, mark as conditional pass with explicit runtime validation recommendation. +If a class uses runtime-determined argument mapping or dynamic method dispatch that cannot be proven statically, mark as conditional pass with explicit runtime validation recommendation. diff --git a/.github/skills/debug-model/SKILL.md b/.github/skills/debug-model/SKILL.md index 709c05392..3afabb541 100644 --- a/.github/skills/debug-model/SKILL.md +++ b/.github/skills/debug-model/SKILL.md @@ -21,15 +21,11 @@ Read the error, identify which integration layer is failing, isolate the smalles Always debug from the outside in. -1. Detection and routing -2. Loader arguments and component selection -3. Checkpoint path and artifact layout -4. Weight loading and key mapping -5. Prompt encoding -6. Sampling forward path -7. Output postprocessing and SD.Next task integration +1. Integration entry checks: detection and routing. +2. Load path checks: loader arguments and component selection, checkpoint path and artifact layout, then weight loading and key mapping. +3. Runtime path checks: prompt encoding, sampling forward path, and output postprocessing plus SD.Next task integration. -Do not start by rewriting the architecture if the failure is likely in detection, loader wiring, or output handling. +Do not start by rewriting the architecture when the failure appears in detection, loader wiring, or output handling. Only consider architecture rewrites after these layers are validated and the root cause is confirmed to be architectural. ## Files To Check First diff --git a/.github/skills/diffusers-code/SKILL.md b/.github/skills/diffusers-code/SKILL.md index 41b93c6e2..062eb1f3d 100644 --- a/.github/skills/diffusers-code/SKILL.md +++ b/.github/skills/diffusers-code/SKILL.md @@ -20,7 +20,7 @@ Use this skill to implement, edit, review, and prepare pull-request-ready change 1. Keep behavior explicit, minimal, and inference-focused. 2. Match existing diffusers architecture and code patterns. -3. Preserve numerical behavior unless a behavior change is explicitly required. +3. Preserve numerical behavior unless a behavior change is explicitly documented in the task requirements. 4. Produce change sets that are clean, reviewable, and PR-ready. ## Hard Rules @@ -35,6 +35,13 @@ Use this skill to implement, edit, review, and prepare pull-request-ready change ## Code Structure Rules +Apply these grouped checks in priority order: + +1. Model-level structure and forward-path clarity. +2. Attention and processor integration consistency. +3. Pipeline runtime behavior and inference API expectations. +4. Scheduler config and mixin conformance. + ### Models - Use ModelMixin patterns and register constructor args with register_to_config. diff --git a/.github/skills/fix-lint/SKILL.md b/.github/skills/fix-lint/SKILL.md index 2bfb725de..af5113500 100644 --- a/.github/skills/fix-lint/SKILL.md +++ b/.github/skills/fix-lint/SKILL.md @@ -1,6 +1,6 @@ --- name: fix-lint -description: "Run SD.Next lint workflow tools in order and fix issues as needed, while ignoring lint findings explicitly marked with TODO." +description: "Run the SD.Next lint workflow in phased order: execute tools in sequence first, then apply minimal fixes, while ignoring findings explicitly marked by TODO comments in code." argument-hint: "Optionally focus on a subset of tools or files, otherwise run full workflow" --- @@ -43,7 +43,7 @@ Note that `pylint` can run for considerable time, so run with no timeouts. ## Fix Policy - Fix issues reported by each tool before moving on. -- Ignore lint issues explicitly marked with `TODO`. +- Ignore lint issues explicitly marked by `TODO` comments in code. - Do not suppress errors globally just to pass checks. - Keep fixes minimal and targeted to reported findings. - Preserve existing project conventions and avoid unrelated refactors. diff --git a/.github/skills/github-features/SKILL.md b/.github/skills/github-features/SKILL.md index 278506030..b7ef0eb9e 100644 --- a/.github/skills/github-features/SKILL.md +++ b/.github/skills/github-features/SKILL.md @@ -1,6 +1,6 @@ --- name: github-features -description: "Read SD.Next GitHub issues with [Feature] in the title and generate a markdown report with short summary, status, and suggested next steps per issue." +description: "Read SD.Next GitHub issues with [Feature] in the title and produce a phased markdown report: short summary first, then status, then suggested next steps per issue." argument-hint: "Optionally specify state (open/closed/all), max issues, and whether to include labels/assignees" --- @@ -89,7 +89,7 @@ If there are many issues, keep summaries short and prioritize clarity. - Keep each issue summary concise and actionable. - Do not invent facts not present in issue data. -- If issue body is sparse, state assumptions explicitly. +- If issue body is sparse, explicitly list assumptions about issue intent or context in 1-2 sentences. - If no matching issues are found, output a clear "no matches" report. ## Pass Criteria diff --git a/.github/skills/github-issues/SKILL.md b/.github/skills/github-issues/SKILL.md index ad8bd88f8..cb1bc99a9 100644 --- a/.github/skills/github-issues/SKILL.md +++ b/.github/skills/github-issues/SKILL.md @@ -1,12 +1,12 @@ --- name: github-issues -description: "Read SD.Next GitHub issues with [Issue] in the title and generate a markdown report with short summary, status, and suggested next steps per issue." +description: "Read SD.Next GitHub issues with the literal string [Issue] in the title and produce a phased markdown report: short summary first, then status, then suggested next steps per issue." argument-hint: "Optionally specify state (open/closed/all), max issues, and whether to include labels/assignees" --- # Summarize SD.Next [Issues] GitHub Issues -Fetch issues from the SD.Next GitHub repository that contain `[Issue]` in the title, then produce a concise markdown report with one entry per issue. +Fetch issues from the SD.Next GitHub repository that contain the literal string `[Issue]` in the title, then produce a concise markdown report with one entry per issue. ## When To Use @@ -87,6 +87,7 @@ If there are many issues, keep summaries short and prioritize clarity. ## Reporting Rules +- Prioritize outputs in this order: accuracy first, then concise summaries and actions, then markdown presentation. - Keep each issue summary concise and actionable. - Do not invent facts not present in issue data. - If issue body is sparse, state assumptions explicitly. diff --git a/.github/skills/port-model/SKILL.md b/.github/skills/port-model/SKILL.md index fe1f0a2ac..4cadbee61 100644 --- a/.github/skills/port-model/SKILL.md +++ b/.github/skills/port-model/SKILL.md @@ -1,12 +1,12 @@ --- name: port-model -description: "Port or add a model to SD.Next using existing Diffusers and custom pipeline patterns. Use when implementing a new model loader, custom pipeline, checkpoint conversion path, or SD.Next model-type integration." +description: "Port or add a model to SD.Next using a phased integration flow: select the least-new-code path that follows SD.Next conventions, then implement loader and routing updates, then validate." argument-hint: "Describe the source model, target task, checkpoint format, and whether the model already has a Diffusers pipeline" --- # Port Model To SD.Next And Diffusers -Read the task, identify the model architecture and artifact layout, choose the narrowest integration path that matches existing SD.Next patterns, implement the loader and pipeline wiring, and validate the result. +Read the task, identify the model architecture and artifact layout, choose the integration path that requires the least amount of new code while adhering to SD.Next patterns, implement the loader and pipeline wiring, and validate the result. ## When To Use @@ -88,6 +88,12 @@ Useful examples by pattern: ## Integration Decision Tree +Use this quick order before diving into detailed path requirements: + +1. If an upstream Diffusers pipeline already covers the model, choose path 1. +2. If upstream support is insufficient but the model can be expressed as a Diffusers-style custom package, choose path 2. +3. If artifacts are raw checkpoints or single-file weights without a usable Diffusers layout, choose path 3. + ### 1. Upstream Diffusers Support Exists Use this path when the model already has a usable Diffusers pipeline and component classes. diff --git a/.github/skills/port-pipeline/SKILL.md b/.github/skills/port-pipeline/SKILL.md index 2b18599c9..95e3b12c1 100644 --- a/.github/skills/port-pipeline/SKILL.md +++ b/.github/skills/port-pipeline/SKILL.md @@ -1,6 +1,6 @@ --- name: port-pipeline -description: "Port custom model pipeline implementations to Diffusers. Use when migrating custom or non-Diffusers pipeline code into SD.Next repo-local pipeline files such as pipelines/model_.py or pipelines//pipeline.py while preserving behavior, avoiding new dependencies, and keeping device/attention handling configurable." +description: "Port custom model pipeline implementations to Diffusers using phased priorities: preserve behavior first, avoid new dependencies second, and keep device/attention handling configurable throughout. Use when migrating custom or non-Diffusers pipeline code into SD.Next repo-local pipeline files such as pipelines/model_.py or pipelines//pipeline.py." argument-hint: "Provide source pipeline path, target SD.Next destination path, and target pipeline class name" --- @@ -26,20 +26,29 @@ Before implementation, confirm these required inputs with the user: 3. Target pipeline class name If any of the above are missing or ambiguous, stop and ask concise clarification questions before writing code. +If the user input is invalid (for example, nonexistent path, non-Python source file, or invalid class name), report the specific validation error and request corrected input before writing code. ## Constraints +Priority 1 - behavior constraints: + +- Preserve externally visible behavior of the source pipeline unless the user asks for intentional changes + +Priority 2 - dependency constraints: + - Do not add new dependencies + +Priority 3 - runtime configurability constraints: + - Do not hard-code device type (`cpu`, `cuda`, `mps`, etc.) - Do not hard-code attention type or backend assumptions -- Preserve externally visible behavior of the source pipeline unless the user asks for intentional changes ## Workflow 1. Collect Inputs - Ask for source path, destination path, and target pipeline name. - Confirm destination is an SD.Next repo-local pipeline location, not an upstream Diffusers repository path. -- Confirm runtime assumptions and expected task type (text-to-image, image-to-image, inpaint, etc.). +- Confirm runtime assumptions, including device configuration, memory constraints, and expected task type (text-to-image, image-to-image, inpaint, etc.). 2. Analyze Source Pipeline - Inspect model loading, prompt processing, denoising or sampling loop, scheduler interactions, and output post-processing. diff --git a/.github/skills/reference-catalog/SKILL.md b/.github/skills/reference-catalog/SKILL.md index 476fc7559..00bdf76ab 100644 --- a/.github/skills/reference-catalog/SKILL.md +++ b/.github/skills/reference-catalog/SKILL.md @@ -6,7 +6,7 @@ argument-hint: "Describe which catalog files to audit (or use all), whether to o # Reference Catalog Maintenance -Use this skill to audit and update SD.Next model reference catalogs with minimal, safe, and deterministic edits. +Use this skill to audit and update SD.Next model reference catalogs using a phased approach: validate structure first, then resolve duplicates/conflicts, then apply minimal deterministic edits. ## When To Use @@ -26,10 +26,19 @@ Use this skill to audit and update SD.Next model reference catalogs with minimal ## Core Rules -- Do not move entries between categories unless explicitly requested or strongly evidenced. +Priority 1 - data safety and category stability: + +- Verify category placement across catalogs and report conflicts first. +- Move entries between categories only when explicitly requested, or when placement is supported by at least two independent metadata sources. - Keep changes targeted to only affected records. + +Priority 2 - schema and formatting consistency: + - Preserve existing field names and conventions used by neighboring entries. - Prefer deterministic normalization (stable key order, consistent value style). + +Priority 3 - assets and size backfill: + - Do not overwrite real thumbnails with placeholders. - For `size` backfill, use `cli/hf-info.py` as the primary source of truth. @@ -47,6 +56,7 @@ Use this skill to audit and update SD.Next model reference catalogs with minimal 3. Cross-catalog consistency - Detect likely duplicates across `reference*.json` files. - Flag conflicting metadata for the same model key/name. +- Resolve duplicates by keeping the most complete record in the correct category, then merge missing non-conflicting metadata from duplicate records. - Report category conflicts; only auto-fix when rules are explicit. 4. Thumbnail alignment diff --git a/.github/skills/todo/SKILL.md b/.github/skills/todo/SKILL.md index 975136ae0..a1ac36a2c 100644 --- a/.github/skills/todo/SKILL.md +++ b/.github/skills/todo/SKILL.md @@ -6,7 +6,7 @@ argument-hint: "Optionally focus on specific folders or TODO categories, otherwi # Audit TODO Markers And Propose Next Steps -Search the repository for TODO markers, collect each actionable item, and produce a markdown report with recommended next steps. +Run this workflow in order: (1) search the repository for TODO markers, (2) deduplicate results, (3) categorize TODOs, (4) propose actionable next steps, and (5) produce a markdown report. ## When To Use @@ -38,7 +38,7 @@ Look for common TODO variants such as: - `/* TODO */` - inline TODO notes in comments or docstrings -Ignore generated/vendor output when clearly not user-maintained. +Ignore files in common generated or vendor directories (for example `node_modules`, `dist`, `build`, `.venv`, `venv`) unless the user explicitly requests including them. ## What To Capture diff --git a/.github/skills/update-docs/SKILL.md b/.github/skills/update-docs/SKILL.md index ca0c728be..dfac7ec5d 100644 --- a/.github/skills/update-docs/SKILL.md +++ b/.github/skills/update-docs/SKILL.md @@ -62,17 +62,27 @@ Use the repo-local validation script before and after doc edits when possible: - `test/check-docs wiki/File.md` to validate one or more specific files - `test/check-docs --fix wiki/File.md` only when a safe markdownlint auto-fix is appropriate -### 1. Confirm Target And Depth +### 1. Confirm Target Extract from user prompt: - target markdown file(s) in `wiki/` -- desired depth: syntax-only, readability, or full pass - constraints (tone, audience, preserve wording, max rewrite level) If targets are missing, ask for paths before editing. -### 2. Read And Diagnose +### 2. Confirm Depth + +Extract from user prompt: + +- desired depth mode: + - syntax-only: fix markdown syntax/rendering issues only; do not rewrite wording or structure beyond what syntax requires + - readability: include syntax fixes plus clarity and scanability edits without broad restructuring + - full pass: include syntax, readability, structure normalization, terminology consistency, and broader doc cleanup + +If depth is missing, default to readability and state that assumption. + +### 3. Read And Diagnose For each target file: @@ -81,7 +91,7 @@ For each target file: - Identify readability pain points (dense blocks, weak headings, mixed terminology) - Note risky sections where edits may alter meaning -### 3. Normalize Heading Hierarchy +### 4. Normalize Heading Hierarchy Apply heading structure rules before deep rewrites: @@ -90,7 +100,7 @@ Apply heading structure rules before deep rewrites: - ensure sibling sections use consistent levels - rename headings only when it improves clarity without changing meaning -### 4. Apply Syntax Fixes First +### 5. Apply Syntax Fixes First Fix rendering/correctness issues first, such as: @@ -101,7 +111,7 @@ Fix rendering/correctness issues first, such as: - inconsistent table delimiter rows - accidental HTML/markdown mixing that breaks rendering -### 5. Apply Readability Improvements +### 6. Apply Readability Improvements Make editorial improvements while preserving meaning: @@ -117,7 +127,7 @@ Apply tone constraints during edits: - approachable wording for normal users - no unexplained technical babble -### 6. Run Link Integrity Pass +### 7. Run Link Integrity Pass Check and fix obvious link issues: @@ -127,7 +137,7 @@ Check and fix obvious link issues: If link targets cannot be verified from repo context, keep the original target and flag it in the report. -### 7. Add Code Block Language Tags +### 8. Add Code Block Language Tags For fenced code blocks: @@ -135,7 +145,7 @@ For fenced code blocks: - correct clearly wrong tags - leave tag blank only when language cannot be inferred safely -### 8. Run Completion Checks +### 9. Run Completion Checks Validate each edited file against this checklist: @@ -147,7 +157,7 @@ Validate each edited file against this checklist: - no factual changes introduced - tone is concise, technical, and approachable -### 9. Report Results +### 10. Report Results Return: diff --git a/TODO.md b/TODO.md index dbc4230e5..fbd1e442b 100644 --- a/TODO.md +++ b/TODO.md @@ -9,6 +9,7 @@ - Reimplement `llama` remover for Kanvas, @vladmandic - Implement [pruna](https://github.com/PrunaAI/pruna), @vladmandic - Change params to default, @vladmandic +- Use `caption` to analyze image consistency/quality, @vladmandic - Detailer postprocessing, @CalamitousFelicitousness - Cloud providers, @CalamitousFelicitousness