mirror of
https://github.com/vladmandic/automatic
synced 2026-09-19 01:04:32 +02:00
add hidream-o1
Co-authored-by: Copilot <copilot@github.com> Signed-off-by: Vladimir Mandic <mandic00@live.com>
This commit is contained in:
@@ -78,6 +78,10 @@ Use these repo-local skills for recurring SD.Next model integration work:
|
||||
File: `.github/skills/port-model/SKILL.md`
|
||||
Use when adding a new model family, porting a standalone script into a Diffusers pipeline, or wiring an upstream Diffusers model into SD.Next.
|
||||
|
||||
- `port-pipeline`
|
||||
File: `.github/skills/port-pipeline/SKILL.md`
|
||||
Use when porting a custom model pipeline implementation to a Diffusers pipeline class with behavior parity and no hard-coded device or attention assumptions.
|
||||
|
||||
- `debug-model`
|
||||
File: `.github/skills/debug-model/SKILL.md`
|
||||
Use when a new or existing SD.Next/Diffusers model integration fails during detection, loading, prompt encoding, sampling, or output handling.
|
||||
|
||||
@@ -8,6 +8,10 @@ This folder contains repo-local Copilot skills for recurring SD.Next tasks.
|
||||
File: `port-model/SKILL.md`
|
||||
Use when adding or porting a model family into SD.Next and Diffusers.
|
||||
|
||||
- `port-pipeline`
|
||||
File: `port-pipeline/SKILL.md`
|
||||
Use when porting a custom pipeline implementation into a Diffusers pipeline class while preserving behavior and avoiding hard-coded runtime assumptions.
|
||||
|
||||
- `debug-model`
|
||||
File: `debug-model/SKILL.md`
|
||||
Use when a new or existing SD.Next/Diffusers model integration fails during detect, load, prompt encode, sample, or output handling.
|
||||
|
||||
@@ -0,0 +1,102 @@
|
||||
---
|
||||
name: port-pipeline
|
||||
description: "Port custom model pipeline implementations to Diffusers. Use when migrating custom or non-Diffusers pipeline code into SD.Next repo-local pipeline files such as pipelines/model_<name>.py or pipelines/<model>/pipeline.py while preserving behavior, avoiding new dependencies, and keeping device/attention handling configurable."
|
||||
argument-hint: "Provide source pipeline path, target SD.Next destination path, and target pipeline class name"
|
||||
---
|
||||
|
||||
# Port Custom Pipeline To Diffusers
|
||||
|
||||
Port an existing custom model pipeline implementation into a Diffusers-compatible pipeline class with behavior parity and SD.Next-friendly conventions.
|
||||
This skill targets SD.Next repo-local pipeline ports only.
|
||||
|
||||
## When To Use
|
||||
|
||||
- A user has a custom pipeline implementation and wants it ported to Diffusers
|
||||
- Existing model code is runnable but not structured as a Diffusers pipeline
|
||||
- The destination is SD.Next pipeline code under `pipelines/model_*.py` or `pipelines/<model>/`
|
||||
- The task requires preserving generation behavior without introducing new dependencies
|
||||
- The task requires removing hard-coded runtime assumptions (device or attention backend)
|
||||
|
||||
## Mandatory Clarification Gate
|
||||
|
||||
Before implementation, confirm these required inputs with the user:
|
||||
|
||||
1. Path to the source custom pipeline implementation
|
||||
2. Destination path in this SD.Next repository (typically under `pipelines/`)
|
||||
3. Target pipeline class name
|
||||
|
||||
If any of the above are missing or ambiguous, stop and ask concise clarification questions before writing code.
|
||||
|
||||
## Constraints
|
||||
|
||||
- Do not add new dependencies
|
||||
- Do not hard-code device type (`cpu`, `cuda`, `mps`, etc.)
|
||||
- Do not hard-code attention type or backend assumptions
|
||||
- Preserve externally visible behavior of the source pipeline unless the user asks for intentional changes
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Collect Inputs
|
||||
- Ask for source path, destination path, and target pipeline name.
|
||||
- Confirm destination is an SD.Next repo-local pipeline location, not an upstream Diffusers repository path.
|
||||
- Confirm runtime assumptions and expected task type (text-to-image, image-to-image, inpaint, etc.).
|
||||
|
||||
2. Analyze Source Pipeline
|
||||
- Inspect model loading, prompt processing, denoising or sampling loop, scheduler interactions, and output post-processing.
|
||||
- Identify all components that must be ported: models, tokenizers or processors, schedulers, adapters, preprocessors, postprocessors, callbacks, and output dataclasses.
|
||||
- Note any hidden global state, side effects, or implicit defaults that must become explicit parameters.
|
||||
|
||||
3. Map To Diffusers Interfaces
|
||||
- Choose the most appropriate Diffusers base class and output type.
|
||||
- Define `__init__`, module registration, `from_pretrained` and `__call__` signatures aligned with existing Diffusers patterns.
|
||||
- Keep parameter names and behavior as close as possible to upstream conventions.
|
||||
- Identify any custom classes needed beyond the pipeline itself: transformer blocks, attention processors, custom schedulers, or output types. Plan a separate module file for each.
|
||||
|
||||
4. Implement Supporting Classes
|
||||
- If the pipeline requires custom model classes (e.g., a custom transformer block, attention module, or other model component), implement each in a **separate module** located in the **same directory** as the main pipeline file (e.g., `pipelines/<model>/transformer.py`, `pipelines/<model>/scheduler.py`).
|
||||
- If the pipeline requires a custom scheduler class, implement it in its own module (e.g., `pipelines/<model>/scheduler_<name>.py`) following Diffusers scheduler conventions (`step`, `add_noise`, `scale_model_input`, etc.).
|
||||
- Each supporting class module must be self-contained: no circular imports, no hidden global state, and no hard-coded device or attention assumptions.
|
||||
- Import supporting classes into the main pipeline module from their respective sibling modules.
|
||||
|
||||
5. Implement Pipeline Class
|
||||
- Create the destination pipeline classes at the user-provided path.
|
||||
- Port logic in small, testable sections: initialization, input validation, prompt encoding, latent preparation, denoising loop, decoding, and output packaging.
|
||||
- Replace hard-coded device and attention logic with runtime-configurable behavior.
|
||||
- Keep imports limited to existing project and Diffusers dependencies.
|
||||
|
||||
6. Lint And Fix
|
||||
- Activate the project venv: `source venv/bin/activate`
|
||||
- Run `ruff` on all newly written files: `pnpm ruff` (or `ruff check <file> --fix` for targeted runs).
|
||||
- Run `pylint` on all newly written files: `pnpm pylint` (or `pylint <file>` for targeted runs).
|
||||
- Fix every reported error or warning that is not explicitly marked with a `TODO` suppression comment in the source.
|
||||
- Re-run both linters after fixes to confirm a clean result before proceeding.
|
||||
|
||||
7. Validate Behavior Parity
|
||||
- Compare source and ported implementations for input-output shape handling, dtype flow, scheduler step ordering, and guidance behavior.
|
||||
- Run focused checks or smoke tests if available in the workspace.
|
||||
- Call out any known differences that were required for Diffusers compatibility.
|
||||
|
||||
8. Report Results
|
||||
- Summarize what was ported and where.
|
||||
- List any unresolved assumptions, risks, or TODOs.
|
||||
- Provide minimal follow-up steps for integration and testing.
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- Required inputs were collected before edits
|
||||
- No new dependency was introduced
|
||||
- No hard-coded device or attention backend remains
|
||||
- Core components from source pipeline were fully mapped
|
||||
- Pipeline class is in requested destination with requested name
|
||||
- Each custom supporting class (transformer, scheduler, etc.) is in its own sibling module
|
||||
- Supporting modules have no circular imports or hidden global state
|
||||
- `ruff` and `pylint` both pass cleanly on all newly written files (venv activated)
|
||||
- Main inference path behavior matches the source implementation
|
||||
|
||||
## Output Expectations
|
||||
|
||||
Final response should include:
|
||||
- Source path, destination path, and final pipeline class name
|
||||
- Brief parity summary of key components ported
|
||||
- Validation performed and any gaps
|
||||
- Explicit note of any assumptions requiring user confirmation
|
||||
Reference in New Issue
Block a user