From 6531fb87c2e383cb7640f808d9ef374e886f5b10510c1b8e0fc4c886cfde71e4 Mon Sep 17 00:00:00 2001 From: Kareem <2+imrayya@noreply.localhost> Date: Fri, 28 Aug 2026 10:31:06 +0200 Subject: [PATCH] doc: Added design doc --- doc/DESIGN-DOC.md | 300 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 300 insertions(+) create mode 100644 doc/DESIGN-DOC.md diff --git a/doc/DESIGN-DOC.md b/doc/DESIGN-DOC.md new file mode 100644 index 0000000..48d2a3a --- /dev/null +++ b/doc/DESIGN-DOC.md @@ -0,0 +1,300 @@ +# Model Manager — Design Doc v1.2 + +## 1. Design Document + +### 1.1 System Overview +The Model Manager is a Windows application with a web-based UI served by a local Python/FastAPI server. It manages large AI model files (5–80 GB) downloaded from Hugging Face. Content is organized as a hierarchy: **families contain models; models contain one or more repos; repos contain files (with quant/variant tags)**. Models are stored on one or more HDD (L3) roots and can be promoted to an SSD (L2) cache. Metadata includes per-file HF content hashes; families share files by hash-verified dedupe; update detection is per-file diff. + +### 1.2 Goals & Non-Goals +**Goals:** +- Download from Hugging Face (public/private with token), resumable across restarts +- **Hierarchy: family → model → repo → files; multiple repos and multiple quants per model; non-standard variant tags supported (user-editable)** +- Multiple HDD roots; promotion to SSD cache; HDD integrity validated before demotion +- Model types: text, image, embedding +- **Shared files at family or model scope (jinja at any level, mmproj typically model)** +- Draft/speed-up models (MTP, DFlash) attachable **same-repo (role) or separate-repo (component) — user choice** +- Per-file update diff (commit hash as fast pre-check); timestamps tracked +- Web UI (editable without recompile), SSE live progress, Logs page +- Graceful shutdown (5-min default wait, force-pause option); abort vs pause distinction +- Retry in seconds + manual "Retry Now"; single-worker disk-copy queue + +**Non-Goals (current version):** +- Automatic family/model detection (suggested from repo org/name, but user confirms) +- Automatic cache eviction; llama.cpp config generation (schema/roles ready; placeholder service) +- Automatic image-gen role detection; multi-user/LAN access +- Local deep-hash verification on every operation (explicit command only) + +### 1.3 Functional Requirements + +#### 1.3.1 Content Model & Storage Layout +**Hierarchy:** `family → model → repo → files`. + +- **Family** (e.g., "Qwen3.5"): groups related models. Required; auto-suggested from repo org at Add New. May contain many models (27B, 35B, 9B…). +- **Model** (e.g., "Qwen3.5-27B"): logical model of a type (`text`/`image`/`embedding`). May aggregate **multiple repos** (official + community quants); UI shows a merged per-model view of all variants across repos. +- **Repo**: one HF repository, attached to exactly one model, with a `component` tag used as its folder segment: `main`, `draft`, `vae`, `lora`, `other` (default `main`). +- **File**: one downloaded file, with `role`, `variant` (quant/variety tag), and `scope`. + +**Variant tags:** parsed from filenames (strip shard suffixes `-NNNNN-of-NNNNN`; match known patterns `Q\d+_…`, `IQ\d…`, `UD-…`; fallback = trailing name segments). **Heuristics will not catch every naming scheme (e.g., `APEX-Compact`, `I-Balanced`)** — the tag is always editable in the preview UI and in file details. Sharded GGUFs share one variant tag. + +**Draft/MTP/DFlash (user-selectable attachment):** +- *Under repo:* draft files inside the main model's repo → role `speedup`, stored with the repo. +- *Under model:* a separate HF repo → its own repo row with `component='draft'`, stored under `{model}/draft/`. +- Either way, the config maker (future) can resolve a draft target for `--model-draft`. + +**Shared-scope files:** each file has `scope`: +| scope | physical location | typical use | +|---|---|---| +| `repo` (default) | `{root}/{model_type}/{fam}/{component}/{model}/{repo}/…` | quants, VAE, CLIP, tokenizers, LoRA | +| `model` | `{root}/{model_type}/{fam}/_shared/{model}/…` | **mmproj (default)**, model-level jinja | +| `family` | `{root}/{model_type}/{fam}/_shared/…` | family-level jinja overrides | + +Files keep their source-repo provenance (FK to repo) even when stored at model/family scope. Defaults at download time: mmproj → `model`, everything else → `repo`; user can elevate jinja to model/family in the preview UI. + +**Folder structure:** `{storage_root}/{text|image|embed}/{family}/{component}/{model}/{repo_name}/{file}` +(plus `{storage_root}/{model_type}/{fam}/_shared/…` and `{storage_root}/_incoming/{repo_id}/` temp areas). +- Model type→folder map: text→`text`, image→`image`, embedding→`embed`. +- Monolithic image repos (diffusers: `unet/`, `vae/`, `text_encoder/` inside one repo) → `component='main'`, **internal repo structure preserved** per file (`rel_file_path`). +- Reserved folder names: `_shared`, `_incoming`; model/repo/family names sanitized (Windows-illegal chars, reserved device names, trailing dots/spaces, length caps). Repo folder = repo name; on collision append a short hash. + +**Storage roots:** multiple HDD roots, removable only when unreferenced. **L2 (SSD):** single cache root in settings; promote/demote operates on **repo folders** (promoting a "model" = queue promote for each of its repos). Demotion validates the HDD copy first (files present, sizes match DB). Shared-scope files are small and resolve from wherever they live; promoting them is a future option. + +**Path handling:** DB stores IDs + relative paths only; resolution joins root + segments and validates containment. Windows long paths enabled and documented; depth is bounded by the 6-level layout (TC-024). + +#### 1.3.2 Download Workflow +1. User pastes **full URL or `org/name` shorthand**; URL cruft stripped; a branch/revision in the URL is pinned and stored. Recent repos remembered in a dropdown. +2. Preview: `model_info(files_metadata=True)` → names, sizes, **LFS sha256 OIDs**. +3. **Entity resolution:** app suggests family (from org) and model (from name, suffix-stripped); user can pick existing family/model or create new. For an existing model, the repo attaches to it; files whose OID matches any family file are flagged "available via family" and skipped by default (small non-LFS files never deduped). +4. Files grouped by role and **variant**; user selects files, assigns roles/scopes/variant fixes, picks HDD root, optionally "Cache to SSD." +5. `families`/`models`/`repos` rows ensured (repo `status='downloading'`), `jobs` row created; download targets `{root}/_incoming/{repo_id}/` on the destination volume. +6. `snapshot_download(local_dir=, allow_patterns, revision, token)`. Direct-to-target volume; no `~/.cache` duplication (asserted at runtime); `hf_transfer` optional toggle. +7. Resumable; retry 5s→15s→45s→2m, cap 5m, `max_retries` default 5, manual "Retry Now." Retries exhausted → `paused`, partials kept, auto-resume offered on next launch. +8. Success: files moved into final locations per scope (temp+rename throughout; cross-volume dedupe → copy, same-volume → hardlink), OIDs/sizes recorded, `status='complete'`. +9. **Cancel** → `_incoming` deleted, repo `cancelled`. **Pause** → partials kept. Crash → startup sweep resumes. +10. "Cache to SSD" → copy job enqueued (§1.3.7); SSD copy also temp-then-rename. + +#### 1.3.3 Family Sharing & Scope Rules +- Dedupe identity = **LFS sha256 OID**, within a family, across models/repos. Matched files are materialized by hardlink (same volume) or copy (cross volume). +- **Shared files are immutable: never rewritten in place** — updates create new files and swap via rename, so hardlink partners are unaffected. +- Small non-LFS files (jinja, configs, tokenizers) are always re-downloaded, never deduped. +- User override: force re-download per file. + +#### 1.3.4 Update Check +Per repo (commit hash + per-file OIDs stored at download). Pre-check commit; if changed, per-file OID diff → added/changed/removed list → user selects → apply job re-downloads changed files to `_incoming`, swaps per file via rename (scope-aware), updates rows + revision. Model-level UI aggregates update state across its repos. + +#### 1.3.5 Web UI +- **Overview:** counts by type, disk usage per root, SSD usage, active jobs, pending updates. +- **Models:** grouped Family → Model → Repos; per-model merged variant list; per-repo cache status. +- **Add New:** URL/shorthand input, preview, family/model resolver, file selection with roles/scopes/variants, recent repos. +- **Settings:** roots, SSD cache root, token, concurrency, retries. +- **Logs:** live tail (SSE) + rotating file log. +- Static `web/` folder, vanilla JS, SSE (`/api/events`, `/api/logs/stream`). No recompile to edit. + +#### 1.3.6 Graceful Shutdown +Ctrl+C or UI quit → stop accepting jobs → wait up to **5 min** (configurable) for active tasks; UI shows "waiting for N tasks"; force-pause keeps partials for next launch; final DB checkpoint (WAL; writes are continuous anyway). + +#### 1.3.7 Copy Queue +Single worker serializes all disk-to-disk ops (promote, demote, update swaps, scope materialization). Network pool separate (default 2). Copies cancelable with try/finally temp cleanup. + +#### 1.3.8 Startup Recovery +Sweep: `repos.status='downloading'` ↔ `_incoming/{repo_id}/` reconciled (resume or clean); orphaned `_incoming` dirs and SSD `.tmp` folders deleted with log entries. + +### 1.4 Non-Functional Requirements +Performance (SSE progress, 2 network workers, 1 copy worker, optional `hf_transfer`); reliability (same-volume atomic renames, DB written at job start/end, WAL); security (loopback-only, `keyring` token, path validation, log redaction); Windows (long paths, short roots, name sanitization). + +### 1.5 Technology Stack +Python 3.11, FastAPI, Uvicorn, SQLAlchemy (SQLite/WAL), `huggingface_hub` (>= 0.23, pinned), `pydantic`, `keyring`, optional `hf_transfer`; static `web/` + vanilla JS; `ThreadPoolExecutor` pools; `filelock`; `shutil`/`robocopy`. + +### 1.6 Data Model + +```sql +CREATE TABLE families ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + name TEXT UNIQUE NOT NULL, -- sanitized; reserved names rejected + description TEXT, + created_at DATETIME +); + +CREATE TABLE models ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + family_id INTEGER NOT NULL, + name TEXT NOT NULL, -- "Qwen3.5-27B" (sanitized) + model_type TEXT NOT NULL CHECK(model_type IN ('text','image','embedding')), + description TEXT, + created_at DATETIME, + UNIQUE(family_id, name), + FOREIGN KEY(family_id) REFERENCES families(id) +); + +CREATE TABLE repos ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + model_id INTEGER NOT NULL, + hf_repo_id TEXT UNIQUE NOT NULL, -- "org/name"; one row per HF repo + component TEXT NOT NULL DEFAULT 'main' + CHECK(component IN ('main','draft','vae','lora','other')), -- folder {component} + revision TEXT, -- commit hash at download + status TEXT NOT NULL DEFAULT 'downloading' + CHECK(status IN ('downloading','complete','paused','error','cancelled')), + storage_location_id INTEGER NOT NULL, + rel_path TEXT NOT NULL, -- {model_type}/{fam}/{component}/{model}/{repo} + hf_last_modified DATETIME, + download_started DATETIME, + download_completed DATETIME, + is_cached INTEGER DEFAULT 0, + cache_path TEXT, -- relative to SSD root + UNIQUE(storage_location_id, rel_path), + FOREIGN KEY(model_id) REFERENCES models(id), + FOREIGN KEY(storage_location_id) REFERENCES storage_locations(id) +); +-- Model status is DERIVED from its repos (UI aggregate). Downloads/caching operate per repo. + +CREATE TABLE model_files ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + repo_id INTEGER NOT NULL, -- provenance (source repo) + rel_file_path TEXT NOT NULL, -- path within its scope folder + role TEXT NOT NULL DEFAULT 'other' CHECK(role IN ( + 'main','mmproj','speedup','jinja','vae','clip','text_encoder', + 'lora','tokenizer','other')), + variant TEXT, -- quant/variety tag; shared by shards; user-editable + scope TEXT NOT NULL DEFAULT 'repo' CHECK(scope IN ('family','model','repo')), + size_bytes INTEGER, + lfs_sha256 TEXT, -- HF OID; NULL for small non-LFS files + UNIQUE(repo_id, rel_file_path), + FOREIGN KEY(repo_id) REFERENCES repos(id) +); + +-- CHANGED: polymorphic target (downloads/caches target repos; checks may target models) +CREATE TABLE jobs ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + job_type TEXT NOT NULL CHECK(job_type IN + ('download','cache_promote','cache_demote','update_check','update_apply','delete')), + target_type TEXT NOT NULL CHECK(target_type IN ('repo','model','family')), + target_id INTEGER NOT NULL, + status TEXT NOT NULL CHECK(status IN + ('queued','running','paused','completed','failed','cancelled')), + detail TEXT, -- JSON + progress REAL, + created_at DATETIME, started_at DATETIME, ended_at DATETIME +); + +CREATE TABLE storage_locations ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + path TEXT UNIQUE NOT NULL, -- HDD roots only + label TEXT, + is_active INTEGER DEFAULT 1 +); + +CREATE TABLE settings (key TEXT PRIMARY KEY, value TEXT); +-- Keys: ssd_cache_root, max_concurrent_downloads, retry_schedule, max_retries, +-- hf_transfer_enabled, recent_repos, shutdown_wait_seconds +-- HF token lives in Credential Manager (keyring), never here. +``` + +**Delete guardrails:** repo delete requires confirm (removes folder + files rows); model delete blocked while repos exist; family delete blocked while models exist. **Same HF repo cannot attach to two models** (global UNIQUE) — flagged as a v1 limitation. + +### 1.7 Architecture Overview +Unchanged components: console entry (signals) → FastAPI (API+SSE+static) → Job Scheduler → Download Manager (network pool) + Copy Queue (1 worker) → Storage Manager (path resolution now hierarchy-aware, temp+rename, hardlink/copy, space/integrity checks) → HF Service → DB → Startup Recovery → Config Service (placeholder). + +### 1.8 Key Design Decisions +- **Hierarchy over flat repos:** families/models are cheap metadata; repos are the unit of download/caching; files carry role+variant+scope. Enables multi-repo models, multi-quant repos, and model/family-shared files without duplication. *(User decision, v1.2.)* +- **Variant = per-file tag**, heuristic-parsed, user-editable; shards grouped by tag. +- **Scope controls placement**, provenance stays on the repo; shared files immutable (rename-swap updates). +- **Draft attachment is a classification choice** (role in-repo vs component='draft' separate repo), not a schema fork. +- Direct-to-target downloads; OID-based dedupe; resumable transfers; state machine (B4); seconds-scale retry + manual retry; single copy queue; startup sweep. (Carried from v1.1.) + +### 1.9 Risks & Mitigations +Carried from v1.1 (large copies → queue; space pre-checks; capped pools; token+backoff; WAL; orphan sweep; long paths; pinned hub version) **plus:** variant mislabeling from heuristics → editable tags + preview review; name collisions/illegal chars → sanitization + reserved-name checks; deep folder nesting → bounded layout + long-path enablement. + +--- + +## 2. Architecture Diagrams + +### 2.0 Entity Relationships (new) +```mermaid +erDiagram + FAMILIES ||--o{ MODELS : contains + MODELS ||--o{ REPOS : aggregates + REPOS ||--o{ MODEL_FILES : contains + STORAGE_LOCATIONS ||--o{ REPOS : hosts + REPOS ||--o{ JOBS : targeted-by + MODELS ||--o{ JOBS : targeted-by + FAMILIES ||--o{ JOBS : targeted-by + MODEL_FILES }o--|| REPOS : "scope folder may live under _shared/" +``` + +### 2.1 Components — unchanged from v1.1 (Storage Manager now hierarchy/path-scope aware). + +### 2.2 Download Flow +```mermaid +sequenceDiagram + participant U as User + participant API as FastAPI + participant JS as Job Scheduler + participant DM as Download Mgr + participant HF as Hugging Face + participant ST as Storage + participant DB as SQLite + + U->>API: GET /api/repo/preview (url or org/name) + API->>HF: model_info(files_metadata=true) + HF-->>API: files + sizes + LFS OIDs + API->>DB: family OID lookup + suggest family/model + API-->>U: grouped list (role, variant, scope, dedupe flags) + U->>API: POST /api/repos (family/model targets, files, roles, scopes, variants, root, cache?) + API->>DB: ensure family/model rows; repo(status=downloading) + job row + API-->>U: job id + JS->>DM: run + DM->>HF: snapshot_download(local_dir=_incoming/{repo_id}, allow_patterns, revision, token) + HF-->>DM: resumable stream + DM->>ST: place files per scope (repo folder / _shared/{model} / _shared) via temp+rename; dedupe by OID (link/copy) + DM->>DB: files rows + repo(status=complete) + alt cache requested + JS->>ST: enqueue promote (copy queue) + ST->>ST: copy → SSD .tmp → rename + ST->>DB: is_cached=1 + end + DM--)U: SSE progress +``` + +### 2.3 Promote / Demote — as v1.1, unit = repo folder; demote gated on HDD validation. + +### 2.4 Update Check — as v1.4 flow, per repo; model page aggregates repo results. + +--- + +## 3. Test Plan & Test Cases + +Strategy unchanged (unit / mock-HF integration / system / performance; retry delays configurable to seconds). TC-001…TC-029 from v1.1 carry over with target renames (model→repo where applicable: TC-016, TC-018, TC-019, TC-026). New/changed: + +| ID | Scenario | +|----|----------| +| TC-030 | Model with 3 repos (official + 2 community) — merged per-model variant list correct; repos distinct folders | +| TC-031 | Non-standard variant names (`APEX-Compact`, `I-Balanced`) — parsed or defaulted, editable, persisted; shards grouped under one tag | +| TC-032 | Draft MTP attached (a) as role=speedup in main repo, (b) as separate repo component='draft' — both resolvable as draft target for future config maker | +| TC-033 | mmproj scope=model — downloaded once; second repo of same model reuses via OID link/copy; stored in `{fam}/_shared/{model}/` | +| TC-034 | jinja scope=family — stored in `{fam}/_shared/`, shared across the family's models | +| TC-035 | Add New suggests family=org, model=name; user can map to existing family/model instead | +| TC-036 | Delete guards: model with repos blocked; family with models blocked; repo delete removes folder+rows | +| TC-037 | Reserved names rejected (`_shared`, `_incoming`, `CON`, trailing dots) for family/model/repo names | +| TC-038 | Two orgs, same repo name, under one model — second gets hash suffix, no overwrite | +| TC-039 | Same HF repo submitted for a second model — rejected cleanly (UNIQUE) with clear message | +| TC-040 | URL with branch (`/tree/dev`) — revision pinned and stored; shorthand `org/name` accepted | + +--- + +## 4. Security & Privacy Assessment + +Carried from v1.1 (loopback-only + Host-header check; `keyring` token; repo-ID regex + relative-path containment; try/finally temp cleanup; pinned deps + `pip-audit`; SQLite perms/WAL; log redaction; deep-verify as explicit command). **Added for v1.2:** folder-name sanitization treats all HF-derived strings (org, repo, filename) as untrusted — illegal chars, reserved device names, trailing dots/spaces, reserved app names (`_shared`, `_incoming`) — before any path is constructed; delete operations require explicit confirmation (destructive scope grew with hierarchy cascades). Privacy unchanged: no telemetry; HF metadata + transfers only; no content inspection beyond names/sizes/hashes. + +--- + +## Open items (v1.2 decisions applied — veto freely) + +1. **Family required**, auto-suggested from org (TC-035) — alternative would be a `_none` pseudo-family. +2. **Monolithic image repos stay whole** under `component='main'` with internal structure preserved (not split into vae/… type folders). +3. **Variant parsing is heuristic + editable** — no perfect auto-detection for arbitrary naming schemes. +4. **Promote unit = repo folder**; shared-scope files (mmproj/jinja) resolve from their live location (L3 is fine — they're small). Promoting `_shared` contents is a future option. +5. **One HF repo = one model attachment** (global UNIQUE) — a repo truly serving two models isn't representable in v1. +6. Repo folder = repo name (+hash on collision); `_shared`/`_incoming` reserved. +7. Config maker still awaiting your details — the schema now gives it everything it needs (main GGUF + variant, mmproj path via scope, draft target via role or component='draft'). + +Natural next steps: (a) implementation plan — module skeleton + build order, or (b) spec the llama.cpp config maker now that the data model can feed it. Which do you want? \ No newline at end of file