ai_model_manager/doc/DESIGN-DOC.md

Model Manager — Design Doc v1.2

1. Design Document

1.1 System Overview

The Model Manager is a Windows application with a web-based UI served by a local Python/FastAPI server. It manages large AI model files (5–80 GB) downloaded from Hugging Face. Content is organized as a hierarchy: families contain models; models contain one or more repos; repos contain files (with quant/variant tags). Models are stored on one or more HDD (L3) roots and can be promoted to an SSD (L2) cache. Metadata includes per-file HF content hashes; families share files by hash-verified dedupe; update detection is per-file diff.

1.2 Goals & Non-Goals

Goals:

• Download from Hugging Face (public/private with token), resumable across restarts
• Hierarchy: family → model → repo → files; multiple repos and multiple quants per model; non-standard variant tags supported (user-editable)
• Multiple HDD roots; promotion to SSD cache; HDD integrity validated before demotion
• Model types: text, image, embedding
• Shared files at family or model scope (jinja at any level, mmproj typically model)
• Draft/speed-up models (MTP, DFlash) attachable same-repo (role) or separate-repo (component) — user choice
• Per-file update diff (commit hash as fast pre-check); timestamps tracked
• Web UI (editable without recompile), SSE live progress, Logs page
• Graceful shutdown (5-min default wait, force-pause option); abort vs pause distinction
• Retry in seconds + manual "Retry Now"; single-worker disk-copy queue

Non-Goals (current version):

• Automatic family/model detection (suggested from repo org/name, but user confirms)
• Automatic cache eviction; llama.cpp config generation (schema/roles ready; placeholder service)
• Automatic image-gen role detection; multi-user/LAN access
• Local deep-hash verification on every operation (explicit command only)

1.3 Functional Requirements

1.3.1 Content Model & Storage Layout

Hierarchy: family → model → repo → files.

• Family (e.g., "Qwen3.5"): groups related models. Required; auto-suggested from repo org at Add New. May contain many models (27B, 35B, 9B…).
• Model (e.g., "Qwen3.5-27B"): logical model of a type (text/image/embedding). May aggregate multiple repos (official + community quants); UI shows a merged per-model view of all variants across repos.
• Repo: one HF repository, attached to exactly one model, with a component tag used as its folder segment: main, draft, vae, lora, other (default main).
• File: one downloaded file, with role, variant (quant/variety tag), and scope.

Variant tags: parsed from filenames (strip shard suffixes -NNNNN-of-NNNNN; match known patterns Q\d+_…, IQ\d…, UD-…; fallback = trailing name segments). Heuristics will not catch every naming scheme (e.g., APEX-Compact, I-Balanced) — the tag is always editable in the preview UI and in file details. Sharded GGUFs share one variant tag.

Draft/MTP/DFlash (user-selectable attachment):

• Under repo: draft files inside the main model's repo → role speedup, stored with the repo.
• Under model: a separate HF repo → its own repo row with component='draft', stored under {model}/draft/.
• Either way, the config maker (future) can resolve a draft target for --model-draft.

Shared-scope files: each file has scope:

scope	physical location	typical use
repo (default)	{root}/{model_type}/{fam}/{component}/{model}/{repo}/…	quants, VAE, CLIP, tokenizers, LoRA
model	{root}/{model_type}/{fam}/_shared/{model}/…	mmproj (default), model-level jinja
family	{root}/{model_type}/{fam}/_shared/…	family-level jinja overrides

Files keep their source-repo provenance (FK to repo) even when stored at model/family scope. Defaults at download time: mmproj → model, everything else → repo; user can elevate jinja to model/family in the preview UI.

Folder structure: {storage_root}/{text|image|embed}/{family}/{component}/{model}/{repo_name}/{file} (plus {storage_root}/{model_type}/{fam}/_shared/… and {storage_root}/_incoming/{repo_id}/ temp areas).

• Model type→folder map: text→text, image→image, embedding→embed.
• Monolithic image repos (diffusers: unet/, vae/, text_encoder/ inside one repo) → component='main', internal repo structure preserved per file (rel_file_path).
• Reserved folder names: _shared, _incoming; model/repo/family names sanitized (Windows-illegal chars, reserved device names, trailing dots/spaces, length caps). Repo folder = repo name; on collision append a short hash.

Storage roots: multiple HDD roots, removable only when unreferenced. L2 (SSD): single cache root in settings; promote/demote operates on repo folders (promoting a "model" = queue promote for each of its repos). Demotion validates the HDD copy first (files present, sizes match DB). Shared-scope files are small and resolve from wherever they live; promoting them is a future option.

Path handling: DB stores IDs + relative paths only; resolution joins root + segments and validates containment. Windows long paths enabled and documented; depth is bounded by the 6-level layout (TC-024).

1.3.2 Download Workflow

1. User pastes full URL or org/name shorthand; URL cruft stripped; a branch/revision in the URL is pinned and stored. Recent repos remembered in a dropdown.
2. Preview: model_info(files_metadata=True) → names, sizes, LFS sha256 OIDs.
3. Entity resolution: app suggests family (from org) and model (from name, suffix-stripped); user can pick existing family/model or create new. For an existing model, the repo attaches to it; files whose OID matches any family file are flagged "available via family" and skipped by default (small non-LFS files never deduped).
4. Files grouped by role and variant; user selects files, assigns roles/scopes/variant fixes, picks HDD root, optionally "Cache to SSD."
5. families/models/repos rows ensured (repo status='downloading'), jobs row created; download targets {root}/_incoming/{repo_id}/ on the destination volume.
6. snapshot_download(local_dir=<incoming>, allow_patterns, revision, token). Direct-to-target volume; no ~/.cache duplication (asserted at runtime); hf_transfer optional toggle.
7. Resumable; retry 5s→15s→45s→2m, cap 5m, max_retries default 5, manual "Retry Now." Retries exhausted → paused, partials kept, auto-resume offered on next launch.
8. Success: files moved into final locations per scope (temp+rename throughout; cross-volume dedupe → copy, same-volume → hardlink), OIDs/sizes recorded, status='complete'.
9. Cancel → _incoming deleted, repo cancelled. Pause → partials kept. Crash → startup sweep resumes.
10. "Cache to SSD" → copy job enqueued (§1.3.7); SSD copy also temp-then-rename.

1.3.3 Family Sharing & Scope Rules

• Dedupe identity = LFS sha256 OID, within a family, across models/repos. Matched files are materialized by hardlink (same volume) or copy (cross volume).
• Shared files are immutable: never rewritten in place — updates create new files and swap via rename, so hardlink partners are unaffected.
• Small non-LFS files (jinja, configs, tokenizers) are always re-downloaded, never deduped.
• User override: force re-download per file.

1.3.4 Update Check

Per repo (commit hash + per-file OIDs stored at download). Pre-check commit; if changed, per-file OID diff → added/changed/removed list → user selects → apply job re-downloads changed files to _incoming, swaps per file via rename (scope-aware), updates rows + revision. Model-level UI aggregates update state across its repos.

1.3.5 Web UI

• Overview: counts by type, disk usage per root, SSD usage, active jobs, pending updates.
• Models: grouped Family → Model → Repos; per-model merged variant list; per-repo cache status.
• Add New: URL/shorthand input, preview, family/model resolver, file selection with roles/scopes/variants, recent repos.
• Settings: roots, SSD cache root, token, concurrency, retries.
• Logs: live tail (SSE) + rotating file log.
• Static web/ folder, vanilla JS, SSE (/api/events, /api/logs/stream). No recompile to edit.

1.3.6 Graceful Shutdown

Ctrl+C or UI quit → stop accepting jobs → wait up to 5 min (configurable) for active tasks; UI shows "waiting for N tasks"; force-pause keeps partials for next launch; final DB checkpoint (WAL; writes are continuous anyway).

1.3.7 Copy Queue

Single worker serializes all disk-to-disk ops (promote, demote, update swaps, scope materialization). Network pool separate (default 2). Copies cancelable with try/finally temp cleanup.

1.3.8 Startup Recovery

Sweep: repos.status='downloading' ↔ _incoming/{repo_id}/ reconciled (resume or clean); orphaned _incoming dirs and SSD .tmp folders deleted with log entries.

1.4 Non-Functional Requirements

Performance (SSE progress, 2 network workers, 1 copy worker, optional hf_transfer); reliability (same-volume atomic renames, DB written at job start/end, WAL); security (loopback-only, keyring token, path validation, log redaction); Windows (long paths, short roots, name sanitization).

1.5 Technology Stack

Python 3.11, FastAPI, Uvicorn, SQLAlchemy (SQLite/WAL), huggingface_hub (>= 0.23, pinned), pydantic, keyring, optional hf_transfer; static web/ + vanilla JS; ThreadPoolExecutor pools; filelock; shutil/robocopy.

1.6 Data Model

CREATE TABLE families (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    name TEXT UNIQUE NOT NULL,          -- sanitized; reserved names rejected
    description TEXT,
    created_at DATETIME
);

CREATE TABLE models (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    family_id INTEGER NOT NULL,
    name TEXT NOT NULL,                 -- "Qwen3.5-27B" (sanitized)
    model_type TEXT NOT NULL CHECK(model_type IN ('text','image','embedding')),
    description TEXT,
    created_at DATETIME,
    UNIQUE(family_id, name),
    FOREIGN KEY(family_id) REFERENCES families(id)
);

CREATE TABLE repos (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    model_id INTEGER NOT NULL,
    hf_repo_id TEXT UNIQUE NOT NULL,    -- "org/name"; one row per HF repo
    component TEXT NOT NULL DEFAULT 'main'
        CHECK(component IN ('main','draft','vae','lora','other')), -- folder {component}
    revision TEXT,                      -- commit hash at download
    status TEXT NOT NULL DEFAULT 'downloading'
        CHECK(status IN ('downloading','complete','paused','error','cancelled')),
    storage_location_id INTEGER NOT NULL,
    rel_path TEXT NOT NULL,             -- {model_type}/{fam}/{component}/{model}/{repo}
    hf_last_modified DATETIME,
    download_started DATETIME,
    download_completed DATETIME,
    is_cached INTEGER DEFAULT 0,
    cache_path TEXT,                    -- relative to SSD root
    UNIQUE(storage_location_id, rel_path),
    FOREIGN KEY(model_id) REFERENCES models(id),
    FOREIGN KEY(storage_location_id) REFERENCES storage_locations(id)
);
-- Model status is DERIVED from its repos (UI aggregate). Downloads/caching operate per repo.

CREATE TABLE model_files (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    repo_id INTEGER NOT NULL,           -- provenance (source repo)
    rel_file_path TEXT NOT NULL,        -- path within its scope folder
    role TEXT NOT NULL DEFAULT 'other' CHECK(role IN (
        'main','mmproj','speedup','jinja','vae','clip','text_encoder',
        'lora','tokenizer','other')),
    variant TEXT,                       -- quant/variety tag; shared by shards; user-editable
    scope TEXT NOT NULL DEFAULT 'repo' CHECK(scope IN ('family','model','repo')),
    size_bytes INTEGER,
    lfs_sha256 TEXT,                    -- HF OID; NULL for small non-LFS files
    UNIQUE(repo_id, rel_file_path),
    FOREIGN KEY(repo_id) REFERENCES repos(id)
);

-- CHANGED: polymorphic target (downloads/caches target repos; checks may target models)
CREATE TABLE jobs (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    job_type TEXT NOT NULL CHECK(job_type IN
        ('download','cache_promote','cache_demote','update_check','update_apply','delete')),
    target_type TEXT NOT NULL CHECK(target_type IN ('repo','model','family')),
    target_id INTEGER NOT NULL,
    status TEXT NOT NULL CHECK(status IN
        ('queued','running','paused','completed','failed','cancelled')),
    detail TEXT,                        -- JSON
    progress REAL,
    created_at DATETIME, started_at DATETIME, ended_at DATETIME
);

CREATE TABLE storage_locations (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    path TEXT UNIQUE NOT NULL,          -- HDD roots only
    label TEXT,
    is_active INTEGER DEFAULT 1
);

CREATE TABLE settings (key TEXT PRIMARY KEY, value TEXT);
-- Keys: ssd_cache_root, max_concurrent_downloads, retry_schedule, max_retries,
-- hf_transfer_enabled, recent_repos, shutdown_wait_seconds
-- HF token lives in Credential Manager (keyring), never here.

Delete guardrails: repo delete requires confirm (removes folder + files rows); model delete blocked while repos exist; family delete blocked while models exist. Same HF repo cannot attach to two models (global UNIQUE) — flagged as a v1 limitation.

1.7 Architecture Overview

Unchanged components: console entry (signals) → FastAPI (API+SSE+static) → Job Scheduler → Download Manager (network pool) + Copy Queue (1 worker) → Storage Manager (path resolution now hierarchy-aware, temp+rename, hardlink/copy, space/integrity checks) → HF Service → DB → Startup Recovery → Config Service (placeholder).

1.8 Key Design Decisions

• Hierarchy over flat repos: families/models are cheap metadata; repos are the unit of download/caching; files carry role+variant+scope. Enables multi-repo models, multi-quant repos, and model/family-shared files without duplication. (User decision, v1.2.)
• Variant = per-file tag, heuristic-parsed, user-editable; shards grouped by tag.
• Scope controls placement, provenance stays on the repo; shared files immutable (rename-swap updates).
• Draft attachment is a classification choice (role in-repo vs component='draft' separate repo), not a schema fork.
• Direct-to-target downloads; OID-based dedupe; resumable transfers; state machine (B4); seconds-scale retry + manual retry; single copy queue; startup sweep. (Carried from v1.1.)

1.9 Risks & Mitigations

Carried from v1.1 (large copies → queue; space pre-checks; capped pools; token+backoff; WAL; orphan sweep; long paths; pinned hub version) plus: variant mislabeling from heuristics → editable tags + preview review; name collisions/illegal chars → sanitization + reserved-name checks; deep folder nesting → bounded layout + long-path enablement.

---

2. Architecture Diagrams

2.0 Entity Relationships

erDiagram
    FAMILIES ||--o{ MODELS : contains
    MODELS  ||--o{ REPOS   : aggregates
    REPOS   ||--o{ MODEL_FILES : contains
    STORAGE_LOCATIONS ||--o{ REPOS : hosts
    REPOS   ||--o{ JOBS    : targeted-by
    MODELS  ||--o{ JOBS    : targeted-by
    FAMILIES ||--o{ JOBS   : targeted-by
    MODEL_FILES }o--|| REPOS : "scope folder may live under _shared/"

2.1 Components

2.2 Download Flow

sequenceDiagram
    autonumber
    participant U as User
    participant API as FastAPI
    participant JS as Job Scheduler
    participant DM as Download Mgr
    participant HF as Hugging Face
    participant CQ as Copy Queue
    participant ST as Storage Mgr
    participant DB as SQLite

    U->>API: GET /api/repo/preview (url or org/name)
    API->>HF: model_info(files_metadata=true)
    HF-->>API: files, sizes, LFS sha256 OIDs
    API->>DB: family OID lookup, suggest family/model
    API-->>U: grouped list (role, variant, scope, dedupe flags)
    U->>API: POST /api/repos (family, model, files, roles, scopes, variants, root, cache)
    API->>DB: ensure family/model rows
    API->>DB: create repo (status=downloading) and job row
    API-->>U: job id
    JS->>DM: run download job
    DM->>HF: snapshot_download(local_dir=_incoming, allow_patterns, revision, token)
    HF-->>DM: resumable stream into _incoming
    loop until download completes
        DM-->>U: SSE progress (bytes, speed)
    end
    DM->>DM: verify sizes and OIDs
    DM->>ST: place files per scope (repo folder or _shared) via temp+rename
    DM->>ST: dedupe by OID (hardlink same volume, copy cross volume)
    DM->>DB: insert file rows, repo (status=complete)
    alt cache requested
        JS->>CQ: enqueue promote
        CQ->>ST: copy repo folder to SSD .tmp
        CQ->>ST: rename .tmp to final
        CQ->>DB: is_cached = 1
    end

2.3 Promote / Demote

sequenceDiagram
    autonumber
    participant U as User
    participant API as FastAPI
    participant JS as Job Scheduler
    participant CQ as Copy Queue
    participant ST as Storage Mgr
    participant DB as SQLite

    U->>API: POST /api/repos/123/cache (promote or demote)
    API->>DB: create job row (queued)
    API->>JS: enqueue on copy queue
    API-->>U: accepted (job id)
    alt promote
        JS->>CQ: run promote
        CQ->>ST: free-space check on SSD
        alt insufficient space
            CQ->>DB: job (failed)
            CQ-->>U: SSE error (insufficient SSD space)
        else space ok
            CQ->>ST: copy repo folder to SSD .tmp
            CQ->>ST: rename .tmp to final
            CQ->>DB: is_cached = 1, set cache_path
            CQ-->>U: SSE completion
        end
    else demote
        JS->>CQ: run demote
        CQ->>ST: validate HDD copy (files present, sizes match DB)
        alt HDD invalid
            CQ->>DB: job (failed, demote blocked)
            CQ-->>U: SSE error (HDD copy missing or incomplete)
        else HDD valid
            CQ->>ST: delete SSD folder
            CQ->>DB: is_cached = 0, cache_path = null
            CQ-->>U: SSE completion
        end
    end

2.4 Update Check

sequenceDiagram
    autonumber
    participant U as User
    participant API as FastAPI
    participant HF as Hugging Face
    participant DB as SQLite

    U->>API: POST /api/repos/123/check-update
    API->>DB: load stored revision (commit hash)
    API->>HF: repo info (current sha)
    HF-->>API: current commit
    API->>API: compare commits
    alt same commit
        API-->>U: up to date
    else commit differs
        API->>HF: model_info(files_metadata=true)
        HF-->>API: current files, sizes, OIDs
        API->>DB: diff per-file OIDs against stored
        API-->>U: list added, changed, removed
        U->>API: POST /api/repos/123/apply-update (selected files)
        API->>DB: create update_apply job
        API-->>U: accepted (job id)
        Note over API,DB: apply job downloads changed files to _incoming, swaps per file via temp+rename (scope-aware), updates rows and revision
    end
    Note over U,DB: Models page aggregates update state across all repos of a model

---

3. Test Plan & Test Cases

Strategy unchanged (unit / mock-HF integration / system / performance; retry delays configurable to seconds). TC-001…TC-029 from v1.1 carry over with target renames (model→repo where applicable: TC-016, TC-018, TC-019, TC-026). New/changed:

ID	Scenario
TC-030	Model with 3 repos (official + 2 community) — merged per-model variant list correct; repos distinct folders
TC-031	Non-standard variant names (APEX-Compact, I-Balanced) — parsed or defaulted, editable, persisted; shards grouped under one tag
TC-032	Draft MTP attached (a) as role=speedup in main repo, (b) as separate repo component='draft' — both resolvable as draft target for future config maker
TC-033	mmproj scope=model — downloaded once; second repo of same model reuses via OID link/copy; stored in {fam}/_shared/{model}/
TC-034	jinja scope=family — stored in {fam}/_shared/, shared across the family's models
TC-035	Add New suggests family=org, model=name; user can map to existing family/model instead
TC-036	Delete guards: model with repos blocked; family with models blocked; repo delete removes folder+rows
TC-037	Reserved names rejected (_shared, _incoming, CON, trailing dots) for family/model/repo names
TC-038	Two orgs, same repo name, under one model — second gets hash suffix, no overwrite
TC-039	Same HF repo submitted for a second model — rejected cleanly (UNIQUE) with clear message
TC-040	URL with branch (/tree/dev) — revision pinned and stored; shorthand org/name accepted

---

4. Security & Privacy Assessment

Carried from v1.1 (loopback-only + Host-header check; keyring token; repo-ID regex + relative-path containment; try/finally temp cleanup; pinned deps + pip-audit; SQLite perms/WAL; log redaction; deep-verify as explicit command). Added for v1.2: folder-name sanitization treats all HF-derived strings (org, repo, filename) as untrusted — illegal chars, reserved device names, trailing dots/spaces, reserved app names (_shared, _incoming) — before any path is constructed; delete operations require explicit confirmation (destructive scope grew with hierarchy cascades). Privacy unchanged: no telemetry; HF metadata + transfers only; no content inspection beyond names/sizes/hashes.

---

Open items (v1.2 decisions applied — veto freely)

1. Family required, auto-suggested from org (TC-035) — alternative would be a _none pseudo-family.
2. Monolithic image repos stay whole under component='main' with internal structure preserved (not split into vae/… type folders).
3. Variant parsing is heuristic + editable — no perfect auto-detection for arbitrary naming schemes.
4. Promote unit = repo folder; shared-scope files (mmproj/jinja) resolve from their live location (L3 is fine — they're small). Promoting _shared contents is a future option.
5. One HF repo = one model attachment (global UNIQUE) — a repo truly serving two models isn't representable in v1.
6. Repo folder = repo name (+hash on collision); _shared/_incoming reserved.
7. Config maker still awaiting your details — the schema now gives it everything it needs (main GGUF + variant, mmproj path via scope, draft target via role or component='draft').

Natural next steps: (a) implementation plan — module skeleton + build order, or (b) spec the llama.cpp config maker now that the data model can feed it. Which do you want?