Imrayya/ai_model_manager
SHA256
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update doc/DESIGN-DOC.md

Browse Source
This commit is contained in:
Kareem
2026-09-02 12:35:08 +02:00
parent ae42966acf
commit 091457ae8a
1 changed files with 264 additions and 287 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/DESIGN-DOC.md
+264 -287
View File
@@ -1,377 +1,354 @@
# Model Manager — Design Doc v1.2
## 1. Design Document ## 1. Design Document
### 1.1 System Overview ### 1.1 System Overview
The Model Manager is a Windows application with a web-based UI served by a local Python/FastAPI server. It manages large AI model files (5–80 GB) downloaded from Hugging Face. Content is organized as a hierarchy: **families contain models; models contain one or more repos; repos contain files (with quant/variant tags)**. Models are stored on one or more HDD (L3) roots and can be promoted to an SSD (L2) cache. Metadata includes per-file HF content hashes; families share files by hash-verified dedupe; update detection is per-file diff. The Model Manager is a Windows application with a web-based UI served by a local Python/FastAPI server. It manages large AI model files (5–80 GB) downloaded from Hugging Face. Models are stored on one or more HDD (Level 3) storage locations, and can be manually promoted to an SSD (Level 2) cache folder for faster access. The application tracks metadata, supports family grouping (manual), checks for updates, and handles downloads with retry and checksum verification.
### 1.2 Goals & Non-Goals ### 1.2 Goals & Non-Goals
**Goals:** **Goals:**
- Download from Hugging Face (public/private with token), resumable across restarts - Download models from Hugging Face (public/private with token).
- **Hierarchy: family → model → repo → files; multiple repos and multiple quants per model; non-standard variant tags supported (user-editable)** - Store on configurable HDD storage locations (multiple supported).
- Multiple HDD roots; promotion to SSD cache; HDD integrity validated before demotion - Manual promotion to SSD cache via UI checkbox.
- Model types: text, image, embedding - Manage model types: Text-Gen (GGUF), Image-Gen (safetensors, etc.), Embedding (GGUF).
- **Shared files at family or model scope (jinja at any level, mmproj typically model)** - Support manual family grouping to share files (e.g., mmproj).
- Draft/speed-up models (MTP, DFlash) attachable **same-repo (role) or separate-repo (component) — user choice** - Track download/upload timestamps and suggest updates.
- Per-file update diff (commit hash as fast pre-check); timestamps tracked - Provide a responsive web UI (editable without recompilation).
- Web UI (editable without recompile), SSE live progress, Logs page - Handle graceful shutdown (finish downloads, flush database).
- Graceful shutdown (5-min default wait, force-pause option); abort vs pause distinction - Support checksum verification and exponential retry for failed downloads.
- Retry in seconds + manual "Retry Now"; single-worker disk-copy queue
**Non-Goals (current version):** **Non-Goals (current version):**
- Automatic family/model detection (suggested from repo org/name, but user confirms) - Automatic family detection (manual only).
- Automatic cache eviction; llama.cpp config generation (schema/roles ready; placeholder service) - Automatic cache eviction (manual promotion only).
- Automatic image-gen role detection; multi-user/LAN access - Full config generation for llama.cpp (placeholder only).
- Local deep-hash verification on every operation (explicit command only) - Automatic image-gen file role detection (manual classification during download).
- Multi-user support, remote access (local only).
- Pause/resume downloads (not required, but may be added later).
### 1.3 Functional Requirements ### 1.3 Functional Requirements
#### 1.3.1 Content Model & Storage Layout #### 1.3.1 Storage Management
**Hierarchy:** `family → model → repo → files`. - **Level 3 (HDD)**: One or more root folders (e.g., `D:\Models`, `E:\Models`). User can add/remove storage locations in Settings. Each download is placed into a selected (or automatically chosen) HDD root.
- **Level 2 (SSD)**: A single cache folder (e.g., `C:\ModelCache`). When user checks "Cache to SSD" for a model, the entire model folder is copied from HDD to SSD cache. Original remains on HDD. Unchecking removes the cached copy.
- **Family** (e.g., "Qwen3.5"): groups related models. Required; auto-suggested from repo org at Add New. May contain many models (27B, 35B, 9B…). - **File Organization**: Each model stored in its own subfolder (e.g., `{storage_root}/{model_type}/{hf_repo_name}/`). Within, files are kept as downloaded from HF (preserve repository structure).
- **Model** (e.g., "Qwen3.5-27B"): logical model of a type (`text`/`image`/`embedding`). May aggregate **multiple repos** (official + community quants); UI shows a merged per-model view of all variants across repos.
- **Repo**: one HF repository, attached to exactly one model, with a `component` tag used as its folder segment: `main`, `draft`, `vae`, `lora`, `other` (default `main`).
- **File**: one downloaded file, with `role`, `variant` (quant/variety tag), and `scope`.
**Variant tags:** parsed from filenames (strip shard suffixes `-NNNNN-of-NNNNN`; match known patterns `Q\d+_…`, `IQ\d…`, `UD-…`; fallback = trailing name segments). **Heuristics will not catch every naming scheme (e.g., `APEX-Compact`, `I-Balanced`)** — the tag is always editable in the preview UI and in file details. Sharded GGUFs share one variant tag.
**Draft/MTP/DFlash (user-selectable attachment):**
- *Under repo:* draft files inside the main model's repo → role `speedup`, stored with the repo.
- *Under model:* a separate HF repo → its own repo row with `component='draft'`, stored under `{model}/draft/`.
- Either way, the config maker (future) can resolve a draft target for `--model-draft`.
**Shared-scope files:** each file has `scope`:
| scope | physical location | typical use |
|---|---|---|
| `repo` (default) | `{root}/{model_type}/{fam}/{component}/{model}/{repo}/…` | quants, VAE, CLIP, tokenizers, LoRA |
| `model` | `{root}/{model_type}/{fam}/_shared/{model}/…` | **mmproj (default)**, model-level jinja |
| `family` | `{root}/{model_type}/{fam}/_shared/…` | family-level jinja overrides |
Files keep their source-repo provenance (FK to repo) even when stored at model/family scope. Defaults at download time: mmproj → `model`, everything else → `repo`; user can elevate jinja to model/family in the preview UI.
**Folder structure:** `{storage_root}/{text|image|embed}/{family}/{component}/{model}/{repo_name}/{file}`
(plus `{storage_root}/{model_type}/{fam}/_shared/…` and `{storage_root}/_incoming/{repo_id}/` temp areas).
- Model type→folder map: text→`text`, image→`image`, embedding→`embed`.
- Monolithic image repos (diffusers: `unet/`, `vae/`, `text_encoder/` inside one repo) → `component='main'`, **internal repo structure preserved** per file (`rel_file_path`).
- Reserved folder names: `_shared`, `_incoming`; model/repo/family names sanitized (Windows-illegal chars, reserved device names, trailing dots/spaces, length caps). Repo folder = repo name; on collision append a short hash.
**Storage roots:** multiple HDD roots, removable only when unreferenced. **L2 (SSD):** single cache root in settings; promote/demote operates on **repo folders** (promoting a "model" = queue promote for each of its repos). Demotion validates the HDD copy first (files present, sizes match DB). Shared-scope files are small and resolve from wherever they live; promoting them is a future option.
**Path handling:** DB stores IDs + relative paths only; resolution joins root + segments and validates containment. Windows long paths enabled and documented; depth is bounded by the 6-level layout (TC-024).
#### 1.3.2 Download Workflow #### 1.3.2 Download Workflow
1. User pastes **full URL or `org/name` shorthand**; URL cruft stripped; a branch/revision in the URL is pinned and stored. Recent repos remembered in a dropdown. 1. User provides Hugging Face repo ID (e.g., `org/model-name`) or URL.
2. Preview: `model_info(files_metadata=True)` → names, sizes, **LFS sha256 OIDs**. 2. App lists files in the repo using `huggingface_hub.list_repo_files`.
3. **Entity resolution:** app suggests family (from org) and model (from name, suffix-stripped); user can pick existing family/model or create new. For an existing model, the repo attaches to it; files whose OID matches any family file are flagged "available via family" and skipped by default (small non-LFS files never deduped). 3. App classifies files by extension and filename patterns (GGUF, safetensors, etc.) and presents them to the user, grouped by inferred role (main, mmproj, VAE, etc.). Since detection is manual, the user can select which files to download and optionally assign roles.
4. Files grouped by role and **variant**; user selects files, assigns roles/scopes/variant fixes, picks HDD root, optionally "Cache to SSD." 4. User selects variant(s) (e.g., specific GGUF quantization) and target HDD storage location.
5. `families`/`models`/`repos` rows ensured (repo `status='downloading'`), `jobs` row created; download targets `{root}/_incoming/{repo_id}/` on the destination volume. 5. App downloads selected files to a temporary folder within the target HDD. It uses `huggingface_hub.snapshot_download` or `hf_hub_download` with `allow_patterns` to filter. If token is provided, it is used.
6. `snapshot_download(local_dir=<incoming>, allow_patterns, revision, token)`. Direct-to-target volume; no `~/.cache` duplication (asserted at runtime); `hf_transfer` optional toggle. 6. After download completes, checksums are verified (if available from HF or computed locally).
7. Resumable; retry 5s→15s→45s→2m, cap 5m, `max_retries` default 5, manual "Retry Now." Retries exhausted → `paused`, partials kept, auto-resume offered on next launch. 7. The temp folder is atomically renamed to the final model folder.
8. Success: files moved into final locations per scope (temp+rename throughout; cross-volume dedupe → copy, same-volume → hardlink), OIDs/sizes recorded, `status='complete'`. 8. Database entry is created with metadata: repo ID, local path, file list with roles, HF commit hash, timestamps, etc.
9. **Cancel** → `_incoming` deleted, repo `cancelled`. **Pause** → partials kept. Crash → startup sweep resumes. 9. If "Cache to SSD" was checked, a background copy to SSD cache is started.
10. "Cache to SSD" → copy job enqueued (§1.3.7); SSD copy also temp-then-rename.
#### 1.3.3 Family Sharing & Scope Rules #### 1.3.3 Family Management
- Dedupe identity = **LFS sha256 OID**, within a family, across models/repos. Matched files are materialized by hardlink (same volume) or copy (cross volume). - User can create a **family** (e.g., "Llama-3-Vision") with a name and description.
- **Shared files are immutable: never rewritten in place** — updates create new files and swap via rename, so hardlink partners are unaffected. - When downloading a model, user can assign it to an existing family. The app will compare the new model's files with already downloaded files from the same family.
- Small non-LFS files (jinja, configs, tokenizers) are always re-downloaded, never deduped. - If a file with the same filename and role already exists locally (from another model in the family), the app can avoid re-downloading it by creating a **hard link** or **copy** (configurable) from the existing file to the new model's folder. This is especially useful for shared `mmproj` or `jinja` files.
- User override: force re-download per file. - Manual override: user can force download even if file exists.
#### 1.3.4 Update Check #### 1.3.4 Update Check
Per repo (commit hash + per-file OIDs stored at download). Pre-check commit; if changed, per-file OID diff → added/changed/removed list → user selects → apply job re-downloads changed files to `_incoming`, swaps per file via rename (scope-aware), updates rows + revision. Model-level UI aggregates update state across its repos. - The app stores the HF repository commit hash at download time.
- On demand (UI button or scheduled), the app queries the HF API for the current commit hash of the repo.
- If the hash differs, the UI shows an "Update available" indicator. The user can initiate a re-download of changed files.
- For family-shared files, update check can be done per file if desired, but initial version uses repo-level hash.
#### 1.3.5 Web UI #### 1.3.5 Web UI
- **Overview:** counts by type, disk usage per root, SSD usage, active jobs, pending updates. - **Pages**: Overview (dashboard), Models (list/filter by type), Add New, Settings, Logs.
- **Models:** grouped Family → Model → Repos; per-model merged variant list; per-repo cache status. - UI assets (HTML/CSS/JS) are stored in a `web/` folder next to the application. They can be edited without recompiling the Python code.
- **Add New:** URL/shorthand input, preview, family/model resolver, file selection with roles/scopes/variants, recent repos. - All interactions go through a REST API; frontend is a simple SPA using Vue.js (CDN) or vanilla JS.
- **Settings:** roots, SSD cache root, token, concurrency, retries.
- **Logs:** live tail (SSE) + rotating file log.
- Static `web/` folder, vanilla JS, SSE (`/api/events`, `/api/logs/stream`). No recompile to edit.
#### 1.3.6 Graceful Shutdown #### 1.3.6 Graceful Shutdown
Ctrl+C or UI quit → stop accepting jobs → wait up to **5 min** (configurable) for active tasks; UI shows "waiting for N tasks"; force-pause keeps partials for next launch; final DB checkpoint (WAL; writes are continuous anyway). - The console process captures `CTRL_C_EVENT` or receives a shutdown command from the UI.
- It stops accepting new API requests, waits for active background tasks (downloads, copies) to complete (with a configurable timeout), then closes database connections and exits.
#### 1.3.7 Copy Queue - In-progress downloads are allowed to finish; if timeout exceeded, they are aborted and database remains consistent.
Single worker serializes all disk-to-disk ops (promote, demote, update swaps, scope materialization). Network pool separate (default 2). Copies cancelable with try/finally temp cleanup.
#### 1.3.8 Startup Recovery
Sweep: `repos.status='downloading'` ↔ `_incoming/{repo_id}/` reconciled (resume or clean); orphaned `_incoming` dirs and SSD `.tmp` folders deleted with log entries.
### 1.4 Non-Functional Requirements ### 1.4 Non-Functional Requirements
Performance (SSE progress, 2 network workers, 1 copy worker, optional `hf_transfer`); reliability (same-volume atomic renames, DB written at job start/end, WAL); security (loopback-only, `keyring` token, path validation, log redaction); Windows (long paths, short roots, name sanitization). - **Performance**: Downloads use streaming with progress; UI remains responsive via background threads. Multiple concurrent downloads limited (configurable, default 2).
- **Reliability**: Atomic file moves, database transactions, checksum verification, exponential retry with configurable max attempts.
- **Security**: Local-only server (bind 127.0.0.1), token stored encrypted using Windows DPAPI, input validation to prevent path traversal.
- **Maintainability**: Modular services, decoupled UI, clear separation of concerns.
### 1.5 Technology Stack ### 1.5 Technology Stack
Python 3.11, FastAPI, Uvicorn, SQLAlchemy (SQLite/WAL), `huggingface_hub` (>= 0.23, pinned), `pydantic`, `keyring`, optional `hf_transfer`; static `web/` + vanilla JS; `ThreadPoolExecutor` pools; `filelock`; `shutil`/`robocopy`. - **Backend**: Python 3.11, FastAPI, Uvicorn, SQLAlchemy (SQLite), `huggingface_hub`, `pydantic`, `cryptography` (for DPAPI encryption).
- **Frontend**: Static HTML/JS/CSS served by FastAPI; optional Vue.js via CDN.
- **Background Tasks**: FastAPI `BackgroundTasks` or a custom thread pool for long operations.
- **File Operations**: `shutil` for copy/move, `os` for hard links, `filelock` for safe concurrent access.
### 1.6 Data Model ### 1.6 Data Model (Database Schema)
```sql ```sql
-- families
CREATE TABLE families ( CREATE TABLE families (
id INTEGER PRIMARY KEY AUTOINCREMENT, id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT UNIQUE NOT NULL, -- sanitized; reserved names rejected name TEXT UNIQUE NOT NULL,
description TEXT, description TEXT
created_at DATETIME
);
CREATE TABLE models (
id INTEGER PRIMARY KEY AUTOINCREMENT,
family_id INTEGER NOT NULL,
name TEXT NOT NULL, -- "Qwen3.5-27B" (sanitized)
model_type TEXT NOT NULL CHECK(model_type IN ('text','image','embedding')),
description TEXT,
created_at DATETIME,
UNIQUE(family_id, name),
FOREIGN KEY(family_id) REFERENCES families(id)
);
CREATE TABLE repos (
id INTEGER PRIMARY KEY AUTOINCREMENT,
model_id INTEGER NOT NULL,
hf_repo_id TEXT UNIQUE NOT NULL, -- "org/name"; one row per HF repo
component TEXT NOT NULL DEFAULT 'main'
CHECK(component IN ('main','draft','vae','lora','other')), -- folder {component}
revision TEXT, -- commit hash at download
status TEXT NOT NULL DEFAULT 'downloading'
CHECK(status IN ('downloading','complete','paused','error','cancelled')),
storage_location_id INTEGER NOT NULL,
rel_path TEXT NOT NULL, -- {model_type}/{fam}/{component}/{model}/{repo}
hf_last_modified DATETIME,
download_started DATETIME,
download_completed DATETIME,
is_cached INTEGER DEFAULT 0,
cache_path TEXT, -- relative to SSD root
UNIQUE(storage_location_id, rel_path),
FOREIGN KEY(model_id) REFERENCES models(id),
FOREIGN KEY(storage_location_id) REFERENCES storage_locations(id)
);
-- Model status is DERIVED from its repos (UI aggregate). Downloads/caching operate per repo.
CREATE TABLE model_files (
id INTEGER PRIMARY KEY AUTOINCREMENT,
repo_id INTEGER NOT NULL, -- provenance (source repo)
rel_file_path TEXT NOT NULL, -- path within its scope folder
role TEXT NOT NULL DEFAULT 'other' CHECK(role IN (
'main','mmproj','speedup','jinja','vae','clip','text_encoder',
'lora','tokenizer','other')),
variant TEXT, -- quant/variety tag; shared by shards; user-editable
scope TEXT NOT NULL DEFAULT 'repo' CHECK(scope IN ('family','model','repo')),
size_bytes INTEGER,
lfs_sha256 TEXT, -- HF OID; NULL for small non-LFS files
UNIQUE(repo_id, rel_file_path),
FOREIGN KEY(repo_id) REFERENCES repos(id)
);
-- CHANGED: polymorphic target (downloads/caches target repos; checks may target models)
CREATE TABLE jobs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
job_type TEXT NOT NULL CHECK(job_type IN
('download','cache_promote','cache_demote','update_check','update_apply','delete')),
target_type TEXT NOT NULL CHECK(target_type IN ('repo','model','family')),
target_id INTEGER NOT NULL,
status TEXT NOT NULL CHECK(status IN
('queued','running','paused','completed','failed','cancelled')),
detail TEXT, -- JSON
progress REAL,
created_at DATETIME, started_at DATETIME, ended_at DATETIME
); );
-- storage_locations (HDD roots)
CREATE TABLE storage_locations ( CREATE TABLE storage_locations (
id INTEGER PRIMARY KEY AUTOINCREMENT, id INTEGER PRIMARY KEY AUTOINCREMENT,
path TEXT UNIQUE NOT NULL, -- HDD roots only path TEXT UNIQUE NOT NULL,
label TEXT, type TEXT DEFAULT 'hdd', -- 'hdd' or 'ssd'
is_active INTEGER DEFAULT 1 is_active INTEGER DEFAULT 1
); );
CREATE TABLE settings (key TEXT PRIMARY KEY, value TEXT); -- models
-- Keys: ssd_cache_root, max_concurrent_downloads, retry_schedule, max_retries, CREATE TABLE models (
-- hf_transfer_enabled, recent_repos, shutdown_wait_seconds id INTEGER PRIMARY KEY AUTOINCREMENT,
-- HF token lives in Credential Manager (keyring), never here. hf_repo_id TEXT NOT NULL,
local_path TEXT NOT NULL UNIQUE, -- absolute path to model folder on HDD
model_type TEXT NOT NULL, -- 'text', 'image', 'embedding'
family_id INTEGER,
storage_location_id INTEGER NOT NULL,
download_timestamp DATETIME,
hf_commit_hash TEXT,
upload_timestamp DATETIME, -- last commit date from HF
is_cached INTEGER DEFAULT 0, -- 1 if cached on SSD
cache_path TEXT, -- absolute path to cached folder
FOREIGN KEY(family_id) REFERENCES families(id),
FOREIGN KEY(storage_location_id) REFERENCES storage_locations(id)
);
-- model_files
CREATE TABLE model_files (
id INTEGER PRIMARY KEY AUTOINCREMENT,
model_id INTEGER NOT NULL,
filename TEXT NOT NULL, -- relative path within model folder
role TEXT, -- 'main', 'mmproj', 'vae', 'clip', 'text_encoder', 'lora', 'other'
size_bytes INTEGER,
sha256 TEXT,
FOREIGN KEY(model_id) REFERENCES models(id)
);
-- settings (key-value)
CREATE TABLE settings (
key TEXT PRIMARY KEY,
value TEXT
);
``` ```
**Delete guardrails:** repo delete requires confirm (removes folder + files rows); model delete blocked while repos exist; family delete blocked while models exist. **Same HF repo cannot attach to two models** (global UNIQUE) — flagged as a v1 limitation.
### 1.7 Architecture Overview ### 1.7 Architecture Overview
Unchanged components: console entry (signals) → FastAPI (API+SSE+static) → Job Scheduler → Download Manager (network pool) + Copy Queue (1 worker) → Storage Manager (path resolution now hierarchy-aware, temp+rename, hardlink/copy, space/integrity checks) → HF Service → DB → Startup Recovery → Config Service (placeholder).
The application consists of:
- **Console Entry Point**: Starts Uvicorn server, handles signals.
- **FastAPI Application**: Serves API and static UI files.
- **Background Task Manager**: Uses a `ThreadPoolExecutor` to run downloads, copies, and update checks.
- **Download Manager**: Handles HF interactions, retries, checksum verification.
- **Storage Manager**: Manages HDD roots, SSD cache, file copying/moving, hard links.
- **HF Service**: Wraps `huggingface_hub` functions for listing files, downloading, getting metadata.
- **Database Layer**: SQLAlchemy models and CRUD.
- **Config Service**: Placeholder for future llama.cpp config generation.
### 1.8 Key Design Decisions ### 1.8 Key Design Decisions
- **Hierarchy over flat repos:** families/models are cheap metadata; repos are the unit of download/caching; files carry role+variant+scope. Enables multi-repo models, multi-quant repos, and model/family-shared files without duplication. *(User decision, v1.2.)*
- **Variant = per-file tag**, heuristic-parsed, user-editable; shards grouped by tag. - **Download Method**: Use `huggingface_hub.snapshot_download` with `allow_patterns` to download selected files. This automatically verifies checksums if provided by HF. For additional verification, compute SHA256 after download and store.
- **Scope controls placement**, provenance stays on the repo; shared files immutable (rename-swap updates). - **Retry Mechanism**: Implement in Download Manager: on failure (network error, checksum mismatch), retry after delay: 5 min, 15 min, 45 min, 2 hours, etc., up to `max_retries` (default 5). Only retry if error is transient (e.g., network, HTTP 5xx).
- **Draft attachment is a classification choice** (role in-repo vs component='draft' separate repo), not a schema fork. - **Family Sharing**: When assigning a model to a family, after download, for each file, check if an identical file (by relative path and role) exists in another model of the same family. If yes, replace the downloaded file with a hard link to the existing file (to save space). This is safe because models are read-only.
- Direct-to-target downloads; OID-based dedupe; resumable transfers; state machine (B4); seconds-scale retry + manual retry; single copy queue; startup sweep. (Carried from v1.1.) - **Multiple HDD Storage**: Settings page allows adding multiple storage locations. When downloading, user selects which HDD to use (or app chooses one with sufficient free space).
- **Cache Promotion**: Copy entire model folder from HDD to SSD cache. Use `shutil.copytree` with progress callback. Store `cache_path`. On demotion, delete the SSD copy.
### 1.9 Risks & Mitigations ### 1.9 Risks & Mitigations
Carried from v1.1 (large copies → queue; space pre-checks; capped pools; token+backoff; WAL; orphan sweep; long paths; pinned hub version) **plus:** variant mislabeling from heuristics → editable tags + preview review; name collisions/illegal chars → sanitization + reserved-name checks; deep folder nesting → bounded layout + long-path enablement. - **Large File Copy**: Use efficient copy (robocopy or `shutil.copy2` with buffer). Show progress.
- **Disk Space**: Check available space before download/copy; warn user.
- **Concurrency**: Limit simultaneous downloads/copies to avoid disk thrash.
- **HF API Rate Limits**: Use token to increase limits; implement retry with backoff.
- **Database Corruption**: Use SQLite WAL mode; backup on exit.
--- ---
## 2. Architecture Diagrams ## 2. Architecture Diagrams
### 2.0 Entity Relationships ### 2.1 High-Level Component Diagram
```mermaid ```mermaid
erDiagram flowchart TD
FAMILIES ||--o{ MODELS : contains subgraph Console Application
MODELS ||--o{ REPOS : aggregates A[main.py] --> B[Uvicorn/FastAPI Server]
REPOS ||--o{ MODEL_FILES : contains end
STORAGE_LOCATIONS ||--o{ REPOS : hosts
REPOS ||--o{ JOBS : targeted-by subgraph Web Browser
MODELS ||--o{ JOBS : targeted-by C[UI - HTML/JS/CSS]
FAMILIES ||--o{ JOBS : targeted-by end
MODEL_FILES }o--|| REPOS : "scope folder may live under _shared/"
B <--> C
subgraph Background Processing
D[Background Task Manager<br>Thread Pool]
E[Download Manager]
F[Storage Manager]
end
B --> D
D --> E
D --> F
subgraph External
G[Hugging Face Hub API]
end
E <--> G
subgraph Storage
H[HDD Level 3<br>Multiple Roots]
I[SSD Level 2<br>Cache]
end
F --> H
F --> I
subgraph Database
J[SQLite]
end
B --> J
D --> J
``` ```
### 2.1 Components ### 2.2 Download Flow Sequence
### 2.2 Download Flow
```mermaid ```mermaid
sequenceDiagram sequenceDiagram
autonumber participant U as User (Browser)
participant U as User
participant API as FastAPI participant API as FastAPI
participant JS as Job Scheduler participant BW as Background Worker
participant DM as Download Mgr participant HF as Hugging Face API
participant HF as Hugging Face participant ST as Storage (HDD/SSD)
participant CQ as Copy Queue participant DB as Database
participant ST as Storage Mgr
participant DB as SQLite
U->>API: GET /api/repo/preview (url or org/name) U->>API: POST /api/download (repo_id, file selection, options)
API->>HF: model_info(files_metadata=true) API->>BW: Create background task
HF-->>API: files, sizes, LFS sha256 OIDs API-->>U: Return task ID
API->>DB: family OID lookup, suggest family/model
API-->>U: grouped list (role, variant, scope, dedupe flags) BW->>HF: list_repo_files(repo_id)
U->>API: POST /api/repos (family, model, files, roles, scopes, variants, root, cache) HF-->>BW: File list with metadata
API->>DB: ensure family/model rows BW->>BW: Classify files (manual via UI prior)
API->>DB: create repo (status=downloading) and job row BW->>HF: snapshot_download(allow_patterns, token)
API-->>U: job id HF-->>BW: Download files to temp folder
JS->>DM: run download job BW->>BW: Verify checksums
DM->>HF: snapshot_download(local_dir=_incoming, allow_patterns, revision, token) BW->>ST: Move temp folder to final HDD path (atomic rename)
HF-->>DM: resumable stream into _incoming BW->>DB: Insert model record + files
loop until download completes alt if cache requested
DM-->>U: SSE progress (bytes, speed) BW->>ST: Copy folder to SSD cache
end BW->>DB: Update is_cached=1, cache_path
DM->>DM: verify sizes and OIDs
DM->>ST: place files per scope (repo folder or _shared) via temp+rename
DM->>ST: dedupe by OID (hardlink same volume, copy cross volume)
DM->>DB: insert file rows, repo (status=complete)
alt cache requested
JS->>CQ: enqueue promote
CQ->>ST: copy repo folder to SSD .tmp
CQ->>ST: rename .tmp to final
CQ->>DB: is_cached = 1
end end
BW-->>U: (via WebSocket/SSE) Progress and completion
``` ```
### 2.3 Promote / Demote ### 2.3 Cache Promotion Flow
```mermaid ```mermaid
sequenceDiagram sequenceDiagram
autonumber
participant U as User participant U as User
participant API as FastAPI participant API as FastAPI
participant JS as Job Scheduler participant BW as Background Worker
participant CQ as Copy Queue participant ST as Storage
participant ST as Storage Mgr participant DB as Database
participant DB as SQLite
U->>API: POST /api/repos/123/cache (promote or demote) U->>API: POST /api/model/{id}/cache (checkbox)
API->>DB: create job row (queued) API->>BW: Queue cache/demote task
API->>JS: enqueue on copy queue API-->>U: Accepted
API-->>U: accepted (job id)
alt promote alt Cache to SSD
JS->>CQ: run promote BW->>ST: Check available space on SSD
CQ->>ST: free-space check on SSD BW->>ST: Copy model folder from HDD to SSD cache
alt insufficient space BW->>DB: Update is_cached=1, cache_path
CQ->>DB: job (failed) else Demote from SSD
CQ-->>U: SSE error (insufficient SSD space) BW->>ST: Delete cached folder
else space ok BW->>DB: Update is_cached=0, cache_path=NULL
CQ->>ST: copy repo folder to SSD .tmp
CQ->>ST: rename .tmp to final
CQ->>DB: is_cached = 1, set cache_path
CQ-->>U: SSE completion
end
else demote
JS->>CQ: run demote
CQ->>ST: validate HDD copy (files present, sizes match DB)
alt HDD invalid
CQ->>DB: job (failed, demote blocked)
CQ-->>U: SSE error (HDD copy missing or incomplete)
else HDD valid
CQ->>ST: delete SSD folder
CQ->>DB: is_cached = 0, cache_path = null
CQ-->>U: SSE completion
end
end end
BW-->>U: Completion notification
``` ```
### 2.4 Update Check ### 2.4 Update Check Flow
```mermaid ```mermaid
sequenceDiagram sequenceDiagram
autonumber
participant U as User participant U as User
participant API as FastAPI participant API as FastAPI
participant HF as Hugging Face participant HF as Hugging Face API
participant DB as SQLite participant DB as Database
U->>API: POST /api/repos/123/check-update U->>API: POST /api/model/{id}/check-update
API->>DB: load stored revision (commit hash) API->>DB: Retrieve stored HF commit hash
API->>HF: repo info (current sha) API->>HF: get_repo_info(repo_id)
HF-->>API: current commit HF-->>API: Current commit hash, last modified
API->>API: compare commits API->>API: Compare hashes
alt same commit alt hash differs
API-->>U: up to date API-->>U: Update available (new hash)
else commit differs else same
API->>HF: model_info(files_metadata=true) API-->>U: Up-to-date
HF-->>API: current files, sizes, OIDs
API->>DB: diff per-file OIDs against stored
API-->>U: list added, changed, removed
U->>API: POST /api/repos/123/apply-update (selected files)
API->>DB: create update_apply job
API-->>U: accepted (job id)
Note over API,DB: apply job downloads changed files to _incoming, swaps per file via temp+rename (scope-aware), updates rows and revision
end end
Note over U,DB: Models page aggregates update state across all repos of a model
``` ```
---
## 3. Test Plan & Test Cases ## 3. Test Plan & Test Cases
Strategy unchanged (unit / mock-HF integration / system / performance; retry delays configurable to seconds). TC-001…TC-029 from v1.1 carry over with target renames (model→repo where applicable: TC-016, TC-018, TC-019, TC-026). New/changed: ### 3.1 Test Strategy
- **Unit Tests**: For file role classification (basic extension-based), retry logic, family sharing decision, database operations.
- **Integration Tests**: Downloading from a mock HF server (local HTTP), storage operations on temporary directories (tmpfs).
- **System Tests**: End-to-end through API and UI using sample repositories (may be network-dependent; mocks used for speed).
- **Performance Tests**: Simulate large file downloads (using dummy files) to ensure UI responsiveness.
| ID | Scenario | ### 3.2 Test Cases
|----|----------|
| TC-030 | Model with 3 repos (official + 2 community) — merged per-model variant list correct; repos distinct folders | | ID | Test Scenario | Preconditions | Steps | Expected Result |
| TC-031 | Non-standard variant names (`APEX-Compact`, `I-Balanced`) — parsed or defaulted, editable, persisted; shards grouped under one tag | |----|---------------|---------------|-------|-----------------|
| TC-032 | Draft MTP attached (a) as role=speedup in main repo, (b) as separate repo component='draft' — both resolvable as draft target for future config maker | | TC-001 | List files from HF repo | Valid repo ID, mock HF API | Call `/api/repo/{repo_id}/files` | Returns list with filenames, sizes, roles inferred |
| TC-033 | mmproj scope=model — downloaded once; second repo of same model reuses via OID link/copy; stored in `{fam}/_shared/{model}/` | | TC-002 | Download a GGUF text model | Mock HF repo with GGUF file | User selects file, triggers download | File downloaded to chosen HDD, DB entry created, checksum verified |
| TC-034 | jinja scope=family — stored in `{fam}/_shared/`, shared across the family's models | | TC-003 | Download image model with multiple safetensors | Mock repo with main, VAE, CLIP files | User selects all files, assigns roles manually | All files downloaded, roles stored correctly |
| TC-035 | Add New suggests family=org, model=name; user can map to existing family/model instead | | TC-004 | Retry on transient failure | Simulate network error on first attempt, then success | Trigger download | First attempt fails, retry after delay (configurable), second succeeds |
| TC-036 | Delete guards: model with repos blocked; family with models blocked; repo delete removes folder+rows | | TC-005 | Checksum mismatch | Corrupt file after download | Download | App detects mismatch, deletes temp file, retries download |
| TC-037 | Reserved names rejected (`_shared`, `_incoming`, `CON`, trailing dots) for family/model/repo names | | TC-006 | Promote model to SSD cache | Model on HDD | Check "Cache to SSD" in UI | Model folder copied to SSD, DB updated, original intact |
| TC-038 | Two orgs, same repo name, under one model — second gets hash suffix, no overwrite | | TC-007 | Demote from SSD cache | Model cached | Uncheck "Cache to SSD" | SSD copy deleted, DB updated |
| TC-039 | Same HF repo submitted for a second model — rejected cleanly (UNIQUE) with clear message | | TC-008 | Family sharing | Two models in same family; shared mmproj already exists | Download second model | Shared file is hard-linked instead of downloaded again |
| TC-040 | URL with branch (`/tree/dev`) — revision pinned and stored; shorthand `org/name` accepted | | TC-009 | Update check | Model downloaded with older commit hash; HF has newer | Trigger update check | App reports update available |
| TC-010 | Graceful shutdown during download | Active download | Press Ctrl+C | Download completes (if within timeout), DB consistent, process exits |
| TC-011 | Multiple HDD storage | Two HDD roots configured | Download model, choose second root | Model stored in selected root |
| TC-012 | Token protected download | Private repo, valid token | Download with token | Download succeeds; token not logged |
| TC-013 | Invalid repo ID | Nonexistent repo | Provide bad ID | Error message returned, no crash |
| TC-014 | Low disk space | HDD nearly full | Start download | Warning displayed, download aborted before copying |
| TC-015 | Hard link creation on different drives | Attempt to hard link across different drives | Family sharing across storage locations | App falls back to copy (or disallows family sharing if drives differ) |
### 3.3 Test Environment
- Windows 10/11 machine with Python 3.11.
- Use temporary directories to simulate HDD and SSD (different drives if possible; else use folders).
- Mock Hugging Face server (e.g., `responses` library or a local Flask app) to avoid network dependency.
- For UI tests, use browser automation (Selenium or Playwright) to test basic interactions.
--- ---
## 4. Security & Privacy Assessment ## 4. Security & Privacy Assessment
Carried from v1.1 (loopback-only + Host-header check; `keyring` token; repo-ID regex + relative-path containment; try/finally temp cleanup; pinned deps + `pip-audit`; SQLite perms/WAL; log redaction; deep-verify as explicit command). **Added for v1.2:** folder-name sanitization treats all HF-derived strings (org, repo, filename) as untrusted — illegal chars, reserved device names, trailing dots/spaces, reserved app names (`_shared`, `_incoming`) — before any path is constructed; delete operations require explicit confirmation (destructive scope grew with hierarchy cascades). Privacy unchanged: no telemetry; HF metadata + transfers only; no content inspection beyond names/sizes/hashes. ### 4.1 Assets and Threats
- **Assets**: Model files (potentially proprietary or sensitive), Hugging Face token, application configuration, database metadata.
- **Threats**:
- Unauthorized access to web UI if bound to non-localhost interface.
- Token leakage through logs or configuration files.
- Path traversal attacks via malicious repo IDs or filenames.
- Supply chain attacks via compromised HF dependencies.
- Data loss due to race conditions or improper shutdown.
### 4.2 Security Measures
- **Network Binding**: By default, Uvicorn binds to `127.0.0.1` only. Settings can change to `0.0.0.0` with a warning, but no authentication is provided. For this single-user local tool, it's acceptable.
- **Token Storage**: Hugging Face token is stored encrypted using Windows Data Protection API (DPAPI). The encryption key is tied to the Windows user account. Implementation uses `cryptography` with `win32crypt` or `dpapi` package. The token is never logged.
- **Input Validation**: Repo IDs are validated against a strict pattern (e.g., `^[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+$`). File paths are constructed using `os.path.join` and checked to be within the allowed storage root using `os.path.commonpath`.
- **File Operations**: All file moves/copies are performed within known directories. Temporary files are deleted on failure. Use `filelock` to prevent concurrent modifications.
- **Dependency Management**: Use `pip-audit` in CI to scan for known vulnerabilities. Regularly update dependencies.
- **Database**: SQLite database file permissions set to user-only (Windows default). WAL mode ensures durability.
- **Logging**: Logs exclude tokens and full file paths if sensitive. Log level configurable.
### 4.3 Privacy Considerations
- **Data Collection**: The application does not collect or transmit any personal data. It only communicates with Hugging Face API for download and metadata.
- **User Activity**: The UI may store user preferences (e.g., last selected storage location) locally; no analytics.
- **Model Files**: Model files are user-provided content; the app does not inspect their contents beyond file names and sizes.
### 4.4 Compliance
- Not subject to GDPR or similar regulations as no personal data is processed. If private models are used, the token is the only sensitive credential; handled securely.
--- ---
## Open items (v1.2 decisions applied — veto freely) ## Summary
1. **Family required**, auto-suggested from org (TC-035) — alternative would be a `_none` pseudo-family. These documents provide a comprehensive design for the Model Manager as per your requirements. The system is modular, Windows-focused, and easy to extend. If you need further elaboration on any section or adjustments based on additional thoughts, please let me know.
2. **Monolithic image repos stay whole** under `component='main'` with internal structure preserved (not split into vae/… type folders).
3. **Variant parsing is heuristic + editable** — no perfect auto-detection for arbitrary naming schemes.
4. **Promote unit = repo folder**; shared-scope files (mmproj/jinja) resolve from their live location (L3 is fine — they're small). Promoting `_shared` contents is a future option.
5. **One HF repo = one model attachment** (global UNIQUE) — a repo truly serving two models isn't representable in v1.
6. Repo folder = repo name (+hash on collision); `_shared`/`_incoming` reserved.
7. Config maker still awaiting your details — the schema now gives it everything it needs (main GGUF + variant, mmproj path via scope, draft target via role or component='draft').
Natural next steps: (a) implementation plan — module skeleton + build order, or (b) spec the llama.cpp config maker now that the data model can feed it. Which do you want?