Doc: Updated Design Doc

doc/DESIGN-DOC.md

@@ -208,7 +208,7 @@ Carried from v1.1 (large copies → queue; space pre-checks; capped pools; token
 
 ## 2. Architecture Diagrams
 
-### 2.0 Entity Relationships (new)
+### 2.0 Entity Relationships
 ```mermaid
 erDiagram
     FAMILIES ||--o{ MODELS : contains
@@ -221,43 +221,120 @@ erDiagram
     MODEL_FILES }o--|| REPOS : "scope folder may live under _shared/"
 ```
 
-### 2.1 Components — unchanged from v1.1 (Storage Manager now hierarchy/path-scope aware).
+### 2.1 Components
 
 ### 2.2 Download Flow
+
 ```mermaid
 sequenceDiagram
+    autonumber
     participant U as User
     participant API as FastAPI
     participant JS as Job Scheduler
     participant DM as Download Mgr
     participant HF as Hugging Face
-    participant ST as Storage
+    participant CQ as Copy Queue
+    participant ST as Storage Mgr
     participant DB as SQLite
 
     U->>API: GET /api/repo/preview (url or org/name)
     API->>HF: model_info(files_metadata=true)
-    HF-->>API: files + sizes + LFS OIDs
+    HF-->>API: files, sizes, LFS sha256 OIDs
-    API->>DB: family OID lookup + suggest family/model
+    API->>DB: family OID lookup, suggest family/model
     API-->>U: grouped list (role, variant, scope, dedupe flags)
-    U->>API: POST /api/repos (family/model targets, files, roles, scopes, variants, root, cache?)
+    U->>API: POST /api/repos (family, model, files, roles, scopes, variants, root, cache)
-    API->>DB: ensure family/model rows; repo(status=downloading) + job row
+    API->>DB: ensure family/model rows
+    API->>DB: create repo (status=downloading) and job row
     API-->>U: job id
-    JS->>DM: run
+    JS->>DM: run download job
-    DM->>HF: snapshot_download(local_dir=_incoming/{repo_id}, allow_patterns, revision, token)
+    DM->>HF: snapshot_download(local_dir=_incoming, allow_patterns, revision, token)
-    HF-->>DM: resumable stream
+    HF-->>DM: resumable stream into _incoming
-    DM->>ST: place files per scope (repo folder / _shared/{model} / _shared) via temp+rename; dedupe by OID (link/copy)
+    loop until download completes
-    DM->>DB: files rows + repo(status=complete)
+        DM-->>U: SSE progress (bytes, speed)
-    alt cache requested
+    end
-        JS->>ST: enqueue promote (copy queue)
+    DM->>DM: verify sizes and OIDs
-        ST->>ST: copy → SSD .tmp → rename
+    DM->>ST: place files per scope (repo folder or _shared) via temp+rename
-        ST->>DB: is_cached=1
+    DM->>ST: dedupe by OID (hardlink same volume, copy cross volume)
+    DM->>DB: insert file rows, repo (status=complete)
+    alt cache requested
+        JS->>CQ: enqueue promote
+        CQ->>ST: copy repo folder to SSD .tmp
+        CQ->>ST: rename .tmp to final
+        CQ->>DB: is_cached = 1
     end
-    DM--)U: SSE progress
 ```
 
-### 2.3 Promote / Demote — as v1.1, unit = repo folder; demote gated on HDD validation.
+### 2.3 Promote / Demote
 
-### 2.4 Update Check — as v1.4 flow, per repo; model page aggregates repo results.
+```mermaid
+sequenceDiagram
+    autonumber
+    participant U as User
+    participant API as FastAPI
+    participant JS as Job Scheduler
+    participant CQ as Copy Queue
+    participant ST as Storage Mgr
+    participant DB as SQLite
+
+    U->>API: POST /api/repos/123/cache (promote or demote)
+    API->>DB: create job row (queued)
+    API->>JS: enqueue on copy queue
+    API-->>U: accepted (job id)
+    alt promote
+        JS->>CQ: run promote
+        CQ->>ST: free-space check on SSD
+        alt insufficient space
+            CQ->>DB: job (failed)
+            CQ-->>U: SSE error (insufficient SSD space)
+        else space ok
+            CQ->>ST: copy repo folder to SSD .tmp
+            CQ->>ST: rename .tmp to final
+            CQ->>DB: is_cached = 1, set cache_path
+            CQ-->>U: SSE completion
+        end
+    else demote
+        JS->>CQ: run demote
+        CQ->>ST: validate HDD copy (files present, sizes match DB)
+        alt HDD invalid
+            CQ->>DB: job (failed, demote blocked)
+            CQ-->>U: SSE error (HDD copy missing or incomplete)
+        else HDD valid
+            CQ->>ST: delete SSD folder
+            CQ->>DB: is_cached = 0, cache_path = null
+            CQ-->>U: SSE completion
+        end
+    end
+```
+
+### 2.4 Update Check
+
+```mermaid
+sequenceDiagram
+    autonumber
+    participant U as User
+    participant API as FastAPI
+    participant HF as Hugging Face
+    participant DB as SQLite
+
+    U->>API: POST /api/repos/123/check-update
+    API->>DB: load stored revision (commit hash)
+    API->>HF: repo info (current sha)
+    HF-->>API: current commit
+    API->>API: compare commits
+    alt same commit
+        API-->>U: up to date
+    else commit differs
+        API->>HF: model_info(files_metadata=true)
+        HF-->>API: current files, sizes, OIDs
+        API->>DB: diff per-file OIDs against stored
+        API-->>U: list added, changed, removed
+        U->>API: POST /api/repos/123/apply-update (selected files)
+        API->>DB: create update_apply job
+        API-->>U: accepted (job id)
+        Note over API,DB: apply job downloads changed files to _incoming, swaps per file via temp+rename (scope-aware), updates rows and revision
+    end
+    Note over U,DB: Models page aggregates update state across all repos of a model
+```
 
 ---