Commit Graph

1243 Commits

Author SHA1 Message Date
Aleksander Grygier dfd6ed8564 feat: Enable microphone input as default for audio models 2026-08-25 15:34:19 +02:00
Aleksander Grygier 7af64f3a29 ui : add raw model id tooltip to model selector options 2026-08-25 15:34:19 +02:00
Aleksander Grygier 8961962316 ui : make model option hover and focus highlight override the active state 2026-08-25 15:34:06 +02:00
Aleksander Grygier 2fab2f41b2 ui : move model list into a submenu within the model selector 2026-08-25 15:34:06 +02:00
Aleksander Grygier 4d6874bbda ui : add show-org-name-in-trigger display setting 2026-08-25 15:34:01 +02:00
Aleksander Grygier 3d0da6f4eb ui : keep reasoning submenu visible regardless of model state 2026-08-25 15:34:01 +02:00
Aleksander Grygier 2190337d6c ui : show reasoning and modality icons on model options and search by modality 2026-08-25 15:33:42 +02:00
Aleksander Grygier 7c8ece9a3a ui : strip trailing container-format segments from parsed model names 2026-08-25 15:33:37 +02:00
Aleksander Grygier f1357e4998 ui: ESLint config updates (#27700)
* chore: Spacing between sibling elements in html markup

* chore: Formatting and linting rules
2026-08-25 14:34:34 +02:00
YiChen Lv f280b26983 metal : per-device tuned (Q, NE) for flash-attn vec (#26570)
* metal : per-device tuned (Q, NE) for flash-attn vec (#25750)

* rebase Q-generic FA vec body from 01dc93607 (#23114)

* add 53 f16 (Q,NE) flash-attn vec instantiations (vec 80 -> 133)

* add FA vec (Q,NE) tuning table + dispatch wiring + SMEM cap fallback

* add  FA vec (Q,NE) perf sweep

* fill tuning result

* fold family table into a per-family representative SKU

* refactor tuning result format

* extend FA vec tuning to quantized KV caches

* sync fa vec tuner bucketing with runtime, use pointwise tuning regret

* update tuned table

* format and cleanup

* prefix fa_vec tuning procs with ggml_backend_metal_tuning_, drop unused fa_vec_override_active

* add device id -> token lookup for the offline tuning tool

* add ggml-metal-tuning skeleton

* add op-agnostic perf cell + median timing for the tuner

* add FA-vec graph build + tensor init to the tuner

* tools : add FA-vec (Q,NE) sweep, compression and table emit

* cool down and re-measure the dirty window on thermal drift

* test-backend-ops : replace the FA vec tune mode with a bounded (Q,NE) slice

* tools : document the Metal tuner, point the table comment at it

* abort on unknown KV type, single-source fa_vec_legal_ne

* cleanup

* honor -o in the FA vec (Q,NE) slice

* retune FA-vec (Q, NE) under a pointwise no-harm gate

* cont : add fa-vec tunings for M1 Pro, M2 Ultra, M5 Max

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2026-08-24 19:22:27 +03:00
Xuan-Son Nguyen 160c6b0bdd mtmd: video: fix moov atom at the end of file (#27596)
* mtmd: video: fix moov at the end of file

Co-authored-by: rkfg <rkfg@rkfg.me>

* fix SIGPIPE

* windows: handle broken pipe case

---------

Co-authored-by: rkfg <rkfg@rkfg.me>
2026-08-24 09:59:04 +02:00
Aleksander Grygier d05f89562d fix: Change chat tabs nav shortcuts (#27609) 2026-08-23 19:37:19 +02:00
Georgi Gerganov 8d9af25633 test : fix multi-GPU server tests (#27614)
* tests : fix tests for multi-gpu environment

* cont : not needed
2026-08-23 19:59:42 +03:00
Xuan-Son Nguyen 4a08fa2970 test: move tools/parser to tests (#27548) 2026-08-23 18:38:51 +02:00
Xuan-Son Nguyen 56db501e73 mtmd: use pillow-accurate algo, correct resize_algo for all models (#27594)
* mtmd: use pillow-accurate resize algo, correct resize_algo for all models

* speed optimization
2026-08-23 18:35:41 +02:00
Georgi Gerganov e8eed4525a server : add LLAMA_SERVER_SLOTS_N_DIFF (#27600) 2026-08-23 15:55:51 +03:00
Aleksander Grygier 8144f3192e ui: Chat Conversation Tabbed navigation (#27263)
* ui : add browser-style conversation tabs store

Track open conversation tabs in order, persisted to localStorage and
pruned against the loaded conversation list on init. The chat layout
syncs the route's tab on every navigation, so any way of reaching a
conversation opens a tab for it.

* ui : add temporary new-chat tabs

New-chat tabs are unsaved conversations carrying a temporary id used
directly as the route (#/chat/<id>). They live in memory and are only
persisted to the database - keeping the same id so the route and tab
stay stable - when the first message is sent. Deleting one drops it
without confirmation, and deleting conversations now closes their tabs.

* ui : render conversation tab bar in chat layout

Desktop-only tab bar above the chat screen, one tab per open
conversation or new-chat tab. The active tab follows the route id;
clicking navigates, middle-click or the close button closes (switching
to the left neighbor), and a trailing + starts a new chat. Tabs appear
only on chat-id routes; the bare #/ new-chat view has none. The bare
route stays put unless a prompt/model deep-link routes it to a new-chat
tab.

* ui : route new-chat entry points through tabs

The sidebar New chat item, Cmd+Shift+O, the search page and the
arrow-key fallback now open a new-chat tab instead of navigating to the
?new_chat URL, which is removed. New chat is no longer a special route
but a tab like any other conversation.

* ui : track sidebar expanded state in a shared ui store

Move the desktop sidebar expanded/collapsed state out of deviceStore into a
dedicated uiStore so the chat tab bar can react to it.

Assisted-by: pi

* chat : add opt-in conversation tabs setting

Add a Display setting that turns browser-style conversation tabs on or off,
enabled by default.

Assisted-by: pi

* chat : add browser-style conversation tabs with a new-chat screen

Track open conversations as tabs above the chat, one per open chat, plus a
single New chat tab for the bare `#/` route. New chat is just the `#/`
screen - no temporary conversations - and its tab is dropped when navigating
away. Sending the first message creates a real conversation and opens a tab
for it.

Assisted-by: pi

* chat : turn tab bar into a horizontally scrollable carousel

Make the tab bar a horizontally scrollable carousel with edge scroll buttons
and active-tab centering, and align its styling with the sidebar.

Assisted-by: pi

* chat : restyle the scroll-to-bottom button to match tab styling

Assisted-by: pi

* chat : add close-tab keyboard shortcut

Assisted-by: pi

* chat : soften tab bar fade and dim inactive tabs

Assisted-by: pi

* feat: Add stop button to tabs

* refactor: Componentize

* ui : fix carousel scrollability detection

Observe the content wrapper as well as the container, since adding overflowing items does not change the container's own box size. Also expose an onScrollableChange callback.

Assisted-by: pi

* ui : add unified ScrollCarousel component

Single carousel component with top/center variants, gap and scroll options, and hover-revealed chevrons. Rename the HorizontalScrollCarousel accessibility story accordingly.

Assisted-by: pi

* ui : migrate carousels to ScrollCarousel

Switch the settings mobile header, attachments list, thumbnail strip, and MCP resources to the unified component, and drop HorizontalScrollCarousel.

Assisted-by: pi

* ui : improve chat tabs carousel UX

Scroll newly added tabs into view, fade overflowing tabs at the edges, and hide the New chat button while a new-chat tab is open.

Assisted-by: pi

* refactor: Naming

* chat : add keyboard shortcut to jump between conversation tabs

Shift+Cmd/Ctrl+Left/Right cycles the open tabs, mirroring the existing
Shift+Cmd/Ctrl+Up/Down conversation navigation.

Assisted-by: pi

* chat : make the whole tab item act as a link

The full tab is now a link instead of only the inner label button, while
the stop and close buttons stay interactive by swallowing their clicks.

Assisted-by: pi

* chat : adjust tab bar width and use a shared offset variable

Widen the tab bar for the expanded sidebar and rename the tab bar height
variable to --chat-tabs-offset with a smaller value so the chat screen
min-height accounts for the overlay without overshooting.

Assisted-by: pi

* chat : account for the tab bar offset in the assistant min-height

Subtract the tab bar offset when it is shown so the last assistant message
does not overflow the available viewport space.

Assisted-by: pi

* refactor: Post-review fixes

* ui : restore deep links on the chat start page

- handle ?model selection, with ?load=true eager router loading
- ?q now creates a conversation, sends the prompt, and clears the params
- show the not-available-model dialog for unknown models
- never block mount on the conversation list

Assisted-by: pi

* ui : fix tab item link nesting and centralize tab constants

- the tab anchor covers the whole item while stop/close stay siblings,
  so interactive elements are never nested inside the anchor
- cmd/ctrl/middle clicks are left to the browser (new window)
- extract the tab labels, the active-tab data attribute, and the
  sidebar-offset max widths into constants

Assisted-by: pi

* ui : tidy scroll carousel hook and keep mobile header arrows on

- drop the dead scrollLeft/scrollRight helpers and the unused
  onScrollableChange/scrollBy props
- init the carousel once instead of inside a derived
- restore items-start on the center variant
- always show the settings header arrows on touch

Assisted-by: pi

* ui : keep the new-chat tab across reloads and fall back on close

- the new-chat sentinel is no longer pruned on init, so reloading on
  the bare new-chat route keeps the tab the user is on
- closing the active conversation falls back to the new-chat screen
  when Conversation tabs are off

Assisted-by: pi

* ui : don't block startup on the conversation list

- prune persisted tabs after the list loads in the background instead
  of awaiting it during init
- openNewChat now returns void; its return value was never read

Assisted-by: pi

* ui: fix routing nits

* chore: Update doc comments

* refactor: Mark fire-and-forget openNewChat calls as `void`

* chat: fix the deep-linked prompt, the tab width and the tab shortcuts

The chat start page creates the conversation and hands the prompt over
to the chat route, which still sees it in the query string. Sending it
on both sides queues the second copy as a pending message, which shows
up as a stray user bubble once the answer lands and vanishes on reload
since it never reaches the database.

The tab bar takes the max width of the collapsed sidebar while it is
expanded, and the other way round.

The tab list is pruned against a snapshot of the loaded conversations,
so a conversation created while that list is still loading loses its
tab even though the route just opened it. The active tab then falls out
of the list and the cycling shortcut jumps to an edge on every keypress
instead of moving one tab over. Tabs synced from the route are kept as
they are, only the persisted ones are pruned.

The rich chat input claims ctrl or alt with shift and an arrow for its
badge-aware word jump, which now belongs to the tab cycling shortcut.
Holding shift hands the key combination over, the plain word jump is
unchanged.

The close-tab shortcut consumes the event before checking whether the
setting is on, and the logo background loses its importance flag.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-08-23 10:46:49 +02:00
Xuan-Son Nguyen b21e4de745 mtmd: use ggml_rope_set_offset (#27521)
* mtmd: use ggml_rope_set_offset

* add comment
2026-08-22 16:33:47 +02:00
Xuan-Son Nguyen d9f918d2d0 common: add json.h abstraction (#27511)
* add common/json

* migrate common

* adapt jinja

* migrate server

* big wip

* migrate tests

* wip

* revert some excessive changes

* wip

* wip 2

* revert redundant changes

* fix server crash

* various fixes

* fix ci

* harden a bit

* clean up

* rm json-shim

* add some comments

* rm redundant decl
2026-08-22 16:28:28 +02:00
Xuan-Son Nguyen 2fb989b9e7 fit: also take into account n_streams (#27496)
* fit: also take into account n_streams

* server: make the draft context follow the target context

With a non-unified KV cache the target context now holds n_ctx_train
tokens per sequence, while the draft context was still created with
n_ctx = 0 and fell back to n_ctx_train / n_streams per sequence. A slot
filled beyond that point makes the draft batch fail to decode, and the
server answers 500 on the request.

The draft context now takes its size from the target context, so both
hold the same number of tokens per sequence. Contexts that share their
cells with the target no longer need the kv_size override.

The memory reserved for the draft model before fitting is measured at
the largest context the target can take, since the draft context grows
with the target and a fixed byte margin cannot express that.

* fit: take an optional second model into account

Illustrates the alternative discussed on the draft context fix. The
memory of a draft or MTP context is currently handed to the fit as a
fixed byte margin, which cannot express a memory that grows with the
context the fit is still deciding on.

common_fit_params now takes an optional second model that shares the
devices of the main one. Its context follows the main context and its
memory is measured again whenever that context changes, so the reduce
path stays exact instead of conservative. A model that cannot be
measured on its own, such as a shared cell MTP context, is skipped with
a warning and the main model is fitted alone.

This drops the reservation block in the server, which no longer has to
probe the trained context size of the target to guess an upper bound.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-08-22 16:16:06 +02:00
Xuan-Son Nguyen 54ee5ee643 mtmd: support dots3-note vision+audio (#27524)
* text: conversion

* init impl

* mtmd: conversion

* impl mtmd cpp

* Update gguf-py/gguf/tensor_mapping.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-08-22 10:35:50 +02:00
Xuan-Son Nguyen d775b8967a mtmd: support webp via ffmpeg (#27520) 2026-08-22 01:38:05 +02:00
Aleksander Grygier 5b6ddc9675 ui: Settings navigation cleanup (#27241)
* ui : rework the settings registry into ordered raw-data sections

SETTINGS_REGISTRY becomes an ordered SettingsSectionEntry[] array; the
array order is the sidebar display order. Section titles, color mode
options and title radio options are declared inline in their section or
entry. Entries gain showInUi; MCP servers, the system-message toggle and
the title LLM flag become hidden entries of their own section.

Derived values (config defaults, help info, chat sections, numeric field
lists, syncable parameters) are still derived here; they move to their
actual consumers in follow-up commits.

* ui : extract settings localStorage persistence into SettingsService

Stateless load/save of the settings config and user-override keys, plus the
legacy theme key migration. Business logic (default merging, mobile
sendOnEnter default, applying the migrated theme) stays in the store.

* ui : move the settings exit route into ROUTES

SETTINGS_FALLBACK_EXIT_ROUTE is just a route, so it lives with the other
routes as ROUTES.SETTINGS_EXIT.

* ui : derive the syncable parameter list in the parameter sync service

The syncable parameter mapping is only consumed by the sync service, so
derive it there from the registry instead of exporting it from the
constants file.

* ui : restore isPrivate for API key masking

* ui : clean up settings registry and router fetch guard

Drop the per-entry section field (duplicates the parent slug and is
never read) and guard the router model fetch on fields?.length so the
Tools/Import-Export pages with empty fields are excluded again.

Assisted-by: pi

* ui : merge sampling and penalties settings into one section

Assisted-by: pi
2026-08-21 12:30:03 +02:00
Aleksander Grygier 521a64cd01 ui: Stores split refactor (#27240)
* ui: Extract server stream lifecycle from chatStore into ChatStreamManager

Discovery, attach/replay, resume retry and the remote-running snapshot
formed a cohesive cluster inside chatStore. It now lives in
chat-streams.svelte.ts as ChatStreamManager, owned by chatStore, which
keeps the public entry points as delegates so components are
unchanged. chatStore: 2877 -> 2418 lines.

* ui: Extract user interaction gates from agenticStore into AgenticGates

Tool permission requests, turn-limit continue prompts and queued
steering messages are the state the loop waits on between turns. They
had no coupling to session state, so they now live in
agentic-gates.svelte.ts; agenticStore keeps delegates so components
are unchanged. agenticStore: 1196 -> 1073 lines.

* ui: Compose MCP resources under mcpStore.resources

Resource state was a second import scope next to mcpStore. Consumers
now go through mcpStore.resources, so the MCP surface is one store;
mcp-resources.svelte.ts stays a separate file owned by mcpStore.

* ui: Reorganize stores into domain namespaces

* fix: Update stale doc comments

* ui: Consolidate conv running-state into a chat activity ledger

Running-state was split across chatStore.chatLoadingStates (local
pipes), ChatStreamManager.remoteRunningConvs (backend sessions) and
attachingConvs (attach lifecycle), unioned by hand in
getAllLoadingChats and cross-cleaned by setChatLoading calling
streams.clearRemoteRunning - the 'spinner ghosts until tab toggle'
workaround.

chatActivityStore now owns both sets with one transition per event:
markLocal / localEnded (local pipe end also drops the stale remote
hint, no cross-owner call) / applyRemoteSnapshot (diffed). The
sidebar reads chatStore.activity.loadingConvs through the unchanged
getAllLoadingChats entry point.

Consequences:
- isStreamingActive and its five manual writers are gone; isStreaming()
  now reports whether the active conversation has a live streaming
  pipe, which is what all four consumers (assistant row, stop action,
  context gauge, chat screen) actually check
- isLoading/isReasoning become derived from the per-conv maps plus
  the active conversation, dropping the manual resync in
  syncLoadingStateForChat and clearUIState
- attachingConvs and the last-attach coordination disappear from
  ChatStreamManager
- getAllStreamingChats (no consumers) is removed

* ui: Give store collaborators narrow host interfaces

Collaborators took 'host: typeof <store>', i.e. the store's entire
public surface, which is how chatStore's streamChatCompletion,
createAssistantMessage, getApiOptions and setStreamingActive got
widened to public. Replace with per-collaborator interfaces carrying
only the members each one drives:

- ChatStreamHost (chat/streams) - activity, processing, streaming
  states, abort controller, loading/streaming setters
- ChatFlowsHost (chat/flows) - streaming core, message creation,
  per-conv state setters
- McpHealthHost (mcp/health) - connection registry + reconnection
- ModelPropsHost / ModelStatusHost (models) - model rows, feed
  updates; the managers write modalities/status back onto the host's
  rows, so those members stay writable
- ConversationsPreferencesHost (conversations) - the active row and
  the conversation list

The store classes now declare 'implements <Host>' so the contract is
visible at the class level, and the 'import type { <store> }' back
references in the collaborators disappear entirely - the host
contract is local to each collaborator file, and collaborators can
no longer reach around their slice. Members stay public (structural
typing), but the collaborator side is now compiler-enforced.

* test: Chat Activity store test

* refactor: Cleanup

* chore: Remove legacy architecture docs

* ui: Memoize findMessageIndex for the streaming hot path

Streaming looks up the same message index on every chunk, a linear
scan of activeMessages each time. Cache the last lookup and reuse it
after validating the id still sits at the same position (O(1)); any
structural change to the array fails validation and falls back to a
full scan.

* ui: Throttle per-chunk stream state writes to localStorage

saveStreamState ran JSON.stringify + a synchronous localStorage.setItem
on every decoded chunk of the stream. The read loop now goes through a
new saveStreamStateThrottled (one write per conversation per 500ms,
latest value held pending); the public saveStreamState keeps its
immediate-write contract for stream start and pre-fetch, and also
resets the throttle window.

A pending offset is force-flushed at resume boundaries (resumeStream
reads the offset back from localStorage), on visibilitychange->hidden
and on pagehide, so a reload always finds a usable offset. The resume
offset only needs to be roughly current since the server retransmits
from a line boundary and the client discards its partial line.

Adds unit tests for the throttled/flush/clear interplay.

* ui: Compute context gauge timing stats in one pass

currentRead/Fresh/Cache/Output were separate deriveds, each running a
full reverse scan of activeMessages for the last assistant timings,
and cumulative ran its own forward scan plus an agentic filter - 4-5
O(n) passes per chunk while streaming. Replace with a single
summarizeAssistantTimings() pass (last assistant timings, last
agentic llm totals and the cumulative sums) feeding a shared derived
snapshot. Semantics unchanged, including the live-stats overrides and
the agentic llm-totals branch.

* agentic : clear session state when a conversation is deleted

Every conversation that ran an agentic flow left an AgenticSession in the
store forever; clearSession was never called. conversationsStore now
notifies deletion listeners and agenticStore drops the matching sessions,
avoiding a circular import back into conversationsStore.

* chat : extract ChatService.normalizeMessagesForApi

The DB->API message normalization (convert + drop empty system messages)
was duplicated in sendMessage, preEncode and the agentic flow. Extract it
into one shared method and call it from all three.

* sse : share record splitting and data extraction

splitSseRecords and extractSseDataPayload centralize the record-boundary
splitting and data: line extraction used by parseSseJsonStream and the
models status feed. chat.service keeps its own line-based parser for
resume support.

* api : delegate apiFetchWithParams to apiFetch

apiFetchWithParams duplicated apiFetch's headers/fetch/error handling
body-for-body; it only differs in URL construction. Build the URL and
delegate.

* chat flows : dedupe title, timings and cleanup handling

- conversationsStore.applyTitleFromContent centralizes the title-from-first-
  message logic duplicated in 5 places
- ChatProcessingStore.applyStreamTimings centralizes the onTimings handler
  shared by the chat and continue flows
- host.cleanupStreaming centralizes the loading/streaming/processing reset
  repeated across the continue flow's exit paths

* conversations : centralize conversation update mirroring

rename, pin, mcp override, reasoning effort and cwd all repeated the same
write-DB-then-mirror-into-list-and-active dance. A single
applyConversationUpdate(id, updates) on the host collapses all five and
removes the forgot-to-mirror-one-field bug class. Drops the redundant
array reassignment in setCwd (deep  field assignment is reactive).

* mcp : dedupe tool execution, server parsing and tool indexing

- executeTool delegates to executeToolByName (only diff was argument parsing)
- drop the private #parseServerSettings copy; use parseMcpServerSettings
- cache getServers() keyed on the raw config value (hot path)
- indexServerTools() unifies the three identical toolsIndex rebuild loops

Assisted-by: Claude

* mcp : share cursor pagination and tool indexing

- MCPService.paginate() collapses the identical do-while loops in
  listAllResources and listAllResourceTemplates
- promoteHealthCheckToConnection now uses indexServerTools like the other
  connect paths

Assisted-by: Claude

* database : share message parent-child bookkeeping

- addChildToParent() dedups the append-to-children update in createMessageBranch
  and createSystemMessage
- removeChildFromParent() dedups the remove-from-children cleanup in deleteMessage
  and deleteMessageCascading
- bulkAdd the cloned messages when forking a conversation instead of one add
  per message

Assisted-by: Claude

* chore: Lint/format

* fix: `pagehide` event from `window`

* refactor: Api Fetch util

* docs : rewrite architecture sections in README

Update the high-level diagram, routes, hooks, stores, services and data
flow tables to match the current UI structure (mcp/settings/search
routes, agentic/tools/mcp stores, MCPService/ToolsService/SandboxService,
/tools API). Fix stale architectural patterns for per-conversation state
and modality validation.

* chore : add ESLint rule for blank lines between accessors

Enforce a blank line between consecutive class accessors. The core
padding-line-between-statements rule does not cover class members, so a
local rule is needed.

* refactor : reorder store members and unify naming

Order store class members as public fields, private fields, constructor,
getters, public methods, then private methods. Normalize private naming
to the `private` keyword (drop `#` and the `_` prefix where there is no
matching public getter). Rename conversationsStore.init() to
initialize() to match the other stores.

* refactor : prefix lookup methods with get in agentic and chat stores

Unify bare-name lookup methods with the get* prefix used across the
other stores (mcp, models, tools, settings). Renames currentTurn,
totalToolCalls, lastError, streamingToolCall, executingToolCallId,
pendingPermissionRequest, pendingContinueRequest,
pendingSteeringMessageContent, pendingSteeringMessageExtras in the
agentic store and pendingMessageContent, pendingMessageExtras in the
chat store. Updates the two consuming components and a doc comment.

* refactor: Clean up comments in stores' and services' code

* chore : add ESLint rule for class member ordering

Enforce structural order (public fields -> private fields -> constructor ->
getters -> setters -> public methods -> private methods) with alphabetical
sorting within each group via perfectionist/sort-classes. Dependency
detection keeps Svelte $derived fields in a valid dependency order instead
of alphabetizing them, since Svelte rejects forward references.

Assisted-by: Claude

* refactor : reorder class members to match new ESLint rule

Apply the sort-classes rule across stores, services, hooks and utils.
Pure reordering - verified no logic changes by comparing sorted line
multisets before/after. All tests and svelte-check pass.
2026-08-20 19:02:04 +02:00
John-Henry Lim 681c29d36a mtmd: add --mmproj-device argument (#23255)
* feat: add --mmproj-device arg & backwards compatible MTMD_BACKEND_DEVICE env var

* feat: load mmproj device backend immediately, add -mmdev shortflag

* fix: its a pointer now get the name

* clean up

* gen docs

* nits

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-08-20 18:45:37 +02:00
Xuan-Son Nguyen 9855ad69d3 server: (router) lazy-load startup_models after main setup (#27424)
* server: (router) lazy-load startup_models after main setup

* only allow is_first_load to populate it

* nits

* nits 2
2026-08-20 15:22:16 +02:00
Aritro Bandyopadhyay 8a832e4bf3 server : fix --docker-repo being treated as router mode (#27416) 2026-08-20 14:37:14 +02:00
Xuan-Son Nguyen 947fd9bb2b server: refactor sleep handling, allow access /metrics during sleep (#27376)
* add cached responses

* refactor on_sleeping_state

* allow accessing metrics during sleep

* metrics task should not reset timer

* updated docs

* fix

* fix get_res_model_info

* add test

* fix a race condition

* split metrics and slots tasks / results

* should_reset_buckets
2026-08-19 20:48:09 +02:00
s0mecode ee0ea03adf server : make models endpoints private when authentication is enabled (#26347)
* server : make models endpoints private when authentication is enabled

* tests : fix models endpoint auth
2026-08-19 20:44:42 +02:00
Pascal 77acca437f ui: read persisted settings before the API key probe (#27365)
The route loads run ahead of the root layout script, so validateApiKey
read the settings store while it still held factory defaults and probed
/props without the stored key. initStores() now hands the same startup
promise to every caller and the chat loads await it before probing.

The one-time admin baseline no longer overwrites a key the user has
already set: on a first visit the config carries factory values only, so
a diverging key comes from the user and wins.
2026-08-19 14:02:08 +02:00
Xuan-Son Nguyen 95c409c136 mtmd: add mtmd_bitmap_set_mergeable (#27348) 2026-08-19 13:48:22 +02:00
Xuan-Son Nguyen ee4c505a4f server: add dedup-cache-models preset option (#27346) 2026-08-19 11:04:26 +02:00
Xuan-Son Nguyen 6d05498314 server: (cosmetic) do not print cmd_child_to_router messages [no release] (#27347) 2026-08-19 01:45:56 +02:00
Saba Fallah 9731ad3f29 mtmd: deepseek-ocr SAM ggml_conv_2d with the im2col kept in F32 (#26727) 2026-08-18 19:05:19 +02:00
Aleksander Grygier 3dc7285b4f ui: Services consolidation refactor (#27239)
* ui: Move stream lookup and replay fetches into ChatService

chatStore called fetch() directly for /v1/streams/lookup and the
/v1/stream replay. These now live next to the other stream-session
methods in ChatService, so services stay the only API I/O layer.

* ui: Move /models/sse feed reader into ModelsService

ModelsService.watchModelEvents owns the byte stream, reconnect loop
and SSE record parsing; modelsStore keeps only event routing and
state.

* ui: Extract conversation import/export into ConversationTransferService

The JSONL session format, ZIP archiving and browser downloads are
pure I/O with no store state, so they move out of
conversationsStore. The store keeps the DB orchestration
(bulkExportConversations, downloadConversation,
importConversationsData) and delegates the format work.

* ui: Consolidate active model resolution into modelsStore.activeModelId

The same resolution chain was duplicated in useChatScreenActiveModel,
ChatForm, ChatFormActionModels and contextStatsStore, with slight
drift in the single-model fallback. The canonical getter now lives in
modelsStore, and the shared last-assistant-model lookup moved to
utils as getConversationModel.

* ui: Initialize stores explicitly via initStores()

Store constructors and module-level side effects ran migrations and
localStorage reads in import order. Migrations rename and rewrite
localStorage keys, so a settings load racing ahead of them could
clobber migrated values. initStores() is called once from the root
layout and runs migrations first, then the stores that read
localStorage, then the conversations DB load.

* refactor: Constants for stream query params
2026-08-18 16:39:32 +02:00
Aleksander Grygier fdf4c64604 ui: Stores consolidation refactor (#27238)
* ui: Remove dead code from stores

- persisted() helper was exported but never used
- messageUpdateCallback / registerMessageUpdateCallback were never wired up
- conversationsStore.initialize() alias, single caller moved to init()

* ui: Merge device, theme and viewport into a single deviceStore

All three are reactive browser-environment signals, now exposed as one
class store: deviceStore.isMobile, deviceStore.isIOSDevice / isIOSSafari
/ isWKWebView / isStandalone and deviceStore.systemTheme.isDark. The
systemTheme name disambiguates the OS preference from the user theme
preference in settingsStore. Drops the unused viewport export (only
isMobile was consumed).

* ui: Merge build info into version store

One VersionStore class with build (llama.cpp build number from
build.json) and frontend (PWA version from _app/version.json),
matching the class pattern of the other stores.

* ui: Colocate context gauge popup state with its components

The gauge popup state is local UI state shared only by the
ChatFormContextGauge subtree, so it lives next to its consumers
instead of the app-scope stores barrel.
2026-08-18 16:37:26 +02:00
Ed Addario 0596704284 quant : Optimise memory usage by evicting weights after processing each layer (#22877)
* Evict weights from memory after processing each layer

* Revert changes

* Move unmap to libllama

* Unmap weights offloaded to backend

* Change member's constness

* Remove unmap weights offloaded to backend
2026-08-18 16:22:32 +02:00
BlackFoil 7acdbb1f19 mtmd: fix LFM2 image tiling threshold (#27057)
* mtmd: fix LFM2 image tiling threshold

* refactor testing

* fix

* fix on windows

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-08-18 11:11:19 +02:00
Georgi Gerganov 27e345b574 build : fix xcframework + cmake clean-up (#27304)
* xcframework : fix build

* mtmd : remove unused include path

* vendor : use vendor::hash alias target in cmake

CMake reserves "::" in target names for imported/alias targets, so the real
target keeps the name vendor-hash and a vendor::hash ALIAS target is added.
Consumers (mtmd, llama-gguf-hash) now link against the namespaced alias.

Assisted-by: pi:llama.cpp/Qwen3.8-27B

* vendor : add cmake targets for all vendored libs with vendor:: aliases

Add INTERFACE targets for the header-only vendor libs (miniaudio, nlohmann,
sheredom, stb) and ALIAS targets named vendor::<lib> for all of them,
including cpp-httplib and hash. Each exposes the vendor/ root so includes
are namespaced, e.g. <nlohmann/json.hpp>.

Consolidate the per-lib add_subdirectory calls into a single
add_subdirectory(vendor), keeping the cpp-httplib gate on LLAMA_BUILD_COMMON.
Consumers (llama-common, mtmd) now link the aliases instead of relying on
raw vendor/ include paths.

hash: consumers now include via "hash/hash.h"; the vendor/hash dir is kept
as a PRIVATE include so the synced upstream sources compile unmodified.

Assisted-by: pi:llama.cpp/Qwen3.8-27B

* readme : use foo/bar names in acknowledgements

Assisted-by: pi:llama.cpp/Qwen3.8-27B

* ocd : fix valign
2026-08-18 11:16:51 +03:00
Aleksander Grygier 01818e4956 ui: enforce alphabetical enum member ordering (#27272) 2026-08-17 23:52:00 +02:00
Aleksander Grygier 0021a77de0 ui: Refactor Built-In Tools naming (Server/Browser) (#27271)
* server: rename built-in tools to server tools

* ui: rename built-in tools to server/browser tools
2026-08-17 22:23:22 +02:00
Pascal 087f94d82e doc: document MCP stdio servers and CORS defaults in the server README [no release] [no ci] (#26847)
* doc: document MCP stdio servers and CORS defaults in the server README

The MCP arguments were listed but nothing explained what an MCP server
is or how to declare one. Cover the stdio transport, the config keys,
the tool naming, and add a POSIX shell echo server as a minimal
example.

Also document the CORS behavior: the default reflected origin, the
switch to localhost once tools are enabled, and the recommended setting
per deployment.

* doc: drop the inline MCP shell example from the server README

The example parsed JSON-RPC by hand and sat in a page people copy paste
from, into servers spawned with the privileges of llama-server. Point to
the specification instead.

Link the pull request that introduced the feature, and keep a short
mcp.json snippet so the table of configuration keys has a declaration to
refer to.
2026-08-17 21:32:15 +02:00
Xuan-Son Nguyen 533b18257b server: save processed mtmd chunks as placeholder (#27278) 2026-08-17 21:29:50 +02:00
Xuan-Son Nguyen ed1c3a20f5 mtmd: use sha256 for input hashing (#27274)
* mtmd: use sha256 for input hashing

* void conflict with boringssl
2026-08-17 21:20:12 +02:00
Tarek Dakhran b75ecd1971 mtmd : skip thumbnail for non-tiled LFM2 images (#27246)
* mtmd : skip thumbnail for non-tiled LFM2 images

Matches HF behavior.

* Update tools/mtmd/mtmd-image.cpp

---------

Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com>
2026-08-17 19:41:32 +02:00
Xuan-Son Nguyen 666f8898a2 ui: move get_datetime tool to frontend (#27255)
* ui: move get_datetime tool to frontend

* clarify docs

* server: drop the now unused ctime include

strftime() and gmtime_r() were the only users, both went away with the
get_datetime tool. Also make the renderer's catch inert: the browser
executor always emits JSON, so a non-JSON result is no longer a date to
display.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-08-17 14:33:55 +02:00
Xuan-Son Nguyen 7077abbe14 ui: add browser get_info tool (#27251)
* ui: add browser get_info tool

* format

* nits

* Update tools/ui/src/lib/stores/tools.svelte.ts

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>

---------

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
2026-08-17 13:14:33 +02:00
Xuan-Son Nguyen 9f0d017efb mtmd: harden preprocessor_granite (#27235)
* mtmd: harden preprocessor_granite

* simply cap to 1024
2026-08-17 11:30:15 +02:00
Fathi Boudra 4695f001fe llama-bench: fix deprecation warnings missing trailing newline (#27179)
The log output does not append a newline, so the warning ran into the
next line printed on stdout, corrupting the benchmark table header.

Signed-off-by: Fathi Boudra <fathi.boudra@linaro.org>
2026-08-17 07:33:24 +03:00
Cristiano Pinto ece963f41b ui: mask API Key field in settings and error splash to stop browser a… (#26562)
* ui: mask API Key field in settings and error splash to stop browser autofill

* ui: set autocomplete=new-password on private fields

The password input type makes browsers offer to save the API key
in the password manager and autofill saved site credentials into
the field. The new-password autocomplete value disables both.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-08-15 22:52:55 +02:00