mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-09-19 09:15:04 +02:00
ggml: add cross-backend profiler
Add an optional per-op / per-copy profiler to the ggml scheduler that records timed events across all backends of a split graph, so a single run can be inspected end to end (compute kernels, host<->device copies, fusion names, tensor shapes/strides/types, op params). - ggml-profiler.h/.cpp: ggml_profile_record, per-backend profiler interface (enable/reset/get_records), JSON export - ggml-backend.cpp: scheduler-level collection, copy events, backend attribution, mul_mat_id stats, throughput stat, concurrent-mode fix, auto-export via GGML_PROFILE env var - Backend profilers: CPU, CUDA/HIP/MUSA (event-based timing), Vulkan (timestamp queries), BLAS, Metal (tentative); stubs for the remaining backends - llama: expose profiler enable/export; --profile, --profile-output, --with-backends args in common; hooks in server, completion and the debug example - tools/profiler/profiler.py: analysis tool (per-op / per-backend summaries, Chrome trace export) - test-backend-ops / test-export-graph-ops: run perf tests with exactly the tensor shapes recorded in a profile (converged with export-graph-ops) - docs/cross-profiler.md - ggml-cuda: avoid ROCm_Host compute on HIP integrated GPUs Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ney1sm8n1bSjeA3DrrW5ah
This commit is contained in:
@@ -3,6 +3,7 @@
|
||||
|
||||
#include "build-info.h"
|
||||
#include "common.h"
|
||||
#include "ggml-profiler.h"
|
||||
#include "fit.h"
|
||||
#include "log.h"
|
||||
#include "llama.h"
|
||||
@@ -1401,6 +1402,14 @@ common_init_result::common_init_result(common_params & params, bool model_only)
|
||||
return;
|
||||
}
|
||||
|
||||
if (params.profiling) {
|
||||
ggml_backend_sched_t sched = llama_context_get_sched(lctx);
|
||||
if (sched != nullptr) {
|
||||
ggml_backend_sched_set_profiling(sched, true);
|
||||
LOG_INF("%s: profiling enabled\n", __func__);
|
||||
}
|
||||
}
|
||||
|
||||
pimpl->context.reset(lctx);
|
||||
|
||||
set_process_priority(params.cpuparams.priority);
|
||||
|
||||
Reference in New Issue
Block a user