 Xuan-Son Nguyen
|
decaf508bb
|
server: refactor + correctness fixes for metrics (#26920)
* server: refactor metrics
* move most fields to server_slot_stats
* cont
* rm result_timings
* tie stats to batch
* cont
* nits: move place in code
* exclude first generated token
* more accurate batch metrics tracking
* n_predict --> n_gen
* metrics_on_prediction
* metrics_flush_idle
* metrics: seperate cache/processed prompt tokens
* refactor server_task_result_metrics
* add test
* nits
* fix flush before reset()
* cont
* rm dead code
* nits
|
2026-08-13 10:02:01 +02:00 |
|