mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-09-18 00:34:55 +02:00
7a16a6ce32
1) Combine two consecutive lookups (find + insert) into a single insert-attempt/lookup routine so that we don't per form two O(log(n)) lookup operations in a row anymore -- we only need to do it once and then see if the insert succeeded. 2) Instead of copying every potential stack (expensive) and then moving it (cheap) to new_stacks when it's a final output state, we switch the order so that we move every potential stack (cheap), and then only copy it (expensive) to new stacks when it's a final output state. There are a LOT of intermediate states that get generated, and unless they become final output states, then all of these expensive intermediate copies are wasted. Before: lookup -> lookup/insert + copy -> optional move to output New: lookup/insert + move -> optional copy to output