Wagner Bruna 541034d72d sd: sync to master-652-92dc726 (#2234) ...

* sd: sync to master-650-1ceb5bd

* sd: add LongCat-Image support

* sd: sync to master-652-92dc726

* sd: add Lens support

* sd: split gpt-oss vocab and merge files

2026-05-28 01:21:57 +08:00

..

vocab

sd: sync to master-652-92dc726 (#2234)

2026-05-28 01:21:57 +08:00

bpe_tokenizer.cpp

sd: sync to master-621-baf7eda

2026-05-23 09:05:35 -03:00

bpe_tokenizer.h

sd: sync to master-621-baf7eda

2026-05-23 09:05:35 -03:00

clip_tokenizer.cpp

sd: sync to master-591-331cfa5 (#2155)

2026-05-01 16:33:28 +08:00

clip_tokenizer.h

sd: sync to master-582-7023fc4 (#2150)

2026-04-21 23:01:33 +08:00

gemma_tokenizer.cpp

sd: sync to master-621-baf7eda

2026-05-23 09:05:35 -03:00

gemma_tokenizer.h

sd: sync to master-621-baf7eda

2026-05-23 09:05:35 -03:00

gpt_oss_tokenizer.cpp

sd: sync to master-652-92dc726 (#2234)

2026-05-28 01:21:57 +08:00

gpt_oss_tokenizer.h

sd: sync to master-652-92dc726 (#2234)

2026-05-28 01:21:57 +08:00

mistral_tokenizer.cpp

sd: sync to master-591-331cfa5 (#2155)

2026-05-01 16:33:28 +08:00

mistral_tokenizer.h

sd: sync to master-582-7023fc4 (#2150)

2026-04-21 23:01:33 +08:00

qwen2_tokenizer.cpp

sd: sync to master-607-fd1a279 (#2212)

2026-05-17 11:37:16 +08:00

qwen2_tokenizer.h

sd: sync to master-582-7023fc4 (#2150)

2026-04-21 23:01:33 +08:00

t5_unigram_tokenizer.cpp

sd: sync to master-582-7023fc4 (#2150)

2026-04-21 23:01:33 +08:00

t5_unigram_tokenizer.h

sd: sync to master-582-7023fc4 (#2150)

2026-04-21 23:01:33 +08:00

tokenize_util.cpp

sd: sync to master-582-7023fc4 (#2150)

2026-04-21 23:01:33 +08:00

tokenize_util.h

sd: sync to master-582-7023fc4 (#2150)

2026-04-21 23:01:33 +08:00

tokenizer.cpp

sd: sync to master-582-7023fc4 (#2150)

2026-04-21 23:01:33 +08:00

tokenizer.h

sd: sync to master-582-7023fc4 (#2150)

2026-04-21 23:01:33 +08:00