Files
oh-my-pi/packages/coding-agent/test/plan-mode
can1357 12238f55ca feat: implemented native ctok tokenization engine with model scopes
- Implemented the `ctok` Rust native tokenization engine with offline support for Claude V3, V47, V5, and V5Sonnet families.
- Replaced global token estimation with model-scoped `Tokenizer` instances and provider-anchored transcript accounting across packages.
- Added vocabulary generation scripts, test fixtures, and comprehensive unit tests for tokenizer routing and matching modes.
2026-08-19 23:27:29 +02:00
..
2026-07-27 16:43:53 +02:00