feat(crush): add initial session-level local usage support by IvGolovach · Pull Request #346 · junhoyeo/tokscale

IvGolovach · 2026-03-20T06:22:38Z

Summary

add initial local-only Crush support by discovering project databases from projects.json and parsing crush.db
expose Crush in CLI and TUI local client filters while keeping default submit behavior unchanged for existing users

Why

Crush is a good fit for Tokscale, but its storage model differs from existing clients. It stores usage in per-project SQLite databases and exposes reliable session-level totals, not clean per-message token accounting. This PR adds a narrow, honest v1 integration that imports only the data Crush stores reliably instead of fabricating model-level attribution.

Diff scope

add crush as a first-class local client and discover per-project crush.db files from the global project registry plus current-workspace fallback
parse root Crush sessions into local usage records using stored cost and created_at
represent Crush imports as session-level records with model = session-total and zero token breakdown, because Crush does not expose reliable per-message or per-model token totals
surface --crush through CLI and TUI client selection and display paths
exclude Crush from the default submit client set so session totals are not uploaded unless explicitly requested
harden affected core and CLI tests so they remain hermetic under the new discovery path

Test proof

cargo test -p tokscale-core
- 384 passed, 0 failed, 1 ignored
CARGO_TARGET_DIR=/tmp/tokscale-crush-check cargo test -p tokscale-cli
- unit tests: 294 passed, 0 failed, 1 ignored
- integration tests: 66 passed, 0 failed
CARGO_TARGET_DIR=/tmp/tokscale-crush-check cargo clippy -p tokscale-core --all-features -- -D warnings
- passed
CARGO_TARGET_DIR=/tmp/tokscale-crush-check cargo clippy -p tokscale-cli --all-features -- -D warnings
- passed
cargo fmt --all --check
- passed

Verification-pack proof

Not applicable - core and CLI local parsing changes only.

Migration notes

Not applicable - no schema or data migration.

CI context confirmation

CI context names unchanged.

Rollback plan

if merged with a merge commit: git revert <merge_commit_sha>
if merged as a squash commit: git revert <squash_commit_sha>
no DB downgrade required

Known residual risks

this is intentionally an initial Crush integration: it imports session-level totals only and does not attempt per-message or per-model token attribution
daily attribution uses the Crush session created_at timestamp, so long-running sessions are not split across multiple days
project discovery depends on registry entries plus current-workspace fallback, so unregistered projects outside the current workspace may remain undiscovered

…m host filesystem After rebasing onto main (which added Kilo at index 14), Crush moved to index 15 bumping the total client count. The all-clients filter test asserted 16 but now needs 17 (16 ClientId variants + synthetic). The crush scanner test picked up a real .crush/crush.db from the host machine because find_current_workspace_crush_db() walks from CWD. On assertion failure the manual env var restore was skipped, leaking XDG_DATA_HOME and breaking 4 downstream source-cache tests. Fixed by changing CWD to an isolated temp directory during the test.

junhoyeo · 2026-03-24T22:29:29Z

Maintainer rebase + analysis

I've rebased this branch onto current main and resolved all merge conflicts. Here's what changed and my review findings.

Rebase resolution

PR #353 added Kilo = 14 (Kilo CLI client) to main after this branch was created. The rebase:

Moved Crush from index 14 → 15 (16 total clients)
Added submit_default: true to the Kilo entry (new field from this PR)
Updated all count assertions (ClientId::COUNT → 16, all-clients filter → 17)
Kept both kilo_db and crush_dbs fields in ScanResult
Preserved both Kilo and Crush parsing blocks in parse_local_clients and parse_all_messages_with_pricing

Test fix

The Crush scanner test test_scan_all_clients_crush_populates_crush_db_paths was picking up a real .crush/crush.db from the host machine because find_current_workspace_crush_db() walks up from CWD. On assertion failure, the manual restore_env() call was skipped (panic unwind), leaking XDG_DATA_HOME to a deleted temp dir and breaking 4 downstream source-cache tests. Fixed by changing CWD to an isolated temp directory during the test.

Verification

After rebase:

cargo clippy -p tokscale-core --all-features -- -D warnings ✅
cargo clippy -p tokscale-cli --all-features -- -D warnings ✅
cargo test -p tokscale-core — 424 passed, 0 failed, 1 ignored ✅
cargo test -p tokscale-cli — 297 passed (unit) + 66 passed (integration), 0 failed ✅
cargo fmt --all --check ✅

Integration completeness

Crush is wired into every required integration point — matching the pattern of Mux and other clients:

Integration Point	Status
`define_clients!` macro (`Crush = 15`)	✅
Session parser (`sessions/crush.rs`)	✅
Scanner discovery (registry + workspace)	✅
`parse_all_messages_with_pricing`	✅
`parse_local_clients`	✅
CLI flag (`--crush`) across all subcommands	✅
`ClientFlags` / `build_client_filter`	✅
TUI `CLIENT_UI` entry (hotkey `h`)	✅
Wrapped display name + logo	✅
`submit_default: false` (excluded from default submit)	✅

Data quality note

Crush provides session-level cost only — no per-message tokens, no model attribution. This is an honest v1 that imports what Crush reliably exposes. The submit_default: false exclusion is the right call since session-level totals would distort per-model analytics on the leaderboard.

Remaining gap

README.md is not updated — Crush is missing from the client support table, filtering examples, data sources section, and Windows paths table. This could be addressed in a follow-up.

vercel · 2026-03-24T22:30:46Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
tokscale	Ignored	Preview	Mar 24, 2026 10:30pm

Both clients were missing from documentation across all 4 locales. Added to: client support table, multi-platform feature list, filtering examples, frontend source filtering, Windows paths, and data sources sections.

devin-ai-integration

Devin Review found 1 new potential issue.

⚠️ 1 issue in files not directly in the diff

⚠️ CACHE_SCHEMA_VERSION not bumped after adding `message_count` to bincode-serialized UnifiedMessage (`crates/tokscale-core/src/message_cache.rs:14`)

The PR adds a new message_count: i32 field to UnifiedMessage (crates/tokscale-core/src/sessions/mod.rs:37), which is serialized into the on-disk message cache via bincode (crates/tokscale-core/src/message_cache.rs:401). Bincode is a positional binary format that does not honour #[serde(default)] for missing fields — so any cache written by the previous release (schema version 4, without message_count) will fail to deserialize under the new struct layout. The CACHE_SCHEMA_VERSION at crates/tokscale-core/src/message_cache.rs:14 remains 4 instead of being bumped to 5. Because deserialize_from(...).ok()? gracefully returns None on failure (crates/tokscale-core/src/message_cache.rs:434-437), the stale cache is silently discarded and data is re-parsed — so there is no crash or data loss. However, the schema-version mechanism exists precisely to short-circuit this: without the bump the code reads and attempts to parse the entire (potentially large) cache file before giving up, instead of rejecting it immediately at the version check.

View 10 additional findings in Devin Review.

…pport-refresh

devin-ai-integration

Devin Review found 1 new potential issue.

🐛 1 issue in files not directly in the diff

🐛 KiloCode parser outputs `client = "kilo"` instead of `"kilocode"`, breaking the KiloCode/Kilo CLI split (`crates/tokscale-core/src/sessions/kilocode.rs:10`)

After splitting the old combined Kilo client into KiloCode (VS Code extension, id "kilocode") and Kilo (CLI, id "kilo"), the parse_kilocode_file function at crates/tokscale-core/src/sessions/kilocode.rs:10 still passes "kilo" as the source/client string to parse_roo_kilo_file. This means all VS Code KiloCode messages will have client = "kilo" — identical to Kilo CLI messages — instead of client = "kilocode".

This causes three failures:

--kilocode filter returns nothing: The filter sends "kilocode" to retain_for_requested_clients (crates/tokscale-core/src/lib.rs:63) which checks requested.contains(client). Since KiloCode messages have client = "kilo", they won't match.
KiloCode data mixed into Kilo CLI: --kilo will return both Kilo CLI AND KiloCode VS Code data since both have client = "kilo".
Wrong display names: client_display_name("kilo") returns "Kilo CLI" (crates/tokscale-cli/src/commands/wrapped.rs:1363), so VS Code extension usage will be mislabeled as "Kilo CLI" in wrapped images and reports.

View 14 additional findings in Devin Review.

…pport # Conflicts: # crates/tokscale-cli/src/main.rs # crates/tokscale-core/src/clients.rs # crates/tokscale-core/src/scanner.rs

… data loader

Resolve new PR junhoyeo#346 review findings by removing legacy kilocode->kilo normalization, propagating use_env_roots to Crush DB discovery, and including Kilo/Mux in --test-data diagnostics.

junhoyeo · 2026-03-31T17:14:02Z

🚀 Released in tokscale@v2.0.16! Thanks for the contribution! cc @IvGolovach @Jelloeater

Closes #278. Adds Charmbracelet Crush as a lazy-loaded provider: - src/providers/crush.ts: walks ~/.local/share/crush/projects.json (XDG_DATA_HOME and CRUSH_GLOBAL_DATA aware), opens each project's crush.db read-only, queries root sessions where parent_session_id IS NULL. Emits one ParsedProviderCall per session with real prompt_tokens, completion_tokens, cost (dollars), and the dominant model resolved from messages.model. - src/providers/index.ts: register crush alongside cursor, goose, opencode, antigravity, cursor-agent in the lazy import path. - tests/providers/crush.test.ts: 10 fixture-based tests covering discovery, parsing, missing-registry, malformed JSON, missing db, child session exclusion, dominant model selection, dedup, and array-shaped legacy registry. Schema source: charmbracelet/crush@v0.66.1 internal/db/migrations/20250424200609_initial.sql, verified by spawning a research agent against upstream. The schema *comments* in that migration claim millisecond timestamps but every actual INSERT/UPDATE uses strftime('%s', 'now') which returns Unix seconds; the parser treats values as seconds. Tokscale's parser (junhoyeo/tokscale#346) gets this wrong and is off by 1000x, plus its parser misses the prompt_tokens/completion_tokens columns that exist in Crush's schema. Our integration uses both, so Crush sessions get real per-model attribution. Menubar: - mac/Sources/CodeBurnMenubar/AppStore.swift: add .crush case to ProviderFilter and its cliArg switch. - mac/Sources/CodeBurnMenubar/Views/AgentTabStrip.swift: add Crush color to the per-tab color extension. The visibleFilters computed property already filters by detected providers, so the Crush tab appears automatically when a user has Crush data. README: - Replace the provider table with an icon-led layout. Icons live under assets/providers/<name>.<ext>. 14 icons sourced from junhoyeo/tokscale (MIT) under nominative fair use, 4 sourced separately: codex (OpenAI org avatar), cursor-agent (reuses the Cursor icon), kiro (kiro.dev favicon, ico->png via sips), omp (can1357/oh-my-pi icon.svg, MIT). Attribution line added. - Add Crush row. Docs: - docs/providers/crush.md: full per-provider doc with verified schema excerpt, the seconds-vs-milliseconds quirk, and a "when fixing a bug here" checklist. - docs/architecture.md: provider count 17 -> 18, test count 41 -> 42, and crush in the lazy list. - docs/providers/README.md: add Crush row to the lazy index. - CONTRIBUTING.md: bump test count to 568 (was 558). All 568 tests pass locally; swift build clean.

feat(crush): add initial session-level local usage support

897f0ff

This comment was marked as resolved.

Sign in to view

fix(crush): harden discovery and session import

04c1f13

This comment was marked as resolved.

Sign in to view

fix(crush): use a project-owned logo asset

6b483d0

junhoyeo approved these changes Mar 24, 2026

View reviewed changes

IvGolovach and others added 4 commits March 25, 2026 07:16

feat(crush): add initial session-level local usage support

2739869

fix(crush): harden discovery and session import

6b969da

fix(crush): use a project-owned logo asset

38266fc

junhoyeo force-pushed the feat/crush-session-support branch from 6b483d0 to 6ad18ca Compare March 24, 2026 22:26

This comment was marked as resolved.

Sign in to view

junhoyeo and others added 2 commits March 25, 2026 07:39

docs: add Kilo CLI and Crush to all README translations

73c88b8

Both clients were missing from documentation across all 4 locales. Added to: client support table, multi-platform feature list, filtering examples, frontend source filtering, Windows paths, and data sources sections.

fix(crush): merge latest main and preserve daily session totals

0d504db

This comment was marked as resolved.

Sign in to view

IvGolovach added 2 commits March 24, 2026 15:44

chore(crush): sync remote branch updates

1f60ddc

docs(crush): document xdg fallback paths

e834189

This comment was marked as resolved.

Sign in to view

fix(wrapped): add Kilo CLI display metadata

31de6d9

devin-ai-integration Bot reviewed Mar 25, 2026

View reviewed changes

fix(cache): bump message cache schema version

0a289a9

junhoyeo force-pushed the main branch from 63e6d84 to bef1da8 Compare March 25, 2026 22:13

IvGolovach requested a review from junhoyeo March 26, 2026 22:39

Merge remote-tracking branch 'origin/main' into feat-crush-session-su…

b4b160d

…pport-refresh

This comment was marked as resolved.

Sign in to view

fix(crush): refresh branch against main

a93bfe6

devin-ai-integration Bot reviewed Mar 29, 2026

View reviewed changes

IvGolovach and others added 2 commits March 29, 2026 15:49

fix(kilocode): preserve distinct client identity

e730bf0

Merge remote-tracking branch 'origin/main' into feat/crush-session-su…

e414381

…pport # Conflicts: # crates/tokscale-cli/src/main.rs # crates/tokscale-core/src/clients.rs # crates/tokscale-core/src/scanner.rs

This comment was marked as resolved.

Sign in to view

junhoyeo added 2 commits March 31, 2026 23:48

fix(cli): add missing use_env_roots field to LocalParseOptions in TUI…

83d8e86

… data loader

fix(core): honor distinct kilocode ID and crush env-root strategy

036a111

Resolve new PR junhoyeo#346 review findings by removing legacy kilocode->kilo normalization, propagating use_env_roots to Crush DB discovery, and including Kilo/Mux in --test-data diagnostics.

junhoyeo merged commit 3ae3bf5 into junhoyeo:main Mar 31, 2026
13 of 14 checks passed

IvGolovach deleted the feat/crush-session-support branch March 31, 2026 17:55

Jelloeater mentioned this pull request May 9, 2026

feat: Add support for Crush agent metrics getagentseal/codeburn#278

Closed

iamtoruk mentioned this pull request May 10, 2026

feat(providers): add Crush + provider icons in README (closes #278) getagentseal/codeburn#286

Merged

5 tasks

Conversation

IvGolovach commented Mar 20, 2026 • edited by devin-ai-integration Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why

Diff scope

Test proof

Verification-pack proof

Migration notes

CI context confirmation

Rollback plan

Known residual risks

Related

Uh oh!

vercel Bot commented Mar 20, 2026

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

Jelloeater commented Mar 20, 2026

Uh oh!

Jelloeater commented Mar 23, 2026

Uh oh!

IvGolovach commented Mar 23, 2026

Uh oh!

junhoyeo left a comment

Choose a reason for hiding this comment

Uh oh!

junhoyeo commented Mar 24, 2026

Maintainer rebase + analysis

Rebase resolution

Test fix

Verification

Integration completeness

Data quality note

Remaining gap

Uh oh!

This comment was marked as resolved.

Uh oh!

vercel Bot commented Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

⚠️ CACHE_SCHEMA_VERSION not bumped after adding message_count to bincode-serialized UnifiedMessage (crates/tokscale-core/src/message_cache.rs:14)

Uh oh!

This comment was marked as resolved.

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

🐛 KiloCode parser outputs client = "kilo" instead of "kilocode", breaking the KiloCode/Kilo CLI split (crates/tokscale-core/src/sessions/kilocode.rs:10)

Uh oh!

This comment was marked as resolved.

Uh oh!

Uh oh!

junhoyeo commented Mar 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

IvGolovach commented Mar 20, 2026 •

edited by devin-ai-integration Bot

Loading

vercel Bot commented Mar 24, 2026 •

edited

Loading

⚠️ CACHE_SCHEMA_VERSION not bumped after adding `message_count` to bincode-serialized UnifiedMessage (`crates/tokscale-core/src/message_cache.rs:14`)

🐛 KiloCode parser outputs `client = "kilo"` instead of `"kilocode"`, breaking the KiloCode/Kilo CLI split (`crates/tokscale-core/src/sessions/kilocode.rs:10`)