Token amounts not accurate

[Joon]

We are rolling out codeburn for agentic coding tracking for our developer population and have feedback from many of them that the numbers shown are not accurate.

I tested this morning and codeburn is definitely not picking up all usage.

We use Cursor as the coding tool, and most of our devs use macbooks.

Test evidence:
Cursor token usage screenshot from admin portal for the current user (Millions of tokens shown):

Codeburn daily usage screen for the current user (thousands of tokens shown):

[iamtoruk]

Thanks again for the clear report, the screenshots made this easy to chase. Here's the honest picture, plus a branch to test.

What's happening: Cursor stores each chat in a local SQLite DB, but on current builds the per-message token count is written as {input: 0, output: 0}. So codeburn was estimating Cursor tokens rather than reading them, which is why the numbers came in low.

Cursor does record its own tokenizer-accurate context size per conversation in composerData.promptTokenBreakdown (the number behind the in-app context-window bar). codeburn just wasn't reading it.

The honest limit: we searched the entire local Cursor footprint (global and per-workspace state.vscdb, ai-code-tracking.db, logs, LevelDB, ~/.cursor). The exact billed figures are not stored on disk anywhere. Only the input/context count is local; output tokens, cache tokens, and the cumulative-per-turn total your admin portal bills are computed server-side and never written locally. That is inherent to usage-based billing.

What we changed (branch in #575, draft):

• Input tokens now come from promptTokenBreakdown instead of an estimate. On a test machine, one day's Cursor input went from ~44.9k to ~168.5k tokens, matching the real per-conversation context.
• Tools + bash: codeburn now surfaces the agent's tool calls (Read, Grep, Glob, Shell) and the actual shell commands it ran, pulled from the local agent log.
• Pricing: composer-2.5 was missing a price mapping (showed $0); it is now priced like composer-2.

It gets much closer but will not match the portal exactly: it is the latest context snapshot (not cumulative-per-turn), output stays a reply-text estimate, and cache tokens are not on disk.

You can check the real on-disk context total yourself (read-only, no chat content):

DB="$HOME/Library/Application Support/Cursor/User/globalStorage/state.vscdb"
sqlite3 "$DB" "SELECT COALESCE(SUM(CAST(json_extract(value,'$.promptTokenBreakdown.totalUsedTokens') AS INT)),0) FROM cursorDiskKV WHERE key LIKE 'composerData:%';"

Question for you: to match your admin portal exactly, the only path is reading Cursor's Team/Admin API with an API key. That is opt-in, since it leaves the machine, unlike everything else codeburn does locally. You are rolling this out to a developer population and already have admin access, so would an opt-in Cursor API-key integration be worth it for you? If yes, we will build it. If the local improvement above is close enough, we will keep it local-first.

Either way, please try the #575 branch and let us know how the numbers compare to your portal.

[hiSandog]

For token amount mismatches, it would help to split the report by source of truth: provider usage API, local transcript tokenization, or normalized model family totals. If the issue is that some providers report cached/input/output/reasoning tokens differently, a small table with raw provider payload versus Codeburn's displayed totals would make it much easier to see where the conversion is drifting.

[iamtoruk]

@hiSandog this is exactly the right instinct, and it's now built: codeburn audit (#578).

It prints a per-(provider, model) table of the raw token fields each provider records next to what codeburn displays and prices, so any drift is visible in one place:

Provider  Model          Calls    Input   Output   Reason  Cache wr   Cache rd      Cost
Claude    Opus 4.8      27,900    15.5M    42.7M        0    160.0M    8155.6M  $6569.89
Codex     GPT-5.5        3,835    64.6M     2.0M   876.8K         0     459.7M   $639.72
Cursor    composer-2.5       4   151.6K        0        0         0          0     $0.00

--format json adds the conversion detail you asked for: the per-component cost, the exact rates applied, and both raw cache-read fields.

"raw":       { "outputTokens": 2000000, "reasoningTokens": 876800, "cacheReadInputTokens": "...", "cachedInputTokens": "..." },
"displayed": { "outputTokens": 2876800 },          // reasoning folds into output
"rates":     { "inputCostPerToken": 3e-6, "cacheWriteCostPerToken": 3.75e-6, "cacheReadCostPerToken": 3e-7 },
"cost":      { "input": "...", "cacheRead": "...", "recomputedTotalUSD": "..." },
"attributedCostUSD": "..."

One small adjustment to the framing: codeburn is local-first, so for most providers the "source of truth" isn't a usage API, it's the provider's own usage object recorded in the local transcript. Cursor is the exception (no real per-token on disk, so it estimates). The audit makes both modes' conversions explicit: reasoning folding into output, cache read = max(Anthropic cacheReadInput, OpenAI cached), and the 1.25x / 0.1x cache multipliers, which is where most "the numbers look off" reports actually trace back to.

Is this what you had in mind? Happy to add a per-call mode or more columns if you want it more granular.

[Joon]

This is awesome team, thank you so much! I will test this branch today

[Joon]

Tested and it's not working (or I tested incorrectly)

I left detailed comments with screenshots in the PR #575