60 Commits

Author SHA1 Message Date
can1357 5cbdd740bf feat(stats): added token aggregation and updated window grouping logic
- Added `fetchUsageData` and fleet token summation to aggregate token burn across clients.
- Changed window grouping logic from window labels to limit IDs for distinct separation.
- Updated `ProviderWindowInsight` properties and added tests for label suffixing and token summation.
2026-08-19 16:00:13 +02:00
Anatoli Tsinovoy 16ad301117 fix(stats): restore configurable dashboard bind host 2026-08-17 09:46:49 +03:00
can1357 1132c3e31c feat: implemented anthropic keep-alive and migration support for updates
- Added Anthropic prompt-cache refresh scheduling and state management to keep prompts warm across idle sessions.
- Updated pricing models and database stats tracking to calculate and store cost-weighted cache savings.
- Integrated cache savings metrics and efficiency displays into the stats CLI, dashboard routes, and UI components.
- Added support for package renaming, manifest pointer tracking, and installation migration during CLI updates.
2026-08-13 04:59:33 +02:00
can1357 b60bef961c feat: introduced nix packaging and path-based binary resolution
- Add comprehensive Nix flake definitions, derivations, modules, and CI workflows.
- Update tests and executables to resolve binaries from PATH rather than absolute paths.
- Ensure byte reproducibility and zeroed timestamps in embedded dashboard archives.
- Add handling for Nix-managed installations in CLI update checks.
2026-08-13 03:52:47 +02:00
can1357 e9888367d1 refactor: migrated packages to internal utility modules and removed external dependencies
- Implemented in-house, zero-dependency utility modules in `pi-utils` covering DOM manipulation, markdown parsing, templating, browser automation helpers, and terminal buffers.
- Migrated packages across the repository to consume the new internal utilities and `omptype` schema validators instead of external dependencies.
- Removed multiple external runtime and development dependencies including Zod, Marked, LRU cache, Turndown, and Puppeteer browser packages.
2026-08-05 13:39:09 +02:00
can1357 cab9f62400 test(stats): isolated port-conflict suite from the real stats db
- server-port-conflict opened the module-global db against the live agent stats.db, poisoning later files (sync-serial read 2.28M real rows); installStatsTestIsolation gives it a temp agent dir and closes the handle in teardown
2026-08-05 01:22:36 +02:00
can1357 ece9245a0a style: applied biome formatting to merged changes 2026-08-05 01:15:59 +02:00
can1357 191d7f4051 fix(stats): reclaim insecure wildcard dashboards 2026-08-05 01:11:58 +02:00
roboomp 6e3718675e fix(stats): rejected insecure dashboard reuse
Versioned the dashboard identity header and required the current security version with no CORS permission before reusing a listener.

Older versioned and headerless stats dashboards are now identified by their process and restarted so loopback-only binding takes effect immediately.

Fixes #7633
2026-08-04 15:45:45 +00:00
roboomp d8c7089963 test(stats): probed loopback bind over direct tcp
Replaced the 127.0.0.2 fetch, which a configured HTTP proxy could intercept, with a direct Bun.connect TCP probe so the loopback-only bind is asserted against the real listener.

Fixes #7633
2026-08-04 15:38:37 +00:00
roboomp 70dd7a318e fix(stats): restricted dashboard access
Bound the dashboard and reuse probe to IPv4 loopback, removed wildcard CORS, and reported the actual listening hostname.

Added regression coverage for non-loopback refusal and absent cross-origin access.

Fixes #7633
2026-08-04 15:26:48 +00:00
can1357 455493dfad feat: introduced native file lock bindings for cross-process advisory locking
- Added native `FileLock` bindings supporting cross-process advisory locking on Linux, Unix, and Windows.
- Replaced directory-based file locking and custom stale-lock reclamation with OS-backed native locks.
- Updated TypeScript declarations, native bindings, and package documentation for the new API.
- Added comprehensive unit tests and fixtures validating single-owner constraints and process death handoff.
2026-08-03 16:09:09 +02:00
can1357 39bc9de52f style: applied biome import order and formatting 2026-08-03 15:26:20 +02:00
can1357 9fb082bf14 refactor(utils): consolidated file locking into pi-utils file-lock
- Moved the coding-agent lock-directory primitive to @oh-my-pi/pi-utils/file-lock
  and migrated settings, MCP config-writer, and security store imports.
- Replaced the stats aggregator's parallel ~200-line token/breaker lock protocol
  with the shared primitive: dead owners reclaimed immediately, live-but-wedged
  owners after STATS_SYNC_LOCK_STALE_MS, unstamped acquisitions after the new
  acquireStaleMs grace (10s).
- Shared primitive now treats EPERM kill probes as live owners.
- Rewrote the stats lock-reclamation regressions against the shared protocol
  and moved the file-lock contract test into pi-utils.
2026-08-03 15:25:03 +02:00
can1357 0faa145985 fix(stats): reclaim interrupted sync locks promptly 2026-08-03 15:15:36 +02:00
can1357 c53a47f97d Merge PR #7287: fix(coding-agent): prune archived session stats (@alphastorm)
# Conflicts:
#	packages/coding-agent/src/session/session-manager.ts
2026-08-03 15:15:36 +02:00
roboomp b1ce143a3c fix(stats): added timeout to usage snapshot reads
Installed a five-second SQLite busy timeout before the read-only usage query and covered lock contention with a subprocess-backed regression test.

Fixes #7300
2026-08-01 18:50:54 +00:00
Sunil Srivatsa f3a555ac78 fix(coding-agent): reconcile archived session stats 2026-08-01 14:43:59 -04:00
can1357 30994dafb2 feat(stats): added Providers dashboard with per-provider analytics and utilization insights
- Added `/api/stats/providers` endpoint and database queries returning per-provider request/token/cost totals, hourly burn, and time-series data.
- Created ProvidersRoute UI component with totals panel, trend charts, peak-hours histogram, and window-insights visualizations.
- Added usage-window analytics computing peak utilization fractions, ideal account counts, and exhaustion events from usage snapshots.
- Included comprehensive tests for the aggregation logic, API, and window-stat computations.
2026-07-24 14:41:55 +02:00
roboomp bc5906aeb4 fix(stats): prevented malformed entries from stalling sync
- Guarded malformed persisted content blocks so later project files continue ingesting.
- Marked full-session migrations complete only after a successful sync pass.

Fixes #6373
2026-07-23 18:47:35 +00:00
Wolfgang Schoenberger e77b687a68 test(stats): derive expected conversation total 2026-07-18 16:55:03 -07:00
Wolfgang Schoenberger 9d6e8c2174 fix(stats): clarify overview token totals 2026-07-18 16:40:15 -07:00
can1357 b7c8fce83c fix(stats): bind port-conflict test listeners to the wildcard address
On macOS SO_REUSEADDR lets startServer's wildcard bind coexist with a
127.0.0.1-only listener, so the EADDRINUSE path was never exercised and
three of the four conflict tests failed. Wildcard-bind the holders and
the reused dashboard so the conflict is real on every platform.
2026-07-18 19:42:51 +02:00
roboomp 4ef017232e fix(stats): restricted port reclamation to stats processes
Recorded listener command lines across supported platforms and required an omp-stats-specific command identity before terminating Bun, Node, or omp holders. Unrelated runtimes now take the refusal path without receiving a signal.

Fixes #5970
2026-07-18 07:16:34 +00:00
roboomp 4010bef985 fix(stats): verified dashboard identity before port reuse
Stamped an x-omp-stats-dashboard header on dashboard responses and required it (or the models JSON-array shape) in the reuse probe, so a foreign 200 responder such as an SPA dev server catch-all is no longer mistaken for a live dashboard.

Fixes #5970
2026-07-18 07:10:54 +00:00
roboomp 477112e81d fix(stats): recovered occupied dashboard port
Reused live stats dashboards sharing the requested port and reclaimed stale Bun, Node, or omp listeners after a failed health probe. Foreign listeners now produce an ownership-specific error.

Fixes #5970
2026-07-18 07:04:01 +00:00
roboomp cc9977cce4 fix(advisor): anchored context maintenance on provider usage
- Anchored advisor compaction on provider-reported context usage (cached
  input + generated output) floored by a full local estimate including the
  advisor system prompt and tool schemas, so a near-full cached context is no
  longer undercounted by the per-message estimate.
- Rejected stale provider usage retained across advisor compaction via a
  runtime-only usage-anchor boundary recorded on the summary message.
- Recovered provider overflow by clearing only the advisor's own context at
  the current primary cursor, retrying the bounded failing batch once against
  a fresh context without replaying old primary history, and keeping later
  updates eligible.
- Threaded the selected dashboard range through the stats Recent Errors UI,
  API, and database timestamp filter before ordering and the 50-row limit.

Fixes #5282
2026-07-14 17:58:19 +00:00
can1357 1b0b18c7a6 fix(stats): handled malformed session entries to prevent sync crashes
- Coerce missing `stopReason`, token counts, and timestamps at the parser boundary to satisfy database NOT NULL constraints.
- Skip assistant entries missing essential model attribution or usage data instead of aborting the sync process.
- Filter invalid tool call blocks that lack required identifiers to ensure successful database insertion.
2026-07-12 12:42:37 +02:00
can1357 7df2ac297e fix(stats): handled legacy session entries with missing cost data
- Resolved a crash occurring when syncing legacy session files that lack usage cost breakdowns.
- Updated cost resolution to fallback to catalog pricing when stored cost is unavailable or zero.
2026-07-11 00:08:53 +02:00
can1357 c68a01f597 fix(stats): refined behavior metric extraction and filtering logic
- Adjusted yelling, profanity, and anguish criteria to reduce false-positive metrics.
- Implemented refined regex patterns for negation, blame, and noise reduction.
- Incremented the backfill key to v8 to accommodate updated signal definitions.
- Expanded test suite to validate new constraints for yelling, exclusion words, and sentiment triggers.
2026-07-06 03:55:43 +02:00
can1357 0d3d4c6362 test(stats): covered anthropic priority api classification
- Allowed the priority premium-request fixture to set assistant message `api` values.
- Covered direct Anthropic priority traffic with `api: "anthropic-messages"` so premium-request backfill follows API-family classification.
2026-07-02 08:31:36 +02:00
can1357 fbf3a6eb3f feat(stats): added tool usage dashboard
- Parsed assistant `toolCall` blocks and `toolResult` messages into persisted `tool_calls` rows with one-shot historical backfill.
- Added tool aggregate queries, model breakdowns, call time-series data, and the `/api/stats/tools` dashboard endpoint.
- Added the `/#/tools` route with summary metrics, stacked calls-over-time chart, per-tool table, and model breakdown panel.
- Added end-to-end stats coverage for tool ingestion, result/error linkage, fork deduplication, incremental updates, and dashboard shaping.
2026-07-02 08:31:18 +02:00
can1357 b0420f1053 test(stats): align large-session newOffset with lenient parser (PR #3735 merge) 2026-06-29 16:53:40 +02:00
can1357 a402486d9a Merge PR #3735: fix(stats): avoid macOS stats sync worker abort (@roboomp)
# Conflicts:
#	packages/stats/src/parser.ts
#	packages/stats/test/parser-large-session.test.ts
2026-06-29 16:48:59 +02:00
roboomp 7d5544493e fix(stats): skipped sync worker smoke on darwin
smokeTestSyncWorker spawned a worker via createSyncWorker on every platform, so omp --smoke-test (and the macOS signing pre-launch run in scripts/ci-macos-sign.sh) still hit the Bun-worker re-entry path the serial macOS sync default avoids. Early-return on darwin and document the carve-out.

Fixes #3733
2026-06-29 11:06:17 +00:00
roboomp 9abea02420 test(stats): isolated stats tests from xdg-shared db
Stats tests set PI_CONFIG_DIR + setAgentDir(home/<config>/agent), which resolves equal to the default agent dir and routes stats.db through $XDG_DATA_HOME/omp/stats.db whenever an XDG var is set, leaking rows across tests. The shared installStatsTestIsolation helper now snapshots and clears XDG_*_HOME with PI_CONFIG_DIR on each test, points the agent dir at a fresh TempDir, closes the DB handle, and restores the prior env after the test.
2026-06-29 10:59:06 +00:00
can1357 de9a1f79f9 fix-stats-large-jsonl-parse 2026-06-28 18:41:07 +02:00
roboomp 234c1919f5 fix(stats): kept worker pool for one-file syncs
Keyed the serial-parser branch off the caller-requested worker count instead of the per-pool clamp, so a one-file sync still fans out to workers when the caller (or default) asked for more than one.

Fixes #3733
2026-06-28 16:31:17 +00:00
roboomp efb96b123f fix(stats): avoided native jsonl parser trap
Replaced the stats session parser's Bun.JSONL.parseChunk path with a lenient JS line scanner so large session files do not enter the native parser before /stats launches.

Fixes #3733
2026-06-28 16:29:19 +00:00
roboomp b48d488f2c fix(stats): avoided macos stats sync worker abort
Kept the documented workers: 1 path inline and defaulted macOS stats sync to that serial parser path so /stats dashboard launches do not re-enter Bun workers on macOS.

Fixes #3733
2026-06-28 16:21:59 +00:00
can1357 fbad280b57 feat: implemented multi-advisor concurrent runtime with tui management
- Introduced comprehensive support for multiple concurrent, independently-configured advisors via `WATCHDOG.yml` files.
- Implemented a full-screen TUI overlay for managing advisor rosters, models, tools, and instructions.
- Added session-wide advisor initialization, telemetry aggregation, and named transcript isolation.
- Enhanced advisor security and observability with secret redaction in tool results and secure XML attribute encoding.
2026-06-28 12:55:09 +02:00
can1357 577d2a8eb8 style: biome format/organize-imports across integrated PRs 2026-06-27 02:06:38 +02:00
can1357 5a1c17c1cb Fix gain dashboard project scoping 2026-06-27 02:04:51 +02:00
can1357 c03a78917a test(coding-agent): disabled fsmonitor during test repo initialization
- Added core.fsmonitor configuration to the git initialization process in test setup to prevent potential environment-specific conflicts.
2026-06-25 12:55:01 +02:00
roboomp b5769a4e10 fix(stats): dedupe forked-session entries to stop double-counting
SessionManager.fork() and createBranchedSession() deep-copy a parent's
entries into the child JSONL — same entry_id, timestamp, model,
responseId, token counts, cost. Stats keyed uniqueness on
(session_file, entry_id), so both files contributed to request, token,
and cost totals.

insertMessageStats and insertUserMessageStats now skip rows whose
(entry_id, timestamp) already exists under a different session_file
(first-write-wins). A one-shot migration on initDb, gated by the
fork_dedupe_v1 meta sentinel, collapses pre-existing duplicate rows
in messages and user_messages by keeping the lowest-id row per
(entry_id, timestamp) group.

Fixes #3370
2026-06-24 14:14:25 +00:00
can1357 153fd78345 feat(stats): added token usage breakdown by agent type
- Implemented agent type classification and persistence in the database, including a backfill for legacy records.
- Added `AgentType` categorization and aggregated token usage metrics to the backend services.
- Created an `AgentTokenShare` visualization component to display usage distribution on the overview dashboard.
- Integrated agent-based tracking into existing data pipelines, view models, and testing suites.
2026-06-22 17:42:58 +02:00
lycaon 525f4baa64 feat(stats): redesign the omp stats dashboard
Rewrite the local `omp stats` web UI on the OMP brand palette with a left nav, focused per-section views, a new Projects view, a system-aware light/dark toggle, and flicker-free navigation. Same data, endpoints, and command; the only server change is a 1h time-range bucketing fix in the aggregator.
2026-06-16 23:30:50 -06:00
can1357 34c74c4f52 test(stats): align backfill test with v6 key
The PR bumped USER_MESSAGES_BACKFILL_KEY to user_messages_v6, but the
behavior-backfill regression still seeded the dead user_messages_v5
sentinel, so it no longer exercised the user-messages backfill reset
path. Point both fixtures at v6.

Addresses review feedback on #2457.
2026-06-14 17:29:29 +02:00
can1357 3cb7b8786b test(stats): pin git-not-profanity regression
Adds a computeUserMessageMetrics() assertion that ordinary `git`
prose scores profanity: 0, so the dropped token cannot be silently
reintroduced by a future profanity-list edit.

Addresses review feedback on #2457.
2026-06-14 17:29:29 +02:00
can1357 ae415199dc feat: added build-time compatibility in ModelSpec/buildModel pipeline
- Centralized catalog and registry handling on `ModelSpec` and `buildModel`, resolving compatibility at model build time.
- Removed runtime compatibility detectors and switched provider request flows to direct `model.compat` reads.
- Added compat fields (`supportsReasoningParams`, `alwaysSendMaxTokens`, `strictResponsesPairing`, `whenThinking`).
- Persisted explicit compatibility overrides through `compatConfig` in discovery and cache merge paths.
2026-06-10 06:20:51 +02:00