smashimo/oh-my-pi
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test(mnemopi): refresh bench artifact on an idle host and sync changelog claims

Browse Source
This commit is contained in:
Wolfgang Schoenberger
2026-07-22 05:18:28 -07:00
parent 8047bedaa4
commit 72394db605
3 changed files with 37 additions and 144 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/mnemopi/CHANGELOG.md
+1 -1
View File
@@ -4,7 +4,7 @@
### Changed
- Recall hot loops (exact vector-index search, binary vector search, SHMR similarity clustering, and default-similarity MMR rerank) now run on native batch kernels with one N-API crossing per operation. Crossing-inclusive speedups at dim=384 (see `bench/native-vectors.bench.json`): 1.6-3.2x cosine scoring, 1.8-2x top-K search, 1.6x pairwise clustering scans, 3-10x Hamming search, and 21-39x MMR rerank. Custom `similarityFn` MMR reranks and incremental per-row scoring stay on the TypeScript implementations ([#6280](https://github.com/can1357/oh-my-pi/pull/6280) by [@wolfiesch](https://github.com/wolfiesch)).
- Recall hot loops (exact vector-index search, SHMR similarity clustering, and default-similarity MMR rerank) now run on native batch kernels with one N-API crossing per operation. Wrapper-inclusive speedups at dim=384 on Apple M1 (see `bench/native-vectors.bench.json`): 1.6-1.8x top-K search, 2.4x pairwise clustering scans at the measured 100- and 1000-candidate points (0.86x at the measured 10-candidate point, where the whole scan costs ~20µs either way), and 22-36x MMR rerank. Binary vector search stays on the TypeScript loops (per-call packing measured as a wash), as do custom `similarityFn` MMR reranks and incremental per-row scoring ([#6280](https://github.com/can1357/oh-my-pi/pull/6280) by [@wolfiesch](https://github.com/wolfiesch)).
## [17.0.4] - 2026-07-18
packages/mnemopi/bench/native-vectors.bench.json
+35 -142
View File
@@ -1,207 +1,100 @@
{
"sha": "3c84e2c410afae163fb1a778a4f7011354cc893c",
"date": "2026-07-22T11:33:29.804Z",
"sha": "8047bedaa46276710c9544d08f12bb13996da858",
"date": "2026-07-22T12:17:28.977Z",
"scenario": "dim=384, stride=48B, warmup=20, iterations=200 (adaptive for O(n²) rows, see per-row fields), crossing-inclusive",
"runtime": "bun 1.3.14",
"host": "Apple M1, darwin-arm64",
"rows": [
{
"kernel": "cosineSimilarityBatch",
"kernel": "searchExactVectorIndex (topK wrapper)",
"count": 10,
"ts_us": 9.85,
"native_us": 3.2,
"speedup": 3.08,
"ts_us": 17.34,
"native_us": 9.49,
"speedup": 1.83,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "cosineSimilarityBatch",
"kernel": "searchExactVectorIndex (topK wrapper)",
"count": 100,
"ts_us": 41.15,
"native_us": 25.18,
"speedup": 1.63,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "cosineSimilarityBatch",
"count": 1000,
"ts_us": 398.55,
"native_us": 242.88,
"speedup": 1.64,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "cosineSimilarityBatch",
"count": 10000,
"ts_us": 3936.21,
"native_us": 2431.53,
"ts_us": 56,
"native_us": 34.65,
"speedup": 1.62,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "searchExactVectorIndex (topK wrapper)",
"count": 10,
"ts_us": 5.82,
"native_us": 6.44,
"speedup": 0.9,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "searchExactVectorIndex (topK wrapper)",
"count": 100,
"ts_us": 36.99,
"native_us": 19.48,
"speedup": 1.9,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "searchExactVectorIndex (topK wrapper)",
"count": 1000,
"ts_us": 310.52,
"native_us": 165.81,
"speedup": 1.87,
"ts_us": 508.91,
"native_us": 300.93,
"speedup": 1.69,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "searchExactVectorIndex (topK wrapper)",
"count": 10000,
"ts_us": 3513.57,
"native_us": 1884.54,
"speedup": 1.86,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "hammingDistanceBatch (incl. packing)",
"count": 10,
"ts_us": 3.07,
"native_us": 1.39,
"speedup": 2.21,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "hammingDistanceBatch (incl. packing)",
"count": 100,
"ts_us": 7.09,
"native_us": 3.97,
"speedup": 1.79,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "hammingDistanceBatch (incl. packing)",
"count": 1000,
"ts_us": 34.25,
"native_us": 26.11,
"speedup": 1.31,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "hammingDistanceBatch (incl. packing)",
"count": 10000,
"ts_us": 221.43,
"native_us": 207.45,
"speedup": 1.07,
"ts_us": 5562.41,
"native_us": 3437.82,
"speedup": 1.62,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "cosineSimilarityPairs (incl. flatten+adjacency)",
"count": 10,
"ts_us": 246.34,
"native_us": 22.08,
"speedup": 11.16,
"ts_us": 26.42,
"native_us": 30.67,
"speedup": 0.86,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "cosineSimilarityPairs (incl. flatten+adjacency)",
"count": 100,
"ts_us": 5767.19,
"native_us": 1381.02,
"speedup": 4.18,
"ts_us": 4418.75,
"native_us": 1866.25,
"speedup": 2.37,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "cosineSimilarityPairs (incl. flatten+adjacency)",
"count": 1000,
"ts_us": 530658.9,
"native_us": 131050.53,
"speedup": 4.05,
"ts_us": 445625.44,
"native_us": 179685.28,
"speedup": 2.48,
"ts_iterations": 10,
"native_iterations": 10
},
{
"kernel": "hammingDistanceForDimBatch (incl. packing)",
"count": 10,
"ts_us": 2.52,
"native_us": 1.95,
"speedup": 1.29,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "hammingDistanceForDimBatch (incl. packing)",
"count": 100,
"ts_us": 5.88,
"native_us": 4.16,
"speedup": 1.41,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "hammingDistanceForDimBatch (incl. packing)",
"count": 1000,
"ts_us": 27.69,
"native_us": 28.69,
"speedup": 0.97,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "hammingDistanceForDimBatch (incl. packing)",
"count": 10000,
"ts_us": 231.52,
"native_us": 235.98,
"speedup": 0.98,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "mmrRerankIndices (via mmrRerank)",
"count": 10,
"ts_us": 331.56,
"native_us": 15.1,
"speedup": 21.95,
"ts_us": 395.5,
"native_us": 17.62,
"speedup": 22.44,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "mmrRerankIndices (via mmrRerank)",
"count": 100,
"ts_us": 7965.61,
"native_us": 220.67,
"speedup": 36.1,
"ts_us": 10439.19,
"native_us": 287.99,
"speedup": 36.25,
"ts_iterations": 200,
"native_iterations": 200
},
{
"kernel": "mmrRerankIndices (via mmrRerank)",
"count": 1000,
"ts_us": 78042.9,
"native_us": 2639.47,
"speedup": 29.57,
"ts_us": 103521.56,
"native_us": 3430.07,
"speedup": 30.18,
"ts_iterations": 200,
"native_iterations": 200
}
],
"sink": 823377224.8853176
"sink": 15782.734317403312
}
packages/natives/CHANGELOG.md
+1 -1
View File
@@ -4,7 +4,7 @@
### Added
- Added batch vector kernels for mnemopi recall paths: `cosineSimilarityBatch`, `cosineSimilarityPairs`, `vectorIndexTopK`, `hammingDistanceBatch`, `hammingDistanceForDimBatch`, and `mmrRerankIndices` ([#6280](https://github.com/can1357/oh-my-pi/pull/6280) by [@wolfiesch](https://github.com/wolfiesch)).
- Added batch vector kernels for mnemopi recall paths: `cosineSimilarityPairs`, `vectorIndexTopK`, and `mmrRerankIndices` ([#6280](https://github.com/can1357/oh-my-pi/pull/6280) by [@wolfiesch](https://github.com/wolfiesch)).
## [17.0.5] - 2026-07-18