Some tests check task completion and reproducibility; others measure modeling quality, decisions and token use. Each result links to its scope and source records.
01
Usage & reliability
Task delivery, tool integration, saved-model reuse and consistency across versions.
All 30 tasks reproduced. Mean model quality preserved.
All 30 tasks and 472 candidates were checked against the original TabArena binary-classification protocol. Final model selections, candidate metrics and prediction rankings matched, preserving the historical macro ROC-AUC.
3 October 2026 · validation for the r7 release
Macro ROC-AUC
81.669%
Original model quality retained
100%
Original protocol versus pre-release reproduction: macro average across 30 tasks.
Metric
Original result
Reproduction
ROC-AUC ↑
81.669%
81.669%
Mean quality change
—
0 percentage points
Task coverage
30 / 30
30 / 30
ROC-AUC is displayed as a percentage, not classification accuracy. The reproduced value follows from identical prediction rankings on the same labels; hidden labels were not rescored.
Execution scope and numerical agreement+
The original TabArena v0.1 binary-30 subset, r0/f0/s0. The saved application and original SDK associated with the r7 release work ran on DGX ARM64. Raw scores were byte-identical on 28 tasks; the other two had a maximum difference of about 2.04×10⁻¹⁴. All rankings matched, and all 30 models reproduced their saved predictions byte for byte. Final Windows/Linux archive acceptance was separate.
This replay did not collect new DSH token usage. The separate five-task efficiency study below reports its own model, quality and usage; its token reduction is not assigned to Full-30.
18 / 18Prediction deliveries across ordinary DSH and DSH + RISA
Each path completed three tasks, with three sessions per task. All nine RISA runs saved a model; reloaded predictions matched the delivered files exactly.
Matching RISA model replays
9 / 9
Sessions / task / path
3
This historical v6 integration check used three development tasks, real model requests and a local authorization fixture. Quality and token comparisons remain in the detailed record below.
200 / 200LongBench v1 HotpotQA · all questions completed
YunXiaoHe 0.3.7 completed all 200 questions zero-shot, with no terminal failures and a QA-F1 of 74.56. Per-question answers, usage and scoring code are public.
v2 completed / all questions
499 / 503
v2 correct / all questions
337 / 503
v2 accuracy is 67.0%, counting four unfinished questions as incorrect. It is a published provisional, self-reported result, not an official leaderboard submission. The v1 and v2 metrics are not directly comparable.
Detailed record: early DSH integration and quality comparisons+
DSH & RISA · 2 OCTOBER 2026 · HISTORICAL v6
Ordinary DSH and the RISA integration
Both paths received the same request and data and used DeepSeek Flash. Ordinary DSH could write code, select models and tune them; the integrated path used the then-current RISA balanced optimizer. Each of three tasks ran three times, with model-usage receipts, delivered predictions and saved-model checks.
Token use fell substantially. Iris accuracy was comparable; errors were higher on both regression tasks. These usage figures do not describe the later accuracy-first or r7 releases.
Historical v6: means of three sessions per task. Both paths used the same fixed data split.
Task & metric
Ordinary DSH tokens
DSH + RISA tokens
Token reduction
Ordinary DSH → with RISA
Iris Accuracy ↑
399,363
11,263
97.18%
98.92% → 100.00%
Diabetes RMSE ↓
701,886
15,149
97.84%
53.1941 → 58.5824
Friedman1 RMSE ↓
676,825
16,187
97.61%
1.0162 → 3.5323
How were these figures calculated?+
Token totals include input, output, cache reads/writes and auxiliary calls such as session titles. Reasoning is already counted in output. The reduction is calculated from the two token totals; it is not a billing reduction. Repetitions are agent sessions, not independent datasets or splits.
These were fixed development tasks. Tool and context organization differed, YH authorization used a local fixture, and model-provider requests were real. A later accuracy-first revision reduced regression errors in a separate native test, without remeasuring tokens; those results are not combined here.
An independent record of 60 Vision cases: 52 completed reports, 7 genuine HOLDs, and 1 original route failure. Model-judged content quality averaged 0.661442 over completed reports; interim Penalty-SAR was 0.872222 over 59 observed original traces. The amended per-case L1 mean SAR was 0.929722 with an explicit zero for the original failure. The original scorer gate remains FAILED; no official Overall score is available.
Completed-report content quality · 52 cases
0.661442
Full-batch content quality · 60 cases
0.573250
Interim Penalty-SAR · 59 traces
0.872222
Amended L1 mean SAR · 60 slots
0.929722
Original case outcomes
52 completed · 7 HOLD · 1 route failure
Includes a bilingual account, 60 rows of derived case scores, and an aggregate summary. The separate retry did not replace the original failure; metric definitions and source hashes are linked.
02
Capabilities
Modeling quality, decision scores and token use, with the baselines, datasets and metrics kept explicit.
RISA · TABARENA
Tabular modeling, across 30 datasets.
0.816690Macro ROC-AUC ↑
RISA produced 241,578 predictions on 30 public binary-classification datasets from TabArena v0.1. Macro ROC-AUC improved by 3.60 percentage points over our previous result on the same set, with gains on 24 datasets.
Split
r0 / f0 / s0
Versus our prior result: wins / ties / losses
24 / 0 / 6
The result covers the 30-dataset binary subset, not the full TabArena leaderboard.
YunXiaoHe’s Supply Chain Analysis Agent, powered by Avalon r4, completed decisions step by step in the frozen environment: 36 actions per scenario, all 16 runs completed.
Completed scenarios
16 / 16
Completed actions
576 / 576
The same frozen environment, seeds and cost formula were used. The score is in benchmark units, not percent or dollars.
62.7% fewer tokens. Accuracy within one percentage point.
Both paths used DeepSeek Pro. With RISA-G1, mean accuracy across five tasks went from 83.87% to 82.97%, while mean token use fell from 683,477 to 254,678.
Published record · 6 September 2026
Reduction in mean tokens
62.7%
Baseline accuracy retained
98.9%
One model, two task-processing paths. Values are means across five tasks.
Metric
DeepSeek Pro Native path
DeepSeek Pro + RISA-G1
LLM tokens ↓
683,477
254,678
Accuracy ↑
83.87%
82.97%
Macro F1 ↑
0.7952
0.7545
Time (seconds) ↓
223.9
241.9
98.9% is the share of baseline accuracy retained, not the final accuracy. The absolute difference is 0.90 percentage points. Token use fell in this study; elapsed time did not.
Tasks, comparison scope and source records+
Five tasks: Adult Income, Bank Marketing, Credit-G, Spambase, Steel Plates.
This historical same-model efficiency study is separate from the 30-dataset TabArena evaluation. The token reduction is not a billing percentage or a promise for every task on the current download. The separate Flash-agent row in the public CSV uses a different model and metering scope and is not pooled here.
The system produced 241,578 predictions with a mean ROC-AUC of 0.8167. This improved on our earlier 0.7807 result on the same set, with gains on 24 datasets.
Thirty binary-classification datasets from TabArena. ROC-AUC measures how well a model separates the two classes.YUNXIAOHE + AVALON / SUPPLYCHAINBENCH
576sequential decisions
Avalon completed 16 frozen scenarios with 36 actions per scenario and received an environment cost score of 54.81.
Results were checked against the public supply-chain simulation environment and its frozen scorer.
Shared settings: distilled LTX-2.5 22B, original BF16 weights, no additional quantization, 704 × 448 pixels, 121 frames at 24 fps, and about 5.04 seconds of audio and video.
RTX 4050 Laptop
6 GB VRAM
One sample took about 48.12 minutes.
Total staged time from text condition to completed video.
RTX A6000
48 GB VRAM
Ten concurrent jobs averaged about 46.23 minutes per batch.
Three rounds, 30 tasks, and 29 complete videos.
DGX Spark · GB10
128 GB unified memory
Serial execution averaged about 8.74 minutes per item.
All 30 tasks produced complete videos, including dynamic text encoding.
Execution methods and timing boundaries differ across the devices. A complete video means that the media file could be decoded; image quality and prompt alignment require separate evaluation.
Complete records and scoring methods+
Data and evaluation
YH / Benchmark
Hugging FaceBenchmark
RISA × Avalon
Public evaluation of YunXiaoHe 0.3.9.8
Public results, figures, and evaluation evidence covering 30 tabular datasets and 16 sequential supply-chain runs.
Two evaluations of YunXiaoHe 0.3.9.8. RISA performs tabular modeling on 30 public datasets from TabArena v0.1 binary r0f0s0. The supply-chain agent Avalon r4 completes sequential decisions across 16 seeds in the frozen SupplyChainBench environment.
TabArena, 30 datasets · macro ROC-AUC
0.816690
Compared with our prior result on the same set
+3.60 percentage points · wins on 24/30
SupplyChainBench score
54.810907
Avalon completion
16/16 runs · 576/576 actions
Includes a technical report, client-facing results brief, paired score CSV, frozen leaderboard, figures, and evidence hashes. A separate historical five-task token-efficiency comparison is also included. Implementation code is outside the scope of this release.
YH / Evaluation data
Hugging FaceEvaluation data
YunXiaoHe
LongBench evaluation release
YunXiaoHe 0.3.7 on LongBench in a zero-shot setting, with per-item records and a reproducible scoring script.
A complete zero-shot evaluation of YunXiaoHe 0.3.7 on LongBench v1 HotpotQA. Per-item answers, scores, latency, and usage records are public, along with a script that recomputes the metric.
Completed samples
200 / 200
LongBench QA-F1
74.56
Includes JSONL records, a metric recomputation script, figures, and a technical report.
Files, licenses and evaluation methods are in the linked repositories. Earlier entries were checked on September 17, 2026; the Vision record was added on September 21, 2026. These are dated results, not new tests of the current download. Some Hugging Face entries show downloads in the last 30 days, synchronized hourly. Counts do not represent unique users. See the definitions for models and datasets.
These records describe their tested dates and versions, not current service uptime. For research methods and publications, see Research papers →
