A strong Project2 result spread across platforms. Independent experiments confirm that the first-request tool schema changes trajectories, but do not establish a stable engineering-ability gain.
Automatically summarized from source discussions. Sources and the full record are available below.
Evidence & timeline
Current conclusion
The trajectory effect is strong; the ability gain is unresolved. They are not interchangeable.
Maintainer response
No verified maintainer response found
Untriaged
From trajectory effect to ability claim
Observed signal
The original Project2 runs scored 99/96, and independent schema experiments separated Minimal-like trajectories in 9/9 and 5/5 runs.
Current boundary
The trajectory effect is strong; the ability gain is unresolved. They are not interchangeable.
Decision threshold
Freeze every protocol field and run a preregistered paired Minimal, Standard, and Anchored comparison on a second public engineering task, reporting both ability and cost.
Trajectory separation has crossed its evidence threshold; stable ability gain has not.
Revision criteria
Stable cross-task paired gains with auditable trajectories would upgrade the ability claim; continued trajectory separation without ability differences would narrow the conclusion to an interface-behavior effect.
Current conclusion
The original Project2 result is directionally strong, but independent runs do not stably confirm the ability magnitude; the evidence remains mixed.
Scope
V4 Pro on Project2 and adjacent engineering tasks; different protocols must be interpreted separately.
Verification
Mixed evidence
Maintainer response
Untriaged
Author response
Author update · Aug 17, 2026, 02:13 AM
Product impact
Affects preset selection, community recommendations, and benchmark interpretation.
Suggested next action
Freeze DSH, model route, tool schema, and task version, then run a preregistered paired replication on a second engineering task.
Lifecycle
Active
Source scope
Public GitHub and Bilibili sources reviewed through 2026-08-22
Attribution confidence · 98%
First found by this site · Aug 22, 2026, 03:30 PM
Status change log
Verification · Supported → Mixed evidence · Aug 16, 2026, 04:13 PM
Multi-environment runs did not reproduce 98/99, and the randomized ability difference was not significant.
Open evidence ↗Verification · Not yet verified → Supported · Aug 14, 2026, 03:15 AM
Project2 showed a strong within-environment direction with few runs.
Open evidence ↗Supporting evidence
The original Project2 runs scored 99/96, and independent schema experiments separated Minimal-like trajectories in 9/9 and 5/5 runs.
Conflicting evidence
Multi-environment and randomized independent runs did not stably reproduce the 98/99 ability magnitude; the n=3 comparison was not significant.
Key counterargument
Independent tasks, environments, and samples remain limited. Failure to detect a stable gain is not proof of no gain, and interface alignment may matter only for specific models and tasks.
Missing evidence
Cross-task paired repetitions with a frozen model revision, harness commit, task version, and complete raw trajectories are missing.
Verification plan
Freeze every protocol field and run a preregistered paired Minimal, Standard, and Anchored comparison on a second public engineering task, reporting both ability and cost.
Revision criteria
Stable cross-task paired gains with auditable trajectories would upgrade the ability claim; continued trajectory separation without ability differences would narrow the conclusion to an interface-behavior effect.
Scope & boundaries
The assessment covers only reviewed V4 Pro, Project2, and adjacent engineering experiments and does not generalize to every model, task, or DSH version.
| Source | Contributor | Stance | Independent sources | Exact evidence |
|---|---|---|---|---|
| Author response to the independent replication Aug 17, 2026, 02:13 AM · Version 1 | xiaobright Resolver | Corrects | Source-author response | “the mechanism claim (trajectory anchoring) is strong; the ability claim is unresolved.” |
| Publish DeepSeek V4 Pro harness comparison results Aug 14, 2026, 03:15 AM · Version 1 | xiaobright Originator | Originates | Original Project2 experiment and propagation | “DSH minimal + max 两跑为 99/96；同一 WSL/max 环境中，minimal 为 99/96，standard 为 91，PTC 为 92。” |
| windows下使用标准工具和极简模式思维链实现极简模式跑分 Aug 14, 2026, 02:13 PM · Version 1 | 小明XBright Amplifier | Supports | Original Project2 experiment and propagation | “官方 Standard/PTC 只有 91/92 分，Minimal 是 99/96，但只有两个工具。” |
| Multi-env test report: Ability 85–90, 98/99 not reproduced Aug 16, 2026, 02:40 AM · Version 1 | JimMilk Challenger | Challenges | Multi-environment reproduction | “Ability 85–90，未复现 README 的 98/99。” |
| Independent replication: trajectory reproduces, ability gap does not Aug 16, 2026, 04:13 PM · Version 1 | TipsyDrifter Challenger | Challenges | Randomized-block reproduction | “The ability advantage did not reproduce at n=3. Means were standard 90.0, anchored-standard 93.3, whale-standard 90.7. One-way ANOVA F(2,6) = 1.05 — not significant.” |
| DeepSeek Harness 极简模式真能让 V4 Pro 变强？44 次工程实测！ Aug 17, 2026, 12:00 AM · Version 1 | 程序员阿江-Relakkes Challenger | Challenges | Public DeepSWE and Harbor challenge | “现有结果还不足以证明它能稳定增强工程能力。” |
程序员阿江-Relakkes · Not reproduced
| Protocol field | Original run | Reproduction | Comparability |
|---|---|---|---|
| Model provider• | DeepSeek official API | Unknown | Missing from reproduction run |
| Model• | deepseek-v4-pro | deepseek-v4-pro | Match |
| Model revision• | Unknown | Unknown | Missing from original run |
| Serving method• | official API; route id not published | Unknown | Missing from reproduction run |
| Harness commit• | 47f9438 lineage | Unknown | Missing from reproduction run |
| Preset• | minimal | minimal versus standard | Mismatch |
| Reasoning effort | max | Unknown | Missing from reproduction run |
| Benchmark• | Project2 V4.1b | DeepSWE and Harbor | Mismatch |
| Benchmark commit• | 04255b55f16c4439e538239fb9783070c4165081 | Unknown | Missing from reproduction run |
| Task set• | one private project task | 22 comparisons; task-level artifacts not public | Mismatch |
| Scorer version• | Unknown | Unknown | Missing from original run |
| Run count | 2 | 44 | Mismatch |
JimMilk · Not reproduced
| Protocol field | Original run | Reproduction | Comparability |
|---|---|---|---|
| Model provider• | DeepSeek official API | Unknown | Missing from reproduction run |
| Model• | deepseek-v4-pro | deepseek-v4-pro | Match |
| Model revision• | Unknown | Unknown | Missing from original run |
| Serving method• | official API; route id not published | official endpoint; route id not published | Mismatch |
| Harness commit• | 47f9438 lineage | DSH 0.1.0-rc.5 | Mismatch |
| Preset• | minimal | multiple minimal, standard, and anchored snapshots | Mismatch |
| Reasoning effort | max | max | Match |
| Benchmark• | Project2 V4.1b | Project2 V4.1b | Match |
| Benchmark commit• | 04255b55f16c4439e538239fb9783070c4165081 | Unknown | Missing from reproduction run |
| Task set• | one private project task | same project; environment-specific limits | Mismatch |
| Scorer version• | Unknown | Unknown | Missing from original run |
| Hardware | Unknown | macOS, Windows 11 ARM64 VM, Linux Lima aarch64 | Missing from original run |
| Run count | 2 | 11 | Mismatch |
TipsyDrifter · Inconclusive
| Protocol field | Original run | Reproduction | Comparability |
|---|---|---|---|
| Model provider• | DeepSeek official API | Unknown | Missing from reproduction run |
| Model• | deepseek-v4-pro | deepseek-v4-pro | Match |
| Model revision• | Unknown | Unknown | Missing from original run |
| Serving method• | official API; route id not published | official endpoint; route id not published | Mismatch |
| Harness commit• | 47f9438 lineage | DSH 0.1.0-rc.6 | Mismatch |
| Preset• | minimal | anchored-standard | Mismatch |
| Reasoning effort | max | max | Match |
| Benchmark• | Project2 V4.1b | Project2 V4.1b | Match |
| Benchmark commit• | 04255b55f16c4439e538239fb9783070c4165081 | 04255b55f16c4439e538239fb9783070c4165081 | Match |
| Task set• | one private project task | same project without ESP-IDF build | Mismatch |
| Scorer version• | Unknown | Unknown | Missing from original run |
| Hardware | Unknown | Windows 11 | Missing from original run |
| Run count | 2 | 3 | Mismatch |
Discovery
An early public showcase of the V4 formal preview; no evidence shows that this source originated the DSH Minimal ability claim.
Discovery
An early monitored publication of the Minimal, Standard, and PTC Project2 comparison.
Amplification
A Bilibili video amplified the Anchored 98/99 result and publicly linked both GitHub repositories.
Failed reproduction
Eleven runs across three environments scored 85–90 and did not reproduce 98/99.
Reproduction
The randomized-block study reproduced trajectory separation, while the ability confidence interval crossed zero.
Challenge
An aggregate of 44 DeepSWE/Harbor runs did not show a stable Minimal engineering-ability gain.
Author update
The source author accepted the bounded conclusion: strong trajectory mechanism, unresolved ability claim.
Current conclusion
Official code confirms that Minimal reproduces the RL prompt/schema, and two independent experiments reproduce tool-surface trajectory separation. This does not establish an ability gain.
Scope
V4 Pro, DSH 0.1.0-rc.5/rc.6, Project2, and first-request probes.
Verification
Reproduced
Maintainer response
Untriaged
Author response
No response recorded
Product impact
Affects presets, tool schemas, the agent loop, and trajectory diagnostics.
Suggested next action
Within one version and task, separately ablate the system prompt, tool schema, and compaction.
Lifecycle
Active
Source scope
Public GitHub and Bilibili sources reviewed through 2026-08-22
Attribution confidence · 95%
First found by this site · Aug 22, 2026, 03:30 PM
Status change log
Verification · Supported → Reproduced · Aug 16, 2026, 04:13 PM
Two independent experiments reproduced tool-surface trajectory separation under bounded protocols.
Open evidence ↗| Source | Contributor | Stance | Independent sources | Exact evidence |
|---|---|---|---|---|
| 首轮工具 schema 是锚定的决定变量 Aug 15, 2026, 04:26 AM · Version 1 | Rtyyy233 Reproducer | Reproduces | Controlled first-request schema experiment | “minimal 的工具 schema 在 256000 下无需任何 maxTokens 封顶就能稳定锚定（5/5）。” |
| Minimal preset snapshot test at 47f9438 Aug 13, 2026, 11:56 AM · Version 1 | DeepSeek Harness Maintainers Synthesizer | Neutral | Official Minimal configuration fact | “sends the exact RL prompt and schemas” |
| Independent replication: trajectory reproduces, ability gap does not Aug 16, 2026, 04:13 PM · Version 1 | TipsyDrifter Reproducer | Reproduces | Whole-trajectory randomized-block reproduction | “The trajectory anchoring is real and very robust. 9/9 runs separated perfectly by preset.” |
Rtyyy233 · Supports claim
| Protocol field | Original run | Reproduction | Comparability |
|---|---|---|---|
| Model provider• | Unknown | DeepSeek official API | Missing from original run |
| Model• | deepseek-v4-pro | deepseek-v4-pro | Match |
| Model revision• | Unknown | Unknown | Missing from original run |
| Serving method• | Unknown | Unknown | Missing from original run |
| Harness commit• | 47f9438 lineage | DSH 0.1.0-rc.5 package | Mismatch |
| Preset• | minimal | minimal versus standard-family bootstrap | Mismatch |
| Reasoning effort | max | max | Match |
| Tool schema | official minimal pair at 47f9438 | Unknown | Missing from reproduction run |
| Benchmark• | Project2 V4.1b and trajectory probes | first-request trajectory probe | Mismatch |
| Benchmark commit• | Unknown | Unknown | Missing from original run |
| Task set• | Unknown | English probe, Project2 prompt, and Chinese task | Missing from original run |
| Scorer version• | Unknown | Unknown | Missing from original run |
| Run count | Unknown | 45 | Missing from original run |
TipsyDrifter · Supports claim
| Protocol field | Original run | Reproduction | Comparability |
|---|---|---|---|
| Model provider• | Unknown | Unknown | Missing from original run |
| Model• | deepseek-v4-pro | deepseek-v4-pro | Match |
| Model revision• | Unknown | Unknown | Missing from original run |
| Serving method• | Unknown | Unknown | Missing from original run |
| Harness commit• | 47f9438 lineage | DSH 0.1.0-rc.6 | Mismatch |
| Preset• | minimal | standard versus anchored-standard versus whale-standard | Mismatch |
| Reasoning effort | max | max | Match |
| Tool schema | official minimal pair at 47f9438 | Unknown | Missing from reproduction run |
| Benchmark• | Project2 V4.1b and trajectory probes | Project2 V4.1b | Mismatch |
| Benchmark commit• | Unknown | 04255b55f16c4439e538239fb9783070c4165081 | Missing from original run |
| Task set• | Unknown | Unknown | Missing from original run |
| Scorer version• | Unknown | Unknown | Missing from original run |
| Run count | Unknown | 9 | Missing from original run |
Reproduction
Forty-five controlled sessions supported first-request tool-schema trajectory separation.
Current conclusion
Fixed repository history and two run reviews agree; the source author updated the README.
Scope
The historical Anchored Project2 runs from 2026-08-14, not the current preset.
Verification
Supported
Maintainer response
Untriaged
Author response
Documentation change · Aug 17, 2026, 02:15 AM
Product impact
Prevents the current exact-Minimal tool pair from inheriting historical 98/99 scores.
Suggested next action
Keep the preset commit and first-request tool catalog beside every future score.
Lifecycle
Resolved
Source scope
GitHub history and fixed review files examined through 2026-08-22
Attribution confidence · 100%
First found by this site · Aug 22, 2026, 03:30 PM
Status change log
Lifecycle · Active → Resolved · Aug 17, 2026, 02:15 AM
The source author added the provenance note to the README.
Open evidence ↗Verification · Not yet verified → Supported · Aug 16, 2026, 12:27 PM
Fixed history and two run reviews consistently identify the historical tool surface.
Open evidence ↗| Source | Contributor | Stance | Independent sources | Exact evidence |
|---|---|---|---|---|
| Attribute the 98/99 runs to the legacy pwsh/read configuration Aug 16, 2026, 12:27 PM · Version 1 | MolecularFullerene Synthesizer | Corrects | Historical configuration audit and author correction | “Those runs used the Minimal system prompt, but the first-request Windows tool surface was pwsh + read. After the first durable tool/call, the catalog changed to the full 25-tool Standard surface.” |
| Author confirms the provenance correction Aug 17, 2026, 02:15 AM · Version 1 | xiaobright Resolver | Supports | Historical configuration audit and author correction | “明确 98/99/99 三轮使用的是 Minimal 系统提示 + pwsh/read 首请求面 + 完整 25 工具晋升目录” |
No reproduction recorded.
Correction
Fixed history showed that 98/99 used the historical pwsh/read → 25 tools configuration.
Documentation change
The README was updated with the historical configuration provenance for 98/99.
Minimal preset snapshot test at 47f9438
Open source Opens in a new tabPublish DeepSeek V4 Pro harness comparison results
Open source Opens in a new tabwindows下使用标准工具和极简模式思维链实现极简模式跑分
Open source Opens in a new tabMulti-env test report: Ability 85–90, 98/99 not reproduced
Open source Opens in a new tabAttribute the 98/99 runs to the legacy pwsh/read configuration
Open source Opens in a new tabIndependent replication: trajectory reproduces, ability gap does not
Open source Opens in a new tabDeepSeek Harness 极简模式真能让 V4 Pro 变强？44 次工程实测！
Open source Opens in a new tabAuthor response to the independent replication
Open source Opens in a new tabAuthor confirms the provenance correction
Open source Opens in a new tab