Reproducibility
What was frozen, when, and how to check it.
Every configuration and pass/fail rule was committed to git before the data that tested it was downloaded. This page lists the periods, hashes, commits, seeds and the order of operations.
The final test was frozen before download
final config committed (git)2026-09-26T13:03:26-04:00
first test image written (file system)2026-09-26T17:03:50.836025+00:00
config committed before the first test imageyes
final configconfig/phase3_final_test_config.json
final config hash6f35d7fc205cdffe4a971974362cf77a5bbeb7a6ce518e8831b172292539b49e
config commit → result commit96e0694 → fdc2122
Dataset periods
Four periods, never pooled
| period | sols | role | dataset manifest | manifest SHA-256 | run |
|---|---|---|---|---|---|
| development | 412–430 | tuning and method design | data/manifests/navcam_development.json | 1d59a989c117b9c2… | artifacts/phase3_3/20260926T134654-phase3.3-dev-32ec |
| validation1 | 779–820 | first out-of-sample check (informed post-validation simplification; not evidence for later claims) | data/manifests/navcam_validation.json | 7007c544ed4f0bae… | artifacts/phase3_3/20260926T142033-phase3.3-val-2a31 |
| validation2 | 1100–1129 | fresh validation of the simplified claim (selected from PDS listings only) | data/manifests/navcam_validation2.json | 0be8dfa5dab17814… | artifacts/phase3_4/20260926T165307-phase3.4-val2-1248 |
| test | 950–979 | final held-out test — config frozen before download, analysed once | data/manifests/navcam_test.json | 7fb27a574f5d9771… | artifacts/phase3_final/20260926T174018-phase3-final-test-e4e2 |
Configurations
Frozen configuration hashes
config/phase3_3_validation_config.json87e52b59c77269fc0bb860e5a34a91a3c905ca8cb535cd383e6ffaca1f635ca0
config/phase3_4_validation_config.json3498febc3a4c87d516ab779871104576405294f3fae756d9c50f23a66a5ee0b8
config/phase3_final_test_config.json6f35d7fc205cdffe4a971974362cf77a5bbeb7a6ce518e8831b172292539b49e
data/splits/phase3_splits.jsonbd927885d8fd04f903f7ab8475213d0541cb88e11a11642268edf284498757f8
data/splits/phase3_validation2.json90e9de3f3fa6bcc18193fd60147933cf8f2712ba3b6dac3c6767a6b0206632c2
Each runner recomputes its config hash and the SHA-256 of every listed source file and refuses to start if anything differs.
Run order
Git tags, in order
| # | tag | commit | committed | stage |
|---|---|---|---|---|
| 1 | v0.1-baseline | 90c7f4a | 2026-09-24T15:23:51-04:00 | Phase 1 · telemetry triage prototype |
| 2 | phase2-complete | 58725e0 | 2026-09-25T06:19:44-04:00 | Phase 2 · Jev semantic decision evaluation |
| 3 | phase3-baseline | 577d5be | 2026-09-25T09:25:06-04:00 | Phase 3 · Navcam downlink baseline |
| 4 | phase3.1-complete | af661c6 | 2026-09-25T09:41:06-04:00 | Phase 3.1 · controlled visual tests |
| 5 | phase3.2-complete | 6b913fa | 2026-09-25T10:07:43-04:00 | Phase 3.2 · stereo-safe scheduler |
| 6 | phase3.3-config-frozen | 6bce97d | 2026-09-26T09:44:00-04:00 | Phase 3.3 · config + rules frozen before validation1 download |
| 7 | phase3.3-complete | e78aaff | 2026-09-26T10:24:57-04:00 | Phase 3.3 · validation1 (779–820) |
| 8 | phase3.4-config-frozen | 566c323 | 2026-09-26T12:31:02-04:00 | Phase 3.4 · claim + decision rule pre-registered |
| 9 | phase3.4-validation2-frozen | f800550 | 2026-09-26T12:31:53-04:00 | Phase 3.4 · validation2 interval frozen from listings |
| 10 | phase3.4-complete | 33ef7d8 | 2026-09-26T12:54:53-04:00 | Phase 3.4 · fresh validation (1100–1129) |
| 11 | phase3-final-test-config-frozen | 96e0694 | 2026-09-26T13:03:26-04:00 | Final test config frozen before download |
| 12 | phase3-final-test-complete | fdc2122 | 2026-09-26T13:41:29-04:00 | Final held-out test (950–979) |
Random seeds
Every stochastic component is seeded
{
"phase3_3": {
"random_baseline_order": 0,
"synthetic_seed_tag": {
"development": "DEVREPRO",
"validation": "VALIDATION"
},
"synthetic_source_sampling": "random.Random(seed_for('SYNTH_V2', tag, family)) — one stream per family",
"near_duplicate_sampling": "random.Random(seed_for('SYNTH_V2', tag, 'NEAR_DUPLICATE'))",
"synthetic_control": "numpy default_rng(seed_for('SYNTH_V2', tag, synthetic_id)) inside synthetic_v2.perturb / near_duplicate",
"embedding_random_pairs": 11,
"phash_false_merge_examples": 3,
"bootstrap": null,
"resolved": {
"synthetic_source_sampling": {
"BLUR": 884331856,
"EXPOSURE": 1683474208,
"MISSING_REGION": 503821405,
"STRIPING": 3477892241,
"FRAME_DROPOUT": 626936839,
"LOCALIZED_STRUCTURE": 712103420,
"TEXTURE_CHANGE": 3401736209
},
"near_duplicate_sampling": 4124308942
}
},
"phase3_4": {
"random_order": 0,
"bootstrap": 20260926,
"synthetic_tag": "VALIDATION2",
"embedding_random_pairs": 11
},
"final": {
"random_order": 0,
"bootstrap": 20260926
}
}Integrity
Scientific artifacts are hash-locked
manifestdocs/release/science-artifacts.json
artifacts covered70
aggregate SHA-256eaffef74a4f94dccb2a56a7a75ca8c2cc4c7fae4c7b84b583e01c67c48591068
Commands
uv run python scripts/release_integrity.py --verify uv run python scripts/run_phase3_final_test.py --stage analysis --run artifacts/phase3_final/20260926T174018-phase3-final-test-e4e2 uv run python scripts/build_release_data.py uv run python scripts/make_release_figures.py
Raw PDS products are not committed (they are re-downloadable from the URLs in each manifest and verified by SHA-256); manifests, configs, run artifacts and reports are.