docs(overhaul): record cathedral GPU recurrence and isolated checks

This commit is contained in:
Erik 2026-09-05 17:01:44 +02:00
parent f4bf5ffab8
commit 620540aba2
4 changed files with 217 additions and 1 deletions

View file

@ -101,7 +101,7 @@ High-profile indoor authored-light/material path needs comparison with retail.
blindly raise global ambient or change the owner's brightness settings.
Evidence: `research/2026-09-01-overhaul/475-owner-regate.md`.
## #477 — AMD watchdog / Vulkan device-loss crash while jumping in Neftet
## #477 — AMD watchdog / Vulkan device-loss crashes in Neftet and cathedral
**Status:** IN-PROGRESS, crash independently confirmed2026-09-05 on clean
`de427d2c0`. Owner saw an AMD crash notice while jumping around Neftet.
@ -135,6 +135,19 @@ The ordinary short Neftet run at1e376c244 exited0/gracefully after12jump-input
pairs; this is not12proven airborne jumps, a long soak or proof against the
owner's original recurrence. Owner stability acceptance remains open.
**Owner recurrence2026-09-05 at16:41:** exact green Releaseb2bdb09cc,
ordinary1280x720 High/MSAA4 run PID7852 crashed in the cathedral. Same
abandoned-frame ErrorDeviceLost and Windows LiveKernelEvent141; initiating
GPU fault still unknown. Cathedral reveal had completed. Retail was also
running, but coexistence is not proven causal; owner closed it for isolated
tests. Corrected core/sync and separate GPU-assisted cathedral routes both
completed/exit0/gracefully with no reported validation faults or new Windows
GPU reports;14PNGs inspected narrow PASS/provisional. The discarded first
attempt was a script spelling error, owner-authorized forced close, not a GPU
reproduction. These short instrumented runs do not establish recurrence safety.
`research/2026-09-01-overhaul/477-cathedral-recurrence.md` records exact paths,
hashes and boundaries. #477/G4 remain open; FPS deferred, no production change.
## #476 — First post-resize graphical gate PNG is horizontally corrupted
**Status:** IN-PROGRESS — capture and resize-boundary repairs integrated,

View file

@ -850,6 +850,18 @@ all retail outgoing paths/containment. No automatic four-cell golden or
production fix; request explicit scope expansion before AP159 repair (or
explicit owner decision to narrow the witness). FPS deferred, G4 unpassed.
**Owner cathedral crash and isolated checks2026-09-05:** after reporting the
visible fixes looked good (and questioning the Rynthid report), owner had a
confirmed AMD/LiveKernelEvent141 recurrence at16:41 on exact Releaseb2bdb09cc.
G4 stability FAIL. Corrected isolated core/sync and separate GPU-assisted
cathedral routes both completed/exit0/gracefully, no reported native faults or
new Windows GPU events;14PNGs opened narrow PASS/provisional. First attempt
discarded for lead's invalid input verb; only its client was force-stopped
after explicit owner authorization. Both clients now closed, ACE stays up.
No production/FPS changes and no new intentional deviation. Full evidence:
`477-cathedral-recurrence.md`. Actual initiating GPU fault and C1a retail
membership closure remain unresolved; short clean runs do not close #477/G4.
## 14. Risk register (condensed)
- **R1 second scene graph.** Graph records own only topology/membership facts;

View file

@ -0,0 +1,165 @@
# #477 — cathedral AMD recurrence and isolated diagnostics
2026-09-05. **Owner stability gate FAIL; no production repair in this record.**
The owner first said the visible fixes looked good and withdrew confidence in
the Rynthid placement report, then reported another AMD crash in the cathedral.
Do not treat the earlier visual acceptance as crash/stability acceptance.
FPS/#481/C2 performance work remains explicitly deferred; no main merge.
## Owner crash: confirmed, initiating GPU fault still unknown
Ordinary visible owner run `logs/owner-gate-20260905-162519-478-final/`,
PID7852. At16:41:20+02 Windows .NET Runtime1026 recorded unhandled
`ErrorDeviceLost` from `vkQueueSubmit2 (abandoned frame timeline signal)`.
Stderr also records secondary failed teardown waiting for timeline178412.
Windows Error Reporting1001 at16:41:28 and16:41:57 records LiveKernelEvent141,
bucket `LKD_0x141_Tdr:C_AppFault_IMAGE_amdkmdag.sys-PF:1-HWS:1-SCH-PS:95d63b9d-CS:6708cc79-GS:62c333ab`,
report b3e69f70-71fc-418c-b9e8-686a6bdaf552. WER names
`C:/WINDOWS/LiveKernelReports/WATCHDOG/WATCHDOG-20260905-1641.dmp`; its content
has NOT been opened or analyzed here. Two WER entries are one report, not two
separate GPU resets. RX9070XT, Vulkan1.4.349/driver2.0.395, 1280x720,
High/4xMSAA; ordinary profile without requested validation.
Generation12 at F4180104 completed and became visible before the crash.
The observed arrival was (36.9,48.7,169.805); last input log includes turning,
combat toggle and forward movement. The exact crash pose/trajectory is unknown.
Subsequent test login was saved at F418010F; arrival is not an exact crash pose.
Retail PID32728 also existed during the owner session; the owner closed it
before the isolated tests. Its coexistence is context, not a demonstrated cause.
Same failure family as Neftet, not proven same initiating fault. Source still
collapses fatal acquire and ordinary unavailability into false in
`VulkanGraphicsContext.SwapchainBackbuffer.TryAcquire`; the empty timeline
submit therefore may only expose an earlier device loss. No evidence connects
this crash to C1a's three-versus-four-cell membership question, which remains
unresolved. No geometry golden, AP159 repair, driver setting, TDR setting,
shader, synchronization rule, frame pacing, or exception suppression changed.
Owner logs SHA256: stdout
2B70E77CCDF9D4DFBB482143F49127216FF806A105984A30812A690B235C7A79;
stderr BC3C98C31F0B95606BC717ADDD202BB095F57C7F26822C82AAEFF8400E742FD4.
## Exact unchanged binary and diagnostic scope
Both tests use Release built at b2bdb09ccbd515f180b4fb3b7159de4d3913e68d.
Its locked restore/build is green0W/0E and literal-hermetic17028/0/0,
`artifacts/s5-final-hermetic-expectations/release-gate-summary.json`.
Source HEADf4bf5ffab was clean at launch and differs only in docs since that
build. App/Core DLL hashes were checked before each launch:
- App5692DF7F1495D235351CF87C8741781260B4243E6922476F7A23B8F2B025F305.
- CoreD4F15FEB44F99DBC257DF5911D302E82FC7FAB9BB549590B06FFE28E8685BF9B.
Same `logs/s5-c5-session-v10.json` and recipe10 package, High/MSAA4,
1280x720. ACE UDP9000PID13340 checked up; both client names checked absent
before each launch. One run per background command, no builds/tests beside
graphics. Automation enables checkpoint/screenshot machinery and borderless
exact-framebuffer mode; these are not identical to the ordinary owner's
window/session, nor are validation runs FPS measurements.
The corrected81-command route is archived as `route.txt` in each successful
run. SHA25692F5797C23455D569630C557A6237D8FFADC0636F26641D1B2FFE24BB7FEA9C7.
Lead checked all verb and InputAction names against production source,
balanced every held input, and checked the terminal close-client. Workload:
saved cathedral login, F4180104 arrival, two forward/back and turning input
sets/two combat toggles, F4180113 stairwell, eight -20 mouse-look steps,
return tilt/right camera turn, and30seconds settled. Input pairs are workload,
not proof of exact world displacement or a particular combat result.
## Discarded first attempt — test-script failure, not a GPU reproduction
`logs/selfgate-20260905-164528-477-cathedral-validation-r1-b2bdb09cc/`,
PID7444, aborted at line30: unknown `MovementBackward` (correct enum is
`MovementBackup`). Only login and arrival screenshots were captured. This is
**FAIL/incomplete**, never a passing route. Original route is preserved there;
SHA256A05CB6933ABD0C06467A639593AB6AA857753467F3A5106373B8BB5BB59D8882.
CloseMainWindow returned false; computer-use found no targetable hidden window.
After explicit owner authorization, only identity-checked PID7444 was forcibly
stopped at16:50:43; client exit-1. Nothing else was killed. Waited over three
minutes for ACE's stale session before retry. No production fix round consumed.
## Corrected core + synchronization validation — PASS, narrow/provisional
`logs/selfgate-20260905-165355-477-cathedral-validation-r2-b2bdb09cc/`,
PID7976, start16:53:55.855+02, settled checkpoint16:55:41.434+02.
Process-only VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation,
VK_LAYER_VALIDATE_SYNC=1, VK_LAYER_GPUAV_ENABLE=0, ACDREAM_DEVTOOLS=1.
Lead observed loaded C:/VulkanSDK/1.4.350.0/Bin/VkLayer_khronos_validation.dll.
All81commands completed; client/runner exit0, graceful logout confirmed,
both clients absent afterward. Zero matches across both logs for SYNC-HAZARD,
Validation Error, VUID, ErrorDeviceLost, unhandled exception, timeout, invalid
input or abandoned teardown. Stderr contains one known unsupported VFX emitter
warning. Windows Application1000/1001/1026 matching AcDream/LiveKernelEvent/
WATCHDOG in16:53:5516:56:10: zero.
All six checkpoints: completed/visible true, wait cue false, invariant failures0,
transit idle. Final staged uploads/live teardowns/landblock retirement/warmup0.
All seven1280x720 PNGs were opened by the lead: narrow **PASS, provisional**
for coherent scene/current screenshot and no stuck reveal. Not an independent
retail comparison, complete cathedral visibility proof, or crash-recurrence
guarantee. PNG paths below are relative to this run's `artifacts/screenshots/`:
| PNG | SHA256 |
| --- | --- |
|477-cathedral-00-login.png|D7A2677CE1683B9B77041C3EA05D12CE9D019DF5FB56D359034198C668C334E7|
|477-cathedral-01-arrival.png|79F74A4228776ADB5BCD7389CD7EFE069E5A73AE8D099DF1C2DA58849EBB8097|
|477-cathedral-02-movement.png|91021D332DE8C9CE774B00900EE2C66F65D0E713D45162E985C2A47BBB6BEB31|
|477-cathedral-03-stairwell.png|BC32D524A699DE7055C1F3277C2862E47512A002C1845530BD81FB1535ACDBA2|
|477-cathedral-04-stairwell-tilt.png|A8D1F31F94FA5D6BF9C80D13F5C8E29DCA1C4ABB83727F6466DB03DB4227BD4D|
|477-cathedral-05-stairwell-turn.png|47510A14361801B3F1F6B255AAAC67E033312602C2D49D2B1CD7F96D33E6A83C|
|477-cathedral-06-settled.png|FFA32856E18A21ECB01CA768A1F8EADEB737EBDE9E769F8A4BD5E84EBC16E8C6|
stdout C8428F1DAFA603626118EF1F5A7A78ECC35D452642F7115C3C60F7BB3B84507D;
stderr4D40C9BEEA7C8FFDB7ED9C2AF70273714FF592E2BED2320E57D7FD87A56D2F87.
## GPU-assisted follow-up
Same corrected route, separate process, aimed at shader-memory errors rather
than repeating core validation. Process-only GPUAV_ENABLE=1 with VALIDATE_CORE=0
and VALIDATE_SYNC=0, using the full VK_LAYER_ prefixes, DEVTOOLS=1 and the same
explicit layer. The environment mapping and separate-mode recommendation are
from [Khronos](https://github.com/KhronosGroup/Vulkan-ValidationLayers/blob/main/docs/updating_from_VK_EXT_validation_features.md)
and [LunarG](https://vulkan.lunarg.com/doc/view/latest/windows/gpu_validation.html).
GPU-assisted instrumentation changes execution and overhead; it is neither
a performance comparison nor a guarantee of every shader path being checked.
`logs/selfgate-20260905-165627-477-cathedral-gpuav-r3-b2bdb09cc/`, PID2156,
start16:56:27.991+02, settled checkpoint16:59:01.266+02. Lead observed the
same loaded validation DLL. No reported GPU-AV initialization/disable warnings;
this is not a negative-control proof of every instrumented shader path.
All81commands completed, client/runner exit0, graceful logout confirmed,
both clients absent afterward. Zero fault matches across both logs using the
same checks as r2. Stderr contains one known unsupported VFX-emitter warning.
Windows Application1000/1001/1026 matching AcDream/LiveKernelEvent/WATCHDOG
in16:56:2717:00:03+02: zero. System4101 across16:53:5517:00:03: zero.
All six checkpoints completed/visible, invariant failures0 and transit idle.
Login checkpoint has waitCueShown=true (reported, not concealed); subsequent
five are false and its adjacent PNG has no portal caption. Final uploads,
live teardown, landblock retirement and warmup0. All seven1280x720 PNGs below
were opened: **PASS, narrow/provisional**, with the same scope limits as r2.
Paths are relative to this run's `artifacts/screenshots/`:
| PNG | SHA256 |
| --- | --- |
|477-cathedral-00-login.png|E93C8495590D455D4EE72575F0B821FF15A823959D13988ABBF2722C5BCDBF44|
|477-cathedral-01-arrival.png|0D74AE19EA868380D2C35400845D7AE03039DFE5325A282612BAF648429030C2|
|477-cathedral-02-movement.png|CF6995FD988896A05CF33E589AA9079BA2ADDD5D9C83130BD6989E571734EC66|
|477-cathedral-03-stairwell.png|E3378BC67AEB038EABF6FC98ED569BED165C2E8651E2F5D9B95933ECDE42E208|
|477-cathedral-04-stairwell-tilt.png|B7CD111D3DCFD1421B52A5D1A023A61B467BC6909E8E963701ABDB7E56563A96|
|477-cathedral-05-stairwell-turn.png|750388A582BE4841C53D39AFDFFC32091DB68F561304082DED135BDC58BEF534|
|477-cathedral-06-settled.png|6B2844528F10EB0D31C52E5B3F74CF3F2582C15C5C35A3E2EC76FA44A491C202|
stdout BC5F97025E57FF886746C8F45F025AC5D5F517371D4FDBB634AE477CDAF3D38B;
stderr BAF767100B9E5CEA9C34FED950F01A6D25F0B430C20D73F178A1D820D08AE293.
## Conclusion and next diagnostic
Neither bounded isolated mode reproduced the owner's confirmed AMD reset.
The original roughly16-minute ordinary session and these roughly23-minute
instrumented routes are not equivalent; do not close #477/G4 or blame the
driver, retail coexistence, or cathedral membership without further evidence.
Next useful diagnostic is an ordinary-profile longer reproduction with the
first acquire failure retained, not another identical short validation loop.
Any code change to preserve that result needs its own bounded contract; no
speculative GPU repair is authorized by a clean test. C1a's exact retail
membership comparison remains separate and incomplete. FPS remains deferred.

View file

@ -5029,3 +5029,29 @@ Original assertion/hash placeholder and scratch hashes remain unchanged.
A production AP159 repair needs a separately expanded contract; narrowing
the witness needs explicit owner approval. No option assumed. Scope stop,
not static-impossibility or a proved Rynthid cause. FPS deferred, G4FAIL.
## 60. Cathedral AMD recurrence and isolated validation — 2026-09-05
Owner first reported the visible fixes looked good and questioned the Rynthid
placement report, then crashed in the cathedral at16:41. Exact ordinary
Releaseb2bdb09cc/PID7852: abandoned-frame ErrorDeviceLost and Windows141.
Reveal had completed; this does not identify an initiating GPU fault or connect
the crash to C1a/AP159. Owner closed retail, authorized isolated testing and,
after the lead's initial script-verb error, explicitly authorized stopping the
stuck diagnostic client. No unrelated app or ACE stopped.
Corrected81-command route preflight checks every verb/InputAction, balanced
holds and terminal close-client. Core/sync run7976 and separate GPU-assisted
run2156 use the unchanged green Release binary and v10 package. Both completed,
exited0/gracefully, no reported validation/native failures or new Windows GPU
events. Fourteen PNGs opened narrow PASS/provisional, not retail parity proof.
GPU-assisted login checkpoint retained a wait-cue flag; later five false and
all six completed/visible/invariants0. First attempt7444 is discarded/incomplete,
not a passing route. Exact paths/hashes/profile and limits:
`477-cathedral-recurrence.md`. Both clients now absent; ACE9000 stays up.
No source, shader, package, driver, TDR, quality or FPS changes. #477/G4 remain
FAIL/open: roughly23-minute diagnostic routes cannot clear the observed
roughly16-minute ordinary crash. Next useful evidence is longer ordinary
reproduction plus first-acquire-failure preservation under a bounded contract,
not repeating the same short clean route. C1a remains independently unclosed.