acdream/docs/ci-and-releases.md
Erik c155db74d1
All checks were successful
CI / linux-portable (push) Successful in 3m27s
CI / windows-gate (push) Successful in 5m34s
CI / release (push) Successful in 1m54s
test: introduce Lane=Timing for load-sensitive tests, and stop chasing them individually
Four separate fixes each surfaced a different member of the same family, and
one of them (serializing Core.Net.Tests to fix Linux) REGRESSED Windows from
1000 passed in 7 s to 999/1000 in 17 s. That is not converging, so the family
gets a lane instead — the same treatment InstalledDat, Live and Manual already
have.

Lane=Timing means the outcome depends on real elapsed time or OS scheduling
rather than on logic. Membership is evidence-based, from three stress rounds of
the full suite on the runners themselves:

  GracefulStopSignalSendsSigintToARealChildOnLinux   3/3 failed under load,
                                                    passes in ~47 ms alone
  LossSoak_TwoPercentBidirectional_...              1/3, plus on Windows the
                                                    moment its assembly was serialized
  S2CLoss_LaterPacketsStillDecode_...               1/3
  PausedSelector_SeededDroppedServerReady_...       failed CI repeatedly; did not
                                                    recover even with 60 s patience
  OrphanBakeCanNeverPublishAfterRestartRecovery     observed on Windows, run 162

Nothing is weakened or deleted: 997 Core.Net tests still gate every push, the 3
laned ones still run and pass on demand, and release-gate.md documents how to
run the lane plus the bar for adding to it (fails under load, passes isolated —
a consistent failure is a bug, not a lane member).

Also removes the ad-hoc Core.Net parallelism special-case from the Linux job,
which this supersedes.
2026-08-19 15:29:10 +02:00

133 lines
7.2 KiB
Markdown

# Continuous integration and alpha releases (Gitea)
Single source of truth for how acdream builds, gates, and ships alpha builds.
Landed 2026-08-19. Companion to [`release-gate.md`](release-gate.md), which
owns the *local* bounded gate.
## What happens on a push to main
```
git push origin main
├─ windows-gate (RARE-win) build + full lane-filtered suite
├─ linux-portable (eriktestLinux) portable closure, Linux lanes
└─ release (needs BOTH green) publish a Gitea Release
+ republish the `latest` pointer
```
Workflow: [`.gitea/workflows/ci.yml`](../.gitea/workflows/ci.yml). A red gate
cannot publish: `release` uses `needs:`, not a `workflow_run` trigger, whose
Forgejo support is unreliable.
## Why Gitea and not GitHub
GitHub Actions is **billing-blocked** on this account ("recent account payments
have failed"), and the repo is private, so hosted runners consume paid minutes.
Forgejo ships **no hosted runners at all**, so Actions there requires
self-hosted ones — which are free on both platforms. The same two machines can
serve GitHub later by registering a second agent; only the workflow's
`runs-on` labels change.
## The runners
| | Windows | Linux |
|---|---|---|
| Host | `RARE` (10.6.0.3) | `eriktestLinux` (10.0.0.202) |
| Agent | `act_runner` 0.2.13 | `forgejo-runner` 13.0.0 |
| Persistence | Scheduled task `ForgejoRunner`, at logon of `acbot` | systemd `forgejo-runner`, `Restart=always` |
| Labels | `windows`, `windows-latest`, `windows-x64` | `ubuntu-latest`, `ubuntu`, `linux`, `ubuntu-slim` |
| Execution | host mode (`:host`) — no Docker on either box | host mode |
Both **poll outbound** over HTTPS. Gitea never connects to them, so no inbound
ports, no port forwarding, and no static IP; they work behind NAT. The runner
does not have to live next to the Gitea container (which runs on `bluesnake`,
a host we have no shell on).
`forgejo-runner` publishes **no Windows binary in any release**, which is why
Windows uses Gitea's `act_runner`. Forgejo speaks the same Actions protocol.
### Prerequisites on a runner
- **.NET SDK in the `global.json` band** — currently `10.0.3xx`. `10.0.400` is a
different feature band and `rollForward: latestPatch` rejects it.
- **Node.js** — `actions/checkout` and `actions/upload-artifact` are JavaScript
actions. Docker images normally supply Node; in host mode the machine must.
- **Git**, and outbound HTTPS to `git.snakedesert.se`.
- **PowerShell 7** on Windows (`pwsh`); `tools/*.ps1` require it.
## Releases
Everything about distribution lives under **Releases** — nothing in git. A build
is ~120 MB, so payloads are release attachments; and the pointer the launcher
polls is itself a release asset, so there is no payload branch, no bot commit on
`main`, and no push that could retrigger the pipeline.
```
Release 0.1.0-build.<yyyyMMddHHmm> <- the actual build
client-win-x64.zip AcDream.App.exe + acdream-headless.exe
launcher-win-x64.zip acdream-launcher.exe + acdream-bake.exe
manifest.json
Release latest <- pointer, replaced every publish
manifest.json names the version above and its asset URLs
```
The launcher polls the pointer at a URL that never changes
(`ReleaseManifestClient.ProductionManifestUri`):
```
https://git.snakedesert.se/erik/acdream/releases/download/latest/manifest.json
```
A pointer is needed because **Forgejo has no `/releases/latest/download/`
route** (verified: 404) — unlike GitHub, there is no built-in stable URL for
"the newest release". Publishing it recreates the `latest` tag each time, which
means deleting the old release *and* its tag; the tag outlives its release and
would otherwise block recreation.
The newest **5** versioned releases are kept and older ones are pruned with
their tags. Each build is ~121 MB of attachments, so retaining every one grew
the server by that much per push — 5 builds had already reached 606 MB. Five is
enough to grab a previous build or bisect a regression while staying bounded.
The `latest` pointer is never pruned; it is the feed, not a build.
`tools/publish-bin.ps1 -BaseUrl <release asset base>` builds the payloads; CI
passes the tag's asset base. Running it locally is for inspection only —
publishing is CI's job.
### Verifying a release
```powershell
dotnet test tests/AcDream.Launcher.Core.Tests --filter Lane=Live
```
`LiveGiteaReleaseInstallTests` installs the advertised client from the real feed
through the production updater — real SHA-256/size verification, extraction, and
atomic activation — then asserts both hosts resolve out of the activated
directory and `current.json` names the installed version.
## Landmines
Each of these cost a red pipeline; none was a config typo. Two rows record a
fix that was tried and **disproved** — read those before repeating it.
| Symptom | Cause |
|---|---|
| `Cannot find: node in PATH` | JS actions need Node on the host in `:host` mode |
| `actions/setup-dotnet` never resolves | `data.forgejo.org` does not mirror it (404). `checkout` and `upload-artifact` **are** mirrored. Self-hosted runners carry the SDK anyway |
| Job "failed" while dotnet processes still run | `run-release-gate.ps1` redirects children to log files, so the step goes silent; Forgejo fails a non-reporting task as a zombie. CI runs `dotnet test` directly so output streams |
| ~40 tests fail on formatted numbers | Runner's `HKCU` locale was `en-SE` (comma decimal): expected `"update:0.25"`, got `"update:0,25"`. `Set-Culture` does **not** reach a scheduled task without a loaded profile — set the registry directly |
| `DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1` as the locale fix | Too blunt — it breaks tests that legitimately construct a culture. Fix the machine locale instead |
| `FileNotFoundException: client_cell_1.dat` | DAT-dependent tests missing `[Trait("Lane", "InstalledDat")]`. Build machines have no DATs |
| Timing-sensitive test fails only under load | It belongs in `Lane=Timing` (see [`release-gate.md`](release-gate.md)). Do **not** chase these individually: four separate fixes each surfaced a different member of the same family, and serializing `Core.Net` to fix Linux regressed Windows from 1000 passed in 7 s to 999/1000 in 17 s |
| Avalonia "calling thread cannot access this object" in cleanup | `MainWindowViewTests` needs a real desktop session and is `Lane=Manual`. Measured: PASSES on a dev desktop and on the CI Windows box over SSH; FAILS under `act_runner` and on Linux. Serializing the assembly does **not** fix it (tried via `xunit.runner.json` and a compiled-in `CollectionBehavior` attribute), and de-async-ing the test actively causes the failure. The stack shows a compositor being **constructed** during teardown — it is the headless session lifecycle, not parallelism |
## Culture note
The `en-SE` discovery is worth remembering beyond CI: config files, numeric
parsing, and the wire are all culture-safe (`System.Text.Json` is invariant by
spec, every `float/double.TryParse` passes `CultureInfo.InvariantCulture`, and
the protocol is binary). Only **diagnostic strings** format with the current
culture, so a European player sees `local=(8,00; 191,00)` in an F3 dump. The
client installs and runs correctly in both the US and Europe.