20905e16fc
A stress run left going on the Windows runner kept 17 dotnet processes alive and competed with CI for the same machine for roughly half an hour, slowing every job and making the load-sensitive failures it was meant to diagnose more likely. Records the cleanup commands and the caveat that the runner agent itself should be left alone.
148 lines
7.8 KiB
Markdown
148 lines
7.8 KiB
Markdown
# Continuous integration and alpha releases (Gitea)
|
|
|
|
Single source of truth for how acdream builds, gates, and ships alpha builds.
|
|
Landed 2026-08-19. Companion to [`release-gate.md`](release-gate.md), which
|
|
owns the *local* bounded gate.
|
|
|
|
## What happens on a push to main
|
|
|
|
```
|
|
git push origin main
|
|
│
|
|
├─ windows-gate (RARE-win) build + full lane-filtered suite
|
|
├─ linux-portable (eriktestLinux) portable closure, Linux lanes
|
|
│
|
|
└─ release (needs BOTH green) publish a Gitea Release
|
|
+ republish the `latest` pointer
|
|
```
|
|
|
|
Workflow: [`.gitea/workflows/ci.yml`](../.gitea/workflows/ci.yml). A red gate
|
|
cannot publish: `release` uses `needs:`, not a `workflow_run` trigger, whose
|
|
Forgejo support is unreliable.
|
|
|
|
## Why Gitea and not GitHub
|
|
|
|
GitHub Actions is **billing-blocked** on this account ("recent account payments
|
|
have failed"), and the repo is private, so hosted runners consume paid minutes.
|
|
Forgejo ships **no hosted runners at all**, so Actions there requires
|
|
self-hosted ones — which are free on both platforms. The same two machines can
|
|
serve GitHub later by registering a second agent; only the workflow's
|
|
`runs-on` labels change.
|
|
|
|
## The runners
|
|
|
|
| | Windows | Linux |
|
|
|---|---|---|
|
|
| Host | `RARE` (10.6.0.3) | `eriktestLinux` (10.0.0.202) |
|
|
| Agent | `act_runner` 0.2.13 | `forgejo-runner` 13.0.0 |
|
|
| Persistence | Scheduled task `ForgejoRunner`, at logon of `acbot` | systemd `forgejo-runner`, `Restart=always` |
|
|
| Labels | `windows`, `windows-latest`, `windows-x64` | `ubuntu-latest`, `ubuntu`, `linux`, `ubuntu-slim` |
|
|
| Execution | host mode (`:host`) — no Docker on either box | host mode |
|
|
|
|
Both **poll outbound** over HTTPS. Gitea never connects to them, so no inbound
|
|
ports, no port forwarding, and no static IP; they work behind NAT. The runner
|
|
does not have to live next to the Gitea container (which runs on `bluesnake`,
|
|
a host we have no shell on).
|
|
|
|
`forgejo-runner` publishes **no Windows binary in any release**, which is why
|
|
Windows uses Gitea's `act_runner`. Forgejo speaks the same Actions protocol.
|
|
|
|
### Prerequisites on a runner
|
|
|
|
- **.NET SDK in the `global.json` band** — currently `10.0.3xx`. `10.0.400` is a
|
|
different feature band and `rollForward: latestPatch` rejects it.
|
|
- **Node.js** — `actions/checkout` and `actions/upload-artifact` are JavaScript
|
|
actions. Docker images normally supply Node; in host mode the machine must.
|
|
- **Git**, and outbound HTTPS to `git.snakedesert.se`.
|
|
- **PowerShell 7** on Windows (`pwsh`); `tools/*.ps1` require it.
|
|
|
|
## Releases
|
|
|
|
Everything about distribution lives under **Releases** — nothing in git. A build
|
|
is ~120 MB, so payloads are release attachments; and the pointer the launcher
|
|
polls is itself a release asset, so there is no payload branch, no bot commit on
|
|
`main`, and no push that could retrigger the pipeline.
|
|
|
|
```
|
|
Release 0.1.0-build.<yyyyMMddHHmm> <- the actual build
|
|
client-win-x64.zip AcDream.App.exe + acdream-headless.exe
|
|
launcher-win-x64.zip acdream-launcher.exe + acdream-bake.exe
|
|
manifest.json
|
|
|
|
Release latest <- pointer, replaced every publish
|
|
manifest.json names the version above and its asset URLs
|
|
```
|
|
|
|
The launcher polls the pointer at a URL that never changes
|
|
(`ReleaseManifestClient.ProductionManifestUri`):
|
|
|
|
```
|
|
https://git.snakedesert.se/erik/acdream/releases/download/latest/manifest.json
|
|
```
|
|
|
|
A pointer is needed because **Forgejo has no `/releases/latest/download/`
|
|
route** (verified: 404) — unlike GitHub, there is no built-in stable URL for
|
|
"the newest release". Publishing it recreates the `latest` tag each time, which
|
|
means deleting the old release *and* its tag; the tag outlives its release and
|
|
would otherwise block recreation.
|
|
|
|
The newest **5** versioned releases are kept and older ones are pruned with
|
|
their tags. Each build is ~121 MB of attachments, so retaining every one grew
|
|
the server by that much per push — 5 builds had already reached 606 MB. Five is
|
|
enough to grab a previous build or bisect a regression while staying bounded.
|
|
The `latest` pointer is never pruned; it is the feed, not a build.
|
|
|
|
`tools/publish-bin.ps1 -BaseUrl <release asset base>` builds the payloads; CI
|
|
passes the tag's asset base. Running it locally is for inspection only —
|
|
publishing is CI's job.
|
|
|
|
### Verifying a release
|
|
|
|
```powershell
|
|
dotnet test tests/AcDream.Launcher.Core.Tests --filter Lane=Live
|
|
```
|
|
|
|
`LiveGiteaReleaseInstallTests` installs the advertised client from the real feed
|
|
through the production updater — real SHA-256/size verification, extraction, and
|
|
atomic activation — then asserts both hosts resolve out of the activated
|
|
directory and `current.json` names the installed version.
|
|
|
|
## Landmines
|
|
|
|
Each of these cost a red pipeline; none was a config typo. Two rows record a
|
|
fix that was tried and **disproved** — read those before repeating it.
|
|
|
|
| Symptom | Cause |
|
|
|---|---|
|
|
| `Cannot find: node in PATH` | JS actions need Node on the host in `:host` mode |
|
|
| `actions/setup-dotnet` never resolves | `data.forgejo.org` does not mirror it (404). `checkout` and `upload-artifact` **are** mirrored. Self-hosted runners carry the SDK anyway |
|
|
| Job "failed" while dotnet processes still run | `run-release-gate.ps1` redirects children to log files, so the step goes silent; Forgejo fails a non-reporting task as a zombie. CI runs `dotnet test` directly so output streams |
|
|
| ~40 tests fail on formatted numbers | Runner's `HKCU` locale was `en-SE` (comma decimal): expected `"update:0.25"`, got `"update:0,25"`. `Set-Culture` does **not** reach a scheduled task without a loaded profile — set the registry directly |
|
|
| `DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1` as the locale fix | Too blunt — it breaks tests that legitimately construct a culture. Fix the machine locale instead |
|
|
| `FileNotFoundException: client_cell_1.dat` | DAT-dependent tests missing `[Trait("Lane", "InstalledDat")]`. Build machines have no DATs |
|
|
| Timing-sensitive test fails only under load | It belongs in `Lane=Timing` (see [`release-gate.md`](release-gate.md)). Do **not** chase these individually: four separate fixes each surfaced a different member of the same family, and serializing `Core.Net` to fix Linux regressed Windows from 1000 passed in 7 s to 999/1000 in 17 s |
|
|
| Avalonia "calling thread cannot access this object" in cleanup | `MainWindowViewTests` needs a real desktop session and is `Lane=Manual`. Measured: PASSES on a dev desktop and on the CI Windows box over SSH; FAILS under `act_runner` and on Linux. Serializing the assembly does **not** fix it (tried via `xunit.runner.json` and a compiled-in `CollectionBehavior` attribute), and de-async-ing the test actively causes the failure. The stack shows a compositor being **constructed** during teardown — it is the headless session lifecycle, not parallelism |
|
|
|
|
## Do not leave load on a runner
|
|
|
|
A stress/diagnostic run left going on a runner competes with CI for the same
|
|
machine and makes every job slower and more likely to trip a load-sensitive
|
|
test — the exact failures you would then be trying to diagnose. Kill background
|
|
work before trusting a timing result:
|
|
|
|
```powershell
|
|
Get-Process dotnet -ErrorAction SilentlyContinue | Stop-Process -Force # Windows
|
|
pkill -9 dotnet # Linux
|
|
```
|
|
|
|
Leave `act_runner` / `forgejo-runner` itself alone; killing those unregisters
|
|
nothing but stops the machine picking up jobs until it restarts.
|
|
|
|
## Culture note
|
|
|
|
The `en-SE` discovery is worth remembering beyond CI: config files, numeric
|
|
parsing, and the wire are all culture-safe (`System.Text.Json` is invariant by
|
|
spec, every `float/double.TryParse` passes `CultureInfo.InvariantCulture`, and
|
|
the protocol is binary). Only **diagnostic strings** format with the current
|
|
culture, so a European player sees `local=(8,00; 191,00)` in an F3 dump. The
|
|
client installs and runs correctly in both the US and Europe.
|