Development¶
This page shows you how to build Occamy from source, run its tests, try
a change to the web client against a running occamyd, preview this
website, and cut a release. Run each command from the repository's
root unless it says otherwise. CONTRIBUTING.md
explains how to send a change, and Architecture how
the code fits together.
What you need¶
| What | Version | For |
|---|---|---|
| Go | 1.26.4 or later, as go.mod says |
occamyd and the Go tests |
| A C compiler, such as gcc | Any | The race detector, -race, on Linux and Windows. macOS needs none |
| Podman with a compose provider, or Docker with Compose v2 | Any recent | The image, the demo stack, and the end-to-end and integration tests |
| Node.js with npm | 22.13 or later in 22, any 24, or 26 or later | The web client |
| golangci-lint | v2. CI runs v2.15.0 | make lint |
| Chrome | Any recent | One end-to-end test, which skips without it |
| Xcode | 26 or later | Occamy for macOS |
| Python | 3 | Previewing the website, and drawing its figures |
make runs Podman unless CONTAINER_ENGINE names another engine.
Give every make command the same engine, for example by exporting it:
export CONTAINER_ENGINE=docker
The integration tests read OCCAMY_CONTAINER_ENGINE instead.
Build¶
make compile # ./occamyd
make client # the web client, in client/occamy-web/dist
make build # the image occamy:latest
make compile runs CGO_ENABLED=0 go build -trimpath, which needs no
C toolchain on Linux, macOS or Windows. make client runs npm ci and
npm run build in client/occamy-web. make build builds both inside
the image, from docker/Dockerfile, and tags it
occamy:latest and occamy:<version>, from git describe, or as
OCCAMY_IMAGE names it
(Next to another stack). To run the
occamyd you built, give -conf a configuration
whose web_client is client/occamy-web/dist.
Run the tests¶
You can run each of CI's checks on your own machine:
| Suite | Command | What it needs | CI job |
|---|---|---|---|
| Unit tests | go test -race ./... |
Go | go |
| Lint | make lint |
golangci-lint v2 | lint |
| The web client's tests | npm test in client/occamy-web |
Node.js and Go | web |
| End-to-end tests | make e2e |
The end-to-end stack | e2e |
| Pressure test | make e2e OCCAMY_PRESSURE_N=10 |
The end-to-end stack, with its debug listener | e2e |
| Occamy for macOS | make swift-test |
Xcode | macos |
| Soak | make soak |
The end-to-end stack, with its debug listener | None |
| Integration tests | go test -tags … |
A container engine | None |
| Fuzz tests | go test -fuzz … |
Go | Each night, in fuzz.yml |
The last three take long or need servers of their own, so no pull
request runs them. CI also builds occamyd on macOS and Windows,
builds the image, and runs a short
benchmark to keep the harness
working.
Unit tests¶
go test -race ./...
The unit tests need no stack or container engine, and take a few
minutes with the race detector; the end-to-end tests in tests/ skip
without a stack. Without a C compiler on Linux or Windows, -race
stops with -race requires cgo or C compiler "gcc" not found, so
leave it out.
CI also tests bench/codecs, a Go module of its own, vets the code,
checks the dependencies' vulnerabilities and licences, and tests the
benchmark's analysis on results with known answers:
(cd bench/codecs && go test ./...)
go vet ./...
go run golang.org/x/vuln/cmd/govulncheck@v1.9.0 ./...
go run github.com/google/go-licenses/v2@v2.0.1 check ./... --disallowed_types=forbidden,restricted
uv run bench/report_test.py # needs uv
Lint¶
golangci-lint version # must say version 2
make lint
make lint runs golangci-lint with .golangci.yml,
which v1 can't read, then again on bench/codecs with its measure
build tag. The configuration also checks the code's gofmt formatting
and goimports grouping. CI lints the RDP integration tests too, which
build only under their tag:
golangci-lint run --build-tags=rdp_integration ./internal/rdp/... ./internal/backend/rdp/...
The web client's tests¶
The web client's interop tests run the TypeScript client against
occamyd's own stream code, which client/internal/webinterop serves.
Build that server with the race detector, so that a data race fails the
test, then run the tests as CI does:
go build -race -o "${TMPDIR:-/tmp}/occamy-webinterop" ./client/internal/webinterop
cd client/occamy-web
npm ci
npm run lint
npm run typecheck
OCCAMY_INTEROP_SERVER="${TMPDIR:-/tmp}/occamy-webinterop" GORACE=halt_on_error=1 npm test
npm run build
cd ../..
Without a C compiler, leave out -race. Without
OCCAMY_INTEROP_SERVER, npm test skips the interop tests; with it,
it still skips the two that check that CI has the interop server.
End-to-end tests¶
The end-to-end tests open real sessions through occamyd, with the Go
client of protocol v1, on the end-to-end stack: the demo stack with
the tests' configuration,
docker/occamy-v1-e2e.yaml, which
docker/docker-compose.e2e.yaml
mounts in place of the demo's.
flowchart LR
tests["go test ./tests<br>(make e2e)"]
chrome["Chrome,<br>for TestWebScreen"]
subgraph stack ["The end-to-end stack: make run E2E=1"]
occamy["occamy-v1<br>occamyd on port 5637"]
desktop["desktop<br>XFCE, over VNC and RDP"]
ssh["ssh<br>OpenSSH"]
vnc["vnc<br>x11vnc"]
tigervnc["tigervnc<br>TigerVNC"]
rdp["rdp<br>xrdp"]
end
tests -- "protocol v1" --> occamy
tests -- "starts" --> chrome
chrome -- "the web client" --> occamy
tests -. "compose: kill a process,<br>read logs, recreate" .-> stack
occamy --> desktop & ssh & vnc & tigervnc & rdp
To run them:
make build
make run E2E=1
make e2e
make run E2E=1 also switches a running demo stack to the end-to-end
one, make run switches back, and make stop stops either. Give
make e2e the engine that make run used: some tests stop processes
in the rdp and vnc containers, or recreate occamy-v1, through its
compose.
| Test | What it checks |
|---|---|
TestE2EV1 |
Both demo desktops arrive twice in a row over VNC and RDP, and input reaches them without ending the session |
TestE2EV1SSH |
Two SSH sessions in a row each open a shell that answers a command |
TestDemoDesktop |
On both demo desktops, XFCE draws its panel and wallpaper, and the panel follows the screen when it grows |
TestE2EV1RDP |
On rdp: the clipboard both ways, a resize, a viewer who joins by a share, the owner leaving and coming back, and the clipboard again after its process in the rdp container is killed |
TestE2ETokenSurvivesRestart |
An access token still works after occamy-v1 is recreated, because the signing key is on a volume |
TestX11vncClipboardRequestorGone |
x11vnc survives a clipboard request from a window that has gone |
TestVNCParity |
Each VNC feature on x11vnc and on TigerVNC. It logs a table, and writes it to the file that OCCAMY_PARITY_MATRIX names |
TestWebScreen |
In Chrome, on tigervnc and rdp, the web client asks for the window's size in CSS pixels at device pixel ratios of 2, 1 and 1.5, when the session opens and when the window changes. It finds Chrome on the host, or at OCCAMY_E2E_CHROME, and skips without it, except in CI |
A run takes a few minutes on an idle host, mostly the parity matrix. To run one test, such as the owner's return:
OCCAMY_E2E_V1_ENDPOINT=localhost:5637 go test -count=1 -v -run 'TestE2EV1RDP/reconnect' ./tests/
When a test fails, the services' logs usually say why:
(cd docker && podman compose logs rdp). An RDP clipboard check that
times out with error in trans_connect chan in the rdp log means
that xrdp found no channel server, the process that carries the
clipboard, for the session's display. The image starts one on the next
connection, and logs starting xrdp-chansrv again.
The test servers¶
Under make run E2E=1, the web client lists 14 connections: the
demo's three, then the tests' servers, which show bare X desktops.
| Connection | What you see |
|---|---|
| x11vnc | A black desktop with two xterms and no window manager. The small one at the top left is a probe: lines typed there reach the clipboard as hex. x11vnc keeps its 1024×768 screen, and the page scales it to the window. Keys go to the window under the pointer |
| TigerVNC | The same desktop, served by TigerVNC, which resizes its screen to the window |
| RDP | xrdp's X session: one xterm with the prompt occamy@…:~$. It resizes to the window. The first connection after the stack starts creates the X session, so wait a few seconds before you type |
The other connections reach the same servers with no password, over
VeNCrypt, and with the German and French keyboard layouts.
TigerVNC, asking for the password asks for a password first, which
is vncpassword.
In the xterms, echo hello | xclip -selection clipboard puts text on
the remote clipboard, and xclip -o -selection clipboard; echo prints
it. The servers also answer text that you send from the Clipboard
panel:
| Connection | Send | The remote clipboard soon holds |
|---|---|---|
| x11vnc, TigerVNC | owner: hi |
echo: hi |
| RDP | occamy-e2e:hi |
echo:occamy-e2e:hi |
x11vnc ignores the browser's text until its latest client has been connected for 45 seconds, so on a fresh connection, wait and send it again.
Pressure test and soak¶
Both tests read occamyd's goroutines, heap and open files from its
debug listener, so start the end-to-end stack with the listener on:
make run E2E=1 OCCAMY_DEBUG=1
The pressure test runs with the end-to-end tests when
OCCAMY_PRESSURE_N gives its rounds, 10 in CI. Each round opens two
sessions at once to each of ssh, vnc and rdp, one of which sends
garbage once it's welcomed, and within a minute after the rounds,
occamyd's goroutines must be back to their number before them.
make e2e OCCAMY_PRESSURE_N=10
The soak runs sessions that type and move the pointer for a long
time, and samples the same figures. After a warm-up, it fails when the
median of a figure's last quarter of samples exceeds its first
quarter's by more than the figure's bound, when a figure isn't back
within its bound of where it started a minute after the last session
closed, or when a session fails. tests/soak_test.go describes its
variables:
| Variable | Default | What it sets |
|---|---|---|
OCCAMY_SOAK_SESSIONS |
8 | Sessions at once |
OCCAMY_SOAK_DURATION |
1h |
How long the soak runs |
OCCAMY_SOAK_CONNECTIONS |
tigervnc |
The connections that the sessions take in turn |
OCCAMY_SOAK_LIFETIME |
10m |
How long a session lasts before another replaces it; 0 keeps each for the whole run |
OCCAMY_SOAK_INTERVAL |
30s |
How often it samples |
OCCAMY_SOAK_WARMUP |
A tenth of the duration | How long before samples count |
OCCAMY_SOAK_BOUNDS |
goroutines=50,heap=32,fds=25 |
The growth allowed, with the heap in MiB |
A short soak of 5 minutes, and the full soak of 20 sessions for 8 hours:
make soak OCCAMY_SOAK_DURATION=5m OCCAMY_SOAK_INTERVAL=10s OCCAMY_SOAK_LIFETIME=2m
make soak OCCAMY_SOAK_SESSIONS=20 OCCAMY_SOAK_DURATION=8h
Integration tests¶
The integration tests run a backend against a real server in a
container that they build, start and remove themselves, without the
demo stack. Build tags keep them out of go test ./..., and each
command sets a -timeout beyond the default 10 minutes, which building
an image can outlast.
| Build tag | Command | What it tests |
|---|---|---|
rdp_integration |
go test -tags rdp_integration -run Integration -timeout 30m ./internal/rdp |
The RDP client, against xrdp in the bench desktop image |
rdp_integration |
go test -tags rdp_integration -run Integration -timeout 30m ./internal/backend/rdp |
The RDP backend, against the same |
kubernetes_integration |
go test -tags kubernetes_integration -run Integration -timeout 15m ./internal/backend/kubernetes |
The Kubernetes backend, against k3s with a busybox pod. It pulls busybox, so it needs the network |
integration |
go test -tags integration -run TestTigerVNC -timeout 30m ./internal/backend/vnc |
The VNC backend over VeNCrypt, against the demo's TigerVNC image |
The test files describe their settings in their comments. Three more
tests of the integration tag run the VNC backend against a server
that you start yourself:
| Test | Server | At |
|---|---|---|
TestBenchDesktop |
x11vnc on the bench desktop | OCCAMY_VNC_ADDR |
TestEncodings |
The same, asking for one encoding at a time | OCCAMY_VNC_ADDR |
TestMacOS |
A Mac with Screen Sharing on, set up as the comment in internal/backend/vnc/ard_integration_test.go says |
OCCAMY_VNC_ARD_ADDR |
For example, with names of your own on a machine that others share:
podman build -t occamy-bench-desktop -f docker/bench/desktop/Dockerfile .
podman run -d --rm --name occamy-vnc-test -p 127.0.0.1:15900:5900 occamy-bench-desktop vnc
OCCAMY_VNC_ADDR=127.0.0.1:15900 go test -tags integration -run TestBenchDesktop ./internal/backend/vnc
podman rm -f occamy-vnc-test
Fuzz tests¶
Every parser of what arrives over a network has a fuzz target, which
go test -list '^Fuzz' ./... lists by package. Fuzz one target at a
time, with four workers to leave the machine usable:
go test -run '^$' -fuzz '^FuzzSession$' -fuzztime 30s -parallel 4 ./client
CI fuzzes every target for 30 seconds each night (fuzz.yml). To do the same on your machine, which takes about 20 minutes:
for pkg in $(go list ./...); do
for target in $(go test -list '^Fuzz' "$pkg" | grep '^Fuzz'); do
go test -run '^$' -fuzz "^$target\$" -fuzztime 30s -parallel 4 "$pkg" || break 2
done
done
A failure writes its input to the package's testdata/fuzz/<target>/,
and once you commit that file, every go test replays it.
Run the web client against a local occamyd¶
To work on the web client, run Vite's dev server, which reloads the
page as you edit, against an occamyd such as the demo stack's:
make build && make run # an occamyd on localhost:5637
cd client/occamy-web
npm ci
npm run dev
The dev server passes /api and its WebSockets on to the occamyd at
http://localhost:5637, or at the URL in OCCAMY_DEV_SERVER. Open the
address that it prints, and sign in as demo with the password
occamy.
Turn on the debug listener¶
The demo stack's debug listener
serves expvar and pprof without authentication, so it's off unless you
give make run OCCAMY_DEBUG=1. It then listens on localhost:5638,
on the loopback interface only, whatever OCCAMY_BIND says:
make run OCCAMY_DEBUG=1
curl -s localhost:5638/debug/vars | head -c 200; echo
curl -s -o /dev/null -w '%{http_code}\n' localhost:5638/debug/pprof/ # 200
make run without it turns the listener off again, whatever the shell
or docker/.env holds, and occamy-v1 logs debug listener off, as
go test -run TestDemoDebugGating ./internal/config checks.
Build Occamy for macOS¶
The Mac app links the Go client as a C library, which make swift-core
builds as a universal archive for Apple silicon and Intel:
make swift-core
swift run --package-path client/occamy-swift Occamy # run it from source
make swift-test # its tests, which also start the interop server
make swift-app # client/occamy-swift/build/app/Occamy.app
make swift-dmg # a disk image of it, beside it
The app has the hardened runtime. Without a Developer ID, the build
signs it ad hoc: it runs on the Mac that built it, but Gatekeeper
refuses a copy downloaded on another Mac. For other Macs, name your
Developer ID certificate, and for Apple to notarize the disk image, the
profile that xcrun notarytool store-credentials saved:
make swift-notarize DEVELOPER_ID="Developer ID Application: Your Name (TEAMID)" NOTARY_PROFILE=occamy
Preview the website¶
The README and docs/ are also the website,
https://changkun.github.io/occamy/, which MkDocs builds from
mkdocs.yml. To preview it as you edit:
python3 -m venv .venv && .venv/bin/pip install -r scripts/docs-requirements.txt
.venv/bin/mkdocs serve
To check it as the Pages workflow does on every pull request that
changes the docs, before it publishes the site from master, build it
with --strict, which fails on a link or an anchor that doesn't
resolve:
.venv/bin/mkdocs build --strict
That build doesn't check links into the docs from specs/,
CHANGELOG.md or the code, so search for a heading's anchor before you
rename it.
A module of scripts/figures/ named after the page draws its figures
other than measurements and Mermaid diagrams, as light and dark SVGs.
After you change one, draw them all again:
python3 scripts/docs_figures.py
Cut a release¶
A release is a tag vMAJOR.MINOR.PATCH, with a suffix such as
-beta.1 for a prerelease. Pushing it runs
release.yml, which publishes
everything but Occamy for macOS, which you build on a Mac:
flowchart LR
tag["You push<br>the tag"]
subgraph wf ["release.yml"]
image["image<br>amd64 and arm64"]
manifest["manifest<br>the multi-arch image<br>and its provenance"]
web["web<br>the web client's archive"]
bins["binaries<br>occamyd's archives,<br>with GoReleaser"]
release["release<br>SHA256SUMS, provenance,<br>and the notes from<br>CHANGELOG.md"]
end
mac["On a Mac:<br>make swift-notarize,<br>gh release upload"]
tag --> image --> manifest --> release
tag --> web --> release
tag --> bins --> release
release --> mac
- Move the
Unreleasedentries inCHANGELOG.mdunder the new version and today's date, and update the comparison links at the end of the file. - Merge that change, and wait for CI to pass on
master. -
Tag the merge commit, and push the tag:
git tag vMAJOR.MINOR.PATCH git push origin vMAJOR.MINOR.PATCHThe image
ghcr.io/changkun/occamygets the tag's name, andlatestunless a suffix marks a prerelease. GoReleaser builds the archives from .goreleaser.yaml. The workflow fails ifCHANGELOG.mdhas no entries for the version. -
On a Mac with the Developer ID, build Occamy for macOS from the tag, which gives its version, have Apple notarize it, and add the disk image to the release:
git worktree add --detach ../occamy-release vMAJOR.MINOR.PATCH cd ../occamy-release make swift-notarize DEVELOPER_ID="Developer ID Application: …" NOTARY_PROFILE=… gh release upload vMAJOR.MINOR.PATCH client/occamy-swift/build/app/Occamy-MAJOR.MINOR.PATCH.dmgThe disk image isn't in
SHA256SUMS, because its Developer ID signature and Apple's notarization vouch for it.
To rehearse a release, choose Release in GitHub's Actions tab,
then Run workflow: this dry run builds everything and publishes
nothing. A pull request that changes the workflow, .goreleaser.yaml,
docker/Dockerfile or scripts/third-party-licenses.sh runs it by
itself.
Clean up¶
Tear down removes the demo stack and its images. To remove what the builds and tests left in the checkout:
rm -f occamyd
rm -rf client/occamy-web/dist client/occamy-web/node_modules
rm -f "${TMPDIR:-/tmp}/occamy-webinterop"
podman rmi occamy-bench-desktop removes the bench desktop's image,
which TestBenchDesktop needs.