Behind a reverse proxy (HTTPS)
YoCache itself speaks plain HTTP. To serve it over HTTPS, or to run several
instances under one hostname (https://cache.example.com/ci/,
https://cache.example.com/dev/), put a reverse proxy in front of it.
Everything a build does goes through the one HTTP port, 6768: downloads,
sstate, uploads, the dashboard, and hash equivalence over WebSocket. So one
proxy rule per instance covers it all.
nginx example
Section titled “nginx example”Two instances under prefixes on one HTTPS hostname, nothing served at the
root. The site itself, as conf.d/cache.example.com.conf:
map $http_upgrade $connection_upgrade { default upgrade; '' ''; }
server { listen 443 ssl; server_name cache.example.com; ssl_certificate /etc/ssl/cache.example.com.pem; ssl_certificate_key /etc/ssl/cache.example.com.key;
client_max_body_size 0; # sstate archives can be hundreds of MB proxy_read_timeout 1h; # hash-equivalence WebSockets idle between tasks
location /ci/ { include snippets/proxy-params.conf; proxy_pass http://yocache-ci:6768/; } location /dev/ { include snippets/proxy-params.conf; proxy_pass http://yocache-dev:6768/; } location / { return 404; }}and the proxy settings every instance’s location includes, as
snippets/proxy-params.conf:
proxy_http_version 1.1; # needed for WebSocketproxy_set_header Upgrade $http_upgrade; # pass the WebSocket upgradeproxy_set_header Connection $connection_upgrade;proxy_set_header Host $host;proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; # see belowproxy_request_buffering off; # stream uploads throughFor a single instance on its own hostname, use location / { … } with the
same settings.
Complete example with Docker Compose
Section titled “Complete example with Docker Compose”Both instances and nginx in one Compose project, with the two nginx files above next to it:
.├── compose.yaml├── cache.example.com.conf├── proxy-params.conf└── certs/ ├── cache.example.com.pem └── cache.example.com.keyservices: yocache-ci: image: ghcr.io/bmoczulski/yocache:latest restart: unless-stopped environment: YOCACHE_HASHEQUIV_ADDR: "" # raw TCP can't go through the proxy YOCACHE_TRUSTED_PROXY: 172.30.0.10 # nginx, below volumes: - yocache-ci-data:/var/lib/yocache networks: [yocache]
yocache-dev: image: ghcr.io/bmoczulski/yocache:latest restart: unless-stopped environment: YOCACHE_HASHEQUIV_ADDR: "" YOCACHE_TRUSTED_PROXY: 172.30.0.10 volumes: - yocache-dev-data:/var/lib/yocache networks: [yocache]
nginx: image: nginx:1.27-alpine restart: unless-stopped depends_on: [yocache-ci, yocache-dev] ports: - "443:443" volumes: - ./cache.example.com.conf:/etc/nginx/conf.d/cache.example.com.conf:ro - ./proxy-params.conf:/etc/nginx/snippets/proxy-params.conf:ro - ./certs:/etc/ssl:ro networks: yocache: ipv4_address: 172.30.0.10 # fixed, so YoCache can trust it
networks: yocache: ipam: config: - subnet: 172.30.0.0/24
volumes: yocache-ci-data: yocache-dev-data:The YoCache containers publish no ports, so builders can only reach them
through nginx. The service names yocache-ci and yocache-dev are the
hostnames the nginx snippet proxies to.
The fixed subnet is there so YOCACHE_TRUSTED_PROXY can name nginx’s
address. Pick one that’s free on the Docker host: VPN clients and company
networks often claim parts of 172.16.0.0/12 or 10.0.0.0/8, and a clash
makes the published port silently unreachable from the host.
Why each setting matters
Section titled “Why each setting matters”client_max_body_size 0: nginx’s default limit is 1 MB, and it rejects anything larger with413before YoCache sees it. Without this line, every sstate archive over 1 MB fails to upload.proxy_http_version 1.1and theUpgrade/Connectionheaders: hash equivalence runs over a WebSocket at/hashequiv, which starts as an HTTP upgrade request. nginx doesn’t forward the upgrade by default. YoCache then answers426 Upgrade Requiredand the build runs without hash equivalence.proxy_read_timeout: nginx closes a WebSocket that has been idle for 60 seconds by default. bitbake reconnects, so builds keep working, but they logError talking to server: Connection closedeach time.proxy_request_buffering off: streams uploads to YoCache instead of writing each archive to the proxy’s disk first.- Trailing slashes on both
location /ci/andproxy_pass …:6768/: this is what strips the prefix, so YoCache receives/sstate/…, not/ci/sstate/…. nginx also redirects a bare/cito/ci/on its own. The dashboard works under a prefix as is.
Client configuration
Section titled “Client configuration”Point YOCACHE_URL at the proxy, prefix included:
YOCACHE_URL = "https://cache.example.com/ci"Mirrors, uploads, the end-of-build summary and hash equivalence all follow
it. The layer derives BB_HASHSERVE = "wss://cache.example.com/ci/hashequiv"
itself.
On releases before Scarthgap, bitbake’s hash-equivalence client can’t speak
WebSocket, and the layer falls back to YoCache’s raw-TCP listener on port
6767. That isn’t HTTP, so it can’t go through the proxy’s location rules,
but nginx’s stream module can relay it as plain TCP:
stream { server { listen 6767; proxy_pass yocache-ci:6767; proxy_timeout 1h; }}The stream block goes at the top level of nginx.conf, next to http.
On these releases the layer defaults BB_HASHSERVE to YOCACHE_URL’s
hostname with port 6767, so a relay on that port needs no client setting;
on any other port, set BB_HASHSERVE = "cache.example.com:<port>". The
relay has three limits: bitbake’s raw-TCP client has no TLS, so the
connection stays unencrypted; TCP has no paths to route by, so each instance
needs its own port; and YoCache sees the relay’s address, not the
builder’s.
Client addresses in logs
Section titled “Client addresses in logs”Behind a proxy, every request reaches YoCache from the proxy’s address, so
that’s what its log lines show (remote=…). To have them name the builder
instead, forward the client address (the X-Forwarded-For line in the
snippet above) and tell YoCache which proxies to believe:
yocache --trusted-proxy 10.0.0.5 # or YOCACHE_TRUSTED_PROXY=10.0.0.5YoCache reads X-Forwarded-For only on connections from a trusted proxy,
and takes the last address in it that isn’t one of your trusted proxies. A
client can’t fake its address by sending the header itself: directly, the
header is ignored, and through the proxy, anything the client put in front
is skipped.
Private or self-signed certificates
Section titled “Private or self-signed certificates”If the proxy’s certificate isn’t from a public CA, the build host has to
trust your CA. Add the CA to the build host’s system trust store
(update-ca-certificates on Debian and Ubuntu), or to the image if you build
in a container such as kas-container. Every tool a build uses reads the
system store, so nothing else is needed.
If you can’t change the trust store, you need both of these, because the two halves of a build look for certificates in different places:
- bitbake’s Python code (mirror checks, uploads, hash equivalence) reads
SSL_CERT_FILE. bitbake removes most environment variables at startup, so let this one through:Terminal window export SSL_CERT_FILE=/path/to/ca.pemexport BB_ENV_PASSTHROUGH_ADDITIONS="$BB_ENV_PASSTHROUGH_ADDITIONS SSL_CERT_FILE" wget, which bitbake runs to download mirror objects, ignoresSSL_CERT_FILEwhen built with GnuTLS, as it is on Debian and Ubuntu. Pass the CA explicitly:Without this line the mirror check still finds the objects, but every download fails withFETCHCMD_wget = "/usr/bin/env wget --tries=2 --timeout=100 --ca-certificate=/path/to/ca.pem"The certificate of '…' is not trustedand bitbake rebuilds them instead.
If you make your own CA, give its certificate the basicConstraints = CA:TRUE
and keyUsage = keyCertSign extensions. Python 3.13 and later reject a CA
certificate without them.
Known limitations
Section titled “Known limitations”- Raw-TCP hash equivalence doesn’t go through the HTTPS proxy. Releases
before Scarthgap can’t use the WebSocket endpoint and need YoCache’s raw-TCP
listener instead. An nginx
streamrelay can carry it (see Client configuration), but only unencrypted, on its own port per instance, and with the relay’s address in YoCache’s logs. - Duplicate uploads still send their data. The uploader asks with
Expect: 100-continue, so that YoCache can refuse an object it already has before the body is sent. nginx answers100 Continueitself, so behind it the whole archive is sent before YoCache gets to refuse it. Uploads stay correct; they just cost more bandwidth when several builders upload the same objects.