Achmad
4cab047432
- control-plane default listen addr is now :3452 (was :8080). An unusual port to avoid collisions on the VM. - agent-micro and agent-gateway default SDP_CP_URL points at ws://localhost:3452/ws/agent. docker-compose.yml updates the control plane command, host port mapping, and agent -cp URLs. - nginx/nginx.conf (the legacy root-mount reference) uses 127.0.0.1:3452 for the upstream. nginx/sandbox.conf is the new deployment config: four location blocks for the /sandbox/credit-card mount — _next/static serves cached chunks, /api/ and /ws/ proxy to 127.0.0.1:3452, /sandbox/credit-card serves the static dashboard with try_files for SPA routing. - scripts/patch-nginx.sh: deleted. The user configures nginx on 186 by hand. scripts/deploy.sh no longer calls it. - AGENTS.md: new file. Documents the build/lint/test commands (with the golang:1.24-alpine container — local Go can't fetch the toolchain), the wire protocol, the Slice-2 conventions (sdp-<repo> container naming, snapshot persistence, PreGitReset/AfterStart hooks), the repo-path gotcha, and the build-artifacts-in-git rationale. - dashboard/out: now tracked in git, alongside bin/. The dashboard static export is scp'd to 186 on deploy; the VMs have no internet so they can't regenerate it. .gitignore comment explains this and warns against re-ignoring. - README.md / REQUIREMENTS.md: status updated to 'Slice 2 done', per-feature checklist marked. Erangel repo path corrected to /var/www/html/erangel-ocean (was wrongly ~/SDP in earlier docs).
192 lines
7.6 KiB
Markdown
192 lines
7.6 KiB
Markdown
# Sandbox Deployment Platform (SDP)
|
|
|
|
Internal deployment platform for Backend/QA. Lets a developer deploy a feature
|
|
branch into an isolated sandbox, with the API Gateway routing selected
|
|
services to the sandbox and the rest to OCP. See
|
|
[REQUIREMENTS.md](REQUIREMENTS.md) for the full spec.
|
|
|
|
## Status (Slice 2 — sandboxes, routes, real auth, all MVP features)
|
|
|
|
`./scripts/build.sh` produces three Linux/amd64 binaries and a static
|
|
dashboard. The full MVP flow works end to end:
|
|
|
|
- Real Bitbucket auth via `git ls-remote` against the api-gateway.
|
|
- Real repo and branch listing via agent WS frames.
|
|
- Sandbox / template / environment CRUD with persisted metadata in
|
|
SQLite.
|
|
- Route overrides per sandbox, with live read-back of the
|
|
`<service>_url` map from the gateway's `config.php` after every
|
|
branch switch. The agent patches the file and gracefully reloads
|
|
apache.
|
|
- Per-deploy port binding: the user picks the host port per service
|
|
(e.g. eredar at `172.18.136.92:9001`), the container's exposed port
|
|
is published to that port.
|
|
- Erangel deploy: `git reset --hard → fetch → checkout → pull →
|
|
composer install → start container → re-apply route overrides`.
|
|
Per-branch OCP-default snapshot persisted to
|
|
`<repo>/.sdp/ocp-defaults.json`.
|
|
|
|
See [REQUIREMENTS.md](REQUIREMENTS.md#status) for the per-feature
|
|
checklist.
|
|
|
|
## Layout
|
|
|
|
```
|
|
.
|
|
├── protocol/ # shared wire types (Event, DeployRequest, RouteOverride, ...)
|
|
├── agentlib/ # Go. Shared agent library: gitutil + deployer (Go/PHP flavours)
|
|
├── control-plane/ # Go. HTTP API + WS hub + SQLite/.log persistence
|
|
├── agent-micro/ # Go. Runs on 172.18.136.92, deploys Go microservices
|
|
├── agent-gateway/ # Go. Runs on 172.18.139.186, deploys the PHP API Gateway
|
|
├── dashboard/ # NextJS static export, served by nginx
|
|
├── nginx/ # reference nginx config (manually applied on 186)
|
|
├── scripts/ # build, deploy, ssh wrappers
|
|
├── docker-compose.yml # all three services on alpine:latest
|
|
├── go.work # Go workspace — one build, five modules
|
|
└── bin/ # build output (gitignored)
|
|
```
|
|
|
|
`agentlib/` is a shared library used by both agents. It owns the git
|
|
helpers and the per-deployment state machine, which has two constructors
|
|
for two build flavours:
|
|
|
|
- **`NewGo`** — for microservices. Runs `go build` on the host, then
|
|
`docker run alpine:3.20` with the host repo bind-mounted at `/src` and
|
|
the binary as the container command. `alpine:3.20` must be pre-loaded
|
|
on the host (see [Offline VMs](#offline-vms)).
|
|
- **`NewPHP`** — for the API Gateway (erangel). Runs
|
|
`git reset --hard → fetch → checkout → pull → composer install
|
|
(best-effort) → docker run php:8.3-apache`, with the repo
|
|
bind-mounted at `/var/www/html/erangel-ocean` and
|
|
`APACHE_DOCUMENT_ROOT=/var/www/html/erangel-ocean` so the gateway is
|
|
served at `/erangel/`, mirroring production. After the container is
|
|
up, the agent's `AfterStart` callback re-applies the active route
|
|
overrides and reloads apache. `php:8.3-apache` must be pre-loaded on
|
|
the host. The agent is written in Go; the thing it deploys is a
|
|
PHP project.
|
|
|
|
## Prerequisites
|
|
|
|
- Docker (for the build container)
|
|
- Node 18+ (for the dashboard)
|
|
- `sshpass` (for the deploy scripts: `brew install sshpass`)
|
|
|
|
No Go install needed locally — `scripts/build.sh` cross-compiles inside
|
|
`golang:1.24-alpine`.
|
|
|
|
## Build
|
|
|
|
```bash
|
|
./scripts/build.sh
|
|
```
|
|
|
|
Outputs:
|
|
- `bin/control-plane`, `bin/agent-micro`, `bin/agent-gateway` (Linux/amd64
|
|
ELF, statically linked)
|
|
- `dashboard/out/` (NextJS static export)
|
|
|
|
The build script:
|
|
1. Starts a `golang:1.24-alpine` container with the repo bind-mounted.
|
|
2. `apk add git` (the base image has none).
|
|
3. Configures `safe.directory /src` so the container's root user can
|
|
read the bind-mounted host tree.
|
|
4. Cross-compiles all three binaries with `GOOS=linux GOARCH=amd64
|
|
CGO_ENABLED=0`, `-trimpath` (reproducible builds) and
|
|
`-ldflags="-s -w"` (strip debug info).
|
|
5. `chmod +x` the binaries inside the container (the host user can't
|
|
chmod files written by the container's root).
|
|
6. Builds the Next.js dashboard with `npm install && npm run build`.
|
|
|
|
The script verifies each binary with `file` to catch a missing
|
|
`GOOS`/`GOARCH`.
|
|
|
|
## Deploy
|
|
|
|
```bash
|
|
./scripts/deploy.sh
|
|
```
|
|
|
|
This script:
|
|
1. SSHs to **172.18.136.92** (`administrator`) and pushes `bin/agent-micro`
|
|
to `~/SDP/bin/`
|
|
2. SSHs to **172.18.139.186** (`administrator`) and pushes
|
|
`bin/control-plane`, `bin/agent-gateway`, and `dashboard/out/` to
|
|
`~/SDP/`
|
|
|
|
Nginx on 186 is configured by hand; the dashboard ends up at
|
|
`/home/administrator/SDP/dashboard/`. The required location block is
|
|
in [nginx/nginx.conf](nginx/nginx.conf).
|
|
|
|
Override the creds via `SDP_92_PASS` / `SDP_186_PASS` env vars.
|
|
|
|
## Local dev (docker compose)
|
|
|
|
For dev on a single host (e.g. a laptop with Docker):
|
|
|
|
```bash
|
|
./scripts/build.sh
|
|
docker compose up -d
|
|
```
|
|
|
|
Three services come up on `alpine:latest`:
|
|
- `control-plane` → `:3452` (an unusual port to avoid collisions)
|
|
- `agent-micro` (connects to control plane, has docker socket + repos mounted)
|
|
- `agent-gateway` (same shape)
|
|
|
|
## Architecture notes
|
|
|
|
- **Pass-through creds.** Bitbucket credentials travel with each deploy
|
|
request from control plane to agent, are used once for `git fetch`/`checkout`/
|
|
`pull`, and are never logged or persisted on the agent.
|
|
- **No Dockerfile build on the agent.** Each agent does the language
|
|
build on the host (Go or composer), then `docker run <base-image>`
|
|
with the host repo bind-mounted and the binary / apache as the
|
|
container command. The base image must be pre-loaded.
|
|
- **Offline VMs.** `alpine:3.20` and `php:8.3-apache` are pre-loaded
|
|
via `docker load`. The dashboard is a static export, no runtime
|
|
fetches.
|
|
- **Persistence.** Deployment progress goes to SQLite
|
|
(`<data>/sdp.db`). Log lines go to append-only
|
|
`<data>/logs/<deploymentId>.log`. SQLite uses `modernc.org/sqlite`
|
|
(pure Go, no cgo) so the control plane binary stays statically
|
|
linkable. The driver name is `sqlite` (not `sqlite3`).
|
|
- **Docker SDK.** The agents use the official Moby Go SDK at
|
|
`github.com/moby/moby/client` v0.5.0.
|
|
- **Realtime transport.** WebSocket end-to-end. Agents connect to
|
|
`/ws/agent` on the control plane; the dashboard subscribes to
|
|
`/ws/deployments/{id}`.
|
|
|
|
## MVP stubs (intentional, deferred)
|
|
|
|
These are marked with `ponytail:` comments in the code and are
|
|
scheduled for later slices.
|
|
|
|
- `CheckOrigin` in the WS upgrader — open CORS, intentional for an
|
|
internal tool.
|
|
- "Drop on backpressure" policy for slow WS subscribers — replace with
|
|
flow control or persistent event log if the dashboard ever needs
|
|
catch-up replay.
|
|
- O(n) log tail scan in `store.TailLogs` — fine for tail use; swap to
|
|
a ring buffer if logs get huge.
|
|
|
|
## Slice 2 dashboard
|
|
|
|
The dashboard has these pages:
|
|
|
|
- `/` — login (real git-ls-remote via the gateway agent).
|
|
- `/dashboard` — quick deploy (ad-hoc single-service deploy).
|
|
- `/dashboard/sandboxes` — list, create, clone-from-template.
|
|
- `/dashboard/sandboxes/{id}` — sandbox detail. Live routes from the
|
|
gateway's `config.php`, per-route toggle (OCP / sandbox override),
|
|
microservice deploys with per-service host port and env.
|
|
- `/dashboard/templates` — template CRUD.
|
|
- `/dashboard/environments` — env CRUD.
|
|
- `/dashboard/history` — deployment history (filterable by sandbox).
|
|
|
|
## See also
|
|
|
|
- [REQUIREMENTS.md](REQUIREMENTS.md) — full spec, infra, MVP success criteria,
|
|
per-feature status checklist
|
|
- [nginx/nginx.conf](nginx/nginx.conf) — reference nginx config
|
|
- [docker-compose.yml](docker-compose.yml) — three-service dev stack
|