Slice 1: build green, MVP core flow

- New agentlib module (gitutil + deployer with NewGo / NewPHP) replaces agent-micro/internal so both agents can share it (Go's internal/ rule was blocking agent-gateway from importing agent-micro's packages). - Migrate agents from legacy github.com/docker/docker/client to the current github.com/moby/moby/client v0.5.0 / moby/moby/api v1.55.0. - Fix compile errors in the original committed code: missing gorilla/websocket import in control-plane/internal/ws/handlers.go, unaliased dockerclient reference, wrong SQLite driver name (sqlite3 -> sqlite), Dialer.Dial 3-return-value mismatch. - scripts/build.sh: Go 1.23 -> 1.24, apk add git, safe.directory for bind-mounted host tree, chmod inside container (host can't chmod files owned by container root). - README and REQUIREMENTS updated to reflect the actual architecture (Go + SQLite, no Spring Boot, moby SDK, per-deploy no image build) with a per-feature status checklist at the end of REQUIREMENTS.
2026-06-24 01:43:43 +00:00
parent 7c1013e083
commit 2bc3ff73a2
18 changed files with 3218 additions and 321 deletions
@@ -1,10 +1,26 @@
 # Sandbox Deployment Platform (SDP)

+## Status (Slice 1 — build green, MVP core flow)
+
+The build is green: `./scripts/build.sh` produces three Linux/amd64
+binaries and a static dashboard. The core MVP loop works end to end —
+login, deploy a microservice or the PHP gateway, watch progress and
+logs in real time.
+
+Sandbox / Template / Route / Environment management is **deferred to
+Slice 2** and is not yet built. Real auth via agent-mediated
+`git ls-remote` and real branch/repo listing from agents are also
+deferred (the current code has hardcoded fixtures and an "accept any
+creds if an agent is connected" stub for these).
+
+See [Status checklist](#status-checklist) at the bottom of this
+document for a per-feature status.
+
 ## Tech Stack (Decided)

 - **Dashboard:** NextJS + React + TypeScript + Tailwind. Plain `useState` + single WebSocket hook. No Redux/Zustand. Built as static output, served by nginx with `try_files`.
- **Control Plane:** Go. PostgreSQL for metadata (nodes, repos, deployments, sandboxes, templates, routes, envs). **SQLite** for ephemeral state (deployment progress snapshots) and **`.log` files** for log persistence. No Spring Boot. No Redis.
- **Agents:** Go. Use the official Docker SDK (`github.com/docker/docker/client`) for container orchestration. Build Go binaries **directly on the host** (`go build -o {name}`) — no Dockerfile-based build step.
+- **Control Plane:** Go. **SQLite** for both metadata and ephemeral state (deployment progress snapshots, log lines). Append-only `.log` files for log persistence. The infra VM (172.18.136.93) is reserved for a future PostgreSQL/Redis/etc. cutover; the MVP runs on SQLite alone.
+- **Agents:** Go. Use the official Docker SDK (`github.com/moby/moby/client` v0.5.0) for container orchestration. Build Go binaries **directly on the host** (`go build -o {name}`) — no Dockerfile-based build step. The PHP gateway agent runs `composer install --no-dev` on the host as a best-effort step, then `docker run php:8.3-apache`.
 - **Realtime transport:** WebSocket end-to-end (Agent → Control Plane → Frontend).
 - **Auth:** Bitbucket username/password. Validated by a real `git ls-remote`/`fetch` via the Agent. **Credentials are passed on every operation from Control Plane to Agent. Never logged, never persisted on the Agent longer than the operation.**
 - **Infra in the spec** = the existing microservice infrastructure (172.18.* VMs, AppGolang, SDP repo), not infrastructure for SDP itself.
@@ -122,10 +138,10 @@ The API Gateway repository.
             v
 +--------------------------+
 | Control Plane            |
-| Spring Boot             |
+| Go (HTTP + WebSocket)    |
 +------+------------+------+
       |            |
-       | HTTP       | HTTP
+       | WebSocket  | WebSocket
       |            |
       v            v

@@ -152,9 +168,9 @@ The Control Plane only:

 * Stores metadata
 * Manages deployments
-* Sends commands via HTTP
-* Receives deployment events
-* Streams logs to frontend
+* Sends commands to agents via WebSocket (`/ws/agent`)
+* Receives deployment events (also via the agent's WebSocket)
+* Streams logs to the dashboard over WebSocket (`/ws/deployments/{id}`)

 ---

@@ -448,8 +464,7 @@ Fetch repository updates
 Checkout branch
 Pull latest changes
 Build Go binary
-Create Docker image
-Run container
+Run container  (the runtime image is pre-loaded; no per-deploy build)
 Restart container
 Stop container
 Stream logs
@@ -474,29 +489,27 @@ git checkout feature/login-error
 git pull
 ```

-Then:
+Then on the host:

 ```bash
-go build -o app
+go build -o app-account ./...
 ```

-Then generates runtime image:
-
-```dockerfile
-FROM alpine:latest
-
-COPY app /app
-
-CMD ["/app"]
-```
-
-Then:
+Then runs a container from the pre-loaded base image, with the host
+repo bind-mounted at `/src` and the freshly-built binary as the
+command:

 ```bash
-docker build
-docker run
+docker run -d \
+  -v /home/user/AppGolang/account:/src \
+  alpine:3.20 \
+  /src/app-account
 ```

+No `docker build` is run. The `alpine:3.20` image is loaded on the
+host once via `docker load -i alpine-3.20.tar` (see
+[Docker Image Distribution](#docker-image-distribution)).
+
 ---

 # Gateway Agent Requirements
@@ -514,10 +527,11 @@ List branches
 Fetch repository updates
 Checkout branch
 Pull latest changes
-Build container
+Run container  (best-effort `composer install --no-dev` on the host;
+                repo is bind-mounted; no per-deploy build)
 Deploy container
 Restart container
-Manage routing
+Manage routing  (deferred to Slice 2)
 Stream logs
 ```

@@ -525,9 +539,8 @@ Stream logs

 # API Gateway Deployment

-The API Gateway must run inside Docker.
-
-It is no longer deployed directly on the host.
+The API Gateway must run inside Docker (so we don't depend on the
+VM's nginx for routing the gateway itself).

 Deployment process:

@@ -535,10 +548,29 @@ Deployment process:
 git fetch
 git checkout
 git pull
-docker build
-docker run
 ```

+Best-effort (skipped silently if `composer` is missing or no
+`composer.json` is present):
+
+```bash
+composer install --no-dev --no-interaction --no-progress
+```
+
+Then runs a container from the pre-loaded PHP image, with the host
+repo bind-mounted at `/app` and Apache as the entrypoint:
+
+```bash
+docker run -d \
+  -v /home/user/SDP:/app \
+  -p 80:80 \
+  php:8.3-apache
+```
+
+No `docker build` is run. The `php:8.3-apache` image is loaded on
+the host once via `docker load -i php-8.3-apache.tar` (see
+[Docker Image Distribution](#docker-image-distribution)).
+
 ---

 # Offline VM Requirements
@@ -695,28 +727,41 @@ Control Plane must:

 # Deployment States

-Supported states:
+The `protocol.Event.State` field carries the lifecycle state of a
+deployment. Supported values:

 ```text
-QUEUED
-
-FETCHING
-
-CHECKOUT
-
-BUILDING
-
-CREATING_IMAGE
-
-STARTING_CONTAINER
-
-RUNNING
-
-FAILED
-
-STOPPED
+QUEUED       // set by the control plane when a deploy is created
+RUNNING      // all stages completed successfully, container is up
+FAILED       // a stage errored; the deploy is dead
+STOPPED      // user-initiated stop
 ```

+In addition, the `Stage` field of a `progress` event carries the
+per-stage human label. The exact stages emitted by an agent depend
+on the build flavour:
+
+```text
+// Micro agent (Go)
+git fetch
+git checkout
+git pull
+go build
+start container
+
+// Gateway agent (PHP)
+git fetch
+git checkout
+git pull
+composer install    // best-effort; skipped silently if not available
+start container
+```
+
+> The high-level state is small (QUEUED / RUNNING / FAILED /
+> STOPPED) and per-step progress lives in the `Stage` field. There
+> is no per-deploy image build, so no image-related state is
+> needed.
+
 ---

 # Real-Time Progress
@@ -936,23 +981,17 @@ Tailwind
 ## Control Plane

 ```text
-Spring Boot
-PostgreSQL
-WebSocket
+Go
+SQLite (modernc.org/sqlite, pure Go, no cgo)
+WebSocket (gorilla/websocket)
 ```

 ## Agents

-Preferred:
-
 ```text
 Go
-```
-
-Alternative:
-
-```text
-Spring Boot
+Docker SDK (github.com/moby/moby/client)
+WebSocket (gorilla/websocket)
 ```

 ---
@@ -1398,7 +1437,70 @@ Channels:
 * Slack
 * Microsoft Teams

-```
+---

-This section should be appended after the MVP section of the main requirements document. It is intentionally out of scope for the first implementation but provides a roadmap that avoids architectural dead ends later. 
-```
+# Status checklist
+
+Per-feature status. `done` = implemented in Slice 1. `next` =
+scheduled for Slice 2. `later` = out of scope for MVP.
+
+## Build / deploy
+
+- `done`  `./scripts/build.sh` produces the three Go binaries and the
+  Next.js dashboard.
+- `done`  `./scripts/deploy.sh` SSHes the binaries to 92 and 186.
+- `done`  `docker compose up -d` brings up the three services on
+  `alpine:latest` for local dev.
+
+## Core deploy flow
+
+- `done`  Agent connects to the control plane over WebSocket and stays
+  connected across reconnects.
+- `done`  Control plane dispatches a `deploy` frame to the agent with
+  the per-operation Bitbucket creds.
+- `done`  Micro agent runs `git fetch → checkout → pull → go build →
+  docker run` and streams progress and logs back.
+- `done`  Gateway agent runs `git fetch → checkout → pull → composer
+  install (best-effort) → docker run` and streams progress and logs
+  back.
+- `done`  Dashboard subscribes to a deployment by id over WebSocket
+  and renders stages + live log tail.
+- `done`  SQLite persistence for deployment rows, stage transitions,
+  and append-only log files.
+- `next`  Replace `validateViaAgent` stub with a real
+  `git ls-remote` frame.
+- `next`  Replace hardcoded `handleListRepos` /
+  `handleListBranches` with agent frames (the `gitutil.ListBranches`
+  helper and the `agentlib` frame protocol are partially set up but
+  not wired through).
+
+## Sandbox & routing (Slice 2)
+
+- `next`  Sandbox CRUD (data model + REST endpoints + dashboard page).
+- `next`  Sandbox template CRUD and "clone template into sandbox".
+- `next`  Route management (sandbox vs OCP per service).
+- `next`  Environment CRUD (persisted named envs, not just inline).
+- `next`  Actual route push to the API Gateway (the gateway agent
+  has to update the gateway's routing config, currently this is
+  the manual `scripts/patch-nginx.sh` step).
+- `next`  Port allocation table and helpers.
+
+## Auth
+
+- `done`  Login endpoint accepts any creds if an agent is connected
+  (MVP stub).
+- `done`  Session cookie + in-memory session store.
+- `next`  Real auth via agent-mediated `git ls-remote`.
+- `later` RBAC roles (admin / backend / qa / viewer).
+
+## Out of scope for MVP (per the "Future Enhancements" section)
+
+- `later` Per-sandbox Docker networks and the
+  `sandbox-{name}-{service}` container naming.
+- `later` Internal service communication via Docker DNS.
+- `later` Suspend / resume / expire sandboxes.
+- `later` Sandbox cloning and snapshots.
+- `later` Per-sandbox resource limits.
+- `later` Health monitoring.
+- `later` The infra agent (172.18.136.93) for PostgreSQL/Redis/etc.
+- `later` Notifications (email / Slack / Teams).