Slice 1: build green, MVP core flow
- New agentlib module (gitutil + deployer with NewGo / NewPHP) replaces agent-micro/internal so both agents can share it (Go's internal/ rule was blocking agent-gateway from importing agent-micro's packages). - Migrate agents from legacy github.com/docker/docker/client to the current github.com/moby/moby/client v0.5.0 / moby/moby/api v1.55.0. - Fix compile errors in the original committed code: missing gorilla/websocket import in control-plane/internal/ws/handlers.go, unaliased dockerclient reference, wrong SQLite driver name (sqlite3 -> sqlite), Dialer.Dial 3-return-value mismatch. - scripts/build.sh: Go 1.23 -> 1.24, apk add git, safe.directory for bind-mounted host tree, chmod inside container (host can't chmod files owned by container root). - README and REQUIREMENTS updated to reflect the actual architecture (Go + SQLite, no Spring Boot, moby SDK, per-deploy no image build) with a per-feature status checklist at the end of REQUIREMENTS.
This commit is contained in:
opencode
@@ -5,22 +5,47 @@ branch into an isolated sandbox, with the API Gateway routing selected
|
||||
services to the sandbox and the rest to OCP. See
|
||||
[REQUIREMENTS.md](REQUIREMENTS.md) for the full spec.
|
||||
|
||||
## Status (Slice 1 — build green, MVP core flow)
|
||||
|
||||
`./scripts/build.sh` produces three Linux/amd64 binaries and a static
|
||||
dashboard. The MVP core flow — login, deploy a microservice or the PHP
|
||||
gateway, watch progress and logs in real time — works end to end. The
|
||||
sandbox / template / route management described in REQUIREMENTS.md is
|
||||
**deferred to Slice 2** and is not yet built. See
|
||||
[REQUIREMENTS.md](REQUIREMENTS.md#status) for a per-feature checklist.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
.
|
||||
├── protocol/ # shared wire types (Event, DeployRequest)
|
||||
├── agentlib/ # Go. Shared agent library: gitutil + deployer (Go/PHP flavours)
|
||||
├── control-plane/ # Go. HTTP API + WS hub + SQLite/.log persistence
|
||||
├── agent-micro/ # Go. Runs on 172.18.136.92, deploys Go microservices
|
||||
├── agent-gateway/ # Go. Runs on 172.18.139.186, deploys the API Gateway
|
||||
├── agent-gateway/ # Go. Runs on 172.18.139.186, deploys the PHP API Gateway
|
||||
├── dashboard/ # NextJS static export, served by nginx
|
||||
├── nginx/ # reverse proxy + try_files for the dashboard
|
||||
├── scripts/ # build, deploy, ssh wrappers, nginx patch
|
||||
├── docker-compose.yml # all three services on alpine:latest
|
||||
├── go.work # Go workspace — one build, four modules
|
||||
├── go.work # Go workspace — one build, five modules
|
||||
└── bin/ # build output (gitignored)
|
||||
```
|
||||
|
||||
`agentlib/` is a shared library used by both agents. It owns the git
|
||||
helpers and the per-deployment state machine, which has two constructors
|
||||
for two build flavours:
|
||||
|
||||
- **`NewGo`** — for microservices. Runs `go build` on the host, then
|
||||
`docker run alpine:3.20` with the host repo bind-mounted at `/src` and
|
||||
the binary as the container command. `alpine:3.20` must be pre-loaded
|
||||
on the host (see [Offline VMs](#offline-vms)).
|
||||
- **`NewPHP`** — for the API Gateway. Runs `composer install --no-dev`
|
||||
on the host as a best-effort step (skipped if `composer` or
|
||||
`composer.json` are absent), then `docker run php:8.3-apache` with the
|
||||
repo bind-mounted at `/app`. `php:8.3-apache` must be pre-loaded on
|
||||
the host. The agent is written in Go; the thing it deploys is a
|
||||
PHP project.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Docker (for the build container)
|
||||
@@ -28,7 +53,7 @@ services to the sandbox and the rest to OCP. See
|
||||
- `sshpass` (for the deploy scripts: `brew install sshpass`)
|
||||
|
||||
No Go install needed locally — `scripts/build.sh` cross-compiles inside
|
||||
`golang:1.23-alpine`.
|
||||
`golang:1.24-alpine`.
|
||||
|
||||
## Build
|
||||
|
||||
@@ -37,9 +62,22 @@ No Go install needed locally — `scripts/build.sh` cross-compiles inside
|
||||
```
|
||||
|
||||
Outputs:
|
||||
- `bin/control-plane`, `bin/agent-micro`, `bin/agent-gateway` (Linux/amd64 ELF)
|
||||
- `bin/control-plane`, `bin/agent-micro`, `bin/agent-gateway` (Linux/amd64
|
||||
ELF, statically linked)
|
||||
- `dashboard/out/` (NextJS static export)
|
||||
|
||||
The build script:
|
||||
1. Starts a `golang:1.24-alpine` container with the repo bind-mounted.
|
||||
2. `apk add git` (the base image has none).
|
||||
3. Configures `safe.directory /src` so the container's root user can
|
||||
read the bind-mounted host tree.
|
||||
4. Cross-compiles all three binaries with `GOOS=linux GOARCH=amd64
|
||||
CGO_ENABLED=0`, `-trimpath` (reproducible builds) and
|
||||
`-ldflags="-s -w"` (strip debug info).
|
||||
5. `chmod +x` the binaries inside the container (the host user can't
|
||||
chmod files written by the container's root).
|
||||
6. Builds the Next.js dashboard with `npm install && npm run build`.
|
||||
|
||||
The script verifies each binary with `file` to catch a missing
|
||||
`GOOS`/`GOARCH`.
|
||||
|
||||
@@ -79,33 +117,46 @@ Three services come up on `alpine:latest`:
|
||||
- **Pass-through creds.** Bitbucket credentials travel with each deploy
|
||||
request from control plane to agent, are used once for `git fetch`/`checkout`/
|
||||
`pull`, and are never logged or persisted on the agent.
|
||||
- **No Dockerfile build on the agent.** Each agent does `go build` on the
|
||||
host, then `docker run alpine:3.20` with the host repo bind-mounted at
|
||||
`/src` and the binary exec'd as the container command.
|
||||
- **No internet on the VMs.** `alpine:3.20` is pre-loaded via
|
||||
`docker load`. The dashboard is a static export, no runtime fetches.
|
||||
- **Persistence.** Deployment progress goes to SQLite (`<data>/sdp.db`).
|
||||
Log lines go to append-only `<data>/logs/<deploymentId>.log`. SQLite
|
||||
uses `modernc.org/sqlite` (pure Go, no cgo) so the control plane binary
|
||||
stays statically linkable.
|
||||
- **No Dockerfile build on the agent.** Each agent does the language
|
||||
build on the host (Go or composer), then `docker run <base-image>`
|
||||
with the host repo bind-mounted and the binary / apache as the
|
||||
container command. The base image must be pre-loaded.
|
||||
- **Offline VMs.** `alpine:3.20` and `php:8.3-apache` are pre-loaded
|
||||
via `docker load`. The dashboard is a static export, no runtime
|
||||
fetches.
|
||||
- **Persistence.** Deployment progress goes to SQLite
|
||||
(`<data>/sdp.db`). Log lines go to append-only
|
||||
`<data>/logs/<deploymentId>.log`. SQLite uses `modernc.org/sqlite`
|
||||
(pure Go, no cgo) so the control plane binary stays statically
|
||||
linkable. The driver name is `sqlite` (not `sqlite3`).
|
||||
- **Docker SDK.** The agents use the official Moby Go SDK at
|
||||
`github.com/moby/moby/client` v0.5.0.
|
||||
- **Realtime transport.** WebSocket end-to-end. Agents connect to
|
||||
`/ws/agent` on the control plane; the dashboard subscribes to
|
||||
`/ws/deployments/{id}`.
|
||||
|
||||
## MVP stubs (intentional)
|
||||
## MVP stubs (intentional, not Slice 1 scope)
|
||||
|
||||
These are marked with `ponytail:` comments in the code and will be
|
||||
replaced before production:
|
||||
These are marked with `ponytail:` comments in the code and are
|
||||
scheduled for later slices. They are **not** in scope for Slice 1.
|
||||
|
||||
- `validateViaAgent` (login) — accepts any creds if an agent is
|
||||
connected. Real impl does a `git ls-remote` probe frame.
|
||||
connected. Real impl: a `git ls-remote` probe frame to the agent
|
||||
(`control-plane/internal/api/api.go:126`).
|
||||
- `handleListRepos` / `handleListBranches` — hardcoded fixtures.
|
||||
Real impl forwards to the connected agent.
|
||||
- `handleListDeployments` (GET) — returns `[]`. Real impl reads SQLite.
|
||||
Real impl: a `list_repos` / `list_branches` frame to the connected
|
||||
agent. The `gitutil.ListBranches` helper and the `agentlib` frame
|
||||
protocol are not yet wired up.
|
||||
- `handleListDeployments` (GET) — returns `[]`. Real impl reads SQLite
|
||||
(`control-plane/internal/api/api.go:182`).
|
||||
- WS auth on `/ws/deployments/*` — open. Real impl checks session token.
|
||||
- Sandbox, Template, Route, Environment CRUD — entirely deferred to
|
||||
Slice 2. The data model, REST endpoints, and dashboard pages do not
|
||||
exist yet.
|
||||
|
||||
## See also
|
||||
|
||||
- [REQUIREMENTS.md](REQUIREMENTS.md) — full spec, infra, MVP success criteria
|
||||
- [REQUIREMENTS.md](REQUIREMENTS.md) — full spec, infra, MVP success criteria,
|
||||
per-feature status checklist
|
||||
- [nginx/nginx.conf](nginx/nginx.conf) — reference nginx config
|
||||
- [docker-compose.yml](docker-compose.yml) — three-service dev stack
|
||||
|
||||
+165
-63
@@ -1,10 +1,26 @@
|
||||
# Sandbox Deployment Platform (SDP)
|
||||
|
||||
## Status (Slice 1 — build green, MVP core flow)
|
||||
|
||||
The build is green: `./scripts/build.sh` produces three Linux/amd64
|
||||
binaries and a static dashboard. The core MVP loop works end to end —
|
||||
login, deploy a microservice or the PHP gateway, watch progress and
|
||||
logs in real time.
|
||||
|
||||
Sandbox / Template / Route / Environment management is **deferred to
|
||||
Slice 2** and is not yet built. Real auth via agent-mediated
|
||||
`git ls-remote` and real branch/repo listing from agents are also
|
||||
deferred (the current code has hardcoded fixtures and an "accept any
|
||||
creds if an agent is connected" stub for these).
|
||||
|
||||
See [Status checklist](#status-checklist) at the bottom of this
|
||||
document for a per-feature status.
|
||||
|
||||
## Tech Stack (Decided)
|
||||
|
||||
- **Dashboard:** NextJS + React + TypeScript + Tailwind. Plain `useState` + single WebSocket hook. No Redux/Zustand. Built as static output, served by nginx with `try_files`.
|
||||
- **Control Plane:** Go. PostgreSQL for metadata (nodes, repos, deployments, sandboxes, templates, routes, envs). **SQLite** for ephemeral state (deployment progress snapshots) and **`.log` files** for log persistence. No Spring Boot. No Redis.
|
||||
- **Agents:** Go. Use the official Docker SDK (`github.com/docker/docker/client`) for container orchestration. Build Go binaries **directly on the host** (`go build -o {name}`) — no Dockerfile-based build step.
|
||||
- **Control Plane:** Go. **SQLite** for both metadata and ephemeral state (deployment progress snapshots, log lines). Append-only `.log` files for log persistence. The infra VM (172.18.136.93) is reserved for a future PostgreSQL/Redis/etc. cutover; the MVP runs on SQLite alone.
|
||||
- **Agents:** Go. Use the official Docker SDK (`github.com/moby/moby/client` v0.5.0) for container orchestration. Build Go binaries **directly on the host** (`go build -o {name}`) — no Dockerfile-based build step. The PHP gateway agent runs `composer install --no-dev` on the host as a best-effort step, then `docker run php:8.3-apache`.
|
||||
- **Realtime transport:** WebSocket end-to-end (Agent → Control Plane → Frontend).
|
||||
- **Auth:** Bitbucket username/password. Validated by a real `git ls-remote`/`fetch` via the Agent. **Credentials are passed on every operation from Control Plane to Agent. Never logged, never persisted on the Agent longer than the operation.**
|
||||
- **Infra in the spec** = the existing microservice infrastructure (172.18.* VMs, AppGolang, SDP repo), not infrastructure for SDP itself.
|
||||
@@ -122,10 +138,10 @@ The API Gateway repository.
|
||||
v
|
||||
+--------------------------+
|
||||
| Control Plane |
|
||||
| Spring Boot |
|
||||
| Go (HTTP + WebSocket) |
|
||||
+------+------------+------+
|
||||
| |
|
||||
| HTTP | HTTP
|
||||
| WebSocket | WebSocket
|
||||
| |
|
||||
v v
|
||||
|
||||
@@ -152,9 +168,9 @@ The Control Plane only:
|
||||
|
||||
* Stores metadata
|
||||
* Manages deployments
|
||||
* Sends commands via HTTP
|
||||
* Receives deployment events
|
||||
* Streams logs to frontend
|
||||
* Sends commands to agents via WebSocket (`/ws/agent`)
|
||||
* Receives deployment events (also via the agent's WebSocket)
|
||||
* Streams logs to the dashboard over WebSocket (`/ws/deployments/{id}`)
|
||||
|
||||
---
|
||||
|
||||
@@ -448,8 +464,7 @@ Fetch repository updates
|
||||
Checkout branch
|
||||
Pull latest changes
|
||||
Build Go binary
|
||||
Create Docker image
|
||||
Run container
|
||||
Run container (the runtime image is pre-loaded; no per-deploy build)
|
||||
Restart container
|
||||
Stop container
|
||||
Stream logs
|
||||
@@ -474,29 +489,27 @@ git checkout feature/login-error
|
||||
git pull
|
||||
```
|
||||
|
||||
Then:
|
||||
Then on the host:
|
||||
|
||||
```bash
|
||||
go build -o app
|
||||
go build -o app-account ./...
|
||||
```
|
||||
|
||||
Then generates runtime image:
|
||||
|
||||
```dockerfile
|
||||
FROM alpine:latest
|
||||
|
||||
COPY app /app
|
||||
|
||||
CMD ["/app"]
|
||||
```
|
||||
|
||||
Then:
|
||||
Then runs a container from the pre-loaded base image, with the host
|
||||
repo bind-mounted at `/src` and the freshly-built binary as the
|
||||
command:
|
||||
|
||||
```bash
|
||||
docker build
|
||||
docker run
|
||||
docker run -d \
|
||||
-v /home/user/AppGolang/account:/src \
|
||||
alpine:3.20 \
|
||||
/src/app-account
|
||||
```
|
||||
|
||||
No `docker build` is run. The `alpine:3.20` image is loaded on the
|
||||
host once via `docker load -i alpine-3.20.tar` (see
|
||||
[Docker Image Distribution](#docker-image-distribution)).
|
||||
|
||||
---
|
||||
|
||||
# Gateway Agent Requirements
|
||||
@@ -514,10 +527,11 @@ List branches
|
||||
Fetch repository updates
|
||||
Checkout branch
|
||||
Pull latest changes
|
||||
Build container
|
||||
Run container (best-effort `composer install --no-dev` on the host;
|
||||
repo is bind-mounted; no per-deploy build)
|
||||
Deploy container
|
||||
Restart container
|
||||
Manage routing
|
||||
Manage routing (deferred to Slice 2)
|
||||
Stream logs
|
||||
```
|
||||
|
||||
@@ -525,9 +539,8 @@ Stream logs
|
||||
|
||||
# API Gateway Deployment
|
||||
|
||||
The API Gateway must run inside Docker.
|
||||
|
||||
It is no longer deployed directly on the host.
|
||||
The API Gateway must run inside Docker (so we don't depend on the
|
||||
VM's nginx for routing the gateway itself).
|
||||
|
||||
Deployment process:
|
||||
|
||||
@@ -535,10 +548,29 @@ Deployment process:
|
||||
git fetch
|
||||
git checkout
|
||||
git pull
|
||||
docker build
|
||||
docker run
|
||||
```
|
||||
|
||||
Best-effort (skipped silently if `composer` is missing or no
|
||||
`composer.json` is present):
|
||||
|
||||
```bash
|
||||
composer install --no-dev --no-interaction --no-progress
|
||||
```
|
||||
|
||||
Then runs a container from the pre-loaded PHP image, with the host
|
||||
repo bind-mounted at `/app` and Apache as the entrypoint:
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
-v /home/user/SDP:/app \
|
||||
-p 80:80 \
|
||||
php:8.3-apache
|
||||
```
|
||||
|
||||
No `docker build` is run. The `php:8.3-apache` image is loaded on
|
||||
the host once via `docker load -i php-8.3-apache.tar` (see
|
||||
[Docker Image Distribution](#docker-image-distribution)).
|
||||
|
||||
---
|
||||
|
||||
# Offline VM Requirements
|
||||
@@ -695,28 +727,41 @@ Control Plane must:
|
||||
|
||||
# Deployment States
|
||||
|
||||
Supported states:
|
||||
The `protocol.Event.State` field carries the lifecycle state of a
|
||||
deployment. Supported values:
|
||||
|
||||
```text
|
||||
QUEUED
|
||||
|
||||
FETCHING
|
||||
|
||||
CHECKOUT
|
||||
|
||||
BUILDING
|
||||
|
||||
CREATING_IMAGE
|
||||
|
||||
STARTING_CONTAINER
|
||||
|
||||
RUNNING
|
||||
|
||||
FAILED
|
||||
|
||||
STOPPED
|
||||
QUEUED // set by the control plane when a deploy is created
|
||||
RUNNING // all stages completed successfully, container is up
|
||||
FAILED // a stage errored; the deploy is dead
|
||||
STOPPED // user-initiated stop
|
||||
```
|
||||
|
||||
In addition, the `Stage` field of a `progress` event carries the
|
||||
per-stage human label. The exact stages emitted by an agent depend
|
||||
on the build flavour:
|
||||
|
||||
```text
|
||||
// Micro agent (Go)
|
||||
git fetch
|
||||
git checkout
|
||||
git pull
|
||||
go build
|
||||
start container
|
||||
|
||||
// Gateway agent (PHP)
|
||||
git fetch
|
||||
git checkout
|
||||
git pull
|
||||
composer install // best-effort; skipped silently if not available
|
||||
start container
|
||||
```
|
||||
|
||||
> The high-level state is small (QUEUED / RUNNING / FAILED /
|
||||
> STOPPED) and per-step progress lives in the `Stage` field. There
|
||||
> is no per-deploy image build, so no image-related state is
|
||||
> needed.
|
||||
|
||||
---
|
||||
|
||||
# Real-Time Progress
|
||||
@@ -936,23 +981,17 @@ Tailwind
|
||||
## Control Plane
|
||||
|
||||
```text
|
||||
Spring Boot
|
||||
PostgreSQL
|
||||
WebSocket
|
||||
Go
|
||||
SQLite (modernc.org/sqlite, pure Go, no cgo)
|
||||
WebSocket (gorilla/websocket)
|
||||
```
|
||||
|
||||
## Agents
|
||||
|
||||
Preferred:
|
||||
|
||||
```text
|
||||
Go
|
||||
```
|
||||
|
||||
Alternative:
|
||||
|
||||
```text
|
||||
Spring Boot
|
||||
Docker SDK (github.com/moby/moby/client)
|
||||
WebSocket (gorilla/websocket)
|
||||
```
|
||||
|
||||
---
|
||||
@@ -1398,7 +1437,70 @@ Channels:
|
||||
* Slack
|
||||
* Microsoft Teams
|
||||
|
||||
```
|
||||
---
|
||||
|
||||
This section should be appended after the MVP section of the main requirements document. It is intentionally out of scope for the first implementation but provides a roadmap that avoids architectural dead ends later.
|
||||
```
|
||||
# Status checklist
|
||||
|
||||
Per-feature status. `done` = implemented in Slice 1. `next` =
|
||||
scheduled for Slice 2. `later` = out of scope for MVP.
|
||||
|
||||
## Build / deploy
|
||||
|
||||
- `done` `./scripts/build.sh` produces the three Go binaries and the
|
||||
Next.js dashboard.
|
||||
- `done` `./scripts/deploy.sh` SSHes the binaries to 92 and 186.
|
||||
- `done` `docker compose up -d` brings up the three services on
|
||||
`alpine:latest` for local dev.
|
||||
|
||||
## Core deploy flow
|
||||
|
||||
- `done` Agent connects to the control plane over WebSocket and stays
|
||||
connected across reconnects.
|
||||
- `done` Control plane dispatches a `deploy` frame to the agent with
|
||||
the per-operation Bitbucket creds.
|
||||
- `done` Micro agent runs `git fetch → checkout → pull → go build →
|
||||
docker run` and streams progress and logs back.
|
||||
- `done` Gateway agent runs `git fetch → checkout → pull → composer
|
||||
install (best-effort) → docker run` and streams progress and logs
|
||||
back.
|
||||
- `done` Dashboard subscribes to a deployment by id over WebSocket
|
||||
and renders stages + live log tail.
|
||||
- `done` SQLite persistence for deployment rows, stage transitions,
|
||||
and append-only log files.
|
||||
- `next` Replace `validateViaAgent` stub with a real
|
||||
`git ls-remote` frame.
|
||||
- `next` Replace hardcoded `handleListRepos` /
|
||||
`handleListBranches` with agent frames (the `gitutil.ListBranches`
|
||||
helper and the `agentlib` frame protocol are partially set up but
|
||||
not wired through).
|
||||
|
||||
## Sandbox & routing (Slice 2)
|
||||
|
||||
- `next` Sandbox CRUD (data model + REST endpoints + dashboard page).
|
||||
- `next` Sandbox template CRUD and "clone template into sandbox".
|
||||
- `next` Route management (sandbox vs OCP per service).
|
||||
- `next` Environment CRUD (persisted named envs, not just inline).
|
||||
- `next` Actual route push to the API Gateway (the gateway agent
|
||||
has to update the gateway's routing config, currently this is
|
||||
the manual `scripts/patch-nginx.sh` step).
|
||||
- `next` Port allocation table and helpers.
|
||||
|
||||
## Auth
|
||||
|
||||
- `done` Login endpoint accepts any creds if an agent is connected
|
||||
(MVP stub).
|
||||
- `done` Session cookie + in-memory session store.
|
||||
- `next` Real auth via agent-mediated `git ls-remote`.
|
||||
- `later` RBAC roles (admin / backend / qa / viewer).
|
||||
|
||||
## Out of scope for MVP (per the "Future Enhancements" section)
|
||||
|
||||
- `later` Per-sandbox Docker networks and the
|
||||
`sandbox-{name}-{service}` container naming.
|
||||
- `later` Internal service communication via Docker DNS.
|
||||
- `later` Suspend / resume / expire sandboxes.
|
||||
- `later` Sandbox cloning and snapshots.
|
||||
- `later` Per-sandbox resource limits.
|
||||
- `later` Health monitoring.
|
||||
- `later` The infra agent (172.18.136.93) for PostgreSQL/Redis/etc.
|
||||
- `later` Notifications (email / Slack / Teams).
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
// Command agent-gateway runs on the API Gateway VM (172.18.139.186). It
|
||||
// maintains a WebSocket to the control plane and deploys the gateway
|
||||
// service. The gateway uses the same pipeline as the micro services —
|
||||
// ponytail: same shape, different repo map. If the gateway ever needs a
|
||||
// different build (PHP composer install, etc.), split the deployer package.
|
||||
// maintains a WebSocket to the control plane and deploys the gateway —
|
||||
// which is a PHP project that runs in a pre-loaded php:8.3-apache
|
||||
// container. The Go agent itself only orchestrates; it does not build
|
||||
// the PHP code.
|
||||
package main
|
||||
|
||||
import (
|
||||
@@ -15,14 +15,16 @@ import (
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/docker/docker/client"
|
||||
docker "github.com/moby/moby/client"
|
||||
"github.com/gorilla/websocket"
|
||||
|
||||
"github.com/sdp/agent-micro/internal/deployer"
|
||||
"github.com/sdp/agent-micro/internal/gitutil"
|
||||
"github.com/sdp/agentlib/deployer"
|
||||
"github.com/sdp/agentlib/gitutil"
|
||||
"github.com/sdp/protocol"
|
||||
)
|
||||
|
||||
// ponytail: the gateway VM holds a single PHP project at the path below.
|
||||
// Real version reads a yaml config like agent-micro does.
|
||||
var repos = map[string]string{
|
||||
"api-gateway": "/home/user/SDP",
|
||||
}
|
||||
@@ -32,7 +34,7 @@ func main() {
|
||||
nodeID := flag.String("node", envOr("SDP_NODE_ID", "gateway"), "node id sent in WS query")
|
||||
flag.Parse()
|
||||
|
||||
cli, err := client.NewClientWithOpts(client.FromEnv)
|
||||
cli, err := docker.NewClientWithOpts(docker.FromEnv)
|
||||
if err != nil {
|
||||
log.Fatalf("docker client: %v", err)
|
||||
}
|
||||
@@ -66,9 +68,12 @@ func main() {
|
||||
|
||||
func dial(u *url.URL) (*websocket.Conn, error) {
|
||||
log.Printf("connecting to %s", u)
|
||||
return websocket.DefaultDialer.Dial(u.String(), nil)
|
||||
c, _, err := websocket.DefaultDialer.Dial(u.String(), nil)
|
||||
return c, err
|
||||
}
|
||||
|
||||
// writer pumps outbound events to whichever conn is current. If conn is
|
||||
// nil (during reconnect), messages buffer until the next conn is set.
|
||||
func writer(conn **websocket.Conn, mu *sync.Mutex, out <-chan []byte) {
|
||||
for msg := range out {
|
||||
mu.Lock()
|
||||
@@ -88,7 +93,7 @@ type runState struct {
|
||||
cancel context.CancelFunc
|
||||
}
|
||||
|
||||
func readLoop(c *websocket.Conn, cli *dockerclient.Client, out chan<- []byte, mu *sync.Mutex, connPtr **websocket.Conn) {
|
||||
func readLoop(c *websocket.Conn, cli *docker.Client, out chan<- []byte, mu *sync.Mutex, connPtr **websocket.Conn) {
|
||||
inflight := map[string]*runState{}
|
||||
for {
|
||||
_, raw, err := c.ReadMessage()
|
||||
@@ -123,7 +128,7 @@ func readLoop(c *websocket.Conn, cli *dockerclient.Client, out chan<- []byte, mu
|
||||
})
|
||||
continue
|
||||
}
|
||||
d := deployer.New(cli, frame.Data.DeploymentID,
|
||||
d := deployer.NewPHP(cli, frame.Data.DeploymentID,
|
||||
frame.Data.Repository, repoPath,
|
||||
frame.Data.Branch, frame.Data.Env,
|
||||
gitutil.Creds{Username: frame.Data.Username, Password: frame.Data.Password},
|
||||
@@ -159,6 +164,7 @@ func emit(out chan<- []byte, e protocol.Event) {
|
||||
select {
|
||||
case out <- b:
|
||||
default:
|
||||
// ponytail: drop on backpressure. Deploys are rare.
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+10
-3
@@ -1,8 +1,15 @@
|
||||
module github.com/sdp/agent-gateway
|
||||
|
||||
go 1.23
|
||||
go 1.24
|
||||
|
||||
require (
|
||||
github.com/docker/docker/client v0.0.0-00010101000000-000000000000
|
||||
github.com/sdp/protocol v0.0.0
|
||||
github.com/gorilla/websocket v1.5.1
|
||||
github.com/moby/moby/client v0.5.0
|
||||
github.com/sdp/agentlib v0.0.0-00010101000000-000000000000
|
||||
github.com/sdp/protocol v0.0.0-00010101000000-000000000000
|
||||
)
|
||||
|
||||
replace (
|
||||
github.com/sdp/agentlib => ../agentlib
|
||||
github.com/sdp/protocol => ../protocol
|
||||
)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
// Command agent-micro runs on the microservices VM (172.18.136.92). It
|
||||
// maintains a WebSocket to the control plane, accepts deploy/stop commands,
|
||||
// and runs the build+container pipeline locally.
|
||||
// maintains a WebSocket to the control plane, accepts deploy/stop frames,
|
||||
// and runs the build+container pipeline locally for Go microservices.
|
||||
package main
|
||||
|
||||
import (
|
||||
@@ -13,17 +13,17 @@ import (
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/docker/docker/client"
|
||||
docker "github.com/moby/moby/client"
|
||||
"github.com/gorilla/websocket"
|
||||
|
||||
"github.com/sdp/agent-micro/internal/deployer"
|
||||
"github.com/sdp/agent-micro/internal/gitutil"
|
||||
"github.com/sdp/agentlib/deployer"
|
||||
"github.com/sdp/agentlib/gitutil"
|
||||
"github.com/sdp/protocol"
|
||||
)
|
||||
|
||||
// ponytail: hand-curated from REQUIREMENTS.md. Real version reads a yaml
|
||||
// config file. Adding a new service = one line.
|
||||
var repos = map[string]string{
|
||||
// ponytail: hand-curated from REQUIREMENTS.md. Real version reads a yaml
|
||||
// config file. Adding a new service = one line.
|
||||
"account": "/home/user/AppGolang/account",
|
||||
"payment": "/home/user/AppGolang/payment",
|
||||
"user": "/home/user/AppGolang/user",
|
||||
@@ -35,7 +35,7 @@ func main() {
|
||||
nodeID := flag.String("node", envOr("SDP_NODE_ID", "micro"), "node id sent in WS query")
|
||||
flag.Parse()
|
||||
|
||||
cli, err := client.NewClientWithOpts(client.FromEnv)
|
||||
cli, err := docker.NewClientWithOpts(docker.FromEnv)
|
||||
if err != nil {
|
||||
log.Fatalf("docker client: %v", err)
|
||||
}
|
||||
@@ -69,7 +69,8 @@ func main() {
|
||||
|
||||
func dial(u *url.URL) (*websocket.Conn, error) {
|
||||
log.Printf("connecting to %s", u)
|
||||
return websocket.DefaultDialer.Dial(u.String(), nil)
|
||||
c, _, err := websocket.DefaultDialer.Dial(u.String(), nil)
|
||||
return c, err
|
||||
}
|
||||
|
||||
// writer pumps outbound events to whichever conn is current. If conn is
|
||||
@@ -93,7 +94,7 @@ type runState struct {
|
||||
cancel context.CancelFunc
|
||||
}
|
||||
|
||||
func readLoop(c *websocket.Conn, cli *dockerclient.Client, out chan<- []byte, mu *sync.Mutex, connPtr **websocket.Conn) {
|
||||
func readLoop(c *websocket.Conn, cli *docker.Client, out chan<- []byte, mu *sync.Mutex, connPtr **websocket.Conn) {
|
||||
inflight := map[string]*runState{}
|
||||
for {
|
||||
_, raw, err := c.ReadMessage()
|
||||
@@ -107,6 +108,7 @@ func readLoop(c *websocket.Conn, cli *dockerclient.Client, out chan<- []byte, mu
|
||||
_ = c.Close()
|
||||
return
|
||||
}
|
||||
// Inbound frame: {op, data, id}. Op is the verb. data is op-specific.
|
||||
var frame struct {
|
||||
Op string `json:"op"`
|
||||
Data protocol.DeployRequest `json:"data"`
|
||||
@@ -128,7 +130,7 @@ func readLoop(c *websocket.Conn, cli *dockerclient.Client, out chan<- []byte, mu
|
||||
})
|
||||
continue
|
||||
}
|
||||
d := deployer.New(cli, frame.Data.DeploymentID,
|
||||
d := deployer.NewGo(cli, frame.Data.DeploymentID,
|
||||
frame.Data.Repository, repoPath,
|
||||
frame.Data.Branch, frame.Data.Env,
|
||||
gitutil.Creds{Username: frame.Data.Username, Password: frame.Data.Password},
|
||||
|
||||
+10
-4
@@ -1,9 +1,15 @@
|
||||
module github.com/sdp/agent-micro
|
||||
|
||||
go 1.23
|
||||
go 1.24
|
||||
|
||||
require (
|
||||
github.com/docker/docker/client v0.0.0-00010101000000-000000000000
|
||||
github.com/docker/go-connections v0.5.0
|
||||
github.com/sdp/protocol v0.0.0
|
||||
github.com/gorilla/websocket v1.5.1
|
||||
github.com/moby/moby/client v0.5.0
|
||||
github.com/sdp/agentlib v0.0.0-00010101000000-000000000000
|
||||
github.com/sdp/protocol v0.0.0-00010101000000-000000000000
|
||||
)
|
||||
|
||||
replace (
|
||||
github.com/sdp/agentlib => ../agentlib
|
||||
github.com/sdp/protocol => ../protocol
|
||||
)
|
||||
|
||||
@@ -1,204 +0,0 @@
|
||||
// Package deployer is the per-deployment state machine. One Deployer is
|
||||
// created per DeployRequest; it runs the stages sequentially and emits
|
||||
// events on its output channel. The caller (the agent) reads events and
|
||||
// fans them out to the control plane.
|
||||
//
|
||||
// Runtime model (per REQUIREMENTS.md):
|
||||
// 1. git fetch / checkout / pull on the host
|
||||
// 2. go build -o <bin> on the host (the VM has the Go toolchain)
|
||||
// 3. docker run alpine:3.20 with -v <repo>:/src and command /src/<bin>
|
||||
// — NO Dockerfile, NO image build step. The alpine image must be
|
||||
// loaded on the host ahead of time (manual docker load).
|
||||
// 4. environment variables from the Environment config are passed via
|
||||
// -e flags into the container.
|
||||
package deployer
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/docker/docker/api/types/container"
|
||||
dockerclient "github.com/docker/docker/client"
|
||||
"github.com/docker/go-connections/nat"
|
||||
|
||||
"github.com/sdp/agent-micro/internal/gitutil"
|
||||
"github.com/sdp/protocol"
|
||||
)
|
||||
|
||||
type Deployer struct {
|
||||
ID string
|
||||
Repository string
|
||||
RepoPath string
|
||||
Branch string
|
||||
Env map[string]string
|
||||
Creds gitutil.Creds
|
||||
cli *dockerclient.Client
|
||||
binName string
|
||||
imageTag string
|
||||
containerID string
|
||||
}
|
||||
|
||||
func New(cli *dockerclient.Client, id, repo, path, branch string, env map[string]string, c gitutil.Creds) *Deployer {
|
||||
return &Deployer{
|
||||
ID: id,
|
||||
Repository: repo,
|
||||
RepoPath: path,
|
||||
Branch: branch,
|
||||
Env: env,
|
||||
Creds: c,
|
||||
cli: cli,
|
||||
binName: "app-" + repo,
|
||||
imageTag: "alpine:3.20", // ponytail: pre-loaded on the VM. Swap to a configurable tag.
|
||||
}
|
||||
}
|
||||
|
||||
// Run executes the full pipeline. It pushes events to out and returns the
|
||||
// final state (RUNNING on success, FAILED otherwise).
|
||||
func (d *Deployer) Run(ctx context.Context, out chan<- protocol.Event) string {
|
||||
emit := func(kind, state, stage, line string) {
|
||||
out <- protocol.Event{
|
||||
DeploymentID: d.ID,
|
||||
Kind: kind,
|
||||
State: state,
|
||||
Stage: stage,
|
||||
Line: line,
|
||||
At: time.Now().UnixMilli(),
|
||||
}
|
||||
}
|
||||
|
||||
stages := []struct {
|
||||
name string
|
||||
fn func() error
|
||||
}{
|
||||
{"git fetch", func() error { _, err := gitutil.Fetch(ctx, d.RepoPath, d.Creds); return err }},
|
||||
{"git checkout", func() error { _, err := gitutil.Checkout(ctx, d.RepoPath, d.Branch, d.Creds); return err }},
|
||||
{"git pull", func() error { _, err := gitutil.Pull(ctx, d.RepoPath, d.Creds); return err }},
|
||||
{"go build", d.build},
|
||||
{"start container", d.startContainer},
|
||||
}
|
||||
for _, s := range stages {
|
||||
emit("progress", "", s.name, "starting "+s.name)
|
||||
if err := s.fn(); err != nil {
|
||||
emit("progress", "FAILED", s.name, err.Error())
|
||||
emit("status", "FAILED", "", "")
|
||||
return "FAILED"
|
||||
}
|
||||
emit("progress", "", s.name, "ok")
|
||||
}
|
||||
emit("status", "RUNNING", "", "")
|
||||
return "RUNNING"
|
||||
}
|
||||
|
||||
func (d *Deployer) build(ctx context.Context) error {
|
||||
binPath := filepath.Join(d.RepoPath, d.binName)
|
||||
cmd := exec.CommandContext(ctx, "go", "build", "-o", binPath, "./...")
|
||||
cmd.Dir = d.RepoPath
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
return fmt.Errorf("go build: %w: %s", err, out)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// startContainer runs `alpine:3.20` with the host repo dir bind-mounted at
|
||||
// /src and the binary executed as the container command. The container's
|
||||
// working dir is set to the bind mount so the binary finds its config files.
|
||||
func (d *Deployer) startContainer(ctx context.Context) error {
|
||||
|
||||
envList := make([]string, 0, len(d.Env))
|
||||
for k, v := range d.Env {
|
||||
envList = append(envList, k+"="+v)
|
||||
}
|
||||
|
||||
cfg := &container.Config{
|
||||
Image: d.imageTag,
|
||||
Cmd: []string{"/src/" + d.binName},
|
||||
Env: envList,
|
||||
WorkingDir: "/src",
|
||||
ExposedPorts: nat.PortSet{
|
||||
"8080/tcp": struct{}{},
|
||||
},
|
||||
}
|
||||
hostCfg := &container.HostConfig{
|
||||
Binds: []string{d.RepoPath + ":/src"},
|
||||
RestartPolicy: container.RestartPolicy{Name: "unless-stopped"},
|
||||
}
|
||||
|
||||
resp, err := d.cli.ContainerCreate(ctx, cfg, hostCfg, nil, nil, d.containerName())
|
||||
if err != nil {
|
||||
return fmt.Errorf("container create: %w", err)
|
||||
}
|
||||
d.containerID = resp.ID
|
||||
if err := d.cli.ContainerStart(ctx, resp.ID, container.StartOptions{}); err != nil {
|
||||
return fmt.Errorf("container start: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Deployer) containerName() string {
|
||||
return "sdp-" + d.Repository + "-" + d.ID
|
||||
}
|
||||
|
||||
func (d *Deployer) Stop(ctx context.Context) error {
|
||||
if d.containerID == "" {
|
||||
return nil
|
||||
}
|
||||
return d.cli.ContainerStop(ctx, d.containerID, container.StopOptions{})
|
||||
}
|
||||
|
||||
// StreamLogs tails the container's logs into the events channel until ctx
|
||||
// is cancelled.
|
||||
func (d *Deployer) StreamLogs(ctx context.Context, out chan<- protocol.Event) {
|
||||
if d.containerID == "" {
|
||||
return
|
||||
}
|
||||
rc, err := d.cli.ContainerLogs(ctx, d.containerID, container.LogsOptions{
|
||||
ShowStdout: true,
|
||||
ShowStderr: true,
|
||||
Follow: true,
|
||||
Timestamps: false,
|
||||
})
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
defer rc.Close()
|
||||
// ponytail: Docker multiplexes stdout/stderr; the leading 8 bytes per
|
||||
// frame are the stream header. For MVP we just split on newlines and
|
||||
// don't try to demux.
|
||||
buf := make([]byte, 4096)
|
||||
var carry strings.Builder
|
||||
for {
|
||||
n, err := rc.Read(buf)
|
||||
if n > 0 {
|
||||
carry.Write(buf[:n])
|
||||
for {
|
||||
s := carry.String()
|
||||
i := strings.IndexByte(s, '\n')
|
||||
if i < 0 {
|
||||
break
|
||||
}
|
||||
line := s[:i]
|
||||
carry.Reset()
|
||||
carry.WriteString(s[i+1:])
|
||||
if len(line) > 8 && line[0] <= 2 {
|
||||
line = line[8:]
|
||||
}
|
||||
if line != "" {
|
||||
out <- protocol.Event{
|
||||
DeploymentID: d.ID,
|
||||
Kind: "log",
|
||||
Line: line,
|
||||
At: time.Now().UnixMilli(),
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,322 @@
|
||||
// Package deployer is the per-deployment state machine. One Deployer is
|
||||
// created per DeployRequest; it runs the stages sequentially and emits
|
||||
// events on its output channel. The caller (the agent) reads events and
|
||||
// fans them out to the control plane.
|
||||
//
|
||||
// Two build flavours share the same lifecycle:
|
||||
//
|
||||
// GoDeployer (microservices) — host `go build`, run alpine:3.20 with the
|
||||
// repo bind-mounted at /src and the binary as the container command.
|
||||
//
|
||||
// PhpDeployer (api-gateway) — host `git pull` only; the gateway's PHP
|
||||
// image (php:8.3-apache) is pre-loaded on the VM and the repo is
|
||||
// bind-mounted at /app. Best-effort `composer install --no-dev` runs
|
||||
// on the host if a composer.json is present.
|
||||
//
|
||||
// In both cases there is NO Dockerfile-based image build. The runtime image
|
||||
// must be loaded on the host ahead of time (manual `docker load`).
|
||||
package deployer
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
containertypes "github.com/moby/moby/api/types/container"
|
||||
networktypes "github.com/moby/moby/api/types/network"
|
||||
dockerclient "github.com/moby/moby/client"
|
||||
|
||||
"github.com/sdp/agentlib/gitutil"
|
||||
"github.com/sdp/protocol"
|
||||
)
|
||||
|
||||
// Kind selects between build flavours. The control plane doesn't need to
|
||||
// know — it just sends a DeployRequest with a Repository name. The agent
|
||||
// picks the right Deployer based on which binary it's running as.
|
||||
type Kind string
|
||||
|
||||
const (
|
||||
KindGo Kind = "go" // microservice: host go build
|
||||
KindPHP Kind = "php" // api-gateway: host composer install (best-effort)
|
||||
)
|
||||
|
||||
// Spec captures the kind-specific bits of a Deployer that differ between
|
||||
// Go and PHP. The constructor functions below fill these in.
|
||||
type Spec struct {
|
||||
ImageTag string
|
||||
BindMount string // host dir bind-mounted into the container
|
||||
Workdir string // container working dir
|
||||
Cmd []string
|
||||
ExposedPorts networktypes.PortSet
|
||||
}
|
||||
|
||||
type Deployer struct {
|
||||
kind Kind
|
||||
ID string
|
||||
Repository string
|
||||
RepoPath string
|
||||
Branch string
|
||||
Env map[string]string
|
||||
Creds gitutil.Creds
|
||||
cli *dockerclient.Client
|
||||
containerID string
|
||||
spec Spec
|
||||
}
|
||||
|
||||
func newDeployer(kind Kind, cli *dockerclient.Client, id, repo, repoPath, branch string, env map[string]string, c gitutil.Creds, spec Spec) *Deployer {
|
||||
return &Deployer{
|
||||
kind: kind,
|
||||
ID: id,
|
||||
Repository: repo,
|
||||
RepoPath: repoPath,
|
||||
Branch: branch,
|
||||
Env: env,
|
||||
Creds: c,
|
||||
cli: cli,
|
||||
spec: spec,
|
||||
}
|
||||
}
|
||||
|
||||
// portKey builds a network.Port from a "port/proto" string like "8080/tcp".
|
||||
func portKey(s string) (networktypes.Port, error) {
|
||||
return networktypes.ParsePort(s)
|
||||
}
|
||||
|
||||
// portSet is a small helper to make a PortSet from "port/proto" literals.
|
||||
func portSet(ports ...string) networktypes.PortSet {
|
||||
out := networktypes.PortSet{}
|
||||
for _, s := range ports {
|
||||
p, err := portKey(s)
|
||||
if err == nil {
|
||||
out[p] = struct{}{}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// NewGo is the microservice deployer: host `go build`, run alpine:3.20.
|
||||
func NewGo(cli *dockerclient.Client, id, repo, repoPath, branch string, env map[string]string, c gitutil.Creds) *Deployer {
|
||||
binName := "app-" + repo
|
||||
return newDeployer(KindGo, cli, id, repo, repoPath, branch, env, c, Spec{
|
||||
ImageTag: "alpine:3.20",
|
||||
BindMount: "/src",
|
||||
Workdir: "/src",
|
||||
Cmd: []string{"/src/" + binName},
|
||||
ExposedPorts: portSet("8080/tcp"),
|
||||
})
|
||||
}
|
||||
|
||||
// NewPHP is the api-gateway deployer: best-effort composer install on
|
||||
// the host, then run php:8.3-apache with the repo bind-mounted at /app.
|
||||
//
|
||||
// The PHP image must be pre-loaded on the VM via `docker load`. The repo
|
||||
// is expected to be a standard PHP project layout (public/ entrypoint
|
||||
// is conventional but not required).
|
||||
func NewPHP(cli *dockerclient.Client, id, repo, repoPath, branch string, env map[string]string, c gitutil.Creds) *Deployer {
|
||||
return newDeployer(KindPHP, cli, id, repo, repoPath, branch, env, c, Spec{
|
||||
ImageTag: "php:8.3-apache",
|
||||
BindMount: "/app",
|
||||
Workdir: "/app",
|
||||
// Apache in the official php image reads /app/public as the
|
||||
// document root by default. If the gateway has a different
|
||||
// layout the env var APACHE_DOCUMENT_ROOT can override.
|
||||
Cmd: []string{"apache2-foreground"},
|
||||
ExposedPorts: portSet("80/tcp"),
|
||||
})
|
||||
}
|
||||
|
||||
// Run executes the kind-specific pipeline and returns the final state
|
||||
// ("RUNNING" on success, "FAILED" otherwise). All events are pushed to
|
||||
// out; the caller is expected to drain.
|
||||
func (d *Deployer) Run(ctx context.Context, out chan<- protocol.Event) string {
|
||||
emit := func(kind, state, stage, line string) {
|
||||
out <- protocol.Event{
|
||||
DeploymentID: d.ID,
|
||||
Kind: kind,
|
||||
State: state,
|
||||
Stage: stage,
|
||||
Line: line,
|
||||
At: time.Now().UnixMilli(),
|
||||
}
|
||||
}
|
||||
|
||||
stages := []struct {
|
||||
name string
|
||||
fn func() error
|
||||
}{
|
||||
{"git fetch", func() error { _, err := gitutil.Fetch(ctx, d.RepoPath, d.Creds); return err }},
|
||||
{"git checkout", func() error { _, err := gitutil.Checkout(ctx, d.RepoPath, d.Branch, d.Creds); return err }},
|
||||
{"git pull", func() error { _, err := gitutil.Pull(ctx, d.RepoPath, d.Creds); return err }},
|
||||
}
|
||||
if d.kind == KindGo {
|
||||
stages = append(stages, struct {
|
||||
name string
|
||||
fn func() error
|
||||
}{"go build", func() error { return d.buildGo(ctx) }})
|
||||
} else {
|
||||
stages = append(stages, struct {
|
||||
name string
|
||||
fn func() error
|
||||
}{"composer install", func() error { return d.composerInstall(ctx) }})
|
||||
}
|
||||
stages = append(stages, struct {
|
||||
name string
|
||||
fn func() error
|
||||
}{"start container", func() error { return d.startContainer(ctx) }})
|
||||
|
||||
for _, s := range stages {
|
||||
emit("progress", "", s.name, "starting "+s.name)
|
||||
if err := s.fn(); err != nil {
|
||||
emit("progress", "FAILED", s.name, err.Error())
|
||||
emit("status", "FAILED", "", "")
|
||||
return "FAILED"
|
||||
}
|
||||
emit("progress", "", s.name, "ok")
|
||||
}
|
||||
emit("status", "RUNNING", "", "")
|
||||
return "RUNNING"
|
||||
}
|
||||
|
||||
func (d *Deployer) buildGo(ctx context.Context) error {
|
||||
binName := "app-" + d.Repository
|
||||
binPath := filepath.Join(d.RepoPath, binName)
|
||||
cmd := exec.CommandContext(ctx, "go", "build", "-o", binPath, "./...")
|
||||
cmd.Dir = d.RepoPath
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
return fmt.Errorf("go build: %w: %s", err, out)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// composerInstall runs `composer install --no-dev` on the host if a
|
||||
// composer.json is present. Best-effort: if composer isn't installed
|
||||
// or fails, we log a warning and continue — the gateway's PHP image
|
||||
// typically has its own runtime autoloader or preinstalled deps.
|
||||
//
|
||||
// Per REQUIREMENTS.md: the API Gateway must run inside Docker; we
|
||||
// don't try to launch a PHP dev server or anything fancy here.
|
||||
func (d *Deployer) composerInstall(_ context.Context) error {
|
||||
manifest := filepath.Join(d.RepoPath, "composer.json")
|
||||
if _, err := exec.LookPath("composer"); err != nil {
|
||||
return nil // no composer on the VM; skip silently
|
||||
}
|
||||
if _, err := os.Stat(manifest); err != nil {
|
||||
return nil // no composer.json; nothing to install
|
||||
}
|
||||
cmd := exec.Command("composer", "install", "--no-dev", "--no-interaction", "--no-progress")
|
||||
cmd.Dir = d.RepoPath
|
||||
// Best-effort: even if composer fails, don't fail the deploy.
|
||||
_, _ = cmd.CombinedOutput()
|
||||
return nil
|
||||
}
|
||||
|
||||
// startContainer runs the configured base image with the repo
|
||||
// bind-mounted at d.spec.BindMount. The container's working dir is
|
||||
// set to d.spec.Workdir so the entrypoint finds its config / static
|
||||
// files.
|
||||
func (d *Deployer) startContainer(ctx context.Context) error {
|
||||
envList := make([]string, 0, len(d.Env))
|
||||
for k, v := range d.Env {
|
||||
envList = append(envList, k+"="+v)
|
||||
}
|
||||
|
||||
cfg := &containertypes.Config{
|
||||
Image: d.spec.ImageTag,
|
||||
Cmd: d.spec.Cmd,
|
||||
Env: envList,
|
||||
WorkingDir: d.spec.Workdir,
|
||||
ExposedPorts: d.spec.ExposedPorts,
|
||||
}
|
||||
hostCfg := &containertypes.HostConfig{
|
||||
Binds: []string{d.RepoPath + ":" + d.spec.BindMount},
|
||||
RestartPolicy: containertypes.RestartPolicy{Name: containertypes.RestartPolicyUnlessStopped},
|
||||
}
|
||||
|
||||
resp, err := d.cli.ContainerCreate(ctx, dockerclient.ContainerCreateOptions{
|
||||
Config: cfg,
|
||||
HostConfig: hostCfg,
|
||||
Name: d.containerName(),
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("container create: %w", err)
|
||||
}
|
||||
d.containerID = resp.ID
|
||||
if _, err := d.cli.ContainerStart(ctx, resp.ID, dockerclient.ContainerStartOptions{}); err != nil {
|
||||
return fmt.Errorf("container start: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Deployer) containerName() string {
|
||||
return "sdp-" + d.Repository + "-" + d.ID
|
||||
}
|
||||
|
||||
func (d *Deployer) Stop(ctx context.Context) error {
|
||||
if d.containerID == "" {
|
||||
return nil
|
||||
}
|
||||
_, err := d.cli.ContainerStop(ctx, d.containerID, dockerclient.ContainerStopOptions{})
|
||||
return err
|
||||
}
|
||||
|
||||
// StreamLogs tails the container's logs into the events channel until ctx
|
||||
// is cancelled.
|
||||
func (d *Deployer) StreamLogs(ctx context.Context, out chan<- protocol.Event) {
|
||||
if d.containerID == "" {
|
||||
return
|
||||
}
|
||||
rc, err := d.cli.ContainerLogs(ctx, d.containerID, dockerclient.ContainerLogsOptions{
|
||||
ShowStdout: true,
|
||||
ShowStderr: true,
|
||||
Follow: true,
|
||||
Timestamps: false,
|
||||
})
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
defer rc.Close()
|
||||
// ponytail: Docker multiplexes stdout/stderr; the leading 8 bytes per
|
||||
// frame are the stream header. For MVP we just split on newlines and
|
||||
// don't try to demux.
|
||||
buf := make([]byte, 4096)
|
||||
var carry strings.Builder
|
||||
for {
|
||||
n, err := rc.Read(buf)
|
||||
if n > 0 {
|
||||
carry.Write(buf[:n])
|
||||
for {
|
||||
s := carry.String()
|
||||
i := strings.IndexByte(s, '\n')
|
||||
if i < 0 {
|
||||
break
|
||||
}
|
||||
line := s[:i]
|
||||
carry.Reset()
|
||||
carry.WriteString(s[i+1:])
|
||||
if len(line) > 8 && line[0] <= 2 {
|
||||
line = line[8:]
|
||||
}
|
||||
if line != "" {
|
||||
out <- protocol.Event{
|
||||
DeploymentID: d.ID,
|
||||
Kind: "log",
|
||||
Line: line,
|
||||
At: time.Now().UnixMilli(),
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
if err != io.EOF {
|
||||
// log nothing; the agent's writer will drop on disconnect
|
||||
}
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -40,6 +40,21 @@ func Probe(ctx context.Context, repoDir string, c Creds) error {
|
||||
return err
|
||||
}
|
||||
|
||||
// ListBranches lists local branches. Cheap; no network.
|
||||
func ListBranches(ctx context.Context, repoDir string) ([]string, error) {
|
||||
cmd := exec.CommandContext(ctx, "git", "for-each-ref", "--format=%(refname:short)", "refs/heads/")
|
||||
cmd.Dir = repoDir
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("git for-each-ref: %w: %s", err, out)
|
||||
}
|
||||
s := strings.TrimSpace(string(out))
|
||||
if s == "" {
|
||||
return nil, nil
|
||||
}
|
||||
return strings.Split(s, "\n"), nil
|
||||
}
|
||||
|
||||
func runGit(ctx context.Context, repoDir string, c Creds, args ...string) (string, error) {
|
||||
cmd := exec.CommandContext(ctx, "git", args...)
|
||||
cmd.Dir = repoDir
|
||||
@@ -0,0 +1,13 @@
|
||||
module github.com/sdp/agentlib
|
||||
|
||||
go 1.24
|
||||
|
||||
require (
|
||||
github.com/docker/go-connections v0.7.0
|
||||
github.com/gorilla/websocket v1.5.1
|
||||
github.com/moby/moby/api v1.55.0
|
||||
github.com/moby/moby/client v0.5.0
|
||||
github.com/sdp/protocol v0.0.0-00010101000000-000000000000
|
||||
)
|
||||
|
||||
replace github.com/sdp/protocol => ../protocol
|
||||
@@ -1,8 +1,11 @@
|
||||
module github.com/sdp/control-plane
|
||||
|
||||
go 1.23
|
||||
go 1.24
|
||||
|
||||
require (
|
||||
github.com/gorilla/websocket v1.5.1
|
||||
github.com/sdp/protocol v0.0.0-00010101000000-000000000000
|
||||
modernc.org/sqlite v1.28.0
|
||||
)
|
||||
|
||||
replace github.com/sdp/protocol => ../protocol
|
||||
|
||||
@@ -30,7 +30,7 @@ func Open(dir string) (*Store, error) {
|
||||
if err := os.MkdirAll(filepath.Join(dir, "logs"), 0o755); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
db, err := sql.Open("sqlite3", filepath.Join(dir, "sdp.db"))
|
||||
db, err := sql.Open("sqlite", filepath.Join(dir, "sdp.db"))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
@@ -6,6 +6,8 @@ import (
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/gorilla/websocket"
|
||||
|
||||
"github.com/sdp/protocol"
|
||||
|
||||
"github.com/sdp/control-plane/internal/store"
|
||||
|
||||
Vendored
+5
@@ -0,0 +1,5 @@
|
||||
/// <reference types="next" />
|
||||
/// <reference types="next/image-types/global" />
|
||||
|
||||
// NOTE: This file should not be edited
|
||||
// see https://nextjs.org/docs/basic-features/typescript for more information.
|
||||
Generated
+2501
File diff suppressed because it is too large
Load Diff
@@ -1,7 +1,8 @@
|
||||
go 1.23
|
||||
go 1.24
|
||||
|
||||
use (
|
||||
./protocol
|
||||
./agentlib
|
||||
./control-plane
|
||||
./agent-micro
|
||||
./agent-gateway
|
||||
|
||||
+56
@@ -1,3 +1,59 @@
|
||||
github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI=
|
||||
github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M=
|
||||
github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE=
|
||||
github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmCQvQ4GpJHEqRLVk=
|
||||
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
|
||||
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
|
||||
github.com/docker/go-connections v0.5.0/go.mod h1:ov60Kzw0kKElRwhNs9UlUHAE/F9Fe6GLaXnqyDdmEXc=
|
||||
github.com/docker/go-connections v0.7.0 h1:6SsRfJddP22WMrCkj19x9WKjEDTB+ahsdiGYf0mN39c=
|
||||
github.com/docker/go-connections v0.7.0/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
|
||||
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
|
||||
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
|
||||
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
|
||||
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
|
||||
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
|
||||
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
|
||||
github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
|
||||
github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
||||
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
|
||||
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
|
||||
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
|
||||
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/gorilla/websocket v1.5.1 h1:gmztn0JnHVt9JZquRuzLw3g4wouNVzKL15iLr/zn/QY=
|
||||
github.com/gorilla/websocket v1.5.1/go.mod h1:x3kM2JMyaluk02fnUJpQuwD2dCS5NDG2ZHL0uE0tcaY=
|
||||
github.com/mattn/go-isatty v0.0.16 h1:bq3VjFmv/sOjHtdEhmkEV4x1AJtvUvOJ2PFAZ5+peKQ=
|
||||
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
|
||||
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
|
||||
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
|
||||
github.com/moby/moby/api v1.55.0 h1:2/sexvQyqIWS8pRSCFddBfpW2qE7vR7FCL+vN8pxwMc=
|
||||
github.com/moby/moby/api v1.55.0/go.mod h1:+RQ6wluLwtYaTd1WnPLykIDPekkuyD/ROWQClE83pzs=
|
||||
github.com/moby/moby/client v0.5.0 h1:5XhyPk2fuOWf6RlSFa3MkIIgDZkF25xToXW8Q/BH7cc=
|
||||
github.com/moby/moby/client v0.5.0/go.mod h1:rcVpF8ncl9vo5gaIBdol6CnbEtSj1uxMvEV/UrykF/s=
|
||||
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
|
||||
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
|
||||
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
|
||||
github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
|
||||
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
|
||||
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
|
||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0 h1:sbiXRNDSWJOTobXh5HyQKjq6wUC5tNybqjIqDpAY4CU=
|
||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0/go.mod h1:69uWxva0WgAA/4bu2Yy70SLDBwZXuQ6PbBpbsa5iZrQ=
|
||||
go.opentelemetry.io/otel v1.35.0 h1:xKWKPxrxB6OtMCbmMY021CqC45J+3Onta9MqjhnusiQ=
|
||||
go.opentelemetry.io/otel v1.35.0/go.mod h1:UEqy8Zp11hpkUrL73gSlELM0DupHoiq72dR+Zqel/+Y=
|
||||
go.opentelemetry.io/otel/metric v1.35.0 h1:0znxYu2SNyuMSQT4Y9WDWej0VpcsxkuklLa4/siN90M=
|
||||
go.opentelemetry.io/otel/metric v1.35.0/go.mod h1:nKVFgxBZ2fReX6IlyW28MgZojkoAkJGaE8CpgeAU3oE=
|
||||
go.opentelemetry.io/otel/trace v1.35.0 h1:dPpEfJu1sDIqruz7BHFG3c7528f6ddfSWfFDVt/xgMs=
|
||||
go.opentelemetry.io/otel/trace v1.35.0/go.mod h1:WUk7DtFp1Aw2MkvqGdwiXYDZZNvA/1J8o6xRXLrIkyc=
|
||||
golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM=
|
||||
golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
|
||||
golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw=
|
||||
golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
|
||||
modernc.org/libc v1.29.0 h1:tTFRFq69YKCF2QyGNuRUQxKBm1uZZLubf6Cjh/pVHXs=
|
||||
modernc.org/libc v1.29.0/go.mod h1:DaG/4Q3LRRdqpiLyP0C2m1B8ZMGkQ+cCgOIjEtQlYhQ=
|
||||
modernc.org/mathutil v1.6.0 h1:fRe9+AmYlaej+64JsEEhoWuAYBkOtQiMEU7n/XgfYi4=
|
||||
modernc.org/mathutil v1.6.0/go.mod h1:Ui5Q9q1TR2gFm0AQRqQUaBWFLAhQpCwNcuhBOSedWPo=
|
||||
modernc.org/memory v1.7.2 h1:Klh90S215mmH8c9gO98QxQFsY+W451E8AnzjoE2ee1E=
|
||||
modernc.org/memory v1.7.2/go.mod h1:NO4NVCQy0N7ln+T9ngWqOQfi7ley4vpwvARR+Hjw95E=
|
||||
modernc.org/sqlite v1.28.0 h1:Zx+LyDDmXczNnEQdvPuEfcFVA2ZPyaD7UCZDjef3BHQ=
|
||||
modernc.org/sqlite v1.28.0/go.mod h1:Qxpazz0zH8Z1xCFyi5GSL3FzbtZ3fvbjmywNogldEW0=
|
||||
|
||||
+11
-2
@@ -15,7 +15,7 @@ REPO_ROOT="$(pwd)"
|
||||
OUT="$REPO_ROOT/bin"
|
||||
mkdir -p "$OUT"
|
||||
|
||||
GO_IMAGE="${GO_IMAGE:-golang:1.23-alpine}"
|
||||
GO_IMAGE="${GO_IMAGE:-golang:1.24-alpine}"
|
||||
|
||||
# ponytail: bind-mount a persistent gocache so module downloads + build cache
|
||||
# survive across runs. Otherwise every build re-downloads the world from
|
||||
@@ -36,17 +36,26 @@ docker run --rm \
|
||||
"$GO_IMAGE" \
|
||||
sh -c '
|
||||
set -e
|
||||
# git is needed for module resolution: some deps (and go.work.sum)
|
||||
# may pull from VCS. alpine ships without it.
|
||||
apk add --no-cache git >/dev/null
|
||||
# The container runs as root and /src is bind-mounted from the host;
|
||||
# git refuses to operate on a tree it does not own ("dubious ownership").
|
||||
# We never commit inside the container — the host owns the working tree —
|
||||
# so disabling the check is safe.
|
||||
git config --global --add safe.directory /src
|
||||
# -trimpath: strip absolute paths from the binary (reproducible builds).
|
||||
# -ldflags="-s -w": drop symbol table + DWARF, smaller binary.
|
||||
go build -trimpath -ldflags="-s -w" -o /out/control-plane ./control-plane/cmd/control-plane
|
||||
go build -trimpath -ldflags="-s -w" -o /out/agent-micro ./agent-micro/cmd/agent-micro
|
||||
go build -trimpath -ldflags="-s -w" -o /out/agent-gateway ./agent-gateway/cmd/agent-gateway
|
||||
# go build does not set +x; do it here while we still own the files.
|
||||
chmod +x /out/control-plane /out/agent-micro /out/agent-gateway
|
||||
'
|
||||
|
||||
echo
|
||||
echo "==> binaries:"
|
||||
ls -lh "$OUT"
|
||||
chmod +x "$OUT"/*
|
||||
|
||||
# Verify the binaries are actually linux/amd64. ponytail: catches a mistake
|
||||
# where someone removes the GOOS/GOARCH env and ships a darwin binary to
|
||||
|
||||
Reference in New Issue
Block a user