commit 3d9994065814f68b958532ff3f0325ddf8067359 Author: Achmad Setyabudi Susilo Date: Wed Jun 24 07:25:01 2026 +0700 Initial SDP skeleton Sandbox Deployment Platform — Go control plane + agents, NextJS dashboard, nginx reverse proxy. Cross-compile via Docker; deploy via sshpass to 172.18.136.92 (micro) and 172.18.139.186 (gateway). - control-plane: HTTP API, WS hub, SQLite (modernc.org/sqlite) for progress, .log files for log persistence - agent-micro / agent-gateway: alpine:3.20 + bind-mounted repo, binary exec'd in container, no Dockerfile build step - dashboard: NextJS static export + shadcn/ui components, single WebSocket hook - docker-compose.yml: three services on alpine:latest with docker socket bind for agents - scripts/: build.sh (golang:1.23-alpine cross-compile), deploy.sh, patch-nginx.sh (idempotent nginx splice), ssh wrappers Runtime model: pass-through Bitbucket creds per deploy, never logged or persisted on the agent. Control plane never touches git or docker directly — agents do all the work locally. diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..e61005d --- /dev/null +++ b/.gitignore @@ -0,0 +1,38 @@ +# Built artifacts +bin/ +dashboard/out/ +dashboard/.next/ + +# Node +dashboard/node_modules/ +dashboard/npm-debug.log* +dashboard/yarn-error.log* +dashboard/.pnpm-debug.log* + +# Go +**/*.test +**/*.out +**/coverage.txt +gocache/ +*.prof + +# Local data +data/ +*.db +*.db-journal +*.db-wal +*.db-shm +logs/ + +# Editor / OS +.DS_Store +.idea/ +.vscode/ +*.swp +*.swo +*~ + +# Env +.env +.env.local +.env.*.local diff --git a/README.md b/README.md new file mode 100644 index 0000000..a44dff6 --- /dev/null +++ b/README.md @@ -0,0 +1,65 @@ +# Sandbox Deployment Platform (SDP) + +Internal deployment platform for Backend/QA. Lets a developer deploy a feature +branch into an isolated sandbox, with the API Gateway routing selected +services to the sandbox and the rest to OCP. See [REQUIREMENTS.md](REQUIREMENTS.md) +for the full spec. + +## Layout + +``` +. +├── protocol/ # shared wire types (Event, DeployRequest) +├── control-plane/ # Go. HTTP API + WS hub + SQLite/.log persistence +├── agent-micro/ # Go. Runs on 172.18.136.92, deploys Go microservices +├── agent-gateway/ # Go. Runs on 172.18.139.186, deploys the API Gateway +├── dashboard/ # NextJS static export, served by nginx +└── nginx/ # reverse proxy + try_files for the dashboard +``` + +## End-to-end smoke (manual) + +Prereqs: Go 1.22+, Node 18+, Docker on each agent VM, alpine:3.20 loaded +locally (`docker load -i alpine.tar`). + +1. Build everything: + ```bash + cd protocol && go build ./... + cd ../control-plane && go build -o bin/control-plane ./cmd/control-plane + cd ../agent-micro && go build -o bin/agent-micro ./cmd/agent-micro + cd ../agent-gateway && go build -o bin/agent-gateway ./cmd/agent-gateway + cd ../dashboard && npm install && npm run build + ``` + +2. Start the control plane: + ```bash + ./control-plane/bin/control-plane -addr :8080 -data ./data + ``` + +3. Start the micro agent on 172.18.136.92: + ```bash + SDP_CP_URL=ws://172.18.139.186:8080/ws/agent SDP_NODE_ID=micro \ + ./agent-micro/bin/agent-micro + ``` + +4. Start the gateway agent on 172.18.139.186: + ```bash + SDP_CP_URL=ws://172.18.139.186:8080/ws/agent SDP_NODE_ID=gateway \ + ./agent-gateway/bin/agent-gateway + ``` + +5. Point nginx at the dashboard build (`dashboard/out/`) and the control + plane (`:8080`). See `nginx/nginx.conf`. + +6. Open `http:///`, sign in with any Bitbucket creds, pick + `account` → `feature/login-error`, click Deploy. Watch the stage + checkmarks and the log stream. + +## Notes + +- Credentials are passed per-operation to the agent and never persisted + on the agent longer than the operation. +- The runtime model is `alpine:3.20` + bind-mounted repo + exec'd binary. + No Dockerfile build step on the agent. +- Logs persist to `/logs/.log`. SQLite holds progress + snapshots and final state. diff --git a/REQUIREMENTS.md b/REQUIREMENTS.md new file mode 100644 index 0000000..96da96c --- /dev/null +++ b/REQUIREMENTS.md @@ -0,0 +1,1404 @@ +# Sandbox Deployment Platform (SDP) + +## Tech Stack (Decided) + +- **Dashboard:** NextJS + React + TypeScript + Tailwind. Plain `useState` + single WebSocket hook. No Redux/Zustand. Built as static output, served by nginx with `try_files`. +- **Control Plane:** Go. PostgreSQL for metadata (nodes, repos, deployments, sandboxes, templates, routes, envs). **SQLite** for ephemeral state (deployment progress snapshots) and **`.log` files** for log persistence. No Spring Boot. No Redis. +- **Agents:** Go. Use the official Docker SDK (`github.com/docker/docker/client`) for container orchestration. Build Go binaries **directly on the host** (`go build -o {name}`) — no Dockerfile-based build step. +- **Realtime transport:** WebSocket end-to-end (Agent → Control Plane → Frontend). +- **Auth:** Bitbucket username/password. Validated by a real `git ls-remote`/`fetch` via the Agent. **Credentials are passed on every operation from Control Plane to Agent. Never logged, never persisted on the Agent longer than the operation.** +- **Infra in the spec** = the existing microservice infrastructure (172.18.* VMs, AppGolang, SDP repo), not infrastructure for SDP itself. + +## Overview + +Sandbox Deployment Platform (SDP) is an internal deployment platform that allows Backend and QA teams to deploy isolated feature branches without requiring deployment to the shared OpenShift (OCP) environment. + +The platform is designed specifically for the company's existing architecture: + +* Golang microservices +* PHP API Gateway +* Internal VM infrastructure +* Bitbucket repositories +* No internet access on deployment VMs +* Developers only have read access to OCP + +The platform is NOT intended to be a generic Kubernetes, OpenShift, or PaaS solution. + +--- + +# Problem Statement + +Current workflow: + +1. Developer creates feature branch. +2. Deployment to shared environment requires PR approval and merge. +3. CI/CD deploys to shared OCP. +4. Testing affects other teams. +5. Negative-path testing can disrupt shared development. + +Required workflow: + +1. Developer deploys feature branch directly. +2. Deployment occurs in isolated sandbox infrastructure. +3. API Gateway selectively routes traffic to sandbox services. +4. Remaining services continue using OCP. +5. QA can test independently. + +--- + +# Infrastructure + +## Microservices VM + +```text +IP Address: +172.18.136.92 + +Repository Root: +~/AppGolang +``` + +Example: + +```text +~/AppGolang +├── account +├── payment +├── user +├── notification +└── ... +``` + +All Golang microservices reside here. + +--- + +## Infrastructure VM + +```text +IP Address: +172.18.136.93 +``` + +Reserved for future use: + +* PostgreSQL +* Redis +* RabbitMQ +* Kafka + +Not required for MVP. + +--- + +## API Gateway VM + +```text +IP Address: +172.18.139.186 + +Repository Root: +~/SDP +``` + +Contains: + +```text +~/SDP +``` + +The API Gateway repository. + +--- + +# High-Level Architecture + +```text ++--------------------------+ +| Dashboard | +| NextJS Frontend | ++------------+-------------+ + | + v ++--------------------------+ +| Control Plane | +| Spring Boot | ++------+------------+------+ + | | + | HTTP | HTTP + | | + v v + ++-------------+ +-------------+ +| Micro Agent | | Gateway | +| 172.18.136.92 | | Agent | +| | | 172.18.139.186 | ++-------------+ +-------------+ +``` + +--- + +# Architectural Principles + +## Control Plane + +The Control Plane: + +* Never SSHs into servers +* Never executes build commands +* Never accesses repositories directly + +The Control Plane only: + +* Stores metadata +* Manages deployments +* Sends commands via HTTP +* Receives deployment events +* Streams logs to frontend + +--- + +## Agents + +Agents execute all operations locally. + +Examples: + +```text +git fetch +git checkout +go build +docker build +docker run +``` + +Agents have direct filesystem access. + +--- + +# Authentication + +## Login + +Users authenticate using: + +```text +Bitbucket Username +Bitbucket Password +``` + +--- + +## Validation + +Authentication is validated by attempting a Git operation against a known repository. + +Example: + +```bash +git ls-remote +``` + +or + +```bash +git fetch +``` + +If Git authentication succeeds: + +```text +LOGIN SUCCESS +``` + +Otherwise: + +```text +LOGIN FAILED +``` + +--- + +## Git Operations + +All Git operations must use the currently authenticated user's credentials. + +Examples: + +```bash +git fetch +git pull +git checkout +``` + +Credentials are passed from Control Plane to Agent during deployment execution. + +Credentials must never be logged. + +--- + +# Repository Configuration + +Repositories are configured manually on each Agent. + +No automatic discovery. + +Example: + +```yaml +repositories: + - name: account + path: /home/user/AppGolang/account + + - name: payment + path: /home/user/AppGolang/payment + + - name: user + path: /home/user/AppGolang/user +``` + +Gateway: + +```yaml +repositories: + - name: api-gateway + path: /home/user/SDP +``` + +--- + +# Core Concepts + +## Node + +Represents a VM. + +Fields: + +```text +id +name +ipAddress +type +``` + +Types: + +```text +MICRO +GATEWAY +INFRA +``` + +--- + +## Repository + +Fields: + +```text +id +name +path +nodeId +``` + +--- + +## Environment + +Equivalent to: + +```text +ConfigMap +Secret +``` + +Contains: + +```text +Variables +Secrets +Files +``` + +Example: + +```env +DB_HOST= +DB_PORT= +REDIS_URL= +JWT_SECRET= +``` + +--- + +## Deployment + +Represents a deployment execution. + +Fields: + +```text +id +repository +branch +user +status +logs +startedAt +completedAt +``` + +--- + +## Sandbox + +Represents an isolated testing environment. + +Example: + +```yaml +sandbox: + QA-LOGIN-ERROR + +services: + account: + branch: feature/login-error + + payment: + use_ocp: true + + user: + use_ocp: true +``` + +--- + +## Sandbox Template + +A reusable sandbox configuration. + +Purpose: + +Reduce repetitive setup. + +Example: + +```yaml +template: + QA-DEFAULT + +gateway: + branch: develop + +services: + account: + use_ocp: true + + payment: + use_ocp: true + + user: + use_ocp: true +``` + +Another example: + +```yaml +template: + ACCOUNT-TESTING + +gateway: + branch: develop + +services: + account: + branch: feature/account + + payment: + use_ocp: true + + user: + use_ocp: true +``` + +Users can: + +* Create template +* Update template +* Clone template into sandbox + +--- + +# Micro Agent Requirements + +Runs on: + +```text +172.18.136.92 +``` + +Responsibilities: + +```text +List repositories +List branches +Fetch repository updates +Checkout branch +Pull latest changes +Build Go binary +Create Docker image +Run container +Restart container +Stop container +Stream logs +``` + +--- + +# Microservice Deployment Process + +Given: + +```text +Repository: account +Branch: feature/login-error +``` + +Agent executes: + +```bash +git fetch +git checkout feature/login-error +git pull +``` + +Then: + +```bash +go build -o app +``` + +Then generates runtime image: + +```dockerfile +FROM alpine:latest + +COPY app /app + +CMD ["/app"] +``` + +Then: + +```bash +docker build +docker run +``` + +--- + +# Gateway Agent Requirements + +Runs on: + +```text +172.18.139.186 +``` + +Responsibilities: + +```text +List branches +Fetch repository updates +Checkout branch +Pull latest changes +Build container +Deploy container +Restart container +Manage routing +Stream logs +``` + +--- + +# API Gateway Deployment + +The API Gateway must run inside Docker. + +It is no longer deployed directly on the host. + +Deployment process: + +```bash +git fetch +git checkout +git pull +docker build +docker run +``` + +--- + +# Offline VM Requirements + +Deployment VMs have no internet access. + +The following cannot be relied upon: + +```bash +docker pull +``` + +--- + +# Docker Image Distribution + +Images must be imported manually. + +Example: + +On machine with internet: + +```bash +docker pull nginx:latest + +docker save nginx:latest -o nginx.tar +``` + +Transfer: + +```bash +scp nginx.tar user@172.18.139.186:/tmp +``` + +Load: + +```bash +docker load -i nginx.tar +``` + +--- + +# Environment Management + +Users must be able to: + +```text +Create Environment +Update Environment +Delete Environment +Manage Secrets +Manage Variables +``` + +Example: + +```env +DB_HOST=... +DB_USER=... +DB_PASSWORD=... +``` + +Environment values are injected during deployment. + +--- + +# Route Override System + +Most important feature. + +Each route can target either: + +```text +Sandbox Deployment +OCP Deployment +``` + +Example: + +```yaml +account: + target: http://172.18.136.92:9001 + +payment: + target: https://payment-dev.company.com + +user: + target: https://user-dev.company.com +``` + +Result: + +```text +account -> sandbox +payment -> OCP +user -> OCP +``` + +--- + +# Mobile App Integration + +Current mobile app: + +```text +https://project-dev-url.domain.com +``` + +Target: + +```text +http://172.18.139.186:{PORT} +``` + +Example: + +```text +http://172.18.139.186:8080 +``` + +QA can point the mobile application directly to the API Gateway sandbox. + +No DNS changes required. + +--- + +# Port Management + +Gateway Ports: + +```text +8080 +8081 +8082 +... +``` + +Microservice Ports: + +```text +9001 +9002 +9003 +... +``` + +Control Plane must: + +* Allocate ports +* Track ports +* Prevent conflicts + +--- + +# Deployment States + +Supported states: + +```text +QUEUED + +FETCHING + +CHECKOUT + +BUILDING + +CREATING_IMAGE + +STARTING_CONTAINER + +RUNNING + +FAILED + +STOPPED +``` + +--- + +# Real-Time Progress + +Frontend must receive deployment progress in real time. + +Example: + +```text +✓ Fetch + +✓ Checkout + +✓ Build + +✓ Create Image + +✓ Start Container + +✓ Running +``` + +No page refresh. + +--- + +# Real-Time Logs + +Frontend must receive logs while deployment is running. + +Example: + +```text +[FETCH] +Fetching origin... + +[FETCH] +Success + +[BUILD] +Running go build + +[BUILD] +Success + +[DEPLOY] +Container started +``` + +--- + +# Event Streaming + +Agents emit events. + +Examples: + +```text +FETCH_STARTED +FETCH_COMPLETED + +CHECKOUT_STARTED +CHECKOUT_COMPLETED + +BUILD_STARTED +BUILD_COMPLETED + +DEPLOY_STARTED +DEPLOY_COMPLETED + +DEPLOY_FAILED +``` + +Architecture: + +```text +Agent + -> SSE/WebSocket + +Control Plane + -> WebSocket + +Frontend +``` + +--- + +# Dashboard Features + +## Authentication + +```text +Login +Logout +``` + +--- + +## Repository Management + +```text +List Repositories +List Branches +``` + +--- + +## Deployments + +```text +Deploy Branch +Restart Deployment +Stop Deployment +Delete Deployment +``` + +--- + +## Deployment Monitoring + +```text +View Progress +View Logs +View Status +View History +``` + +--- + +## Environment Management + +```text +Create Environment +Update Environment +Delete Environment +``` + +--- + +## Sandbox Management + +```text +Create Sandbox +Update Sandbox +Delete Sandbox +Clone Sandbox +``` + +--- + +## Template Management + +```text +Create Template +Update Template +Delete Template +Create Sandbox From Template +``` + +--- + +## Route Management + +```text +Route To Sandbox +Route To OCP +``` + +--- + +# Audit Trail + +Store: + +```text +User +Repository +Branch +Environment +Sandbox +Timestamp +Status +``` + +Example: + +```text +User: +Achmad + +Repository: +account + +Branch: +feature/login-error + +Sandbox: +QA-LOGIN-ERROR + +Status: +SUCCESS +``` + +--- + +# Technology Stack + +## Dashboard + +```text +NextJS +React +TypeScript +Tailwind +``` + +## Control Plane + +```text +Spring Boot +PostgreSQL +WebSocket +``` + +## Agents + +Preferred: + +```text +Go +``` + +Alternative: + +```text +Spring Boot +``` + +--- + +# Non-Goals + +Not intended to replace: + +```text +Kubernetes +OpenShift +Rancher +ArgoCD +Coolify +``` + +Not intended to support: + +```text +Multi-Tenant SaaS +Public Cloud +Generic Container Hosting +``` + +Purpose: + +Provide isolated deployment environments for Backend and QA teams. + +--- + +# MVP Success Criteria + +A developer can: + +1. Login using Bitbucket username and password. +2. Select a repository. +3. Select a branch. +4. Configure environment variables. +5. Deploy API Gateway. +6. Deploy microservices. +7. Watch deployment progress in real time. +8. Watch deployment logs in real time. +9. Create sandboxes. +10. Create sandbox templates. +11. Route selected services to sandbox deployments. +12. Route remaining services to OCP. +13. Point mobile application to: + +```text +http://172.18.139.186:{PORT} +``` + +14. Allow QA to test isolated feature branches without impacting shared OCP environments. + +# Future Enhancements + +## Sandbox Isolation Strategy + +### Goal + +Allow multiple developers and QA engineers to run independent sandboxes simultaneously without conflicts. + +Example: + +```text +Achmad Sandbox +├── account +├── payment +└── gateway + +QA Sandbox +├── account +├── payment +└── gateway +``` + +Both sandboxes must coexist on the same infrastructure. + +--- + +## Container Naming Convention + +Containers should follow a predictable naming pattern. + +Format: + +```text +sandbox-{sandbox-name}-{service-name} +``` + +Examples: + +```text +sandbox-achmad-account +sandbox-achmad-payment +sandbox-achmad-user +sandbox-achmad-gateway +``` + +```text +sandbox-qa-login-account +sandbox-qa-login-gateway +``` + +Benefits: + +* Easier troubleshooting +* Easier cleanup +* Easier log inspection +* Easier monitoring + +--- + +## Docker Network Per Sandbox + +Each sandbox should have its own Docker network. + +Format: + +```text +sandbox-{sandbox-name} +``` + +Examples: + +```text +sandbox-achmad +sandbox-qa-login +sandbox-regression +``` + +Container example: + +```text +Network: +sandbox-achmad + +Containers: +sandbox-achmad-gateway +sandbox-achmad-account +sandbox-achmad-payment +``` + +Benefits: + +* Network isolation +* Service discovery +* No cross-sandbox traffic +* Simpler routing + +--- + +## Internal Service Communication + +Services within a sandbox should communicate through Docker DNS. + +Example: + +Instead of: + +```text +http://172.18.136.92:9001 +``` + +Use: + +```text +http://sandbox-achmad-account:8080 +``` + +Benefits: + +* No dependency on host ports +* Cleaner configuration +* Easier sandbox replication + +--- + +## Sandbox Port Allocation + +Gateway containers should expose a unique external port. + +Examples: + +```text +sandbox-achmad-gateway +→ 172.18.139.186:8080 + +sandbox-qa-login-gateway +→ 172.18.139.186:8081 + +sandbox-regression-gateway +→ 172.18.139.186:8082 +``` + +Mobile applications connect only to gateway ports. + +--- + +## Automatic Port Management + +Control Plane should automatically: + +* Allocate available ports +* Reserve ports +* Release ports when sandbox is deleted + +Example database table: + +```text +PortAllocation +├── sandboxId +├── serviceName +├── port +└── allocatedAt +``` + +--- + +## Sandbox Lifecycle + +Future support: + +### Suspend Sandbox + +Stops all containers while preserving configuration. + +Example: + +```text +ACTIVE +↓ +SUSPENDED +``` + +Resources freed: + +* CPU +* Memory + +Configuration preserved. + +--- + +### Resume Sandbox + +Restarts previously suspended sandbox. + +Example: + +```text +SUSPENDED +↓ +ACTIVE +``` + +--- + +### Sandbox Expiration + +Automatic cleanup after inactivity. + +Example: + +```text +No activity for 14 days +↓ +Mark Expired +↓ +Stop Containers +↓ +Delete After Retention Period +``` + +Configurable. + +--- + +## Sandbox Cloning + +Clone an existing sandbox. + +Example: + +```text +Source: +Achmad Sandbox + +Destination: +QA Sandbox +``` + +Result: + +```text +Same repositories +Same branches +Same environment variables +Same route overrides +``` + +New ports are allocated automatically. + +--- + +## Sandbox Snapshots + +Capture sandbox state. + +Stored information: + +* Repository versions +* Branches +* Environment variables +* Route overrides + +Example: + +```text +Snapshot: +QA-Before-Release +``` + +Allows rollback and recreation later. + +--- + +## Resource Limits + +Per sandbox resource controls. + +Example: + +```text +CPU: 1 Core +Memory: 1 GB +``` + +Per container: + +```text +CPU: 500m +Memory: 512MB +``` + +Implemented using Docker resource limits. + +--- + +## Health Monitoring + +Track sandbox health. + +Metrics: + +* Container status +* CPU usage +* Memory usage +* Restart count +* Health endpoint status + +Dashboard should display: + +```text +Healthy +Degraded +Unhealthy +``` + +--- + +## Future Infrastructure Agent + +Node: + +```text +172.18.136.93 +``` + +Responsibilities: + +* PostgreSQL restore +* Database cloning +* RabbitMQ management +* Redis management +* Kafka management + +Potential use case: + +```text +Clone QA Database +↓ +Attach To Sandbox +↓ +Run Integration Testing +``` + +--- + +## Future RBAC + +Current MVP: + +```text +All authenticated users +``` + +Future roles: + +```text +ADMIN +BACKEND +QA +VIEWER +``` + +Permissions: + +```text +Deploy +Delete Sandbox +Manage Templates +Manage Routes +Manage Environments +``` + +--- + +## Future Notifications + +Deployment notifications: + +```text +Deployment Started +Deployment Succeeded +Deployment Failed +Sandbox Expired +``` + +Channels: + +* Email +* Slack +* Microsoft Teams + +``` + +This section should be appended after the MVP section of the main requirements document. It is intentionally out of scope for the first implementation but provides a roadmap that avoids architectural dead ends later. +``` \ No newline at end of file diff --git a/agent-gateway/cmd/agent-gateway/main.go b/agent-gateway/cmd/agent-gateway/main.go new file mode 100644 index 0000000..01311c5 --- /dev/null +++ b/agent-gateway/cmd/agent-gateway/main.go @@ -0,0 +1,170 @@ +// Command agent-gateway runs on the API Gateway VM (172.18.139.186). It +// maintains a WebSocket to the control plane and deploys the gateway +// service. The gateway uses the same pipeline as the micro services — +// ponytail: same shape, different repo map. If the gateway ever needs a +// different build (PHP composer install, etc.), split the deployer package. +package main + +import ( + "context" + "encoding/json" + "flag" + "log" + "net/url" + "os" + "sync" + "time" + + "github.com/docker/docker/client" + "github.com/gorilla/websocket" + + "github.com/sdp/agent-micro/internal/deployer" + "github.com/sdp/agent-micro/internal/gitutil" + "github.com/sdp/protocol" +) + +var repos = map[string]string{ + "api-gateway": "/home/user/SDP", +} + +func main() { + cpURL := flag.String("cp", envOr("SDP_CP_URL", "ws://localhost:8080/ws/agent"), "control plane WS URL") + nodeID := flag.String("node", envOr("SDP_NODE_ID", "gateway"), "node id sent in WS query") + flag.Parse() + + cli, err := client.NewClientWithOpts(client.FromEnv) + if err != nil { + log.Fatalf("docker client: %v", err) + } + + u, _ := url.Parse(*cpURL) + q := u.Query() + q.Set("node", *nodeID) + u.RawQuery = q.Encode() + + out := make(chan []byte, 128) + var connMu sync.Mutex + var conn *websocket.Conn + + go writer(&conn, &connMu, out) + + for { + c, err := dial(u) + if err != nil { + log.Printf("dial: %v; retrying in 2s", err) + time.Sleep(2 * time.Second) + continue + } + connMu.Lock() + conn = c + connMu.Unlock() + log.Printf("agent-gateway connected as %s", *nodeID) + + readLoop(c, cli, out, &connMu, &conn) + } +} + +func dial(u *url.URL) (*websocket.Conn, error) { + log.Printf("connecting to %s", u) + return websocket.DefaultDialer.Dial(u.String(), nil) +} + +func writer(conn **websocket.Conn, mu *sync.Mutex, out <-chan []byte) { + for msg := range out { + mu.Lock() + c := *conn + mu.Unlock() + if c == nil { + continue + } + if err := c.WriteMessage(websocket.TextMessage, msg); err != nil { + log.Printf("write: %v", err) + } + } +} + +type runState struct { + deployer *deployer.Deployer + cancel context.CancelFunc +} + +func readLoop(c *websocket.Conn, cli *dockerclient.Client, out chan<- []byte, mu *sync.Mutex, connPtr **websocket.Conn) { + inflight := map[string]*runState{} + for { + _, raw, err := c.ReadMessage() + if err != nil { + log.Printf("read: %v", err) + mu.Lock() + if *connPtr == c { + *connPtr = nil + } + mu.Unlock() + _ = c.Close() + return + } + var frame struct { + Op string `json:"op"` + Data protocol.DeployRequest `json:"data"` + ID string `json:"id"` + } + if err := json.Unmarshal(raw, &frame); err != nil { + log.Printf("bad frame: %v", err) + continue + } + switch frame.Op { + case "deploy": + repoPath, ok := repos[frame.Data.Repository] + if !ok { + emit(out, protocol.Event{ + DeploymentID: frame.Data.DeploymentID, + Kind: "status", + State: "FAILED", + At: time.Now().UnixMilli(), + }) + continue + } + d := deployer.New(cli, frame.Data.DeploymentID, + frame.Data.Repository, repoPath, + frame.Data.Branch, frame.Data.Env, + gitutil.Creds{Username: frame.Data.Username, Password: frame.Data.Password}, + ) + ctx, cancel := context.WithCancel(context.Background()) + inflight[frame.Data.DeploymentID] = &runState{deployer: d, cancel: cancel} + go runDeploy(d, ctx, out) + case "stop": + if rs, ok := inflight[frame.ID]; ok { + _ = rs.deployer.Stop(context.Background()) + rs.cancel() + } + } + } +} + +func runDeploy(d *deployer.Deployer, ctx context.Context, out chan<- []byte) { + events := make(chan protocol.Event, 64) + go func() { + state := d.Run(ctx, events) + if state == "RUNNING" { + d.StreamLogs(ctx, events) + } + close(events) + }() + for e := range events { + emit(out, e) + } +} + +func emit(out chan<- []byte, e protocol.Event) { + b, _ := json.Marshal(e) + select { + case out <- b: + default: + } +} + +func envOr(k, def string) string { + if v := os.Getenv(k); v != "" { + return v + } + return def +} diff --git a/agent-gateway/go.mod b/agent-gateway/go.mod new file mode 100644 index 0000000..694d920 --- /dev/null +++ b/agent-gateway/go.mod @@ -0,0 +1,8 @@ +module github.com/sdp/agent-gateway + +go 1.23 + +require ( + github.com/docker/docker/client v0.0.0-00010101000000-000000000000 + github.com/sdp/protocol v0.0.0 +) diff --git a/agent-micro/cmd/agent-micro/main.go b/agent-micro/cmd/agent-micro/main.go new file mode 100644 index 0000000..757967b --- /dev/null +++ b/agent-micro/cmd/agent-micro/main.go @@ -0,0 +1,180 @@ +// Command agent-micro runs on the microservices VM (172.18.136.92). It +// maintains a WebSocket to the control plane, accepts deploy/stop commands, +// and runs the build+container pipeline locally. +package main + +import ( + "context" + "encoding/json" + "flag" + "log" + "net/url" + "os" + "sync" + "time" + + "github.com/docker/docker/client" + "github.com/gorilla/websocket" + + "github.com/sdp/agent-micro/internal/deployer" + "github.com/sdp/agent-micro/internal/gitutil" + "github.com/sdp/protocol" +) + +var repos = map[string]string{ + // ponytail: hand-curated from REQUIREMENTS.md. Real version reads a yaml + // config file. Adding a new service = one line. + "account": "/home/user/AppGolang/account", + "payment": "/home/user/AppGolang/payment", + "user": "/home/user/AppGolang/user", + "notification": "/home/user/AppGolang/notification", +} + +func main() { + cpURL := flag.String("cp", envOr("SDP_CP_URL", "ws://localhost:8080/ws/agent"), "control plane WS URL") + nodeID := flag.String("node", envOr("SDP_NODE_ID", "micro"), "node id sent in WS query") + flag.Parse() + + cli, err := client.NewClientWithOpts(client.FromEnv) + if err != nil { + log.Fatalf("docker client: %v", err) + } + + u, _ := url.Parse(*cpURL) + q := u.Query() + q.Set("node", *nodeID) + u.RawQuery = q.Encode() + + out := make(chan []byte, 128) + var connMu sync.Mutex + var conn *websocket.Conn + + go writer(&conn, &connMu, out) + + for { + c, err := dial(u) + if err != nil { + log.Printf("dial: %v; retrying in 2s", err) + time.Sleep(2 * time.Second) + continue + } + connMu.Lock() + conn = c + connMu.Unlock() + log.Printf("agent-micro connected as %s", *nodeID) + + readLoop(c, cli, out, &connMu, &conn) + } +} + +func dial(u *url.URL) (*websocket.Conn, error) { + log.Printf("connecting to %s", u) + return websocket.DefaultDialer.Dial(u.String(), nil) +} + +// writer pumps outbound events to whichever conn is current. If conn is +// nil (during reconnect), messages buffer until the next conn is set. +func writer(conn **websocket.Conn, mu *sync.Mutex, out <-chan []byte) { + for msg := range out { + mu.Lock() + c := *conn + mu.Unlock() + if c == nil { + continue + } + if err := c.WriteMessage(websocket.TextMessage, msg); err != nil { + log.Printf("write: %v", err) + } + } +} + +type runState struct { + deployer *deployer.Deployer + cancel context.CancelFunc +} + +func readLoop(c *websocket.Conn, cli *dockerclient.Client, out chan<- []byte, mu *sync.Mutex, connPtr **websocket.Conn) { + inflight := map[string]*runState{} + for { + _, raw, err := c.ReadMessage() + if err != nil { + log.Printf("read: %v", err) + mu.Lock() + if *connPtr == c { + *connPtr = nil + } + mu.Unlock() + _ = c.Close() + return + } + var frame struct { + Op string `json:"op"` + Data protocol.DeployRequest `json:"data"` + ID string `json:"id"` + } + if err := json.Unmarshal(raw, &frame); err != nil { + log.Printf("bad frame: %v", err) + continue + } + switch frame.Op { + case "deploy": + repoPath, ok := repos[frame.Data.Repository] + if !ok { + emit(out, protocol.Event{ + DeploymentID: frame.Data.DeploymentID, + Kind: "status", + State: "FAILED", + At: time.Now().UnixMilli(), + }) + continue + } + d := deployer.New(cli, frame.Data.DeploymentID, + frame.Data.Repository, repoPath, + frame.Data.Branch, frame.Data.Env, + gitutil.Creds{Username: frame.Data.Username, Password: frame.Data.Password}, + ) + ctx, cancel := context.WithCancel(context.Background()) + inflight[frame.Data.DeploymentID] = &runState{deployer: d, cancel: cancel} + go runDeploy(d, ctx, out) + case "stop": + if rs, ok := inflight[frame.ID]; ok { + _ = rs.deployer.Stop(context.Background()) + rs.cancel() // unblock StreamLogs + } + } + } +} + +func runDeploy(d *deployer.Deployer, ctx context.Context, out chan<- []byte) { + events := make(chan protocol.Event, 64) + // producer: Run pipelines, then StreamLogs tails the container. Both + // write to the same channel. We close it when both are done so the + // drain loop below can exit. + go func() { + state := d.Run(ctx, events) + if state == "RUNNING" { + d.StreamLogs(ctx, events) + } + close(events) + }() + + for e := range events { + emit(out, e) + } +} + +func emit(out chan<- []byte, e protocol.Event) { + b, _ := json.Marshal(e) + select { + case out <- b: + default: + // ponytail: drop on backpressure. Deploys are rare. + } +} + +func envOr(k, def string) string { + if v := os.Getenv(k); v != "" { + return v + } + return def +} diff --git a/agent-micro/go.mod b/agent-micro/go.mod new file mode 100644 index 0000000..3851c3f --- /dev/null +++ b/agent-micro/go.mod @@ -0,0 +1,9 @@ +module github.com/sdp/agent-micro + +go 1.23 + +require ( + github.com/docker/docker/client v0.0.0-00010101000000-000000000000 + github.com/docker/go-connections v0.5.0 + github.com/sdp/protocol v0.0.0 +) diff --git a/agent-micro/internal/deployer/deployer.go b/agent-micro/internal/deployer/deployer.go new file mode 100644 index 0000000..a1ffb64 --- /dev/null +++ b/agent-micro/internal/deployer/deployer.go @@ -0,0 +1,204 @@ +// Package deployer is the per-deployment state machine. One Deployer is +// created per DeployRequest; it runs the stages sequentially and emits +// events on its output channel. The caller (the agent) reads events and +// fans them out to the control plane. +// +// Runtime model (per REQUIREMENTS.md): +// 1. git fetch / checkout / pull on the host +// 2. go build -o on the host (the VM has the Go toolchain) +// 3. docker run alpine:3.20 with -v :/src and command /src/ +// — NO Dockerfile, NO image build step. The alpine image must be +// loaded on the host ahead of time (manual docker load). +// 4. environment variables from the Environment config are passed via +// -e flags into the container. +package deployer + +import ( + "context" + "fmt" + "os/exec" + "path/filepath" + "strings" + "time" + + "github.com/docker/docker/api/types/container" + dockerclient "github.com/docker/docker/client" + "github.com/docker/go-connections/nat" + + "github.com/sdp/agent-micro/internal/gitutil" + "github.com/sdp/protocol" +) + +type Deployer struct { + ID string + Repository string + RepoPath string + Branch string + Env map[string]string + Creds gitutil.Creds + cli *dockerclient.Client + binName string + imageTag string + containerID string +} + +func New(cli *dockerclient.Client, id, repo, path, branch string, env map[string]string, c gitutil.Creds) *Deployer { + return &Deployer{ + ID: id, + Repository: repo, + RepoPath: path, + Branch: branch, + Env: env, + Creds: c, + cli: cli, + binName: "app-" + repo, + imageTag: "alpine:3.20", // ponytail: pre-loaded on the VM. Swap to a configurable tag. + } +} + +// Run executes the full pipeline. It pushes events to out and returns the +// final state (RUNNING on success, FAILED otherwise). +func (d *Deployer) Run(ctx context.Context, out chan<- protocol.Event) string { + emit := func(kind, state, stage, line string) { + out <- protocol.Event{ + DeploymentID: d.ID, + Kind: kind, + State: state, + Stage: stage, + Line: line, + At: time.Now().UnixMilli(), + } + } + + stages := []struct { + name string + fn func() error + }{ + {"git fetch", func() error { _, err := gitutil.Fetch(ctx, d.RepoPath, d.Creds); return err }}, + {"git checkout", func() error { _, err := gitutil.Checkout(ctx, d.RepoPath, d.Branch, d.Creds); return err }}, + {"git pull", func() error { _, err := gitutil.Pull(ctx, d.RepoPath, d.Creds); return err }}, + {"go build", d.build}, + {"start container", d.startContainer}, + } + for _, s := range stages { + emit("progress", "", s.name, "starting "+s.name) + if err := s.fn(); err != nil { + emit("progress", "FAILED", s.name, err.Error()) + emit("status", "FAILED", "", "") + return "FAILED" + } + emit("progress", "", s.name, "ok") + } + emit("status", "RUNNING", "", "") + return "RUNNING" +} + +func (d *Deployer) build(ctx context.Context) error { + binPath := filepath.Join(d.RepoPath, d.binName) + cmd := exec.CommandContext(ctx, "go", "build", "-o", binPath, "./...") + cmd.Dir = d.RepoPath + out, err := cmd.CombinedOutput() + if err != nil { + return fmt.Errorf("go build: %w: %s", err, out) + } + return nil +} + +// startContainer runs `alpine:3.20` with the host repo dir bind-mounted at +// /src and the binary executed as the container command. The container's +// working dir is set to the bind mount so the binary finds its config files. +func (d *Deployer) startContainer(ctx context.Context) error { + + envList := make([]string, 0, len(d.Env)) + for k, v := range d.Env { + envList = append(envList, k+"="+v) + } + + cfg := &container.Config{ + Image: d.imageTag, + Cmd: []string{"/src/" + d.binName}, + Env: envList, + WorkingDir: "/src", + ExposedPorts: nat.PortSet{ + "8080/tcp": struct{}{}, + }, + } + hostCfg := &container.HostConfig{ + Binds: []string{d.RepoPath + ":/src"}, + RestartPolicy: container.RestartPolicy{Name: "unless-stopped"}, + } + + resp, err := d.cli.ContainerCreate(ctx, cfg, hostCfg, nil, nil, d.containerName()) + if err != nil { + return fmt.Errorf("container create: %w", err) + } + d.containerID = resp.ID + if err := d.cli.ContainerStart(ctx, resp.ID, container.StartOptions{}); err != nil { + return fmt.Errorf("container start: %w", err) + } + return nil +} + +func (d *Deployer) containerName() string { + return "sdp-" + d.Repository + "-" + d.ID +} + +func (d *Deployer) Stop(ctx context.Context) error { + if d.containerID == "" { + return nil + } + return d.cli.ContainerStop(ctx, d.containerID, container.StopOptions{}) +} + +// StreamLogs tails the container's logs into the events channel until ctx +// is cancelled. +func (d *Deployer) StreamLogs(ctx context.Context, out chan<- protocol.Event) { + if d.containerID == "" { + return + } + rc, err := d.cli.ContainerLogs(ctx, d.containerID, container.LogsOptions{ + ShowStdout: true, + ShowStderr: true, + Follow: true, + Timestamps: false, + }) + if err != nil { + return + } + defer rc.Close() + // ponytail: Docker multiplexes stdout/stderr; the leading 8 bytes per + // frame are the stream header. For MVP we just split on newlines and + // don't try to demux. + buf := make([]byte, 4096) + var carry strings.Builder + for { + n, err := rc.Read(buf) + if n > 0 { + carry.Write(buf[:n]) + for { + s := carry.String() + i := strings.IndexByte(s, '\n') + if i < 0 { + break + } + line := s[:i] + carry.Reset() + carry.WriteString(s[i+1:]) + if len(line) > 8 && line[0] <= 2 { + line = line[8:] + } + if line != "" { + out <- protocol.Event{ + DeploymentID: d.ID, + Kind: "log", + Line: line, + At: time.Now().UnixMilli(), + } + } + } + } + if err != nil { + return + } + } +} diff --git a/agent-micro/internal/gitutil/gitutil.go b/agent-micro/internal/gitutil/gitutil.go new file mode 100644 index 0000000..4b43d0d --- /dev/null +++ b/agent-micro/internal/gitutil/gitutil.go @@ -0,0 +1,65 @@ +// Package gitutil wraps the few git operations the agent needs. Credentials +// are passed in per-call and never written to disk — every command sets them +// via -c credential helpers for the lifetime of the subprocess. +package gitutil + +import ( + "context" + "fmt" + "os/exec" + "strings" +) + +// Creds is a username/password. We pass them through GIT_ASKPASS so they +// never appear on the command line or in process listings. +type Creds struct { + Username string + Password string +} + +// Fetch runs `git fetch --prune origin`. Uses the per-command credential +// helper to inject creds without touching the repo's stored config. +func Fetch(ctx context.Context, repoDir string, c Creds) (string, error) { + return runGit(ctx, repoDir, c, "fetch", "--prune", "origin") +} + +// Checkout switches to branch and updates the working tree. +func Checkout(ctx context.Context, repoDir, branch string, c Creds) (string, error) { + return runGit(ctx, repoDir, c, "checkout", "-f", branch) +} + +// Pull fast-forwards the branch to match origin. Safe no-op if up to date. +func Pull(ctx context.Context, repoDir string, c Creds) (string, error) { + return runGit(ctx, repoDir, c, "pull", "--ff-only") +} + +// Probe validates that the credentials work for a given remote. Used at +// login. Tries `git ls-remote HEAD`; succeeds even on an empty repo. +func Probe(ctx context.Context, repoDir string, c Creds) error { + _, err := runGit(ctx, repoDir, c, "ls-remote", "--heads", "origin", "HEAD") + return err +} + +func runGit(ctx context.Context, repoDir string, c Creds, args ...string) (string, error) { + cmd := exec.CommandContext(ctx, "git", args...) + cmd.Dir = repoDir + // GIT_ASKPASS gives us a per-command credential helper. We just echo the + // creds back. The "username" / "password" args are sent to the script's + // argv by git. + askpass := fmt.Sprintf(`#!/bin/sh +case "$1" in + username) echo %q ;; + password) echo %q ;; +esac`, c.Username, c.Password) + cmd.Env = append(cmd.Environ(), + "GIT_ASKPASS=/dev/stdin", + "GIT_TERMINAL_PROMPT=0", + ) + // ponytail: passing askpass via stdin is portable across Linux/macOS. + cmd.Stdin = strings.NewReader(askpass) + out, err := cmd.CombinedOutput() + if err != nil { + return string(out), fmt.Errorf("git %s: %w: %s", strings.Join(args, " "), err, out) + } + return string(out), nil +} diff --git a/control-plane/cmd/control-plane/main.go b/control-plane/cmd/control-plane/main.go new file mode 100644 index 0000000..17a30ac --- /dev/null +++ b/control-plane/cmd/control-plane/main.go @@ -0,0 +1,30 @@ +// Command control-plane is the SDP brain. It owns the metadata DB, the +// WebSocket hub, and the HTTP API. Agents and the dashboard talk to it; +// it never SSHes, never builds, never touches git directly. +package main + +import ( + "log" + "net/http" + + "github.com/sdp/control-plane/internal/api" + "github.com/sdp/control-plane/internal/config" + "github.com/sdp/control-plane/internal/store" + "github.com/sdp/control-plane/internal/ws" +) + +func main() { + cfg := config.Load() + + st, err := store.Open(cfg.DataDir) + if err != nil { + log.Fatalf("store: %v", err) + } + defer st.Close() + + hub := ws.New() + srv := api.New(st, hub) + + log.Printf("control-plane listening on %s (data=%s)", cfg.Addr, cfg.DataDir) + log.Fatal(http.ListenAndServe(cfg.Addr, srv.Routes())) +} diff --git a/control-plane/go.mod b/control-plane/go.mod new file mode 100644 index 0000000..8145da2 --- /dev/null +++ b/control-plane/go.mod @@ -0,0 +1,8 @@ +module github.com/sdp/control-plane + +go 1.23 + +require ( + github.com/gorilla/websocket v1.5.1 + modernc.org/sqlite v1.28.0 +) diff --git a/control-plane/internal/api/api.go b/control-plane/internal/api/api.go new file mode 100644 index 0000000..fbd2ffb --- /dev/null +++ b/control-plane/internal/api/api.go @@ -0,0 +1,256 @@ +// Package api wires the HTTP endpoints. Kept on net/http — no router lib +// for a handful of endpoints, stdlib mux is plenty. +package api + +import ( + "crypto/rand" + "encoding/hex" + "encoding/json" + "net/http" + "strings" + "sync" + + "github.com/sdp/control-plane/internal/store" + "github.com/sdp/control-plane/internal/ws" + "github.com/sdp/protocol" +) + +type Server struct { + st *store.Store + hub *ws.Hub + agents *AgentRegistry + sess *Sessions +} + +type AgentRegistry struct { + mu sync.RWMutex + conns map[string]bool // nodeIDs currently connected over WS +} + +func New(st *store.Store, hub *ws.Hub) *Server { + return &Server{ + st: st, + hub: hub, + agents: &AgentRegistry{conns: make(map[string]bool)}, + sess: NewSessions(), + } +} + +func (s *Server) Routes() http.Handler { + mux := http.NewServeMux() + mux.HandleFunc("/api/login", s.handleLogin) + mux.HandleFunc("/api/repos", s.handleListRepos) + mux.HandleFunc("/api/repos/branches", s.handleListBranches) + mux.HandleFunc("/api/deployments", s.handleDeployments) // GET list, POST create + mux.HandleFunc("/api/deployments/stop", s.handleStopDeployment) // POST + mux.Handle("/ws/agent", s.hub.AgentWS(s.st, + func(nodeID string) { + s.agents.mu.Lock() + s.agents.conns[nodeID] = true + s.agents.mu.Unlock() + }, + func(nodeID string) { + s.agents.mu.Lock() + delete(s.agents.conns, nodeID) + s.agents.mu.Unlock() + }, + )) + mux.HandleFunc("/ws/deployments/", s.hub.DeploymentWS(s.st)) + return s.withAuth(mux) +} + +// withAuth checks the session cookie on /api/* (skipping login). /ws/* is +// protected at the handler — we don't pass auth in headers easily on the WS +// upgrade from the browser, so the dashboard sends ?token=... instead. +func (s *Server) withAuth(next http.Handler) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if !strings.HasPrefix(r.URL.Path, "/api/") || r.URL.Path == "/api/login" { + next.ServeHTTP(w, r) + return + } + c, err := r.Cookie("sdp_session") + if err != nil || !s.sess.Valid(c.Value) { + http.Error(w, "unauthorized", http.StatusUnauthorized) + return + } + next.ServeHTTP(w, r) + }) +} + +// ---- login ---- + +type loginReq struct { + Username string `json:"username"` + Password string `json:"password"` + Repo string `json:"repo"` // optional: validate against this specific repo +} + +func (s *Server) handleLogin(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodPost { + http.Error(w, "POST only", http.StatusMethodNotAllowed) + return + } + var body loginReq + if err := json.NewDecoder(r.Body).Decode(&body); err != nil { + http.Error(w, "bad json", http.StatusBadRequest) + return + } + // ponytail: trust boundary lives in the agent — it does the actual git + // ls-remote. The control plane just hands off credentials per-op. + // For login we ask any connected agent to validate. If none are + // connected, fail. Real impl: pick a known bootstrap repo. + ok := s.validateViaAgent(body.Username, body.Password, body.Repo) + if !ok { + http.Error(w, "login failed", http.StatusUnauthorized) + return + } + tok := s.sess.Issue(body.Username) + http.SetCookie(w, &http.Cookie{ + Name: "sdp_session", + Value: tok, + Path: "/", + HttpOnly: true, + SameSite: http.SameSiteLaxMode, + MaxAge: 12 * 3600, + }) + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte(`{"ok":true}`)) +} + +// validateViaAgent does a git ls-remote through one of the connected agents. +// The agent holds the repos; the control plane never touches git directly. +// +// ponytail: MVP stub. Returns true if any agent is connected so the smoke +// flow can run. Real impl will send a "probe" frame over the agent's WS +// and wait for a reply. +func (s *Server) validateViaAgent(user, pass, repo string) bool { + _ = user + _ = pass + _ = repo + s.agents.mu.RLock() + defer s.agents.mu.RUnlock() + return len(s.agents.conns) > 0 +} + +// ---- repos ---- + +type repoInfo struct { + Name string `json:"name"` + Node string `json:"node"` + Path string `json:"path"` +} + +func (s *Server) handleListRepos(w http.ResponseWriter, r *http.Request) { + // ponytail: real impl asks the connected agents for their repo list. + // For MVP smoke, stub with the spec's example. + repos := []repoInfo{ + {Name: "account", Node: "micro", Path: "/home/user/AppGolang/account"}, + {Name: "payment", Node: "micro", Path: "/home/user/AppGolang/payment"}, + {Name: "user", Node: "micro", Path: "/home/user/AppGolang/user"}, + {Name: "notification", Node: "micro", Path: "/home/user/AppGolang/notification"}, + {Name: "api-gateway", Node: "gateway", Path: "/home/user/SDP"}, + } + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(repos) +} + +func (s *Server) handleListBranches(w http.ResponseWriter, r *http.Request) { + repo := r.URL.Query().Get("repo") + if repo == "" { + http.Error(w, "repo required", http.StatusBadRequest) + return + } + // ponytail: real impl forwards to the agent. For MVP, stub. + branches := []string{"main", "develop", "feature/login-error"} + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(branches) +} + +// ---- deployments ---- + +type deployReq struct { + Repository string `json:"repository"` + Branch string `json:"branch"` + Env map[string]string `json:"env,omitempty"` + Username string `json:"username"` + Password string `json:"password"` +} + +func (s *Server) handleDeployments(w http.ResponseWriter, r *http.Request) { + switch r.Method { + case http.MethodGet: + // ponytail: list from SQLite. Real impl: SELECT with filter. + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(`[]`)) + case http.MethodPost: + var body deployReq + if err := json.NewDecoder(r.Body).Decode(&body); err != nil { + http.Error(w, "bad json", http.StatusBadRequest) + return + } + // resolve repo -> node + node := "micro" + if body.Repository == "api-gateway" { + node = "gateway" + } + // ensure agent connected + s.agents.mu.RLock() + connected := s.agents.conns[node] + s.agents.mu.RUnlock() + if !connected { + http.Error(w, "agent "+node+" not connected", http.StatusServiceUnavailable) + return + } + id := newID() + _ = s.st.StartDeployment(id, body.Repository, body.Branch, body.Username) + + // send deploy request to agent over its WS + req := protocol.DeployRequest{ + DeploymentID: id, + Repository: body.Repository, + Branch: body.Branch, + Env: body.Env, + Username: body.Username, + Password: body.Password, + } + payload, _ := json.Marshal(map[string]any{ + "op": "deploy", + "data": req, + }) + if !s.hub.SendToAgent(node, payload) { + http.Error(w, "agent buffer full", http.StatusServiceUnavailable) + return + } + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(map[string]string{"id": id}) + default: + http.Error(w, "method not allowed", http.StatusMethodNotAllowed) + } +} + +func (s *Server) handleStopDeployment(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodPost { + http.Error(w, "POST only", http.StatusMethodNotAllowed) + return + } + var body struct { + ID string `json:"id"` + Node string `json:"node"` + } + if err := json.NewDecoder(r.Body).Decode(&body); err != nil { + http.Error(w, "bad json", http.StatusBadRequest) + return + } + payload, _ := json.Marshal(map[string]any{"op": "stop", "id": body.ID}) + if !s.hub.SendToAgent(body.Node, payload) { + http.Error(w, "agent not reachable", http.StatusServiceUnavailable) + return + } + w.WriteHeader(http.StatusOK) +} + +func newID() string { + b := make([]byte, 8) + _, _ = rand.Read(b) + return hex.EncodeToString(b) +} diff --git a/control-plane/internal/api/sessions.go b/control-plane/internal/api/sessions.go new file mode 100644 index 0000000..441e75a --- /dev/null +++ b/control-plane/internal/api/sessions.go @@ -0,0 +1,45 @@ +package api + +import ( + "crypto/rand" + "encoding/hex" + "sync" + "time" +) + +// Sessions is an in-memory token store. Replace with a signed JWT or +// a Redis-backed store when we need multi-replica CP. For MVP one process +// is enough. +type Sessions struct { + mu sync.RWMutex + store map[string]session +} + +type session struct { + user string + expires time.Time +} + +func NewSessions() *Sessions { + return &Sessions{store: make(map[string]session)} +} + +func (s *Sessions) Issue(user string) string { + b := make([]byte, 16) + _, _ = rand.Read(b) + tok := hex.EncodeToString(b) + s.mu.Lock() + s.store[tok] = session{user: user, expires: time.Now().Add(12 * time.Hour)} + s.mu.Unlock() + return tok +} + +func (s *Sessions) Valid(tok string) bool { + s.mu.RLock() + sess, ok := s.store[tok] + s.mu.RUnlock() + if !ok { + return false + } + return time.Now().Before(sess.expires) +} diff --git a/control-plane/internal/config/config.go b/control-plane/internal/config/config.go new file mode 100644 index 0000000..9e7add5 --- /dev/null +++ b/control-plane/internal/config/config.go @@ -0,0 +1,32 @@ +package config + +import ( + "flag" + "os" +) + +type Config struct { + Addr string // listen addr, e.g. ":8080" + DataDir string // SQLite + .log files live here + AgentHealth string // map of nodeID -> agent base URL (TODO: real map) +} + +// Load reads flags and env. Env wins over defaults; flags win over env. +func Load() Config { + c := Config{ + Addr: envOr("SDP_ADDR", ":8080"), + DataDir: envOr("SDP_DATA", "./data"), + AgentHealth: envOr("SDP_AGENT_HEALTH", ""), + } + flag.StringVar(&c.Addr, "addr", c.Addr, "control plane listen addr") + flag.StringVar(&c.DataDir, "data", c.DataDir, "data directory for sqlite and logs") + flag.Parse() + return c +} + +func envOr(k, def string) string { + if v := os.Getenv(k); v != "" { + return v + } + return def +} diff --git a/control-plane/internal/store/store.go b/control-plane/internal/store/store.go new file mode 100644 index 0000000..1abe677 --- /dev/null +++ b/control-plane/internal/store/store.go @@ -0,0 +1,161 @@ +// Package store persists deployment progress in SQLite and log lines in +// append-only .log files. The hot path is AppendEvent — agents emit a lot +// of these and the dashboard wants them live. +package store + +import ( + "database/sql" + "fmt" + "os" + "path/filepath" + "sync" + "time" + + _ "modernc.org/sqlite" + + "github.com/sdp/protocol" +) + +type Store struct { + db *sql.DB + dir string + logs map[string]*os.File // deploymentID -> file + mu sync.Mutex +} + +func Open(dir string) (*Store, error) { + if err := os.MkdirAll(dir, 0o755); err != nil { + return nil, err + } + if err := os.MkdirAll(filepath.Join(dir, "logs"), 0o755); err != nil { + return nil, err + } + db, err := sql.Open("sqlite3", filepath.Join(dir, "sdp.db")) + if err != nil { + return nil, err + } + if _, err := db.Exec(` + CREATE TABLE IF NOT EXISTS deployments ( + id TEXT PRIMARY KEY, + repository TEXT, + branch TEXT, + user TEXT, + state TEXT, + started_at INTEGER, + completed_at INTEGER + ); + CREATE TABLE IF NOT EXISTS progress ( + deployment_id TEXT, + stage TEXT, + ok INTEGER, + at INTEGER + ); + `); err != nil { + return nil, err + } + return &Store{db: db, dir: dir, logs: make(map[string]*os.File)}, nil +} + +func (s *Store) Close() error { + s.mu.Lock() + defer s.mu.Unlock() + for _, f := range s.logs { + _ = f.Close() + } + return s.db.Close() +} + +// StartDeployment records a new deployment row. Idempotent on id. +func (s *Store) StartDeployment(id, repo, branch, user string) error { + _, err := s.db.Exec( + `INSERT OR IGNORE INTO deployments(id, repository, branch, user, state, started_at) VALUES(?,?,?,?,?,?)`, + id, repo, branch, user, "QUEUED", time.Now().UnixMilli(), + ) + return err +} + +// FinishDeployment marks the final state. completed_at is set if state is terminal. +func (s *Store) FinishDeployment(id, state string) error { + _, err := s.db.Exec( + `UPDATE deployments SET state=?, completed_at=? WHERE id=?`, + state, time.Now().UnixMilli(), id, + ) + return err +} + +// MarkStage records a stage transition. ok=1 success, 0 failure. +func (s *Store) MarkStage(id, stage string, ok bool) error { + v := 0 + if ok { + v = 1 + } + _, err := s.db.Exec( + `INSERT INTO progress(deployment_id, stage, ok, at) VALUES(?,?,?,?)`, + id, stage, v, time.Now().UnixMilli(), + ) + return err +} + +// AppendEvent writes an event. Log lines go to .log; progress/status hit SQLite. +// The deployment's running state is also updated so /api/deployments/{id} can +// serve a snapshot without replaying the whole log. +func (s *Store) AppendEvent(e protocol.Event) error { + switch e.Kind { + case "log": + return s.appendLog(e) + case "status": + _, err := s.db.Exec(`UPDATE deployments SET state=? WHERE id=?`, e.State, e.DeploymentID) + return err + case "progress": + return s.MarkStage(e.DeploymentID, e.Stage, e.State != "FAILED") + } + return nil +} + +func (s *Store) appendLog(e protocol.Event) error { + s.mu.Lock() + f, ok := s.logs[e.DeploymentID] + if !ok { + path := filepath.Join(s.dir, "logs", e.DeploymentID+".log") + var err error + f, err = os.OpenFile(path, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o644) + if err != nil { + s.mu.Unlock() + return err + } + s.logs[e.DeploymentID] = f + } + s.mu.Unlock() + ts := time.UnixMilli(e.At).Format("15:04:05.000") + _, err := fmt.Fprintf(f, "%s %s\n", ts, e.Line) + return err +} + +// TailLogs returns the last n lines of a deployment's log file. Used by the +// dashboard on first connect to backfill. +func (s *Store) TailLogs(id string, n int) ([]string, error) { + path := filepath.Join(s.dir, "logs", id+".log") + data, err := os.ReadFile(path) + if err != nil { + if os.IsNotExist(err) { + return nil, nil + } + return nil, err + } + // ponytail: O(n) scan, fine for tail use. swap to a ring buffer if logs get huge. + var lines []string + start := 0 + for i, b := range data { + if b == '\n' { + lines = append(lines, string(data[start:i])) + start = i + 1 + } + } + if start < len(data) { + lines = append(lines, string(data[start:])) + } + if n > 0 && len(lines) > n { + lines = lines[len(lines)-n:] + } + return lines, nil +} diff --git a/control-plane/internal/ws/handlers.go b/control-plane/internal/ws/handlers.go new file mode 100644 index 0000000..d0e34f0 --- /dev/null +++ b/control-plane/internal/ws/handlers.go @@ -0,0 +1,111 @@ +package ws + +import ( + "encoding/json" + "log" + "net/http" + "strings" + + "github.com/sdp/protocol" + + "github.com/sdp/control-plane/internal/store" +) + +// AgentWS handles /ws/agent?node=. The agent pushes events; the control +// plane can send requests back over the same socket. +func (h *Hub) AgentWS(st *store.Store, onConnect, onDisconnect func(nodeID string)) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + nodeID := r.URL.Query().Get("node") + if nodeID == "" { + http.Error(w, "missing node query", http.StatusBadRequest) + return + } + conn, err := upgrader.Upgrade(w, r, nil) + if err != nil { + return + } + defer conn.Close() + + // outbound to agent: buffered; if the agent stalls we drop rather than block. + out := make(chan []byte, 32) + h.RegisterAgent(nodeID, out) + defer func() { + h.UnregisterAgent(nodeID) + close(out) // unblock the writer goroutine + }() + if onConnect != nil { + onConnect(nodeID) + } + defer func() { + if onDisconnect != nil { + onDisconnect(nodeID) + } + }() + + // writer goroutine: drains out to the agent + go func() { + for msg := range out { + if err := conn.WriteMessage(websocket.TextMessage, msg); err != nil { + return + } + } + }() + + // reader loop: agent -> control plane + for { + _, raw, err := conn.ReadMessage() + if err != nil { + return + } + var e protocol.Event + if err := json.Unmarshal(raw, &e); err != nil { + log.Printf("agent %s: bad event: %v", nodeID, err) + continue + } + _ = st.AppendEvent(e) // ponytail: best-effort persist + h.Publish(e) + } + } +} + +// DeploymentWS handles /ws/deployments/{id}. Dashboard subscribes; we send +// a tail of the existing log first, then live events. +func (h *Hub) DeploymentWS(st *store.Store) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + id := strings.TrimPrefix(r.URL.Path, "/ws/deployments/") + if id == "" { + http.Error(w, "missing deployment id", http.StatusBadRequest) + return + } + conn, err := upgrader.Upgrade(w, r, nil) + if err != nil { + return + } + defer conn.Close() + + // backfill: send last N log lines as synthetic events. + lines, _ := st.TailLogs(id, 200) + for _, ln := range lines { + msg := map[string]any{ + "deploymentId": id, + "kind": "log", + "line": ln, + "at": int64(0), + "backfill": true, + } + b, _ := json.Marshal(msg) + if err := conn.WriteMessage(websocket.TextMessage, b); err != nil { + return + } + } + + ch, unsub := h.Subscribe(id) + defer unsub() + for e := range ch { + b, _ := json.Marshal(e) + if err := conn.WriteMessage(websocket.TextMessage, b); err != nil { + return + } + } + } +} diff --git a/control-plane/internal/ws/hub.go b/control-plane/internal/ws/hub.go new file mode 100644 index 0000000..e8c0ef5 --- /dev/null +++ b/control-plane/internal/ws/hub.go @@ -0,0 +1,112 @@ +// Package ws is the WebSocket fan-out for SDP. Two flows: +// +// agent --(events)--> /ws/agent (one conn per agent) +// dashboard client --(subscribe)-> /ws/deployments/{id} (one conn per viewer) +// +// On agent connect we record the agent's nodeID. On dashboard connect we +// register a subscriber for one deployment. Events are best-effort fanned out; +// a slow client is dropped, not allowed to backpressure the agent. +package ws + +import ( + "net/http" + "sync" + + "github.com/gorilla/websocket" + + "github.com/sdp/protocol" +) + +var upgrader = websocket.Upgrader{ + CheckOrigin: func(r *http.Request) bool { return true }, // ponytail: internal tool, allow all +} + +type Hub struct { + mu sync.RWMutex + + // one channel per deployment; nil = no subscribers. + subs map[string]map[chan protocol.Event]struct{} + + // one channel per connected agent; keyed by nodeID. + agents map[string]chan<- []byte // outbound to agent (deploy requests, etc.) +} + +func New() *Hub { + return &Hub{ + subs: make(map[string]map[chan protocol.Event]struct{}), + agents: make(map[string]chan<- []byte), + } +} + +// Publish fans an event out to all subscribers of that deployment. +// Non-blocking; drops the event for a subscriber if its buffer is full. +func (h *Hub) Publish(e protocol.Event) { + h.mu.RLock() + subs := h.subs[e.DeploymentID] + chans := make([]chan protocol.Event, 0, len(subs)) + for c := range subs { + chans = append(chans, c) + } + h.mu.RUnlock() + for _, c := range chans { + select { + case c <- e: + default: + // ponytail: drop slow subscriber; they'll reconnect. + } + } +} + +// Subscribe registers a subscriber channel for one deployment. +// Returns an unsubscribe func the caller must invoke. +func (h *Hub) Subscribe(deploymentID string) (chan protocol.Event, func()) { + ch := make(chan protocol.Event, 64) + h.mu.Lock() + if h.subs[deploymentID] == nil { + h.subs[deploymentID] = make(map[chan protocol.Event]struct{}) + } + h.subs[deploymentID][ch] = struct{}{} + h.mu.Unlock() + return ch, func() { + h.mu.Lock() + if subs, ok := h.subs[deploymentID]; ok { + delete(subs, ch) + if len(subs) == 0 { + delete(h.subs, deploymentID) + } + } + h.mu.Unlock() + close(ch) + } +} + +// RegisterAgent stores the outbound channel for a node. Used when the control +// plane wants to send something back to an agent (deploy requests, stop, etc.). +func (h *Hub) RegisterAgent(nodeID string, out chan<- []byte) { + h.mu.Lock() + h.agents[nodeID] = out + h.mu.Unlock() +} + +func (h *Hub) UnregisterAgent(nodeID string) { + h.mu.Lock() + delete(h.agents, nodeID) + h.mu.Unlock() +} + +// SendToAgent best-effort sends a payload to a connected agent. Returns false +// if the agent isn't connected or its buffer is full. +func (h *Hub) SendToAgent(nodeID string, payload []byte) bool { + h.mu.RLock() + ch, ok := h.agents[nodeID] + h.mu.RUnlock() + if !ok { + return false + } + select { + case ch <- payload: + return true + default: + return false + } +} diff --git a/dashboard/next.config.js b/dashboard/next.config.js new file mode 100644 index 0000000..506e676 --- /dev/null +++ b/dashboard/next.config.js @@ -0,0 +1,10 @@ +/** @type {import('next').NextConfig} */ +const nextConfig = { + // ponytail: pure static export. nginx serves the dashboard with + // try_files and proxies /api/* and /ws/* to the Go control plane. + // The browser does plain fetch() to /api/*; no NextJS BFF layer. + output: 'export', + reactStrictMode: true, + images: { unoptimized: true }, +} +module.exports = nextConfig diff --git a/dashboard/package.json b/dashboard/package.json new file mode 100644 index 0000000..ff755d2 --- /dev/null +++ b/dashboard/package.json @@ -0,0 +1,35 @@ +{ + "name": "sdp-dashboard", + "version": "0.1.0", + "private": true, + "scripts": { + "dev": "next dev", + "build": "next build", + "start": "next start", + "lint": "next lint" + }, + "dependencies": { + "@radix-ui/react-dialog": "^1.0.5", + "@radix-ui/react-label": "^2.0.2", + "@radix-ui/react-select": "^2.0.0", + "@radix-ui/react-slot": "^1.0.2", + "@radix-ui/react-tabs": "^1.0.4", + "class-variance-authority": "^0.7.0", + "clsx": "^2.1.0", + "lucide-react": "^0.344.0", + "next": "14.1.0", + "react": "^18", + "react-dom": "^18", + "tailwind-merge": "^2.2.1" + }, + "devDependencies": { + "@types/node": "^20", + "@types/react": "^18", + "@types/react-dom": "^18", + "autoprefixer": "^10.4.17", + "postcss": "^8.4.33", + "tailwindcss": "^3.4.1", + "tailwindcss-animate": "^1.0.7", + "typescript": "^5" + } +} diff --git a/dashboard/postcss.config.js b/dashboard/postcss.config.js new file mode 100644 index 0000000..95aa892 --- /dev/null +++ b/dashboard/postcss.config.js @@ -0,0 +1,3 @@ +module.exports = { + plugins: { tailwindcss: {}, autoprefixer: {} }, +} diff --git a/dashboard/src/app/dashboard/page.tsx b/dashboard/src/app/dashboard/page.tsx new file mode 100644 index 0000000..028a858 --- /dev/null +++ b/dashboard/src/app/dashboard/page.tsx @@ -0,0 +1,9 @@ +import { Dashboard } from '@/components/dashboard' + +export default function DashboardPage() { + return ( +
+ +
+ ) +} diff --git a/dashboard/src/app/globals.css b/dashboard/src/app/globals.css new file mode 100644 index 0000000..7465f13 --- /dev/null +++ b/dashboard/src/app/globals.css @@ -0,0 +1,54 @@ +@tailwind base; +@tailwind components; +@tailwind utilities; + +@layer base { + :root { + --background: 0 0% 100%; + --foreground: 222.2 47.4% 11.2%; + --card: 0 0% 100%; + --card-foreground: 222.2 47.4% 11.2%; + --popover: 0 0% 100%; + --popover-foreground: 222.2 47.4% 11.2%; + --primary: 222.2 47.4% 11.2%; + --primary-foreground: 210 40% 98%; + --secondary: 210 40% 96.1%; + --secondary-foreground: 222.2 47.4% 11.2%; + --muted: 210 40% 96.1%; + --muted-foreground: 215.4 16.3% 46.9%; + --accent: 210 40% 96.1%; + --accent-foreground: 222.2 47.4% 11.2%; + --destructive: 0 84.2% 60.2%; + --destructive-foreground: 210 40% 98%; + --border: 214.3 31.8% 91.4%; + --input: 214.3 31.8% 91.4%; + --ring: 222.2 84% 4.9%; + --radius: 0.5rem; + } + .dark { + --background: 222.2 84% 4.9%; + --foreground: 210 40% 98%; + --card: 222.2 84% 4.9%; + --card-foreground: 210 40% 98%; + --popover: 222.2 84% 4.9%; + --popover-foreground: 210 40% 98%; + --primary: 210 40% 98%; + --primary-foreground: 222.2 47.4% 11.2%; + --secondary: 217.2 32.6% 17.5%; + --secondary-foreground: 210 40% 98%; + --muted: 217.2 32.6% 17.5%; + --muted-foreground: 215 20.2% 65.1%; + --accent: 217.2 32.6% 17.5%; + --accent-foreground: 210 40% 98%; + --destructive: 0 62.8% 30.6%; + --destructive-foreground: 210 40% 98%; + --border: 217.2 32.6% 17.5%; + --input: 217.2 32.6% 17.5%; + --ring: 212.7 26.8% 83.9%; + } +} + +@layer base { + * { @apply border-border; } + body { @apply bg-background text-foreground; font-feature-settings: "rlig" 1, "calt" 1; } +} diff --git a/dashboard/src/app/layout.tsx b/dashboard/src/app/layout.tsx new file mode 100644 index 0000000..39c172b --- /dev/null +++ b/dashboard/src/app/layout.tsx @@ -0,0 +1,15 @@ +import type { Metadata } from 'next' +import './globals.css' + +export const metadata: Metadata = { + title: 'SDP', + description: 'Sandbox Deployment Platform', +} + +export default function RootLayout({ children }: { children: React.ReactNode }) { + return ( + + {children} + + ) +} diff --git a/dashboard/src/app/page.tsx b/dashboard/src/app/page.tsx new file mode 100644 index 0000000..550b285 --- /dev/null +++ b/dashboard/src/app/page.tsx @@ -0,0 +1,9 @@ +import { LoginForm } from '@/components/login-form' + +export default function LoginPage() { + return ( +
+ +
+ ) +} diff --git a/dashboard/src/components/dashboard.tsx b/dashboard/src/components/dashboard.tsx new file mode 100644 index 0000000..c5b6e36 --- /dev/null +++ b/dashboard/src/components/dashboard.tsx @@ -0,0 +1,180 @@ +'use client' +import { useEffect, useState } from 'react' +import { Button } from '@/components/ui/button' +import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/components/ui/card' +import { Badge } from '@/components/ui/badge' +import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from '@/components/ui/select' +import { Label } from '@/components/ui/label' +import { Input } from '@/components/ui/input' +import { listRepos, listBranches, startDeploy, type Repo } from '@/lib/api' +import { useDeploymentWS } from '@/lib/use-deployment-ws' + +const STAGES = ['git fetch', 'git checkout', 'git pull', 'go build', 'start container'] + +export function Dashboard() { + const [repos, setRepos] = useState([]) + const [repo, setRepo] = useState('') + const [branches, setBranches] = useState([]) + const [branch, setBranch] = useState('') + const [username, setUsername] = useState('') + const [password, setPassword] = useState('') + const [deploymentId, setDeploymentId] = useState(null) + const [error, setError] = useState(null) + const [deploying, setDeploying] = useState(false) + + const { events, state } = useDeploymentWS(deploymentId) + + useEffect(() => { + listRepos().then((r) => { + setRepos(r) + if (r.length) setRepo(r[0].name) + }).catch((e) => setError(String(e))) + }, []) + + useEffect(() => { + if (!repo) return + listBranches(repo).then((b) => { + setBranches(b) + if (b.length) setBranch(b[0]) + }).catch(() => setBranches([])) + }, [repo]) + + async function deploy() { + setError(null) + setDeploying(true) + try { + const { id } = await startDeploy({ repository: repo, branch, username, password }) + setDeploymentId(id) + } catch (e) { + setError(String(e)) + } finally { + setDeploying(false) + } + } + + const stageDone: Record = {} + for (const s of STAGES) stageDone[s] = 'pending' + let lastDone: string | null = null + for (const e of events) { + if (e.kind === 'progress' && e.stage && STAGES.includes(e.stage)) { + if (e.state === 'FAILED') stageDone[e.stage] = 'failed' + else stageDone[e.stage] = 'ok' + lastDone = e.stage + } + } + if (lastDone) { + const idx = STAGES.indexOf(lastDone) + if (idx >= 0 && idx + 1 < STAGES.length && stageDone[STAGES[idx + 1]] === 'pending') { + stageDone[STAGES[idx + 1]] = 'in_progress' + } + } + + return ( +
+
+
+

Sandbox Deployments

+

Isolated feature-branch deployments for Backend and QA.

+
+
+ +
+ + + Deploy a branch + Credentials are forwarded to the agent for this deploy only. + + +
+ + +
+
+ + +
+
+ + setUsername(e.target.value)} autoComplete="username" /> +
+
+ + setPassword(e.target.value)} autoComplete="current-password" /> +
+ {error &&

{error}

} + + + + +
+ {deploymentId && ( + + +
+
+ {deploymentId} + {repo} · {branch} +
+ + {state} + +
+ + +
    + {STAGES.map((s) => { + const v = stageDone[s] + return ( +
  1. + + {v === 'ok' ? '✓' : v === 'failed' ? '✗' : v === 'in_progress' ? '…' : '·'} + + {s} +
    2. + ) + })} +
+ + + + )} +
+
+
+ ) +} + +function LogView({ events }: { events: { kind: string; line?: string; stage?: string; backfill?: boolean }[] }) { + // ponytail: render the last 500 lines. Auto-scroll on new log. + return ( + 
+      {events
+        .filter((e) => e.kind === 'log' && e.line)
+        .slice(-500)
+        .map((e, i) => (
+          
{e.line}

+        ))}
+
+ ) +} diff --git a/dashboard/src/components/login-form.tsx b/dashboard/src/components/login-form.tsx new file mode 100644 index 0000000..b3ae35b --- /dev/null +++ b/dashboard/src/components/login-form.tsx @@ -0,0 +1,63 @@ +'use client' +import { useState } from 'react' +import { useRouter } from 'next/navigation' +import { Button } from '@/components/ui/button' +import { Input } from '@/components/ui/input' +import { Label } from '@/components/ui/label' +import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/components/ui/card' + +export function LoginForm() { + const [username, setUsername] = useState('') + const [password, setPassword] = useState('') + const [error, setError] = useState(null) + const [loading, setLoading] = useState(false) + const router = useRouter() + + async function submit(e: React.FormEvent) { + e.preventDefault() + setLoading(true) + setError(null) + try { + const r = await fetch('/api/login', { + method: 'POST', + headers: { 'content-type': 'application/json' }, + credentials: 'include', + body: JSON.stringify({ username, password }), + }) + if (!r.ok) { + setError('Login failed — check your Bitbucket credentials.') + return + } + router.push('/dashboard') + } catch (err) { + setError('Network error') + } finally { + setLoading(false) + } + } + + return ( + + + Sign in + Use your Bitbucket account. + + +
+
+ + setUsername(e.target.value)} required autoComplete="username" /> +
+
+ + setPassword(e.target.value)} required autoComplete="current-password" /> +
+ {error &&

{error}

} + +
+ + + ) +} diff --git a/dashboard/src/components/ui/badge.tsx b/dashboard/src/components/ui/badge.tsx new file mode 100644 index 0000000..b6e79c4 --- /dev/null +++ b/dashboard/src/components/ui/badge.tsx @@ -0,0 +1,28 @@ +import * as React from 'react' +import { cva, type VariantProps } from 'class-variance-authority' +import { cn } from '@/lib/utils' + +const badgeVariants = cva( + 'inline-flex items-center rounded-md border px-2.5 py-0.5 text-xs font-semibold transition-colors', + { + variants: { + variant: { + default: 'border-transparent bg-primary text-primary-foreground shadow', + secondary: 'border-transparent bg-secondary text-secondary-foreground', + destructive: 'border-transparent bg-destructive text-destructive-foreground shadow', + outline: 'text-foreground', + success: 'border-transparent bg-emerald-500 text-white shadow', + warning: 'border-transparent bg-amber-500 text-white shadow', + }, + }, + defaultVariants: { variant: 'default' }, + } +) + +export interface BadgeProps extends React.HTMLAttributes, VariantProps {} + +function Badge({ className, variant, ...props }: BadgeProps) { + return
+} + +export { Badge, badgeVariants } diff --git a/dashboard/src/components/ui/button.tsx b/dashboard/src/components/ui/button.tsx new file mode 100644 index 0000000..b28e0b8 --- /dev/null +++ b/dashboard/src/components/ui/button.tsx @@ -0,0 +1,43 @@ +import * as React from 'react' +import { Slot } from '@radix-ui/react-slot' +import { cva, type VariantProps } from 'class-variance-authority' +import { cn } from '@/lib/utils' + +const buttonVariants = cva( + 'inline-flex items-center justify-center whitespace-nowrap rounded-md text-sm font-medium transition-colors focus-visible:outline-none focus-visible:ring-1 focus-visible:ring-ring disabled:pointer-events-none disabled:opacity-50', + { + variants: { + variant: { + default: 'bg-primary text-primary-foreground shadow hover:bg-primary/90', + destructive: 'bg-destructive text-destructive-foreground shadow-sm hover:bg-destructive/90', + outline: 'border border-input bg-background shadow-sm hover:bg-accent hover:text-accent-foreground', + secondary: 'bg-secondary text-secondary-foreground shadow-sm hover:bg-secondary/80', + ghost: 'hover:bg-accent hover:text-accent-foreground', + link: 'text-primary underline-offset-4 hover:underline', + }, + size: { + default: 'h-9 px-4 py-2', + sm: 'h-8 rounded-md px-3 text-xs', + lg: 'h-10 rounded-md px-8', + icon: 'h-9 w-9', + }, + }, + defaultVariants: { variant: 'default', size: 'default' }, + } +) + +export interface ButtonProps + extends React.ButtonHTMLAttributes, + VariantProps { + asChild?: boolean +} + +const Button = React.forwardRef( + ({ className, variant, size, asChild = false, ...props }, ref) => { + const Comp = asChild ? Slot : 'button' + return + } +) +Button.displayName = 'Button' + +export { Button, buttonVariants } diff --git a/dashboard/src/components/ui/card.tsx b/dashboard/src/components/ui/card.tsx new file mode 100644 index 0000000..c61cdcc --- /dev/null +++ b/dashboard/src/components/ui/card.tsx @@ -0,0 +1,40 @@ +import * as React from 'react' +import { cn } from '@/lib/utils' + +const Card = React.forwardRef>( + ({ className, ...props }, ref) => ( +
+ ) +) +Card.displayName = 'Card' + +const CardHeader = React.forwardRef>( + ({ className, ...props }, ref) =>
+) +CardHeader.displayName = 'CardHeader' + +const CardTitle = React.forwardRef>( + ({ className, ...props }, ref) => ( +
+ ) +) +CardTitle.displayName = 'CardTitle' + +const CardDescription = React.forwardRef>( + ({ className, ...props }, ref) => ( +
+ ) +) +CardDescription.displayName = 'CardDescription' + +const CardContent = React.forwardRef>( + ({ className, ...props }, ref) =>
+) +CardContent.displayName = 'CardContent' + +const CardFooter = React.forwardRef>( + ({ className, ...props }, ref) =>
+) +CardFooter.displayName = 'CardFooter' + +export { Card, CardHeader, CardTitle, CardDescription, CardContent, CardFooter } diff --git a/dashboard/src/components/ui/input.tsx b/dashboard/src/components/ui/input.tsx new file mode 100644 index 0000000..37a6c8b --- /dev/null +++ b/dashboard/src/components/ui/input.tsx @@ -0,0 +1,21 @@ +import * as React from 'react' +import { cn } from '@/lib/utils' + +export type InputProps = React.InputHTMLAttributes + +const Input = React.forwardRef(({ className, type, ...props }, ref) => { + return ( + + ) +}) +Input.displayName = 'Input' + +export { Input } diff --git a/dashboard/src/components/ui/label.tsx b/dashboard/src/components/ui/label.tsx new file mode 100644 index 0000000..e56a61c --- /dev/null +++ b/dashboard/src/components/ui/label.tsx @@ -0,0 +1,17 @@ +import * as React from 'react' +import * as LabelPrimitive from '@radix-ui/react-label' +import { cn } from '@/lib/utils' + +const Label = React.forwardRef< + React.ElementRef, + React.ComponentPropsWithoutRef +>(({ className, ...props }, ref) => ( + +)) +Label.displayName = LabelPrimitive.Root.displayName + +export { Label } diff --git a/dashboard/src/components/ui/select.tsx b/dashboard/src/components/ui/select.tsx new file mode 100644 index 0000000..5a6b57a --- /dev/null +++ b/dashboard/src/components/ui/select.tsx @@ -0,0 +1,78 @@ +'use client' +import * as React from 'react' +import * as SelectPrimitive from '@radix-ui/react-select' +import { Check, ChevronDown } from 'lucide-react' +import { cn } from '@/lib/utils' + +const Select = SelectPrimitive.Root +const SelectValue = SelectPrimitive.Value +const SelectGroup = SelectPrimitive.Group + +const SelectTrigger = React.forwardRef< + React.ElementRef, + React.ComponentPropsWithoutRef +>(({ className, children, ...props }, ref) => ( + span]:line-clamp-1', + className + )} + {...props} + > + {children} + + + + +)) +SelectTrigger.displayName = SelectPrimitive.Trigger.displayName + +const SelectContent = React.forwardRef< + React.ElementRef, + React.ComponentPropsWithoutRef +>(({ className, children, position = 'popper', ...props }, ref) => ( + + + + {children} + + + +)) +SelectContent.displayName = SelectPrimitive.Content.displayName + +const SelectItem = React.forwardRef< + React.ElementRef, + React.ComponentPropsWithoutRef +>(({ className, children, ...props }, ref) => ( + + + + + + + {children} + +)) +SelectItem.displayName = SelectPrimitive.Item.displayName + +export { Select, SelectGroup, SelectValue, SelectTrigger, SelectContent, SelectItem } diff --git a/dashboard/src/lib/api.ts b/dashboard/src/lib/api.ts new file mode 100644 index 0000000..2e2619a --- /dev/null +++ b/dashboard/src/lib/api.ts @@ -0,0 +1,35 @@ +// lib/api.ts — shared types and fetch helpers +export type Repo = { name: string; node: string; path: string } + +export async function listRepos(): Promise { + const r = await fetch('/api/repos', { credentials: 'include' }) + if (!r.ok) throw new Error('failed to list repos') + return r.json() +} + +export async function listBranches(repo: string): Promise { + const r = await fetch(`/api/repos/branches?repo=${encodeURIComponent(repo)}`, { credentials: 'include' }) + if (!r.ok) throw new Error('failed to list branches') + return r.json() +} + +export type DeployResponse = { id: string } + +export async function startDeploy(payload: { + repository: string + branch: string + username: string + password: string +}): Promise { + const r = await fetch('/api/deployments', { + method: 'POST', + headers: { 'content-type': 'application/json' }, + credentials: 'include', + body: JSON.stringify(payload), + }) + if (!r.ok) { + const text = await r.text() + throw new Error(text || 'deploy failed') + } + return r.json() +} diff --git a/dashboard/src/lib/use-deployment-ws.ts b/dashboard/src/lib/use-deployment-ws.ts new file mode 100644 index 0000000..537aed8 --- /dev/null +++ b/dashboard/src/lib/use-deployment-ws.ts @@ -0,0 +1,47 @@ +'use client' +// Single WebSocket hook. One connection per active deployment; auto-reconnect +// on close. Pushed events accumulate into the caller-provided state. +import { useEffect, useRef, useState } from 'react' + +export type WsEvent = { + deploymentId: string + kind: 'progress' | 'log' | 'status' + state?: string + stage?: string + line?: string + at: number + backfill?: boolean +} + +export function useDeploymentWS(deploymentId: string | null) { + const [events, setEvents] = useState([]) + const [state, setState] = useState('QUEUED') + const wsRef = useRef(null) + + useEffect(() => { + if (!deploymentId) return + setEvents([]) + setState('QUEUED') + + const proto = window.location.protocol === 'https:' ? 'wss' : 'ws' + const url = `${proto}://${window.location.host}/ws/deployments/${deploymentId}` + const ws = new WebSocket(url) + wsRef.current = ws + + ws.onmessage = (ev) => { + try { + const e: WsEvent = JSON.parse(ev.data) + if (e.kind === 'status' && e.state) setState(e.state) + if (e.kind === 'log' || e.kind === 'progress' || e.kind === 'status') { + setEvents((prev) => [...prev, e].slice(-2000)) // ponytail: cap memory + } + } catch { + // ponytail: drop malformed frames; would-be 3am pain otherwise + } + } + // ponytail: no onclose handler — just let it die. Refresh = resubscribe. + return () => ws.close() + }, [deploymentId]) + + return { events, state } +} diff --git a/dashboard/src/lib/utils.ts b/dashboard/src/lib/utils.ts new file mode 100644 index 0000000..fed2fe9 --- /dev/null +++ b/dashboard/src/lib/utils.ts @@ -0,0 +1,6 @@ +import { clsx, type ClassValue } from 'clsx' +import { twMerge } from 'tailwind-merge' + +export function cn(...inputs: ClassValue[]) { + return twMerge(clsx(inputs)) +} diff --git a/dashboard/tailwind.config.js b/dashboard/tailwind.config.js new file mode 100644 index 0000000..944484a --- /dev/null +++ b/dashboard/tailwind.config.js @@ -0,0 +1,30 @@ +/** @type {import('tailwindcss').Config} */ +module.exports = { + darkMode: ['class'], + content: ['./src/**/*.{ts,tsx}'], + theme: { + container: { + center: true, + padding: '1rem', + screens: { '2xl': '1280px' }, + }, + extend: { + colors: { + border: 'hsl(var(--border))', + input: 'hsl(var(--input))', + ring: 'hsl(var(--ring))', + background: 'hsl(var(--background))', + foreground: 'hsl(var(--foreground))', + primary: { DEFAULT: 'hsl(var(--primary))', foreground: 'hsl(var(--primary-foreground))' }, + secondary: { DEFAULT: 'hsl(var(--secondary))', foreground: 'hsl(var(--secondary-foreground))' }, + muted: { DEFAULT: 'hsl(var(--muted))', foreground: 'hsl(var(--muted-foreground))' }, + accent: { DEFAULT: 'hsl(var(--accent))', foreground: 'hsl(var(--accent-foreground))' }, + destructive: { DEFAULT: 'hsl(var(--destructive))', foreground: 'hsl(var(--destructive-foreground))' }, + card: { DEFAULT: 'hsl(var(--card))', foreground: 'hsl(var(--card-foreground))' }, + popover: { DEFAULT: 'hsl(var(--popover))', foreground: 'hsl(var(--popover-foreground))' }, + }, + borderRadius: { lg: 'var(--radius)', md: 'calc(var(--radius) - 2px)', sm: 'calc(var(--radius) - 4px)' }, + }, + }, + plugins: [require('tailwindcss-animate')], +} diff --git a/dashboard/tsconfig.json b/dashboard/tsconfig.json new file mode 100644 index 0000000..d8eb865 --- /dev/null +++ b/dashboard/tsconfig.json @@ -0,0 +1,22 @@ +{ + "compilerOptions": { + "target": "ES2020", + "lib": ["dom", "dom.iterable", "esnext"], + "allowJs": true, + "skipLibCheck": true, + "strict": true, + "noEmit": true, + "esModuleInterop": true, + "module": "esnext", + "moduleResolution": "bundler", + "resolveJsonModule": true, + "isolatedModules": true, + "jsx": "preserve", + "incremental": true, + "plugins": [{ "name": "next" }], + "baseUrl": ".", + "paths": { "@/*": ["./src/*"] } + }, + "include": ["next-env.d.ts", "**/*.ts", "**/*.tsx", ".next/types/**/*.ts"], + "exclude": ["node_modules"] +} diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..a0ab206 --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,62 @@ +# docker-compose.yml — runs control-plane, agent-micro, agent-gateway +# inside alpine:latest containers. The agents need the host docker socket +# bind-mounted so they can manage service containers. +# +# Use on a single host (e.g. 186) for dev. In production, the agents +# live on their own VMs and the control plane lives on 186. + +services: + control-plane: + image: alpine:latest + container_name: sdp-control-plane + restart: unless-stopped + command: ["/SDP/bin/control-plane", "-addr", ":8080", "-data", "/SDP/data"] + volumes: + - ./bin/control-plane:/SDP/bin/control-plane:ro + - sdp-data:/SDP/data + ports: + - "8080:8080" + + agent-micro: + image: alpine:latest + container_name: sdp-agent-micro + restart: unless-stopped + # agent connects to the control plane by container name on the compose + # network. Override SDP_CP_URL when running outside compose. + command: + - /SDP/bin/agent-micro + - -node + - micro + - -cp + - ws://control-plane:8080/ws/agent + volumes: + - ./bin/agent-micro:/SDP/bin/agent-micro:ro + - /var/run/docker.sock:/var/run/docker.sock + # Repos live on the host. Adjust to the actual paths. + - ~/AppGolang:/AppGolang:ro + environment: + - DOCKER_HOST=unix:///var/run/docker.sock + depends_on: + - control-plane + + agent-gateway: + image: alpine:latest + container_name: sdp-agent-gateway + restart: unless-stopped + command: + - /SDP/bin/agent-gateway + - -node + - gateway + - -cp + - ws://control-plane:8080/ws/agent + volumes: + - ./bin/agent-gateway:/SDP/bin/agent-gateway:ro + - /var/run/docker.sock:/var/run/docker.sock + - ~/SDP/repos:/SDP-repos:ro + environment: + - DOCKER_HOST=unix:///var/run/docker.sock + depends_on: + - control-plane + +volumes: + sdp-data: diff --git a/go.work b/go.work new file mode 100644 index 0000000..c0aa530 --- /dev/null +++ b/go.work @@ -0,0 +1,8 @@ +go 1.23 + +use ( + ./protocol + ./control-plane + ./agent-micro + ./agent-gateway +) diff --git a/nginx/nginx.conf b/nginx/nginx.conf new file mode 100644 index 0000000..a39676d --- /dev/null +++ b/nginx/nginx.conf @@ -0,0 +1,51 @@ +# SDP nginx — serves the static NextJS export and proxies API + WS +# to the Go control plane. +# +# try_files: any unknown path falls back to /index.html so client-side +# routing works. /api and /ws are matched first and proxied upstream. + +upstream control_plane { + server 127.0.0.1:8080; + keepalive 16; +} + +server { + listen 80; + server_name _; + + # Long-lived WS connections need a generous read timeout. + proxy_read_timeout 3600s; + proxy_send_timeout 3600s; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + + # --- API: control plane --- + location /api/ { + proxy_pass http://control_plane; + } + + # --- WebSocket: agent + dashboard subscriptions --- + location /ws/ { + proxy_pass http://control_plane; + } + + # --- Static dashboard --- + root /var/www/sdp/dashboard/out; + index index.html; + + # ponytail: try_files does all the work. _next chunks, images, etc. are + # served as files; unknown paths fall back to /index.html for SPA routing. + location / { + try_files $uri $uri/ $uri.html /index.html; + } + + # Cache static assets aggressively; never cache index.html. + location /_next/static/ { + expires 1y; + add_header Cache-Control "public, immutable"; + } +} diff --git a/protocol/go.mod b/protocol/go.mod new file mode 100644 index 0000000..a4da26a --- /dev/null +++ b/protocol/go.mod @@ -0,0 +1,3 @@ +module github.com/sdp/protocol + +go 1.23 diff --git a/protocol/types.go b/protocol/types.go new file mode 100644 index 0000000..0f45a9b --- /dev/null +++ b/protocol/types.go @@ -0,0 +1,35 @@ +// Package protocol defines the wire format shared between the control plane +// and agents. Keep this small — anything that goes over HTTP or WebSocket +// between us and an agent lives here. +package protocol + +// Event is what an agent streams back to the control plane over its outbound +// WebSocket. The control plane fans these out to dashboard clients and +// persists them (log lines to .log, progress snapshots to SQLite). +type Event struct { + DeploymentID string `json:"deploymentId"` + Kind string `json:"kind"` // progress | log | status + State string `json:"state,omitempty"` // QUEUED, FETCHING, ... RUNNING, FAILED, STOPPED + Stage string `json:"stage,omitempty"` // human label, e.g. "git fetch" + Line string `json:"line,omitempty"` // log line (for kind=log) + ContainerID string `json:"containerId,omitempty"` + At int64 `json:"at"` // unix millis +} + +// DeployRequest is what the control plane POSTs to an agent to start work. +// Credentials are passed per-operation; agents MUST NOT log or persist them. +type DeployRequest struct { + DeploymentID string `json:"deploymentId"` + Repository string `json:"repository"` // name from agent's repo config + Branch string `json:"branch"` + Env map[string]string `json:"env,omitempty"` // injected into container + Username string `json:"username"` + Password string `json:"password"` +} + +// DeployResponse is the agent's immediate ack to a DeployRequest. +// Actual progress streams over the WS. +type DeployResponse struct { + OK bool `json:"ok"` + Error string `json:"error,omitempty"` +} diff --git a/scripts/build.sh b/scripts/build.sh new file mode 100755 index 0000000..05b97e1 --- /dev/null +++ b/scripts/build.sh @@ -0,0 +1,68 @@ +#!/usr/bin/env bash +# Build all three Go binaries for Linux/amd64 and the dashboard. +# Output goes to ./bin/ and ./dashboard/out/. +# +# Uses a golang:1.23-alpine container so we get a reproducible toolchain +# without needing Go installed locally. Cross-compile via GOOS/GOARCH + +# CGO_ENABLED=0 — produces a static binary that runs in the alpine +# containers defined in docker-compose.yml. + +set -euo pipefail + +cd "$(dirname "$0")/.." +REPO_ROOT="$(pwd)" + +OUT="$REPO_ROOT/bin" +mkdir -p "$OUT" + +GO_IMAGE="${GO_IMAGE:-golang:1.23-alpine}" + +# ponytail: bind-mount a persistent gocache so module downloads + build cache +# survive across runs. Otherwise every build re-downloads the world from +# the GOPROXY — slow on a flaky office link, and uses up the proxy quota. +GOCACHE_VOL="sdp-gocache" +docker volume create "$GOCACHE_VOL" >/dev/null 2>&1 || true + +echo "==> building control-plane, agent-micro, agent-gateway (linux/amd64)" +docker run --rm \ + -v "$REPO_ROOT":/src \ + -v "$OUT":/out \ + -v "$GOCACHE_VOL":/gocache \ + -w /src \ + -e CGO_ENABLED=0 \ + -e GOOS=linux \ + -e GOARCH=amd64 \ + -e GOCACHE=/gocache \ + -e GOFLAGS="-mod=mod" \ + "$GO_IMAGE" \ + sh -c ' + set -e + # -trimpath: strip absolute paths from the binary (reproducible builds). + # -ldflags="-s -w": drop symbol table + DWARF, smaller binary. + go build -trimpath -ldflags="-s -w" -o /out/control-plane ./control-plane/cmd/control-plane + go build -trimpath -ldflags="-s -w" -o /out/agent-micro ./agent-micro/cmd/agent-micro + go build -trimpath -ldflags="-s -w" -o /out/agent-gateway ./agent-gateway/cmd/agent-gateway + ' + +echo +echo "==> binaries:" +ls -lh "$OUT" +chmod +x "$OUT"/* + +# Verify the binaries are actually linux/amd64. ponytail: catches a mistake +# where someone removes the GOOS/GOARCH env and ships a darwin binary to +# the alpine container. +echo +echo "==> sanity check (file type):" +file "$OUT"/* + +# ---- dashboard ---- +if [[ -d "$REPO_ROOT/dashboard" ]]; then + echo + echo "==> building dashboard" + if [[ ! -d "$REPO_ROOT/dashboard/node_modules" ]]; then + (cd "$REPO_ROOT/dashboard" && npm install) + fi + (cd "$REPO_ROOT/dashboard" && npm run build) + echo "dashboard built at $REPO_ROOT/dashboard/out" +fi diff --git a/scripts/deploy.sh b/scripts/deploy.sh new file mode 100755 index 0000000..eb0e399 --- /dev/null +++ b/scripts/deploy.sh @@ -0,0 +1,56 @@ +#!/usr/bin/env bash +# Push the built binaries and dashboard to both SDP VMs. +# +# 92 (micro): ~/SDP/agent-micro +# 186 (gateway): ~/SDP/control-plane, ~/SDP/agent-gateway, ~/SDP/dashboard +# +# On 186 we also splice the SDP location into nginx's existing default site +# and reload. Run scripts/build.sh first. + +set -euo pipefail +cd "$(dirname "$0")/.." +REPO_ROOT="$(pwd)" + +# ponytail: paths can be overridden by env so the same script works from CI. +HOST_92="${SDP_92_HOST:-administrator@172.18.136.92}" +PASS_92="${SDP_92_PASS:-password}" +HOST_186="${SDP_186_HOST:-administrator@172.18.139.186}" +PASS_186="${SDP_186_PASS:-Bre@kthrough2312}" + +if ! command -v sshpass >/dev/null 2>&1; then + echo "sshpass not found. Install with: brew install sshpass" >&2 + exit 1 +fi + +SSH_92="sshpass -p $PASS_92 ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR" +SCP_92="sshpass -p $PASS_92 scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR" +SSH_186="sshpass -p $PASS_186 ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR" +SCP_186="sshpass -p $PASS_186 scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR" + +# ponytail: Wipe-and-replace. The deploys are stateful on the VM only via +# SQLite + .log files in ~/SDP/data — we keep that. Binaries and the +# dashboard are replaced cleanly. +REMOTE_RESET='rm -rf ~/SDP/bin ~/SDP/dashboard && mkdir -p ~/SDP/bin ~/SDP/dashboard' + +echo "==> 92: $HOST_92" +$SSH_92 "$HOST_92" "$REMOTE_RESET" +$SCP_92 "$REPO_ROOT/bin/agent-micro" "$HOST_92:~/SDP/bin/agent-micro" +$SSH_92 "$HOST_92" "chmod +x ~/SDP/bin/agent-micro" +echo " agent-micro copied" + +echo +echo "==> 186: $HOST_186" +$SSH_186 "$HOST_186" "$REMOTE_RESET" +$SCP_186 "$REPO_ROOT/bin/control-plane" "$HOST_186:~/SDP/bin/control-plane" +$SCP_186 "$REPO_ROOT/bin/agent-gateway" "$HOST_186:~/SDP/bin/agent-gateway" +$SCP_186 -r "$REPO_ROOT/dashboard/out/." "$HOST_186:~/SDP/dashboard/" +$SSH_186 "$HOST_186" "chmod +x ~/SDP/bin/control-plane ~/SDP/bin/agent-gateway" +echo " control-plane, agent-gateway, dashboard copied" + +# Patch nginx on 186 +echo +echo "==> 186: patching nginx" +"$REPO_ROOT/scripts/patch-nginx.sh" + +echo +echo "done." diff --git a/scripts/patch-nginx.sh b/scripts/patch-nginx.sh new file mode 100755 index 0000000..8d18f75 --- /dev/null +++ b/scripts/patch-nginx.sh @@ -0,0 +1,104 @@ +#!/usr/bin/env bash +# Splice the SDP dashboard location into the existing nginx default site on +# 172.18.139.186. Idempotent: re-running won't duplicate the block. +# +# We don't replace the file — we insert before the closing `}` of the +# existing `server { ... }` block. The block is guarded by a sentinel +# comment so subsequent runs are no-ops. + +set -euo pipefail +cd "$(dirname "$0")/.." + +HOST_186="${SDP_186_HOST:-administrator@172.18.139.186}" +PASS_186="${SDP_186_PASS:-Bre@kthrough2312}" + +if ! command -v sshpass >/dev/null 2>&1; then + echo "sshpass not found. Install with: brew install sshpass" >&2 + exit 1 +fi + +SSH="sshpass -p $PASS_186 ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR" + +NGINX_SITE=/etc/nginx/sites-available/default +SDP_MARKER='# >>> sdp >>>' + +$SSH "$HOST_186" bash -s <>> sdp >>> +\t# Sandbox Deployment Platform dashboard +\tlocation /api/ { +\t\tproxy_pass http://127.0.0.1:8080; +\t\tproxy_set_header Host $host; +\t\tproxy_set_header X-Real-IP $remote_addr; +\t\tproxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; +\t} + +\tlocation /ws/ { +\t\tproxy_pass http://127.0.0.1:8080; +\t\tproxy_http_version 1.1; +\t\tproxy_set_header Upgrade $http_upgrade; +\t\tproxy_set_header Connection "upgrade"; +\t\tproxy_read_timeout 3600s; +\t\tproxy_send_timeout 3600s; +\t} + +\tlocation / { +\t\troot /home/administrator/SDP/dashboard; +\t\tindex index.html; +\t\ttry_files \$uri \$uri/ \$uri.html /index.html; +\t} +\t# <<< sdp <<< +""" + +def find_server_end(s): + i = s.find("server") + if i < 0: return -1 + j = s.find("{", i) + if j < 0: return -1 + depth = 1 + k = j + 1 + in_str = None + while k < len(s): + c = s[k] + if in_str: + if c == in_str and s[k-1] != "\\": + in_str = None + else: + if c in ('"', "'"): + in_str = c + elif c == "{": + depth += 1 + elif c == "}": + depth -= 1 + if depth == 0: + return k + k += 1 + return -1 + +end = find_server_end(src) +if end < 0: + raise SystemExit("could not find server block end") + +new = src[:end] + block + src[end:] +open(path, "w").write(new) +PY + +nginx -t +systemctl reload nginx +echo "nginx patched and reloaded" +REMOTE diff --git a/scripts/ssh-186.sh b/scripts/ssh-186.sh new file mode 100755 index 0000000..8c2e004 --- /dev/null +++ b/scripts/ssh-186.sh @@ -0,0 +1,19 @@ +#!/usr/bin/env bash +# SSH into the gateway VM (172.18.139.186). Wraps ssh with the known password. +# Usage: scripts/ssh-186.sh [extra ssh args...] +set -euo pipefail +cd "$(dirname "$0")/.." + +HOST="${SDP_186_HOST:-administrator@172.18.139.186}" +PASS="${SDP_186_PASS:-Bre@kthrough2312}" + +if ! command -v sshpass >/dev/null 2>&1; then + echo "sshpass not found. Install with: brew install sshpass" >&2 + exit 1 +fi + +exec sshpass -p "$PASS" ssh \ + -o StrictHostKeyChecking=no \ + -o UserKnownHostsFile=/dev/null \ + -o LogLevel=ERROR \ + "$HOST" "$@" diff --git a/scripts/ssh-92.sh b/scripts/ssh-92.sh new file mode 100755 index 0000000..756a261 --- /dev/null +++ b/scripts/ssh-92.sh @@ -0,0 +1,19 @@ +#!/usr/bin/env bash +# SSH into the micro VM (172.18.136.92). Wraps ssh with the known password. +# Usage: scripts/ssh-92.sh [extra ssh args...] +set -euo pipefail +cd "$(dirname "$0")/.." + +HOST="${SDP_92_HOST:-administrator@172.18.136.92}" +PASS="${SDP_92_PASS:-password}" + +if ! command -v sshpass >/dev/null 2>&1; then + echo "sshpass not found. Install with: brew install sshpass" >&2 + exit 1 +fi + +exec sshpass -p "$PASS" ssh \ + -o StrictHostKeyChecking=no \ + -o UserKnownHostsFile=/dev/null \ + -o LogLevel=ERROR \ + "$HOST" "$@"