feat: polish & windviz & deploy
This commit is contained in:
parent
81b8e763bd
commit
465ad00f7b
78 changed files with 20622 additions and 2154 deletions
88
README.md
88
README.md
|
|
@ -150,7 +150,24 @@ for the configured source. Each `(epoch, subset)` combination is a
|
|||
separate dataset; the loader auto-selects which loaded dataset covers a
|
||||
given prediction query.
|
||||
|
||||
### Metrics
|
||||
### Wind visualization
|
||||
|
||||
`GET /api/v1/wind/field` — a velocity grid in the
|
||||
[wind-js-server](https://github.com/danwild/wind-js-server) / leaflet-velocity
|
||||
format (a two-element `[U, V]` array of `{header, data}` records), suitable for
|
||||
animated particle layers. Query params: `time`, `altitude`, `min_lat`,
|
||||
`max_lat`, `min_lng`, `max_lng`, `step` (degrees, min `0.25`). Responses are
|
||||
cached in memory by parameters.
|
||||
|
||||
`GET /api/v1/wind/meta` — active dataset source, epoch, suggested altitudes,
|
||||
and bounding box.
|
||||
|
||||
A runnable browser client is in [`examples/wind-demo`](examples/wind-demo).
|
||||
|
||||
### Documentation & metrics
|
||||
|
||||
`GET /docs` serves a [ReDoc](https://github.com/Redocly/redoc) rendering of the
|
||||
full OpenAPI spec, which is also available raw at `GET /openapi.yaml`.
|
||||
|
||||
`GET /metrics` — Prometheus text exposition. Counters:
|
||||
`predictor_predictions_total{profile,status}`, `predictor_downloads_total`,
|
||||
|
|
@ -159,31 +176,45 @@ given prediction query.
|
|||
|
||||
## Architecture
|
||||
|
||||
The entire REST API is defined by one OpenAPI spec and served by an
|
||||
[ogen](https://ogen.dev)-generated server; the `internal/api` package only
|
||||
implements the generated `Handler` interface, mapping between the wire types
|
||||
and the engine/dataset/wind subsystems. `/metrics`, `/docs`, and
|
||||
`/openapi.yaml` are mounted on the same `http.ServeMux` alongside it.
|
||||
|
||||
```
|
||||
cmd/
|
||||
predictor/ main server
|
||||
predictor-cli/ HTTP client
|
||||
compare-tawhiri/ end-to-end validation against the public Tawhiri instance
|
||||
api/
|
||||
rest/predictor.swagger.yml OpenAPI 3 spec — ogen input AND served at /openapi.yaml
|
||||
spec.go embeds the spec (go:embed) for the docs handler
|
||||
internal/
|
||||
numerics/ pure numerical primitives (interp, bisect, RK4, refinement)
|
||||
engine/ propagator + constraint system + concrete models + registry
|
||||
weather/ WindField interface; gfs/ — variant-parameterized GFS file format + WindField
|
||||
numerics/ performance-critical core: interpolation, bisection,
|
||||
RK4 + crossing refinement, atmosphere density, vector
|
||||
and polygon math (portable to C/Rust)
|
||||
engine/ propagator + constraint orchestration + registry (thin over numerics)
|
||||
weather/ WindField interface; gfs/ — variant-parameterized GFS cube + sampler
|
||||
datasets/ Source / Storage / Manager + transactional, resumable, subsettable downloads
|
||||
grib/ — shared GRIB downloader skeleton (idx parser, HTTP, parallel blit)
|
||||
gfs/ — GFS Source (URL templating only)
|
||||
gefs/ — GEFS Source (URL templating + member resolution)
|
||||
windviz/ cube-agnostic wind-field rasterizer + cache
|
||||
elevation/ ruaumoko-format ground elevation reader
|
||||
config/ layered file+env+CLI config
|
||||
metrics/ Sink interface + Prometheus text impl
|
||||
api/ HTTP transport
|
||||
tawhiri/ — legacy v1 endpoint via ogen
|
||||
v2/ — synchronous profile-driven endpoint
|
||||
async/ — asynchronous prediction jobs
|
||||
admin/ — dataset + service-status endpoints
|
||||
httpjson/ — tiny JSON response helpers
|
||||
middleware/
|
||||
api/rest/predictor.swagger.yml OpenAPI 3 spec for v1 + /ready
|
||||
pkg/rest/ ogen-generated code (regenerate via `make generate-ogen`)
|
||||
api/ ogen Handler implementation
|
||||
handler.go — composite handler + NewError
|
||||
prediction.go — v1 (Tawhiri), v2, async predictions
|
||||
datasets.go — dataset + job admin + status
|
||||
wind.go — wind visualization endpoints
|
||||
mapping.go — ogen <-> engine conversions
|
||||
async/ — prediction worker pool
|
||||
docs/ — ReDoc page + /openapi.yaml
|
||||
middleware/ — ogen logging, CORS
|
||||
pkg/rest/ ogen-generated server/client/types (regenerate via `make generate-ogen`)
|
||||
examples/wind-demo/ Leaflet + leaflet-velocity sample client
|
||||
docs/numerics.tex end-to-end mathematical reference
|
||||
scripts/build_elevation.py ETOPO 2022 → ruaumoko converter
|
||||
```
|
||||
|
|
@ -200,10 +231,33 @@ right one.
|
|||
|
||||
## Deployment
|
||||
|
||||
Local single instance, Docker container, or load-balanced cluster behind a
|
||||
shared filesystem for the dataset cache. The async API stores results
|
||||
in-memory only; for cluster deployments with sticky sessions, ensure
|
||||
clients poll the same node they submitted to.
|
||||
The service ships as a single static binary in a distroless image and runs in
|
||||
three configurations — see **[DEPLOYMENT.md](DEPLOYMENT.md)** for the full guide.
|
||||
|
||||
| Environment | File |
|
||||
|---|---|
|
||||
| Local dev | `docker compose up --build` (`docker-compose.yml`) |
|
||||
| Staging (single host, + Prometheus) | `docker-compose.staging.yml` |
|
||||
| Production (Docker Swarm) | `docker-compose.swarm.yml` |
|
||||
|
||||
Production runs on Docker Swarm pinned to ≤2 nodes labelled `predictor.data=true`,
|
||||
each holding one copy of the dataset on **node-local disk** (never NFS).
|
||||
Replicas spread across the two nodes for redundancy; multiple replicas per node
|
||||
share the node's dataset and coordinate downloads with a file lock so only one
|
||||
fetches the ~9 GiB cube. The predictor is an internal backend reached by the
|
||||
API gateway over an overlay network; it enforces no auth itself. CI/CD is a
|
||||
Forgejo pipeline that builds, tests, and deploys to Swarmpit
|
||||
(`.forgejo/workflows/ci-cd.yml`).
|
||||
|
||||
The async prediction API stores results in memory only; behind a load balancer,
|
||||
clients must poll the same instance they submitted to (or use the synchronous
|
||||
`/api/v2/prediction`).
|
||||
|
||||
### Health
|
||||
|
||||
- `GET /health` — liveness, always 200 while the process runs (used by the
|
||||
container `HEALTHCHECK` via `predictor -healthcheck`).
|
||||
- `GET /ready` — readiness, 200 only once a dataset is loaded.
|
||||
|
||||
## Validation
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue