Initial commit

README.md

@@ -0,0 +1,89 @@
+# yksa-source-poller
+
+The machine that polls upstreams on a schedule: a source model, a run log, the
+dispatcher and the bookkeeping around one attempt. The cargo — what a fetched
+record *is* and what happens to it — stays in the service.
+
+Extracted from `tdas/yksa_tdas/sources/` and `tle/yksa_tle/sources/`, which were
+the same ~500 lines with different payload types.
+
+## Install
+
+```
+yksa-source-poller @ git+https://git.intra.yksa.space/web/yksa-source-poller.git@v0.1.0
+```
+
+Requires [yksa-django-kit](../yksa-django-kit) for `RunStatus` and `read_secret`.
+It is **not** added to `INSTALLED_APPS`: every model in it is abstract, so it
+ships no tables and needs no app config.
+
+## Use
+
+```python
+# sources/models.py
+class TelemetrySource(AbstractPollingSource):
+    SUBSCRIPTION_ID_FIELDS = {"norad": "norad_cat_id", "satnogs": "satnogs_id"}
+
+    kind = models.CharField(max_length=32, choices=SourceKind.choices)
+    subscription_id_kind = models.CharField(..., default=SubscriptionIdKind.SATNOGS)
+    tracked_satellites = models.ManyToManyField("satellites.Satellite", blank=True)
+
+
+class SourceRun(AbstractSourceRun):
+    source = models.ForeignKey(TelemetrySource, on_delete=models.CASCADE,
+                               related_name="runs")
+```
+
+```python
+# sources/tasks.py
+POLLER = Poller(
+    source_model=TelemetrySource,
+    run_model=SourceRun,
+    adapters=ADAPTERS,
+    persist=_persist_frame,          # -> True when the record is new
+    track=_track_latest_rx,          # optional: extra source fields on success
+    unpollable_kinds=(SourceKind.INTERNAL_PUSH,),
+)
+
+
+@shared_task(bind=True, max_retries=3, default_retry_delay=120)
+def run_source_task(self, source_id):
+    try:
+        return run_source(POLLER, source_id)
+    except Exception as exc:
+        raise self.retry(exc=exc)
+```
+
+The Celery tasks stay in the service on purpose: `CELERY_BEAT_SCHEDULE` names
+them by dotted path, and a task that moved into this package would silently stop
+being scheduled.
+
+## Why the models are abstract
+
+The two concrete source models differ where they have to — which satellite model
+the M2M points at, which identifiers the upstream accepts, and (in tdas) a
+`last_rx_at` watermark. Everything else was identical. Abstract bases share the
+identical part without pretending the rest is the same.
+
+Field definitions here reproduce what the services already had, so adopting the
+package needs **no migration**. Where a service's own definition differed — a
+different `poll_interval_sec` default, different wording — the service overrides
+the field, which Django allows for abstract inheritance. Check with
+`manage.py makemigrations --check --dry-run` after wiring it up.
+
+## What the runner guarantees
+
+Every exit path — success, failure, unconfigured adapter, missing adapter —
+writes `last_status`, updates a counter and re-arms `next_poll_at`. That is the
+reason to share it: a path that forgets leaves the dispatcher either hammering
+the source once a minute or ignoring it forever, and both shipped once.
+
+`NotImplementedError` from an adapter is a *skip*, not a failure: a half-configured
+source must not look like an outage.
+
+## Tests
+
+```sh
+pip install -e ".[test]" ../yksa-django-kit
+python -m pytest
+```