Architecture

A reference for contributors. If you’re using the package, Concepts is more useful.

Package layout

src/constellation_utils/
├── __init__.py
├── cli.py                       # `constellation` CLI (currently: doctor)
├── config/
│   ├── secrets.testing.yaml     # logical name → op:// URI mapping
│   └── secrets.production.yaml
└── secrets/
    ├── __init__.py              # public API: r2(), R2Secrets, ConstellationAuthError
    ├── _backends.py             # OpCLIBackend, EnvBackend, select_backend()
    ├── _config.py               # YAML loading, profile resolution
    └── models.py                # frozen Pydantic models for each secret type

secrets/__init__.py is the entire public surface. Everything with a leading underscore is implementation detail and may change.

Request flow

When user code calls secrets.r2():

user code
   │
   ▼
secrets.r2()                       ← public API, lru_cache wrapped
   │
   ├─► load_profile()              ← reads secrets.{testing,production}.yaml
   │                                 from the active profile (env var)
   │
   ├─► select_backend()            ← picks OpCLIBackend or EnvBackend
   │                                 based on env state
   │
   └─► backend.read_r2(uri_map)    ← fetches each field, returns
                                     R2Secrets(endpoint=..., access_key_id=..., ...)

The backend abstraction means tests can mock subprocess.run to fake op read, or use EnvBackend to skip the CLI entirely. Production code paths only go through OpCLIBackend.

Design choices

Why Pydantic models instead of dicts or dataclasses?

  • Validation at the boundary. A missing field in 1Password fails at the moment of model_validate, with a clear message naming the field. Dicts would defer the error to wherever the value is used.

  • Frozen by default. model_config = ConfigDict(frozen=True, extra="forbid") means consumers can’t accidentally mutate credentials, and rejected typos surface immediately.

  • Schema lock. The R2Secrets field set deliberately mirrors the credential subset of data-engine/uploader/r2_client.py:R2Settings. A test pins the field set so any drift triggers a deliberate review.

Why YAML config files in the package?

Profiles map logical names like r2.endpoint to op://Engineering/R2 Testing/endpoint. That mapping is itself non-secret configuration — there’s no password in there, only a pointer.

Shipping the YAML inside the package (loaded via importlib.resources) gives us:

  • Versioning. A pinned version of constellation-utils carries its own copy of the mappings. Older deployed code keeps working even if main adds new mappings.

  • No “where is the config” question. The package just knows.

  • Testable. tests/test_config.py parametrizes over both profiles and asserts every URI starts with op://Engineering/.

Why an env-var backend at all?

EnvBackend exists for one reason: tests and CI runs that don’t have 1Password access. It is never the production path — the docstring and error messages are explicit about this. If it accidentally activates somewhere it shouldn’t, the missing-var error message tells you to unset R2_ACCESS_KEY_ID and try again.

Why one process-global cache instead of per-call?

secrets.r2() is wrapped in functools.lru_cache(maxsize=1). The first call takes ~200 ms (subprocess to op); every subsequent call is nanoseconds. Since secrets don’t change mid-process, this is the right tradeoff. Tests that need a fresh read use r2.cache_clear().

Why a separate op subprocess instead of the 1Password Python SDK?

Two reasons:

  1. Auth uniformity. op works the same way on a laptop (biometric, via desktop app session) as on a rig (service-account token in env). Calling the SDK directly would require us to recreate the session-detection logic.

  2. Deployability. The op binary is one brew install. The Python SDK is another dependency to vendor through uv and another upgrade lane to maintain. For an MVP, the subprocess is cheaper.

If the cost of subprocess-per-read becomes a problem, the cache makes it a non-issue in practice.

Adding a new secret type

Adding a credential to constellation-utils is a multi-step, reviewed change. See the tutorial for a worked example. The skeleton:

  1. Add an item to the Engineering vault in 1Password with the credential fields.

  2. Add a Pydantic model in secrets/models.py (mirror R2Secrets — frozen, extra-forbid).

  3. Add a top-level block (mirroring the existing r2: block) to both secrets.testing.yaml and secrets.production.yaml, mapping each field to an op://Engineering/... URI.

  4. Add a read_<name>() method on both OpCLIBackend and EnvBackend.

  5. Add a top-level <name>() function in secrets/__init__.py, lru_cache-wrapped.

  6. Add tests in tests/test_models.py, tests/test_backends.py, and tests/test_config.py.

  7. Update README.md and the docs.

Track new secret additions in the Secrets Manager Linear project.

Testing

uv sync --extra dev
uv run pytest        # full suite, ~30 tests
uv run ruff check    # lint
uv run mypy src      # types (strict mode)

The test suite mocks subprocess.run to exercise OpCLIBackend without a real op install. EnvBackend is exercised via monkeypatch.setenv. Both profile YAMLs are validated for shape and op://Engineering/ prefix.

CLI

The constellation CLI is a tyro-based subcommand dispatcher. Currently it has one subcommand:

constellation doctor                       # default profile (testing)
constellation doctor --profile production  # for rig deploys

doctor prints which profile and backend are active, then attempts to read each registered secret. Green ✓ means everything is wired correctly; red ✗ surfaces the underlying error with an actionable next step.

The rig deploy script runs constellation doctor as a smoke check before launching the data-engine daemon. If it fails, the rig won’t start.