This guide explains how the Supermetrics Python SDK generates, filters, and wraps code from upstream OpenAPI specifications.
The SDK uses a multi-stage pipeline to generate type-safe client code from upstream OpenAPI specs:
┌─────────────────────────────────┐ ┌──────────────────────────────────────────────┐
│ openapi-specs/ │ │ scripts/references/ │
│ - openapi-data.yaml │ + │ sdk-endpoint-filters.yaml │
│ - openapi-management.yaml │ │ (endpoint filters & component patches) │
└────────────────┬────────────────┘ └──────────────────────┬───────────────────────┘
│ │
└──────────────────────┬───────────────────────┘
│
python scripts/filter_openapi_spec.py
│
▼
┌──────────────────────────────┐
│ openapi-spec.yaml │
└──────────────┬───────────────┘
│
scripts/regenerate_client.sh
(uvx openapi-python-client, pinned)
│
▼
┌──────────────────────────────┐
│ src/supermetrics/_generated/ │
└──────────────┬───────────────┘
│ (Adapter Pattern)
▼
┌──────────────────────────────┐
│ src/supermetrics/ │
│ - resources/*.py │
│ - client.py / async_client │
└──────────────────────────────┘
Place the upstream OpenAPI YAML files in the openapi-specs/ directory (e.g., openapi-specs/openapi-data.yaml, openapi-specs/openapi-management.yaml).
Note:
openapi-specs/*is listed in.gitignoreso local temporary specs can be tested without cluttering git.
Edit scripts/references/sdk-endpoint-filters.yaml to configure the endpoints and schema patches.
endpoints:
- method: GET
path: /query/data/json
- method: POST
path: /management/connectors
pin_baseline)pin_baseline: openapi-spec.yaml
With this set, any filtered endpoint already present in the committed merged spec is taken
from that file rather than re-read from openapi-specs/. Only genuinely new endpoints
come from upstream, which makes a regeneration purely additive.
This is not a nicety. The upstream specifications drift between publications, and the generator turns some of that drift into breaking changes:
| Upstream drift | What it does to the SDK |
|---|---|
| An operation is retagged | The first tag is the generated API subpackage name, so the module moves and every adapter importing it fails |
| A path is reparameterised | GET /query/data/json became GET /query/data/{context_type}; the old module disappears |
| A shared schema is restructured | Regenerates a model that shipped adapters and their tests are written against |
Absorbing any of that as a side effect of adding one new domain is how a working SDK becomes a broken one. Pinning makes each of those a deliberate, separately reviewable change instead.
Components follow the same rule: for a name defined in both places, the baseline
definition wins, so shared schemas such as ErrorResponse stay byte-identical.
To deliberately refresh a pinned endpoint, delete its entry from openapi-spec.yaml,
or drop pin_baseline for a full upstream refresh — then expect to update the affected
adapters, and their tests, in the same change.
The acceptance check after any regeneration:
git status --porcelain src/supermetrics/_generated | grep -v '^??'
Only new (??) files should appear. Anything modified or deleted means drift got in.
rewrite_path)endpoints:
- method: GET
path: /v1/teams/{team_id}/transfers
rewrite_path: /teams/{team_id}/transfers
The endpoint is matched upstream by path and written into the merged spec under
rewrite_path.
This exists because the upstream specs split a shared prefix inconsistently. Data
Warehouse transfers are declared as /v1/teams/... served from
https://dts-api.supermetrics.com, while backfills, data source connections and
destinations are /teams/... served from https://dts-api.supermetrics.com/v1. Both
resolve to the same URL, but openapi-python-client ignores path-level servers
and simply concatenates base_url + path — so under one base URL one of the two groups
is always wrong, by a duplicated or a missing /v1. Rewriting normalises them onto a
single convention.
Two filter entries that would write to the same merged path is a hard error, so a rewrite cannot silently shadow another endpoint.
Rewriting a path changes the URL the generated client requests. Only use it to correct where the spec puts a prefix that the server also puts in
servers— never to invent a route the API does not serve.
endpoints:
- method: POST
path: /management/connectors
patches:
merge:
description: "Create a custom connector"
tags:
- "Connectors"
Use component_patches to modify shared schemas across all endpoints that reference them:
component_patches:
schemas:
DataResponse:
merge:
properties:
meta:
properties:
result:
properties:
cache_time:
nullable: true
Run the filter script to produce the consolidated openapi-spec.yaml:
uv run python scripts/filter_openapi_spec.py
The script will:
openapi-specs/$ref dependencies and external file referencesopenapi-spec.yamlRun the regeneration script:
./scripts/regenerate_client.sh
Two things it does are load-bearing, and are the reason to use it rather than calling the generator yourself:
It generates into a staging directory and swaps only on success. The committed tree at
src/supermetrics/_generated/ is replaced after the generator has produced a complete
supermetrics_api_client package, not before. An earlier version deleted the tree first
and generated second, so any generator failure left the repository with no client at all.
If generation fails now, the script exits non-zero and the committed tree is untouched.
(The redundant pyproject.toml the generator emits is removed from the staging directory
before the swap.)
It runs the generator on a pinned interpreter, outside the project virtualenv.
openapi-python-client 0.29.0 pulls in a pydantic that raises AssertionError in
_typing_extra.eval_type_backport under Python 3.14 — which is this project’s default
interpreter — so running the generator through uv run in the project environment fails
before it writes anything. The script instead invokes it with uvx on Python 3.12 in a
throwaway environment, at the version read out of the openapi-python-client== pin in
pyproject.toml so the generator and the dev dependency cannot drift. The interpreter
defaults to 3.12 and is overridable — GENERATOR_PYTHON=3.13 ./scripts/regenerate_client.sh
— should a future generator release need a different one.
Running the generator by hand is discouraged for both reasons above. If you must, mirror
what the script does — pin the interpreter and stage the output rather than pointing
--output-path at the committed tree:
# mktemp -d guarantees the parent directory exists. The generator calls
# `mkdir()` without `parents=True`, so pointing --output-path at a path whose
# parent is missing fails with FileNotFoundError before it writes anything.
STAGING="$(mktemp -d)"
uvx --python 3.12 --from "openapi-python-client==0.29.0" \
openapi-python-client generate \
--path openapi-spec.yaml \
--output-path "$STAGING/_generated" \
--config openapi-python-client-config.yaml
# The generated project scaffolding is not part of the SDK.
rm -f "$STAGING/_generated/pyproject.toml"
# Swap only once the generator has succeeded.
rm -rf src/supermetrics/_generated
mv "$STAGING/_generated" src/supermetrics/_generated
The version in the --from pin has to be kept in step with pyproject.toml by hand,
which is the other reason to prefer the script: it reads the pin instead.
Warning: Never hand-edit files in
src/supermetrics/_generated/. Any changes will be overwritten on the next regeneration.
Create or update high-level resource adapters under src/supermetrics/resources/:
supermetrics._generated.supermetrics_api_client.src/supermetrics/resources/_error_handlers.py.SupermetricsClient (src/supermetrics/client.py) and SupermetricsAsyncClient (src/supermetrics/async_client.py).Run tests and quality checks:
# Run tests
just test
# Run full QA suite (format, lint, typecheck, test)
just qa
For real-world reference, see the following PRs in the repository:
$ref file dependencies in the filter script, and regenerated low-level clients.pin_baseline and rewrite_path in the filter script so a regeneration stays additive and the Data Warehouse families can share one base URL.feat/phase3-destinations): added the seven /teams/{team_id}/destinations* endpoints to the filter config, regenerated purely additively on top of the pinned baseline, wrapped them in src/supermetrics/resources/destinations.py, and fixed scripts/regenerate_client.sh as described in Step 4.| Issue | Cause | Solution |
|---|---|---|
Missing endpoint warning during filter |
Endpoint path or HTTP method does not match spec | Check path syntax and casing in sdk-endpoint-filters.yaml |
Unresolved $ref error |
Schema references external file or missing definition | Ensure referenced spec files are present in openapi-specs/ |
Pydantic/typing errors on None values |
API returns null for a non-nullable field in spec |
Add a component_patches entry to set nullable: true |
pyproject.toml generated inside _generated/ |
openapi-python-client creates standalone package by default |
The script removes it from the staging directory before the swap, so it never reaches src/supermetrics/_generated/ |
AssertionError in _typing_extra.eval_type_backport while generating |
The pinned openapi-python-client pulls a pydantic that breaks under Python 3.14, the project’s default interpreter |
Use ./scripts/regenerate_client.sh, which runs the generator on Python 3.12 through uvx; set GENERATOR_PYTHON to pick a different one |
Generation failed and src/supermetrics/_generated/ is gone |
An older version of the script deleted the tree before generating | Restore it with git checkout -- src/supermetrics/_generated, then re-run the current script, which stages and swaps |
The SDK is the first consumer of the canonical specs; the supermetrics-cli chains through
it. The CLI does not read the raw canonical spec — it reads this repo’s filtered,
production-ready openapi-spec.yaml, so the CLI and SDK always expose the same endpoint set.
When openapi-spec.yaml changes on main (i.e. a spec-update PR merges), the
.github/workflows/notify-cli-on-spec-change.yml workflow sends a cross-repo
repository_dispatch to the CLI:
on:
push:
branches: [main]
paths:
- openapi-spec.yaml
workflow_dispatch: # manual re-trigger for the initial backlog sync or recovery
jobs:
notify:
runs-on: ubuntu-latest
steps:
- uses: peter-evans/repository-dispatch@v3
with:
token: $
repository: supermetrics-public/supermetrics-cli
event-type: openapi-spec-updated
The CLI’s own spec-sync.yml handler receives the openapi-spec-updated event, fetches
this repo’s openapi-spec.yaml, regenerates its commands (make generate), and opens a
PR for human review. The automation never pushes to the CLI’s main or auto-merges — a
person reviews and merges every CLI change.
A new endpoint appearing here does not automatically become a CLI command: the CLI
regenerates only the endpoints already listed in its scripts/command-mapping.yaml. Adding
a command is a deliberate, separate change in the CLI repo.
CLI_DISPATCH_TOKEN secretThe dispatch authenticates to the CLI repo with a repo-scoped GitHub PAT stored as the
CLI_DISPATCH_TOKEN secret in this repo’s Actions secrets. The workflow’s GITHUB_TOKEN
cannot reach another repository, so a PAT is required. To (re)create it: generate a
classic PAT with repo scope on supermetrics-public/supermetrics-cli (org admin needed),
store it in 1Password, then add it under Settings → Secrets and variables → Actions.
repository-dispatch exits non-zero if the dispatch call fails or the token is
missing/invalid, so the workflow run turns red in the Actions tab — no silent failure
leaves the CLI on a stale spec. Use the workflow_dispatch trigger to re-fire the dispatch
manually (e.g. the first large backlog sync) without a dummy spec commit.
.github/workflows/sdk-spec-validation.yml guards the generation pipeline. When a PR
touches any generation input — the raw upstream specs (openapi-specs/), the merged spec
(openapi-spec.yaml), the allowlist (scripts/references/sdk-endpoint-filters.yaml), the
pipeline scripts, or the generator config — CI re-runs the pipeline and fails if the
committed artifacts are out of sync — catching a forgotten regeneration, a regeneration from
stale specs, an allowlist edit that wasn’t regenerated, or a hand-edited openapi-spec.yaml.
This is what spec-update PRs from upstream (the api-style-guide → SDK sync) are validated
against before a human reviews them.
The job does exactly what a developer does locally in Step 3 and Step 4:
- run: uv run python scripts/filter_openapi_spec.py # regenerate openapi-spec.yaml
- run: ./scripts/regenerate_client.sh # regenerate src/supermetrics/_generated/
- run: | # fail if either drifted from the commit
CHANGES="$(git status --porcelain -- openapi-spec.yaml src/supermetrics/_generated/)"
[ -z "$CHANGES" ] || { echo "$CHANGES"; exit 1; }
When it fails, the run prints the out-of-sync paths and the diff. The fix is always the same — run the two scripts locally and commit the result:
uv run python scripts/filter_openapi_spec.py
./scripts/regenerate_client.sh
Key properties:
pin_baseline keeps regeneration additive-only (see Step 2 → A1), so a clean PR
reproduces the committed artifacts byte-for-byte.sdk-lint-test.yml (which has no paths filter). This
workflow adds only the drift check and does not re-run just qa.paths filter means PRs that don’t touch a generation input never
start this job. (A bump of the openapi-python-client pin in pyproject.toml is
deliberately not a trigger — it would fire on every unrelated dependency change; such a
bump is instead caught by the next spec PR.)