Skip to content

OpenConceptLab/ocl_online#532 | Capability policy backend, and settings to load extra apps and URLconfs - #963

Draft
paynejd wants to merge 4 commits into
masterfrom
capability-policy-backend
Draft

paynejd wants to merge 4 commits into
masterfrom
capability-policy-backend

Conversation

@paynejd

@paynejd paynejd commented Oct 27, 2026

Copy link
Copy Markdown
Member

OpenConceptLab/ocl_online#532, with the public metering parts of OpenConceptLab/ocl_online#352.

What this does

CAPABILITY_POLICY names the class that decides and records the use of capabilities, as EMAIL_BACKEND does for mail. The default, GroupLimitPolicy, is the per-user override and group limit logic that lived on UserProfile, moved unchanged. UserProfile.get_capability_limit, get_capability_usage and check_and_consume_capability now delegate to it, so their callers don't change.

  • Usage windows. UsageCounter and UsageEvent get window_start. null is the one window that never ends; every existing row and the default policy use it. A policy can count in windows by overriding get_window_start(). A counter is unique per (user, capability, window), with a second, partial constraint because PostgreSQL 14 treats nulls as distinct. Idempotency keys stay unique per (user, capability) across windows.
  • One lock per account. consume() locks the user's row before the counter, so a user's requests serialize even when they count in different windows.
  • Reversal. A failed $match reverses its own usage event (UsageEvent.reverse), which lowers the counter that event counted in and records reversed_units. It never takes back more than the event recorded. The staff refund (POST /capabilities/refund/) stays a counter correction, applied to the policy's current window.
  • Refusals. The policy raises UsageRefused, whose detail is the 403 body. $match and POST /capabilities/consume/ send it as built, adding only their own sentence as detail when the policy gave none.
  • Usage reports. GET /user/?includeCapabilities=true and the consume and refund responses send every field the policy's get_usage() returns.
  • Loader. OCL_EXTRA_APPS and OCL_EXTRA_URLCONFS, comma-separated dotted paths from the environment, empty by default, add Django apps and URLconfs to a build. Extra URLconfs are included after core's own routes, so those keep precedence.
  • used, units and reversed_units are BigIntegerField.

Behaviour under the default

No API response changes: the same limits, the same 403 bodies (detail, error_code, limit, used, with the same sentences and codes), and the same used. The migration gives existing rows window_start = null and keeps every used. The differences are the extra row lock in consume(), reversed_units recorded when a $match search fails, and the Swagger schema typing the capability limit and used as integers.

Tests

  • The full existing suite passes. The two $match integration tests that patched the moved methods now patch GroupLimitPolicy, with every response assertion kept, and the users test that patched get_capability_usage patches GroupLimitPolicy.get_usage.
  • CapabilityPolicyContractTest runs a fake policy, set through CAPABILITY_POLICY, through every call site: the UserProfile delegates, the user's capabilities, consume (success and refusal), $match (refusal, a failed search that's reversed, a search that isn't), and the staff refund.
  • GroupLimitPolicyWindowTest: each window counts from 0, a key replays in a later window, a reversal lowers the window its units counted in, and a reversal never takes back more than the event or the counter holds.
  • GroupLimitPolicyDefaultWindowTest: the default counts in the window that never ends; consume() locks the user row FOR NO KEY UPDATE; an event inserted without the new columns, as the previous release does while a deploy migrates, gets their defaults.
  • ExtraURLConfsTest: an extra URLconf is included after core's routes.

Run locally in an isolated compose stack, as CI runs them (test --parallel=1 --keepdb, PYTHONHASHSEED=2): 2,720 tests OK on master (685f307) and 2,736 OK on this branch (d7cff42). pylint -j0 core/: 10.00, no messages.

Review

Reviewed before opening, with Codex (codex-cli 0.164.0, xhigh) and two fresh Claude reviews. Each Codex pass is posted below.

  • Codex pass 1 (03e82de): one P2. The user-row FOR UPDATE could deadlock a first consume against a staff refund creating the same counter. Fixed in 3765046 (FOR NO KEY UPDATE).
  • Codex pass 2 (3765046): clean.
  • Simplification review: one cut, CapabilityExceeded's unread attributes, applied in 777c70a. Codex check of that commit: clean.
  • Adversarial review: one P1. reversed_units lost its database default after the migration, so the previous release would fail every usage event insert during a rolling deploy. Fixed in d7cff42 (db_default=0). Codex check of that commit: clean. One P2 was accepted: the int-to-bigint rewrite locks two small tables for milliseconds. Six P3s were recorded, not fixed: a rare deadlock between a consume and deleting that same user (on master the same race ends in a 500), and contract edges for other policies and extra URLconfs.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RK9RzEWK5NKU6Wj9WSYGFw

paynejd and others added 4 commits October 24, 2026 09:37
…gs to load extra apps and URLconfs

CAPABILITY_POLICY names the class that decides and records the use of
capabilities, as EMAIL_BACKEND does for mail. The default,
GroupLimitPolicy, is the per-user override and group limit logic that
lived on UserProfile, moved unchanged. UserProfile's three capability
methods now delegate to it.

- Usage counts in windows: UsageCounter and UsageEvent get window_start.
  Null is the one window that never ends, which every existing row and
  the default policy use. consume() locks the user's row first, so a
  user's requests serialize even when they count in different windows.
- A failed $match reverses its own usage event, which lowers the counter
  it counted in and records reversed_units on the event.
- Refusals are UsageRefused, whose detail is the 403 body as the policy
  built it. Usage reports send every field the policy's get_usage()
  returns. Under the default policy every response is unchanged.
- OCL_EXTRA_APPS and OCL_EXTRA_URLCONFS, comma-separated in the
  environment and empty by default, add Django apps and URLconfs to a
  build. Extra URLconfs come after core's own routes.
- A contract test runs a fake policy through every call site.

Refs OpenConceptLab/ocl_online#352

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RK9RzEWK5NKU6Wj9WSYGFw
…EY UPDATE

A plain FOR UPDATE blocks the foreign-key check of another transaction
that inserted a counter for the same user, such as a staff refund
creating the current window's counter, while consume() waits on that
counter's unique entry: a deadlock. FOR NO KEY UPDATE still serializes
a user's consumes and lets the foreign-key check through.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RK9RzEWK5NKU6Wj9WSYGFw
…usal body

Nothing reads its limit, used, requested or capability_name attributes
since the body moved into detail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RK9RzEWK5NKU6Wj9WSYGFw
Django drops a column's default once AddField has filled existing rows,
so code that predates the column, still serving while a deploy migrates,
would fail every usage event insert with a not-null violation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RK9RzEWK5NKU6Wj9WSYGFw
@paynejd

paynejd commented Oct 27, 2026

Copy link
Copy Markdown
Member Author

Codex review, pass 1 (codex-cli 0.164.0, commit 03e82de)

Verdict: 1 finding(s).

  • F1.1 P2 core/capabilities/policy.py:119: The new user-row lock can deadlock a first consume against a staff refund.
findings.json
{
  "reviewer": "codex-cli 0.164.0",
  "perspective": "defects",
  "mode": "full",
  "commit": "03e82de8",
  "verdict": "findings",
  "prior": [],
  "findings": [
    {
      "id": "F1.1",
      "severity": "P2",
      "tag": null,
      "file": "core/capabilities/policy.py",
      "line": 119,
      "claim": "The new user-row lock can deadlock a first consume against a staff refund.",
      "scenario": "When the current window has no counter, a concurrent staff refund can insert it before consume locks the user. Consume then waits on the refund's uncommitted unique entry; the refund's deferred user foreign-key check waits on consume's FOR UPDATE lock. PostgreSQL aborts one transaction, producing an uncaught 500. This follows from static lock-order analysis using PostgreSQL's [uniqueness checks](https://www.postgresql.org/docs/14/index-unique-checks.html) and [foreign-key locking](https://github.com/postgres/postgres/blob/REL_14_STABLE/src/backend/utils/adt/ri_triggers.c).",
      "action": "Use select_for_update(no_key=True) for the user lock, which still serializes consumes while permitting foreign-key checks under PostgreSQL's [lock compatibility rules](https://www.postgresql.org/docs/14/explicit-locking.html#LOCKING-ROWS), or make refund acquire the user lock first. Add a PostgreSQL concurrency regression test for an absent counter.",
      "lines_delta": null,
      "confidence": "high"
    }
  ]
}

Response: F1.1 fixed in 3765046. consume() locks the user row FOR NO KEY UPDATE, which still serializes a user's consumes and lets the foreign-key check through. Pinned by test_consume_locks_the_account_without_blocking_foreign_key_checks.

@paynejd

paynejd commented Oct 27, 2026

Copy link
Copy Markdown
Member Author

Codex review, pass 2 (codex-cli 0.164.0, commit 3765046)

Verdict: clean. Earlier findings: F1.1 resolved.

findings.json
{
  "reviewer": "codex-cli 0.164.0",
  "perspective": "defects",
  "mode": "verify",
  "commit": "3765046e",
  "verdict": "clean",
  "prior": [
    {
      "id": "F1.1",
      "status": "resolved"
    }
  ],
  "findings": []
}

@paynejd

paynejd commented Oct 27, 2026

Copy link
Copy Markdown
Member Author

Codex check of commits 3765046..777c70a (the simplification cut)

Verdict: clean.

findings.json
{
  "reviewer": "codex-cli 0.164.0",
  "perspective": "defects",
  "mode": "cuts",
  "commit": "777c70a2",
  "verdict": "clean",
  "prior": [],
  "findings": []
}

@paynejd

paynejd commented Oct 27, 2026

Copy link
Copy Markdown
Member Author

Codex check of commits 777c70a..d7cff42 (the adversarial review's P1 fix)

Verdict: clean.

findings.json
{
  "reviewer": "codex-cli 0.164.0",
  "perspective": "defects",
  "mode": "cuts",
  "commit": "d7cff42d",
  "verdict": "clean",
  "prior": [],
  "findings": []
}

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant