Error taxonomy

Not every thrown error is a defect. A member typing the wrong password, a clinician opening a member they are not allocated to, an expired session — these are the API working correctly, and they used to land in Datadog Error Tracking identically to a crash. This page is the canonical split between the two, so the backend records errors deliberately rather than by accident.

It covers the backend (apps/perci-platform-backend/functions). The triage workflow that consumes Error Tracking is described in the epic PPL-3265; this page is the input to its severity rubric. For which environments are allowed to report at all — a separate gate, upstream of this one — see Datadog error triage.

The split

	Expected client outcome	Defect
Means	The API correctly rejected the request	Something is broken
Caused by	What the caller sent, or the state they are in	Our code, a client, or a dependency
Status	4xx	4xx (broken client) or 5xx
Log level	warn	error
APM span	tagged perci.client_outcome, not an error	tagged as an error
Error Tracking	no	yes

Expected client outcomes

There is no separate list of these: the branch an error takes in errorHandlingMiddleware is the classification. Anything answered through that middleware's warnExpectedOutcome is an expected outcome. Today that is:

Error	Status	Typical cause
AuthenticationError	401	Missing, expired or tampered session token
AuthorizationError	403	No permission, or no care-team allocation to the member
MfaSetupRequiredError / MfaChallengeRequiredError	403	MFA policy not yet satisfied
MfaApiError	403 / 409 / 429	Already enrolled, already verified, rate limited
ConflictError	409	The request conflicts with current state
LockApiError	409	Another request is already modifying the resource
NotFoundError	404	Unknown id
ValidationError (and CalcomSlotUnavailableError)	400	Invalid input, wrong credentials, slot taken
Malformed JSON body (tagged MalformedJsonBody)	400	express.json() could not parse the body

These are logged at warn by errorHandlingMiddleware with two structured fields — clientOutcome and statusCode — so a spike in rejected logins or permission denials can be queried and alerted on without parsing log messages. That matters for security monitoring: demoting these out of Error Tracking must not make them invisible.

clientOutcome is the error's class name, except where a branch does not map onto a class of its own: the express JSON parser throws a bare SyntaxError, so that branch labels itself MalformedJsonBody instead.

Defects

Everything else, including:

Error	Status	Why it is a defect
ClientContractError	400	The client sent a payload it should never have constructed. Same status as ValidationError, opposite taxonomy
RetryLaterError	503	We asked the caller to back off because something upstream is unhealthy
FetchRetryError	502	A dependency failed after retries
Anything unhandled	500	By definition unexplained

Only two of those tag the span by hand. dd-trace already captured the error from next(error) and marks the request span for any status its validateStatus rejects — by default anything from 500 up — so the 5xx branches need nothing. The two that do tag are ClientContractError, whose 400 no status rule would catch, and the unhandled fallback, which tags explicitly rather than leaning on the response status having been left at 500 (and which is the only way a thrown non-Error keeps its message, since dd-trace ignores non-Error values).

RetryLaterError and FetchRetryError logged at warn before this page existed, which made the rubric read as though the code disagreed with it. They now log at error like every other defect: between them they answer 28 requests a month in prod, so there is no noise argument for keeping them quiet, and both were already in Error Tracking by status.

ClientContractError exists so that a broken client is not hidden by the same demotion that silences a wrong password. Reach for it when the invalid payload could only come from a bug in the caller — a field combination its own UI is supposed to prevent — and raise a ticket against the client at the same time. If a real user can reach the state through normal use, it is a ValidationError.

Why middleware spans are not traced

middlewareTracingEnabled: false, set where the tracer is configured in src/observability/datadogTracing.ts, is load-bearing, not a cost-saving tweak. dd-trace's express instrumentation marks every middleware span in the chain as an error whenever a handler calls next(error), and Error Tracking picks those up. Without it an expected 4xx reaches Error Tracking no matter how errorHandlingMiddleware classifies it. Route tracking (http.route, resource names) is unaffected; only the per-middleware timing spans are lost.

Adding a new error

1. Decide which column of the table above it belongs to. If a normal user can reach it by using the product, it is an expected outcome.
2. Add a branch to errorHandlingMiddleware returning the right status. Expected outcomes log through warnExpectedOutcome, which is also what keeps them off Error Tracking. A defect logs at error, and calls tagSpanAsError if its status is below 500.
3. Add a case to the table in __tests__/errorHandlingMiddleware.test.ts, which asserts the tagging and log level for every outcome in both directions.
4. Update the tables on this page in the same PR.

One warn per outcome. errorHandlingMiddleware emits it, and it is the row carrying clientOutcome and statusCode. A throw site that also logged at warn would put two or three rows in the index for a single 401 and inflate exactly the alert this taxonomy exists to make trustworthy, so throw sites log the detail the middleware cannot see — which Descope error, which id — at info, correlated by trace id. What a throw site must never do is log an expected outcome at error.

When one code path covers both an expected rejection and a genuine dependency failure, branch on it rather than demoting the whole path — isSessionTokenRejection is the worked example. Descope's validateSession throws identically for an expired token and for its own outage, so the predicate recognises token rejections, and anything it does not recognise stays at error. Default to error when unsure: a mislabelled rejection costs one noisy log, a mislabelled outage costs the signal.