Mock Drift Detection

What is mock drift?

Mock drift occurs when a real upstream service evolves (adding fields, changing response codes, modifying headers) but the mock stubs used in your tests remain unchanged. This leads to tests that pass against stale stubs but fail against the real service.

MockServer's drift detection automatically compares forwarded (real) upstream responses against your stub expectations and flags structural differences, giving you early warning that your mocks need updating.

How it works

When MockServer forwards a request to an upstream service (in proxy mode) and there are also response-type stub expectations matching the same request pattern, MockServer compares the real response against each matching stub and records any structural drifts. The analysis runs asynchronously and never slows down the response path.

Structural drift detection (status, schema and header differences) is automatic and requires no configuration — simply have both forwarding expectations and response-type stub expectations for the same request patterns. The two enrichments described later on this page are opt-in: semantic drift analysis (LLM-powered severity classification) and performance drift detection each need to be turned on with a configuration property.

What is detected

Non-semantic headers (date, x-request-id, content-length, transfer-encoding, connection, keep-alive, server) are automatically excluded from comparison.

Retrieving drift records

Use the GET /mockserver/drift endpoint to retrieve detected drifts:

curl http://localhost:1080/mockserver/drift

Response:

{
  "count": 2,
  "drifts": [
    {
      "expectationId": "abc-123",
      "driftType": "STATUS",
      "field": "statusCode",
      "expectedValue": "200",
      "actualValue": "422",
      "confidence": 1.0,
      "epochTimeMs": 1717145600000
    },
    {
      "expectationId": "abc-123",
      "driftType": "SCHEMA_FIELD_ADDED",
      "field": "$.newField",
      "confidence": 0.9,
      "epochTimeMs": 1717145600000
    }
  ]
}

Filtering

Filter by expectation ID:

curl "http://localhost:1080/mockserver/drift?expectationId=abc-123"

Limit the number of results (default 50, max 500):

curl "http://localhost:1080/mockserver/drift?limit=10"

The limit parameter applies only to unfiltered queries (most-recent-first). When you filter by expectationId, all matching records are returned and limit is ignored.

Clearing drift records

To clear all recorded drifts:

curl -X PUT http://localhost:1080/mockserver/drift/clear

Drift records are also cleared automatically when MockServer is reset via PUT /mockserver/reset.

Storage

Drift records are stored in an in-memory LRU (least recently used) buffer with a maximum capacity of 1000 entries. When the buffer is full, the oldest records are evicted to make room for new ones.

Semantic drift analysis (LLM-powered)

When you have a runtime LLM backend configured (see Configuration Properties), you can enable semantic drift analysis to automatically classify each structural drift as BREAKING, WARNING, or INFORMATIONAL.

Enable it with:

-Dmockserver.driftSemanticAnalysisEnabled=true

When enabled, MockServer sends the drift details along with truncated stub and real response bodies to your configured LLM. The LLM classifies each drift and provides a one-sentence explanation. These are added to the drift records as semanticSeverity and semanticExplanation fields.

Semantic analysis is best-effort: if the LLM is unavailable or returns an error, drift records are stored with their original structural data only.

Example response with semantic fields

{
  "count": 1,
  "drifts": [
    {
      "expectationId": "abc-123",
      "driftType": "SCHEMA_FIELD_REMOVED",
      "field": "$.role",
      "expectedValue": "admin",
      "confidence": 0.95,
      "epochTimeMs": 1717145600000,
      "semanticSeverity": "BREAKING",
      "semanticExplanation": "Removing the role field will break clients that depend on it for authorization"
    }
  ]
}

Performance drift detection

MockServer can track response times per expectation and alert you when the p95 (95th percentile) response time exceeds a threshold. This helps detect performance regressions in upstream services.

Enable it by setting a threshold in milliseconds:

-Dmockserver.driftResponseTimeThresholdMs=500

When the p95 response time for any expectation exceeds 500ms, a PERFORMANCE drift record is emitted. MockServer uses a sliding window of the last 100 response time observations per expectation to calculate percentiles.

Set to 0 (the default) to disable performance drift detection.

Baseline Traffic Comparison

The GET /mockserver/drift endpoint detects drift from live forwarded traffic at runtime. For offline CI use — comparing a committed snapshot of expectations against the current set without running a proxy — use the baseline compare endpoint.

Send a PUT request to /mockserver/baseline/compare with a JSON body containing a baseline array of expectations. MockServer compares the baseline against a current array (if supplied) or against its own live active expectations (if current is omitted). It returns HTTP 200 with a structured drift report.

Request body

{
  "baseline": [ <expectations> ],
  "current":  [ <expectations> ]
}

The current field is optional. When omitted, MockServer uses its live active expectations — useful for checking whether newly recorded traffic matches a committed baseline.

Response body

{
  "hasDrift": false,
  "added":   [],
  "removed": [],
  "changed": []
}

Each entry in added and removed contains only the key (matching key, see below). Each entry in changed also carries requestDiffs and responseDiffs arrays of field-level differences, each with a field, diffType (ADDED, REMOVED, or CHANGED), and expectedValue/actualValue.

Matching key

Interactions are correlated across the two sets by a stable key of METHOD normalized-path: method is upper-cased, and a single trailing slash is stripped from the path (but / is preserved). For example, GET /api/users/ and GET /api/users share the same key GET /api/users.

What counts as drift

The comparison is value-insensitive for JSON response bodies: only structural differences are reported. A different value at the same field with the same JSON type is NOT drift. The following ARE drift:

• A new or removed JSON field in the response body
• A JSON type change at any field (e.g. string became number, object became array)
• A response status code change
• A header added, removed, or whose value changed
• A request field change (method, path, headers, query parameters, body)

Non-JSON response bodies fall back to exact-string comparison.

CI usage pattern

A typical CI workflow: commit a baseline expectations file alongside your tests. In CI, start MockServer, run your test suite (which populates expectations via recording or explicit setup), then call PUT /mockserver/baseline/compare with the committed baseline as baseline and no current (so MockServer uses its live expectations). If hasDrift is true, fail the build.

curl -s -X PUT "http://localhost:1080/mockserver/baseline/compare" \
  -H "Content-Type: application/json" \
  -d "{\"baseline\": $(cat baseline-expectations.json)}" \
  | python3 -c "import sys,json; r=json.load(sys.stdin); sys.exit(1 if r['hasDrift'] else 0)"

Response codes

Not the same as LLM cassette drift

The drift detection described on this page is for HTTP mock stubs — it compares forwarded upstream responses against your stub expectations while MockServer proxies live traffic. It is separate from the LLM-specific detect_llm_drift MCP tool, which is a different mechanism: that tool replays a recorded LLM cassette (fixture) against the live LLM provider and reports structural drift (new/removed fields, type changes) in the provider's responses. It requires a configured runtime LLM backend and is intended for an opt-in / scheduled CI lane rather than the per-commit build. If you are validating recorded LLM/MCP traffic against a real provider, use detect_llm_drift; for everything else, use the /mockserver/drift endpoint documented here.