dcp-gateway

DCP Gateway

MCP transparent compression proxy — DCP-encode tool responses without server modification.

Any MCP server → Gateway → 40-70% fewer tokens on repeated calls.

How it works

Client (Claude Code)  ←stdio→  DCP Gateway  ←stdio→  MCP Server (unmodified)

The gateway sits between the MCP client and server as a stdio relay:

• tools/list — strips JSON Schema meta-fields ($schema, additionalProperties, redundant descriptions) to slim tool definitions
• tools/call (1st) — Infers a DCP schema from the response, caches it as provisional v1, and DCP-encodes immediately
• tools/call (2nd+) — DCP-encodes using the cached schema; Validator accumulates samples in the background
• tools/call (N+1) — Validator promotes schema to confirmed v2 (null-rate filtered); subsequent calls use the refined schema

Wrapper unwrapping

Many APIs return { total_count, items: [...] } instead of a flat array. The gateway detects single-array wrapper objects, DCP-encodes the inner array, and prepends scalar fields as a JSON metadata line:

{"total_count":4939,"incomplete_results":false}
["$S","search_repositories:v1","id","name","full_name",...]
[9028031,"DCPathButton","Tangdixi/DCPathButton",...]
[65034998,"dcping","example/dcping",...]

Usage

dcp-gateway --server <command> [args...] [--presets <dir>]

Claude Code settings.json

{
  "mcpServers": {
    "github-via-gateway": {
      "command": "node",
      "args": [
        "/path/to/dcp-gateway/dist/src/cli.js",
        "--server", "npx", "@modelcontextprotocol/server-github"
      ],
      "env": {
        "GITHUB_PERSONAL_ACCESS_TOKEN": "..."
      }
    }
  }
}

Multi-server mode (Phase 4)

Single gateway endpoint fronting N MCP servers. Client sees one unified tool list; tool names are prefixed with alias__ to prevent collisions.

dcp-gateway --servers gateway.json

Config format

// gateway.json
{
  "servers": [
    { "alias": "github", "command": "node", "args": ["/path/to/server-github/dist/index.js"] },
    { "alias": "slack",  "command": "node", "args": ["/path/to/server-slack/dist/index.js"] }
  ]
}

Tool routing

Client calls	Routed to	As
github__search_repositories	github server	search_repositories
slack__list_channels	slack server	list_channels

Aggregation behavior

• tools/list — broadcast to all servers, merge results, compress definitions
• tools/call — route by alias prefix to correct server, DCP-encode response
• initialize / other — broadcast, first response wins
• DCP cache — per-server ResponseInterceptor; alias-prefixed tool names prevent cross-server collisions

Field filtering (planned)

REST APIs often return 60-120+ fields per object, most irrelevant to the caller. DCP encoding alone removes key repetition but still carries every field. Field filtering reduces this further.

Layer model

Layer 0: Raw       — all fields pass through (current behavior, default)
Layer 1: User      — ignore/only list per tool (user writes in presets)
Layer 2: Hard cap  — maxFields safety valve (last resort)

Layer 0 — no filtering. Gateway works as pure DCP encoder. Default.

Layer 1 — User profile (in presets dir, per tool)

// presets/github.filter.json
{
  "search_repositories": {
    "ignore": ["node_id", "owner.gravatar_id", "owner.subscriptions_url"]
  },
  "list_issues": {
    "only": ["id", "title", "state", "user.login", "labels", "created_at"]
  }
}

• ignore — blocklist, drop these fields (dot-notation source paths)
• only — allowlist, keep only these fields
• Mutually exclusive per tool. only takes precedence if both present.

Layer 2 — Hard cap (maxFields in profile or CLI)

After Layer 1, if field count still exceeds cap, truncate. Defaults to Infinity (off).

Why no smart auto-filter

An earlier design included a Layer 1 "smart auto" that cut fields with null rate ≥ 0.9, derived from the Phase 1 sample. This was dropped for the following reasons:

• Phase 1 is a single sample. One response with many nulls (e.g. a sparse result set, an off-peak query, a paginated tail) would permanently exclude fields that are populated in normal usage. There is no statistical basis for a decision at N=1.
• Full output is easier to debug. Seeing all fields in Phase 1 makes it obvious what the API returns. Silently dropping fields obscures this.
• User intent is clearer. A human reading the Phase 1 output and writing ignore: [...] makes a deliberate choice. Auto-filter makes that choice invisibly.

If smart filtering is revisited, the minimum viable approach would be to accumulate N≥10 samples before computing null rates, and to treat the threshold conservatively (= 1.0, i.e. field was null in every observed response). Even then, user confirmation before persisting the filtered schema is advisable.

Schema lifecycle (planned)

Schemas are not static — they are versioned and refined over time by a SchemaValidator that runs alongside the encoding pipeline.

State machine

provisional v1  →  (N samples observed)  →  confirmed v2
                →  (drift detected)       →  provisional v(n+1)
confirmed   vN  →  (major drift)          →  provisional v(N+1)

• provisional — schema inferred from the first response. DCP encoding starts immediately. Good enough for most cases; refinement happens in the background.
• confirmed — schema promoted after N samples. Null-rate filtering applied: fields that were null in all N observed samples are excluded.
• versioned — each promotion increments the version, reflected in the $S header (tool:v1, tool:v2, ...).

Validator responsibilities

Event	Action
N samples accumulated	Compute null rate per field; exclude all-null fields; promote to confirmed
Type drift detected	Invalidate, re-infer from latest sample → new provisional
New field appeared	Invalidate, re-infer from latest sample → new provisional

N defaults to 10 (configurable). Null-rate threshold is 1.0 (field absent in all N samples) — conservative by design. A lower threshold risks discarding fields that are merely sparse in the observed window. See also: Why no smart auto-filter.

Why encode immediately from provisional

Waiting for confirmed before encoding would mean N calls with no compression benefit — the gateway would appear broken. Provisional encoding gives immediate visible effect. The confirmed upgrade is transparent: the client sees a more compact $S from call N+1.

SchemaCacheEntry additions (planned)

interface SchemaCacheEntry {
  schema: DcpSchema;
  mapping: FieldMapping;
  version: number;                      // 1 = provisional, increments on each promotion
  status: "provisional" | "confirmed";
  sampleCount: number;                  // samples observed by Validator
  lastUsed: number;
  hitCount: number;
}

Verification results (GitHub MCP Server)

Tested against @modelcontextprotocol/server-github v0.6.2, 8 tools, 30 items per response.

Tool	JSON (chars)	DCP (chars)	Reduction	Fields
search_repositories	6,201	3,816	-38%	23
search_users	37,388	23,042	-38%	20
search_issues	137,140	92,775	-32%	123
list_issues	153,915	110,081	-29%	61
list_commits	176,221	127,210	-28%	60
list_pull_requests	199,346	135,617	-32%	85
get_file_contents	372,832	372,767	~0%	single obj + base64
search_code	152	152	0%	0 results

• Phase 1 pass-through + Phase 2 DCP encode confirmed for all tools
• Wrapper {total_count, items} correctly unwrapped
• Nested objects flattened with parent-prefix naming (user_url, assignee_login)
• $R nested sub-schemas generated for array-of-objects fields (labels, assignees)

LLM parsing accuracy (Haiku)

Test	Accuracy	Notes
Schema ID reference	100% (3/3 trials)	Cross-schema queries, aggregation, reverse lookup
$R nested structure	100% (3/3 trials)	Empty arrays, flag discrimination, cross-row lookup
Positional 120 fields	No degradation	Field count is not the bottleneck
$S offset error	~15% stochastic	+2 position shift, solved with 2-line format hint

Test suite

npm test   # tsc && node --test dist/**/*.test.js

schema-validator.test.ts — SchemaValidator + ResponseInterceptor unit tests

Suite	What it covers
SchemaValidator — batch mode	N rows at once → promoted to confirmed v2; always-null fields removed; partially-null fields retained
SchemaValidator — incremental mode	stays provisional until N-th sample; does not promote twice
SchemaValidator — drift detection	new field → provisional re-infer; type mismatch → provisional re-infer; fresh sample accumulation toward next promotion after drift
ResponseInterceptor — DCP lifecycle	valid DCP on first call (provisional v1); upgrades at N; batch promotion drops nulls; drift mid-stream stays valid; wrapper unwrap preserved through lifecycle

aggregator.test.ts — AggregatorRelay integration tests

Process-level tests: spawns the compiled gateway with two independent instances of mock-mcp-server.ts (aliases srvA, srvB), communicates via stdio JSON-RPC.

Suite	What it covers
tools/list merges with alias prefix	all 4 tools (2 × 2 servers) present; correct alias__ prefixes; descriptions preserved; no duplicate names
alias__toolname routing	srvA__search_users routes to correct server, returns DCP with expected fields; srvB__get_metrics likewise; cross-alias calls both work; unknown alias → error -32601
ResponseInterceptor isolation per server	schema IDs differ by alias; srvA's schema version advances faster than srvB's (proves separate interceptor instances, not a shared cache); different tool names within the same server are also keyed independently

Design note — isolation assertion: instead of checking for a specific version number (fragile: drift on repeated data causes extra reinfers), the test drives srvA with 9+ more calls than srvB and asserts srvAVersion > srvBVersion. If interceptors were shared, both would sit at the same version — this assertion would fail.

Manual test scripts

test-tavily.mjs — manual integration test for Tavily API shape. Kept at repo root, excluded from npm test via .gitignore (test-*.mjs pattern). Run directly with:

node test-tavily.mjs

Architecture

src/
  cli.ts                  — CLI entry, arg parsing (single + multi-server)
  relay.ts                — Single-server stdio JSON-RPC relay
  aggregator.ts           — Multi-server aggregator (Phase 4)
  server-connection.ts    — Per-server child process wrapper
  interceptor.ts          — Phase 1/2 DCP encoding logic + wrapper unwrap
  schema-validator.ts     — SchemaValidator: provisional→confirmed lifecycle, drift detection
  tools-list-compressor.ts — tools/list definition slimming
  field-filter.ts         — User-defined field filtering (ignore/only lists + null-rate filter)
  schema-cache.ts         — LRU N-limit schema cache (128 entries default)
  preset-loader.ts        — Pre-built schema loading from disk
  types.ts                — Shared types
  schema-validator.test.ts — Unit tests: SchemaValidator + ResponseInterceptor
  aggregator.test.ts      — Integration tests: AggregatorRelay (routing, isolation, merging)
tests/
  mock-mcp-server.ts      — Minimal MCP server (search_users + get_metrics) used by aggregator.test
  integration-test.ts     — End-to-end relay test

Schema registry format

Schemas are persisted as JSON (DcpSchemaDef). Human-readable, directly inspectable.

{
  "$dcp": "schema",
  "id": "search_repositories:v2",
  "fields": ["id", "name", "full_name", "stargazers_count"],
  "fieldCount": 4,
  "types": { "id": { "type": "number" }, ... }
}

Future: DCP-encoded registry

A registry of N schemas is itself a collection of uniform rows — a natural fit for DCP. When the registry grows large enough, encoding it would reduce the token cost of loading all schemas into context.

More importantly, in a multi-agent setting an agent that encounters an unknown tool response could fetch the relevant schema from a DCP-encoded registry directly, decoding it with minimal context overhead. This requires:

• A schema-of-schemas ($S header describing the fields of a DcpSchemaDef row)
• A registry endpoint agents can query (MCP resource, HTTP, or file)
• A decoder available to the agent at query time

The current JSON format is designed to be forward-compatible with this: each registry entry is a uniform object, so the transition to a DCP-encoded registry is a mechanical transformation. No schema redesign needed when the time comes.

Dependencies

• dcp-wrap — Schema inference, DCP encoding/decoding
• Node.js ≥ 20

Platform notes

Windows: .cmd/.bat server commands (e.g. npx.cmd) are spawned with shell: true automatically.

License

Apache-2.0