bulltrackers-module

API v4

Policy-driven Express API for Bulltrackers: centralized auth/capability rules, Zod validation, domain read/write services, and OpenAPI derived from the same handler schemas.

Requirements

Runs as part of bulltrackers-module (parent package.json provides express, firebase-admin, pino, zod, BigQuery/Firestore clients, etc.). Tests expect Node ≥ 20 and Jest 30+.

Quick start

const http = require('http');
const { createApiV4App, attachGracefulShutdown } = require('./core-api');

const clients = {
  firestore: /* admin.firestore() */,
  bigquery: /* BigQuery client */,
  storage: /* optional Storage */,
};

const app = createApiV4App(clients);
const server = http.createServer(app);
attachGracefulShutdown(server); // SIGTERM / SIGINT — use for long-lived processes (Cloud Run, k8s)
server.listen(process.env.PORT || 8080);

Exports from core-api.js:

Export	Purpose
createApiV4App(clients)	Express app with middleware, req.services, routes
createApiV4Server(clients)	Same as createApiV4App today (alias)
attachGracefulShutdown(server, options?)	Drain HTTP on shutdown

Architecture

Request flow

• CORS → request ID (X-Request-ID) + req.log (Pino child) → express.json
• req.services injection (db, dataService, writeService, domains, billingService, authService)
• Routes registered before context: public paths only (ROUTE_KEYS_BEFORE_BUILD_CONTEXT)
• buildContext — Firebase token / test auth, CID lookup, tier, dev/test shadow
• Idempotency (optional) — see below
• Remaining routes: guard → optional capability / feature flags → Zod validate → handler
• 404 → error handler (sanitized messages, structured logs)

Policy table

All routes are declared in config/accessPolicies.js as ROUTE_POLICIES: HTTP method + path string → guard (public | authenticated | verified | pro), optional capability, featureFlag, extraGuards.

Registration order is not manual: registerFromPolicies.js sorts keys so static path segments beat :params and longer paths win (see utils/routeRegistrationSort.js).

Handlers

Handlers live under routes/handlers/, merged by collectHandlers() (routes/handlers/index.js). Each export key must match a ROUTE_POLICIES entry exactly. Parity is enforced by:

• npm test → testing/accessPolicyHandlerParity.test.js
• CLI: node scripts/verify-route-policy-parity.js (from this directory)

Handlers may be a function or { schema: { body|query|params }, handler } using shared Zod pieces in schemas/.

Services & DI

• req.app.locals.config — frozen config from config/index.js
• pathsFromRequest(req) — Firestore path resolution from injected config (config/firestorePaths.js); avoid static require('../config') in handlers
• req.services.dataService / writeService — facades for backward compatibility
• req.services.domains.read — { billing, firestore, computation } (prefer for new features)
• req.services.domains.write — { watchlists, reviews, alerts, analytics, settings, devOverrides, masterList }
• req.services.billingService — Stripe portal + checkout session docs

Docs & typing

• GET /openapi.json — built from policies + handler Zod via utils/generateOpenApi.js (Zod 4 toJSONSchema; transforms may omit some request bodies). Serves a public-only spec (excludes /dev/*, /settings/dev/*, /workspace/admin/*) to avoid leaking internal endpoints. Use scripts/export-openapi-spec.js for full spec (frontend codegen); add --public to export public-only.
• jsconfig.json + types/express-services.d.ts — IDE autocomplete for req.services and req.log without a TS build

Configuration

Primary module: config/index.js (Stripe URLs, BigQuery dataset, Firestore collection map, CORS origins, billing usersCollection override, etc.). Override via environment variables documented in that file.

Useful env vars for ops:

Variable	Effect
LOG_LEVEL	Pino level (info, debug, silent, …)
ENABLE_TEST_AUTH	X-Test-Firebase-User header (CI / emulator only)
UPSTASH_REDIS_*	Redis: tier cache, idempotency, verification tickets
GOOGLE_APPLICATION_CREDENTIALS	GCP clients (outside Firebase Functions)

Operational behavior

Structured logging

Pino logs JSON lines suitable for Cloud Logging / Datadog. Child loggers include requestId and service: 'api-v4'. Sensitive headers are redacted in config.

Idempotency

For POST /sync/request and POST /billing/purchase, if Idempotency-Key is sent (≤ 128 chars) and Upstash Redis is configured, the first successful res.json body is cached 24h and replayed for the same user + path + key (SHA-256 scoped key).

Firestore pagination (alerts / notifications)

Opaque base64url cursors (utils/firestoreTimelineCursor.js) tie-break orderBy(time, 'desc').orderBy(documentId, 'desc'). Legacy plain document IDs still work as cursors. Deploy composite indexes for those query shapes before production.

Graceful shutdown

Use attachGracefulShutdown(server) so SIGTERM stops accepting new connections and finishes in-flight work before exit. Upstash is HTTP-based; no Redis socket to close.

Testing

From functions/api-v4:

npx jest --config jest.config.js testing

testing/setup.js wires theFirestore emulator and test env as needed. Integration tests use supertest against createApiV4App.

Directory map

Path	Role
core-api.js	App factory, service graph, middleware order
config/	accessPolicies, firestorePaths, env-backed index
middleware/	buildContext, guard, validate, errorHandler, idempotency, …
routes/handlers/	Route implementations (keys = policy keys)
routes/registerFromPolicies.js	Binds policies → Express
schemas/	Shared Zod fragments
services/	DataService, WriteService, BillingService, AuthService, data/*, write/*
utils/	OpenAPI gen, route sort, logger, cursors, graceful shutdown
functions/	Side-effect triggers (sync, computation, notifications) used by handlers
websocket/	Verification WS glue (thin)
scripts/	CI helpers (verify-route-policy-parity.js)
testing/	Jest tests + Firestore helpers

Related documentation

• Route ↔ handler parity: run scripts/verify-route-policy-parity.js in CI after changing policies or handlers.
• Collection layout updates (e.g. watchlist membership sharding) may require Core/config/collection_registry.js and downstream ETL alignment.

License / ownership

Part of the Bulltrackers backend monorepo; internal use unless the repository root states otherwise.