@postman/aiei-engine

@postman/aiei-engine

Scoring engine for the AI Engineering Intelligence & Effectiveness Index.

Evaluate organizational AI/agent readiness and measure AI-assisted engineering outcomes across quality, lead time, reliability, cost, and security. This package provides the formulas, connectors, signal routing, and presentation layer that power the CLI, MCP server, and GitHub Action.

Installation

npm install @postman/aiei-engine

Quick Start

Score from a signal payload

import { scoreFromSignals } from '@postman/aiei-engine';

const assessment = scoreFromSignals({
  schemaVersion: '2.0.0',
  orgName: 'your-org',
  signals: {
    defect_rate_ai: { value: 0.07, source: 'measured' },
    defect_rate_nonai: { value: 0.12, source: 'measured' },
    ai_test_pass_rate: { value: 0.91, source: 'measured' },
    rework_rate_ai: { value: 0.16, source: 'measured' },
    cycle_time_ai: { value: 18, source: 'measured' },
    cycle_time_nonai: { value: 42, source: 'measured' },
    deploy_frequency: { value: 3.5, source: 'measured' },
    cfr_ai: { value: 0.08, source: 'measured' },
    slo_attainment: { value: 0.96, source: 'measured' },
    uptime_ai: { value: 0.995, source: 'measured' },
    total_ai_cost: { value: 7200, source: 'estimated' },
    hours_saved_monthly: { value: 126, source: 'estimated' }
  }
}, {
  weightProfile: 'reliability_first',
  hourlyRate: 175
});

console.log(assessment.result.composite);      // 0.68
console.log(assessment.result.dimensions);     // { Q: {...}, L: {...}, R: {...}, C: {...}, S: {...} }
console.log(assessment.presentation?.overall); // score, hours, annualized value, ROI

Run a full assessment with connectors

import { runAssessment } from '@postman/aiei-engine';

const result = await runAssessment({
  orgName: 'your-org',
  weightProfile: 'reliability_first',
  hourlyRate: 175,
  connectors: {
    github: { token: process.env.GITHUB_TOKEN, org: 'your-org', repos: ['api-service'] },
    postman: { apiKey: process.env.POSTMAN_API_KEY, workspaceId: 'ws-id' },
  },
});

console.log(result.composite);
console.log(result.presentation?.overall.value_recovered_annual);

Agent readiness gate

import { assessAgentReadiness, validateSignalPayload } from '@postman/aiei-engine';

const payload = { schemaVersion: '2.0.0', orgName: 'your-org', signals: {} };
const validation = validateSignalPayload(payload);
const readiness = assessAgentReadiness(payload.readinessSignals, payload.runtimeMeasures);
// readiness.gates: discover, understand, execute, recover, govern

Exports

Path	What It Contains
@postman/aiei-engine	Full engine: scoreFromSignals, runAssessment, connectors, types
@postman/aiei-engine/scoring	Dimension scorers: scoreQ, scoreL, scoreR, scoreC, scoreS
@postman/aiei-engine/connectors	Connector registry: ALL_CONNECTORS, SIGNAL_ROUTES, buildCoverageMap
@postman/aiei-engine/presentation	buildPresentation, buildConnectorStory, narrative types

Key Types

import type {
  CompositeResult,       // Full assessment result with dimensions, weights, epistemic notes
  PresentationResult,    // Narratives, benchmarks, AX mapping, connector stories
  ConnectorStoryBlock,   // Per-dimension tool narrative with Postman tie-back
  EpistemicSummary,      // What the score knows and doesn't know
  AXPillarMapping,       // Maps dimensions to Access/Context/Tools/Orchestration
  DimensionNarrative,    // Per-dimension story, formula, factors, benchmarks, action
  EnvironmentConfig,     // .aiei.json config shape
} from '@postman/aiei-engine';

Bundled Files

File	Purpose
openapi.yaml	HTTP contract for the assessment API
llms.txt	Agent-oriented package discovery index
docs/TELEMETRY.md	Telemetry transparency documentation
docs/DEVELOPER_INDEX.md	Entry points for humans and coding agents

40 Connectors

GitHub, GitLab, GitHub Copilot, Postman, Datadog, Dynatrace, Grafana, PagerDuty, Cloudflare, New Relic, Splunk, ServiceNow, Vercel, Supabase, Linear, Jira, Jellyfish, GitHub Actions, OpenAI, Anthropic, Google AI, Portkey, Inngest, SonarQube, 42Crunch, Fern, AWS Bedrock, AWS CloudWatch, AWS CodePipeline, AWS CodeBuild, AWS X-Ray, AWS CodeGuru, AWS Cost Explorer, AWS Step Functions, Azure OpenAI, Azure Monitor, Azure DevOps, Microsoft Defender, OpenTelemetry, Wix.

License

MIT

Built by Postman. Formulas are open, auditable, and versioned.