zod-contract-testing

📁 apankov1/quality-engineering 📅 Today
0
总安装量
1
周安装量
安装命令
npx skills add https://github.com/apankov1/quality-engineering --skill zod-contract-testing

Agent 安装分布

amp 1
cline 1
opencode 1
cursor 1
continue 1
kimi-cli 1

Skill 文档

Zod Contract Testing

Test schemas at boundaries — not just happy-path inputs.

Schemas define contracts between systems. A schema that accepts invalid data is a security hole. A schema that rejects valid data is a broken integration. This skill teaches you to systematically test schemas at system boundaries.

When to use: Testing API request/response schemas, WebSocket message validation, database record parsing, external data ingestion, any Zod schema at a system boundary.

When not to use: Internal type assertions, UI component props, type definitions without runtime validation.

Rationalizations (Do Not Skip)

Rationalization Why It’s Wrong Required Action
“The type system ensures correctness” TypeScript doesn’t exist at runtime Test Zod parsing with real data
“I tested valid inputs” Invalid inputs cause production errors Test rejection of invalid inputs
“Refinements are simple” .refine() failures are easy to miss Test BOTH passing and failing cases
“Optional fields are optional” 2^N combinations have hidden bugs Use compound state matrix

Included Utilities

import {
  testValidInput,
  testInvalidInput,
  testSchemaEvolution,
  testRefinement,
  generateCompoundStateMatrix,
  formatStateMatrix,
  applyCompoundState,
} from './schema-boundary.ts';

Core Workflow

Step 1: Test Valid Inputs

Verify the schema accepts all valid input shapes:

import { z } from 'zod';

const UserSchema = z.object({
  name: z.string(),
  age: z.number().optional(),
});

it('accepts valid user', () => {
  testValidInput(UserSchema, { name: 'Alice', age: 30 });
  testValidInput(UserSchema, { name: 'Bob' });  // age optional
});

Step 2: Test Invalid Inputs

Verify the schema rejects invalid data and errors at the correct path:

it('rejects missing required field', () => {
  testInvalidInput(UserSchema, {}, 'name');  // Error at 'name' path
});

it('rejects wrong type', () => {
  testInvalidInput(UserSchema, { name: 123 }, 'name');
});

it('rejects unknown fields with strict schema', () => {
  const StrictUserSchema = UserSchema.strict();
  testInvalidInput(StrictUserSchema, { name: 'Alice', role: 'admin' });
});

Step 3: Test Schema Evolution

When schemas change, old serialized data must still parse:

// Old schema: { name: string }
// New schema: { name: string, email?: string }

const NewUserSchema = z.object({
  name: z.string(),
  email: z.string().email().optional(),
});

it('backward compatible with old data', () => {
  const oldData = { name: 'Alice' };  // No email field
  testSchemaEvolution(NewUserSchema, oldData);
});

Step 4: Test Refinements

Every .refine() MUST have tests for both passing and failing cases:

const PositiveNumberSchema = z.object({
  value: z.number().refine(n => n > 0, 'Value must be positive'),
});

it('refinement: positive vs non-positive', () => {
  testRefinement(
    PositiveNumberSchema,
    { value: 10 },           // Passing case
    { value: -5 },           // Failing case
    'must be positive',      // Expected error message
  );
});

Step 5: Generate Compound State Matrix

For schemas with N optional fields, there are 2^N possible combinations. For 3-4 fields, test exhaustively. For 5+, switch to pairwise coverage (see decision table below):

// Cell schema: value?, candidates?, isGiven?
// 2^3 = 8 combinations

const matrix = generateCompoundStateMatrix(['value', 'candidates', 'isGiven']);
console.log(formatStateMatrix(matrix));
/*
| # | value | candidates | isGiven | Label |
|---|-------|------------|---------|-------|
| 0 | -     | -          | -       | (empty) |
| 1 | Y     | -          | -       | value |
| 2 | -     | Y          | -       | candidates |
| 3 | Y     | Y          | -       | value + candidates |
| 4 | -     | -          | Y       | isGiven |
| 5 | Y     | -          | Y       | value + isGiven |
| 6 | -     | Y          | Y       | candidates + isGiven |
| 7 | Y     | Y          | Y       | value + candidates + isGiven |
*/

Step 6: Apply Matrix to Schema Tests

Generate test inputs from the matrix:

const CellSchema = z.object({
  value: z.number().optional(),
  candidates: z.array(z.number()).optional(),
  isGiven: z.boolean().optional(),
}).refine(
  cell => !(cell.isGiven && cell.value === undefined),
  'Given cells must have a digit value',
);

describe('cell schema compound states', () => {
  const matrix = generateCompoundStateMatrix(['value', 'candidates', 'isGiven']);
  const template = { value: 5, candidates: [1, 3, 7], isGiven: true };

  for (const entry of matrix) {
    const input = applyCompoundState(entry, template);
    const shouldFail = input.isGiven && input.value === undefined;

    it(`${entry.label}: ${shouldFail ? 'rejects' : 'accepts'}`, () => {
      if (shouldFail) {
        testInvalidInput(CellSchema, input);  // Object-level .refine() reports at root path
      } else {
        testValidInput(CellSchema, input);
      }
    });
  }
});

Compound State Matrix Decision

Optional Fields Combinations Testing Approach
0-2 1-4 Enumerate manually
3 8 Use matrix, manageable
4 16 Use matrix, essential
5+ 32+ Switch to pairwise-test-coverage — covers all field pairs in near-minimal test cases

Violation Rules

missing_invalid_input_test

Every Zod schema MUST have tests for invalid inputs, not just valid ones. Severity: must-fail

missing_refinement_coverage

Every .refine() or .superRefine() MUST have tests for both the passing AND failing case. Severity: must-fail

missing_compound_state_test

Schemas with 3+ optional fields SHOULD use compound state matrix. For 5+ fields, switch to pairwise-test-coverage rather than testing all 2^N individually. Severity: should-fail

schema_not_at_boundary

Zod parsing MUST happen at system boundaries (API handlers, WebSocket messages, database reads), not inside business logic. Severity: should-fail

type_assertion_instead_of_parse

Use Schema.parse() or Schema.safeParse(), NEVER as Type casts for external data. Severity: must-fail


Companion Skills

This skill teaches testing methodology, not Zod API usage. For broader methodology:

  • Search zod on skills.sh for schema authoring, transforms, error handling, and framework integrations
  • Our utilities work with any Zod version (v3, v4) via the ZodLikeSchema interface — no version lock-in
  • Schemas with 5+ optional fields produce 32+ combinations — use pairwise-test-coverage for near-minimal coverage of all field pairs
  • Schema parsing at boundaries often logs errors — use observability-testing to assert structured log output on validation failures
  • Schema evolution testing pairs with resilience — use fault-injection-testing for retry and circuit breaker testing around schema-validated endpoints

Quick Reference

Utility When Example
testValidInput Verify acceptance testValidInput(schema, { name: 'Alice' })
testInvalidInput Verify rejection testInvalidInput(schema, {}, 'name')
testSchemaEvolution Backward compat testSchemaEvolution(newSchema, oldData)
testRefinement Pass + fail testRefinement(schema, passing, failing, message)
generateCompoundStateMatrix Optional fields generateCompoundStateMatrix(['a', 'b', 'c'])
applyCompoundState Generate input applyCompoundState(entry, template)

See patterns.md for Zod-specific patterns, boundary testing methodology, and integration with TypeScript strict mode.