Runtime test interactions

The suite uses a bounded product for the common generated-parser pipeline and focused tests for runtime features that do not fit that product. Matrix size is not a proxy for coverage: every runtime-facing feature still needs an explicit interaction assessment.

contract_version: 1
matrix:
  axes:
    algorithm: [slr, lalr, ielr, lr1]
    table: [plain, compact]
    cst: ["off", "on"]
    locations: ["off", "on"]
    entries: [single, multi, isolated]
  expected_cases: 96
  representative_cases: 12
  roles:
    normal: representative
    scheduled: full
    promotion: full_review
scheduled_long_gates: ["test:matrix:full", "test:adversarial", "fuzz:long"]
golden:
  task: "test:zero_cost"
  policy: feature_off_byte_identity
  digest_path: test/golden/digests.yml
  sources:
    - test/fixtures/compat/calculator.y
    - benchmark/grammars/representative.y
    - gallery/json/grammar.y
  update_command: "bundle exec rake golden:update"
interactions:
  - id: parser_semantics
    axes: [algorithm, table, cst, locations, entries]
    coverage: matrix
    tests: [test/tooling/matrix_runner_test.rb]
  - id: generated_lexer
    axes: [algorithm, table, cst, locations, entries]
    coverage: matrix_and_focused
    tests: [test/codegen/lexer_test.rb, test/frontend/lexer_test.rb]
  - id: semantic_actions
    axes: [table, locations, cst]
    coverage: matrix_and_focused
    tests: [test/runtime/action_contract_test.rb, test/analysis_no_exec_test.rb]
  - id: table_encoding
    axes: [algorithm, table, cst, entries]
    coverage: matrix_and_focused
    tests: [test/tables_test.rb, test/runtime/table_format_test.rb]
  - id: locations
    axes: [locations, table, cst, entries]
    coverage: matrix_and_focused
    tests: [test/location_test.rb, test/runtime/action_contract_test.rb]
  - id: cst
    axes: [cst, algorithm, table, locations, entries]
    coverage: matrix_and_focused
    tests: [test/codegen/cst_runtime_integration_test.rb, test/runtime/cst_serialize_test.rb]
  - id: entry_modes
    axes: [entries, algorithm, table, cst]
    coverage: matrix_and_focused
    tests: [test/codegen/multiple_start_test.rb]
  - id: parser_drivers
    axes: [table, cst, locations]
    coverage: focused_regression
    tests: [test/runtime/parser_test.rb, test/runtime/push_parser_test.rb, test/runtime/parser_table_session_test.rb]
  - id: generated_ast
    axes: [algorithm, table, locations]
    coverage: focused_regression
    tests: [test/codegen/ast_test.rb, test/tooling/matrix_runner_test.rb]
  - id: recovery
    axes: [algorithm, table, cst, entries]
    coverage: focused_regression
    tests: [test/runtime/repair_test.rb, test/runtime/sync_recovery_test.rb]
  - id: resource_limits
    axes: [algorithm, table, cst, entries]
    coverage: matrix_and_focused
    tests: [test/runtime/resource_limits_test.rb]
  - id: observation
    axes: [table, cst, locations]
    coverage: focused_regression
    tests: [test/runtime/observation_test.rb, test/codegen/embedded_tracer_test.rb]
  - id: incremental_cst
    axes: [cst, table, locations]
    coverage: focused_regression
    tests: [test/runtime/cst_incremental_test.rb]
  - id: syntax_session
    axes: [algorithm, table, cst, locations, entries]
    coverage: focused_regression
    tests: [test/runtime/syntax_session_test.rb, test/packaging/runtime_gem_test.rb]
  - id: embedded_runtime
    axes: [table, cst]
    coverage: focused_regression
    tests: [test/packaging/runtime_gem_test.rb, test/codegen/ractor_shareability_test.rb]

The quality validator compares this record with test/matrix.yml, the CI workflow, the golden generator, and the files named by every interaction.

The 96-case product

The full matrix is exactly:

4 algorithms × 2 table encodings × 2 CST states × 2 location states × 3 entry modes = 96

The algorithm axis is SLR, LALR, IELR, and canonical LR(1). The table axis is plain and compact. CST and semantic locations are independently off or on. The entry axis is a single start, shared multi-start automaton, or isolated multi-start construction.

Each case normalizes and round-trips Grammar IR, builds and round-trips Automaton IR, proves deterministic generated bytes, executes valid and invalid input, compares algorithm/table behavior within the same CST/location/entry shape, and enforces a stack resource limit. The runner separately checks generated AST node equivalence for every algorithm/table pair.

The normal pull-request gate chooses twelve deterministic representatives from the ordered product. Twelve is a fast regression net, not a claim of pairwise coverage and not a substitute for feature-focused tests. Scheduled CI runs all 96 cases plus bounded hostile-input tests and the 100,000-sentence fuzz run. Promotion of a Preview or Experimental runtime feature requires a reviewed full-96 result on the candidate revision even if scheduled CI was recently green.

Interaction map and covering rules

The machine-readable map names the current runtime-facing families and their owned tests. matrix means the 96-product is the primary coverage. matrix_and_focused combines product coverage with semantic edge cases. focused_regression means the feature is not an independent matrix axis and the named suite owns its current protocol and edge cases. It does not claim that every listed axis pair is already covered; a change to that row must apply the covering rules below and record the chosen combinations in its tests.

For a new runtime-facing feature:

List every existing axis and feature family whose state or observable result

it can affect.

Add it as a matrix axis only when all values are cheap, orthogonal, and

meaningful for the common pipeline. Update the declared product count in the same change.

Otherwise select a deterministic covering set containing every new value,

every affected existing value, and each pair the implementation says can interact. Record the selection rule and own it in a focused test.

Use feature-specific exhaustive tests for small finite state spaces,

protocol transitions, version readers, and failure-before-input behavior.

Add scheduled long fuzz/property cases only when shorter deterministic tests

cannot provide the same signal. Seeds, counts, and budgets stay fixed.

Pairwise coverage is the minimum for declared interactions, not permission to ignore a known three-way dependency. A bug involving three or more settings is promoted to a permanent focused regression and its interaction is added to the map. Unsupported combinations must be rejected explicitly and tested; omitting them from a generator is not coverage.

Current non-product boundaries include embedded versus installed runtime, pull/yyparse/push driver transitions, action marker versions, recovery and repair state, observation hooks, incremental reuse, serialization, Ractor shareability, and packaging. Their focused suites remain mandatory even though they do not multiply the 96 common cases.

Syntax sessions require CST and a generated lexer, so a disabled-CST value is rejected by the service boundary rather than multiplied into the common product. test/runtime/syntax_session_test.rb exhaustively crosses all four algorithms, both table encodings, and shared versus isolated multi-entry construction (16 cases). A separate four-case cross covers both table encodings with grammar actions that omit or reference semantic locations. Production actions, including their semantic-location references, never execute in a syntax session, so that location-action choice is independent of algorithm and entry construction; CST source locations remain mandatory internally. The remaining tests own trust acknowledgement, fresh-result equivalence, fallback metrics, cancellation, resource bounds, and malformed edit sequences. test/packaging/runtime_gem_test.rb owns the embedded-runtime acquisition path.

Normal, scheduled, and promotion gates

Normal pushes and pull requests run the twelve representative cases and the feature-off golden check as part of the bounded safety net. The weekly schedule runs test:matrix:full, long adversarial settings, and fuzz:long. Manual workflow dispatch does not currently run the full matrix or long fuzz merely by being manual; use the explicit local tasks when preparing a promotion.

A promotion review records the exact candidate revision and includes:

all 96 matrix cases;
every focused suite named by the promoted feature's interaction row;
version-reader, invalid-version, and regeneration diagnostics when an ABI is

involved;

the feature-off golden diff; and
any scheduled long property or fuzz evidence assigned to the feature.

Scheduled success is supporting evidence only. It is not a silent maturity promotion and does not replace review of the exact candidate change.

Golden zero-cost policy

test:zero_cost regenerates three committed parsers and requires both exact bytes and their SHA-256 index to match. When a new feature is disabled, its implementation must not change these reviewed generated bytes. This is a source-size and emitted-code zero-cost contract for the fixed corpus, not a runtime performance or allocation claim.

If a cross-cutting correction intentionally changes feature-off output, run bundle exec rake golden:update and review the generated source diff and digest index in the same pull request. Never update the digest merely to make a failure green. Add a new representative source when the existing three cannot exercise a stable feature-off boundary.

Merge rule

A change under the runtime-facing paths in the ABI policy cannot merge without a valid structured ABI assessment. The automated check verifies the declared versions, matrix arithmetic, scheduled commands, golden inputs, and assessment structure. Human review still decides whether the chosen ABI boundary and interaction set match the behavior of the change.