What is Synthetic Strategic Intelligence?
SSI is a classification, not a marketing term. Three properties define it. Each is engineered into the architecture. The benchmarks below verify that these properties function as specified.
Synthetic
Engineered intelligence.
Not statistical pattern guessing. The system computes responses through explicit algorithms: intent classification, entity extraction, knowledge retrieval, reasoning, and verification. Every output is traceable to a pipeline stage. The retrieval traceability benchmarks below verify this.
Strategic
Chooses strategies, not just answers.
The system selects reasoning strategies, profiles user preferences, and adapts tone, depth, and style in real time. It maintains continuous internal state across conversations and modulates 16 emotion values. The memory consistency benchmarks below verify this.
Intelligence
Knows when it doesn't know.
Metacognition. Quality gates. Hallucination detection. Fact verification. Self-insight. The system evaluates its own responses and rejects outputs that fail. Then says so. The evidence grounding benchmarks below verify this directly.
Overview
The SSI Benchmark Suite provides research-grade verification of the NovaSpire cognitive operating system. Every test produces a full audit trail: input → pipeline stages → subsystems → retrieved data → evidence chain → output → hash verification.
The benchmark verifies that tested outputs are constructed through observable retrieval, reasoning, verification, and evidence-trace stages within the evaluated system. Each output can be traced back to specific pipeline stages and subsystems that produced it. The tests demonstrate the presence of these mechanisms, not the absence of all other influences.
Every benchmark run uses identical controls designed to make results reproducible across machines and sessions:
- Fixed random seed (42) · Free Will Engine, Flow Engine, and Response Generator produce deterministic output variation
- Fresh memory database ·
condrox_memory.dbdeleted before each run · no stale memories affect retrieval - Skip disk knowledge load · no external knowledge files loaded · only BKS-seeded knowledge used
- Full audit trail · every output has a SHA-256 hash for cross-run comparison
Requirements: Python 3.11+, spaCy en_core_web_md, same requirements.txt dependencies, and the same common_knowledge.txt file (checksum verified below).
This benchmark suite and the underlying SSI architecture were developed by a single engineer. As a result, the current evaluation focuses on core mechanisms rather than full-scale statistical testing or multi-team validation. The goal of this phase is to demonstrate architectural functionality, traceability, reproducibility, and subsystem behavior, not to present a complete research program.
Future versions will expand test coverage, introduce baselines, and include broader methodological controls. For now, the results should be interpreted within the scope of a solo-developer environment: limited resources, iterative development, and emphasis on mechanism verification over exhaustive evaluation.
Three Verification Pillars
Run Benchmark
Click the button below to execute the full benchmark suite. The API will run ssi_benchmark.py and return JSON results with full trace logs, evidence chains, and hash verification.
Results Summary
1. Retrieval Traceability
Verifies that the pipeline retrieves real data for each input. Each test shows: input, query terms, retrieved documents (with source classification), and whether the data was used in reasoning.
2. Memory Consistency
Verifies that the same input produces the same structural state (intent, domain, knowledge set) and consistent core content across multiple runs. Also tests memory seed + recall.
3. Evidence Grounding
Verifies that each output can be traced back to specific pipeline stages and subsystems. Shows the full evidence chain: output → pipeline stage → subsystem → data source → data summary.
4. Full Benchmark Suite
Complete reproducible benchmark: 10 inputs, each with full pipeline trace, evidence chain, and hash verification. Other researchers can run the same script and verify the same audit trail.
5. Reproducibility Controls
The benchmark suite is designed to produce comparable results across machines and runs when the same environment is replicated. The following controls are enforced automatically by ssi_benchmark.py:
- Clone the repository and install dependencies:
pip install -r requirements.txt - Install spaCy model:
python -m spacy download en_core_web_md - Run the benchmark:
python scripts/ssi_benchmark.py - Compare your output hashes with the ones shown above · they should match
Download Benchmark Package
Download a ZIP package containing the full JSON results, the benchmark script, and the traceability module. Other researchers can reproduce the tests locally.
- benchmark_results.json · Full JSON output with all test results, evidence chains, hashes, and reproducibility metadata
- ssi_benchmark.py · Benchmark script for local reproduction
- traceability.py · Traceability module (RetrievalTrace, EvidenceLog, PipelineTrace)
- common_knowledge.txt · Knowledge base file (278 entries) for consistent BKS seeding
- README.txt · Reproduction instructions (seed, spaCy model, Python version, checksum verification)