Methodology
How These Numbers Were Produced
All benchmarks were run on the actual Condrox AI system using scripts/benchmark_pipeline.py. The benchmark script sends requests through the full 18-stage pipeline with all 26 conversational intelligence components active, knowledge retrieval enabled, and the memory graph operational. No stages were skipped or simplified.
Each benchmark run executes 1,000 sequential requests with varied input types: questions, code requests, emotional inputs, greetings, and clarifications. Latency is measured per request from input to final output. Throughput is measured as requests per second over the full run. Memory is measured as peak RSS during execution.
Pipeline Latency
Average time from user input to final response, measured across 1,000 requests:
| Metric | Before Optimization | After Optimization | Improvement |
|---|---|---|---|
| Total average latency | 1.513 ms | 0.748 ms | 2.0x faster |
| Median latency | 1.490 ms | 0.720 ms | 2.1x faster |
| 95th percentile | 2.100 ms | 1.050 ms | 2.0x faster |
| 99th percentile | 2.800 ms | 1.400 ms | 2.0x faster |
| Max latency | 3.200 ms | 1.800 ms | 1.8x faster |
For comparison, typical LLM API latencies range from 200 ms to 3,000 ms. Condrox AI achieves sub-millisecond response time through deterministic algorithms and in-memory knowledge retrieval.
Throughput
Requests processed per second on a single machine:
| Configuration | Requests/second | Requests/minute | Requests/hour |
|---|---|---|---|
| Before optimization | 298 | 17,880 | 1,072,800 |
| After optimization | 1,785 | 107,100 | 6,426,000 |
The 6x throughput improvement came from regex pattern caching, embedding cache LRU eviction, and context manager optimization. No stages were removed or simplified.
Memory Usage
| Metric | Before | After | Change |
|---|---|---|---|
| Average peak memory | 0.012 MB | 0.010 MB | 17% reduction |
| Knowledge base (loaded) | ~45 MB | ~45 MB | Unchanged (21,917 items) |
| Memory graph (per session) | ~0.5 MB | ~0.5 MB | Unchanged |
| Embedding cache (LRU) | Unbounded | 256 entries max | Bounded |
LRU eviction for embedding caches was added in Phase 9 (Hardening & Persistence). It bounds memory usage while keeping cache hit rates above 90% in typical workloads.
Per-Stage Latency Breakdown
Average time spent in each pipeline stage (microseconds):
| Stage | Name | Avg time (μs) | % of total |
|---|---|---|---|
| 01 | Preprocessor | 12 | 1.6% |
| 02 | Intent Detection | 45 | 6.0% |
| 02b | Intent Enrichment | 28 | 3.7% |
| 03 | Mode Selector | 8 | 1.1% |
| 04 | Knowledge Retrieval | 180 | 24.1% |
| 05 | Memory Retrieval | 95 | 12.7% |
| 05b | Context Enrichment | 22 | 2.9% |
| 06 | Reasoning Preparation | 15 | 2.0% |
| 07 | Reasoning Core | 110 | 14.7% |
| 08 | Reasoning Synthesis | 35 | 4.7% |
| 08b | Agent Execution | 48 | 6.4% |
| 09 | Response Synthesis | 85 | 11.4% |
| 09b | Response Quality | 18 | 2.4% |
| 10 | Personality | 5 | 0.7% |
| 11 | Ethics | 8 | 1.1% |
| 12 | Security | 10 | 1.3% |
| 13 | Output Builder | 6 | 0.8% |
| 14 | Delivery | 4 | 0.5% |
| Total | 18 stages | 748 μs | 100% |
Knowledge retrieval (Stage 04) is the most expensive stage at 24.1% of total time, followed by reasoning core (Stage 07) at 14.7% and memory retrieval (Stage 05) at 12.7%. These three stages account for over 50% of pipeline latency. This is expected: they perform the core cognitive work.
Test Suite Performance
| Test Category | Tests | Execution Time | Pass Rate |
|---|---|---|---|
| Unit tests | 639 | 1.8s | 100% |
| Integration tests | 90 | 0.6s | 100% |
| API tests | 8 | 0.2s | 100% |
| Storage tests | 11 | 0.1s | 100% |
| Admin tests | 24 | 0.1s | 100% |
| AI-OS tests | 51 | 0.2s | 100% |
| BM25/Conversation tests | 32 | 0.1s | 100% |
| Evidence Engine tests | 83 | ~190s | 100% |
| Total | 938 | ~210s | 100% |
The full test suite runs in approximately 210 seconds. Evidence Engine tests exercise the full BrainController through multi-turn conversations, reasoning tasks, agent invocations, and full-system integration scenarios. That is why they take longer than the unit tests.
Conversation Suite Results
51 conversation tests across 12 categories, all passing:
| Category | Tests | Passed | Failed | Pass Rate |
|---|---|---|---|---|
| Greetings | 5 | 5 | 0 | 100% |
| Factual Knowledge | 8 | 8 | 0 | 100% |
| Conversational Continuity | 5 | 5 | 0 | 100% |
| Emotional Intelligence | 8 | 8 | 0 | 100% |
| Code Mode | 5 | 5 | 0 | 100% |
| Analyze Mode | 5 | 5 | 0 | 100% |
| Architect Mode | 5 | 5 | 0 | 100% |
| Explain Mode | 5 | 5 | 0 | 100% |
| Clarification | 2 | 2 | 0 | 100% |
| Politeness | 1 | 1 | 0 | 100% |
| Edge Cases | 1 | 1 | 0 | 100% |
| Unknown Intent | 1 | 1 | 0 | 100% |
| Total | 51 | 51 | 0 | 100% |
Knowledge Retrieval Performance
BM25 search performance across the 21,917-item knowledge base:
| Metric | Value | Notes |
|---|---|---|
| Total knowledge items | 21,917 | Across 15 domains |
| Inverted index size | ~12,000 unique terms | After stemming and stop-word filtering |
| Average query time | 0.18 ms | For top-5 retrieval |
| Average candidates per query | ~140 documents | Before BM25 scoring |
| BM25 scoring time | ~0.12 ms | For 140 candidates |
| Cache hit rate (embedding) | >90% | With 256-entry LRU cache |
Reproducibility
How to Reproduce These Results
All benchmarks can be reproduced by running the actual scripts on the Condrox AI codebase:
# Run full benchmark
python scripts/benchmark_pipeline.py
# Run full test suite
python -m pytest tests/ -v
# Run conversation suite
python scripts/conversation_suite.py
# Run stress test
python scripts/conversation_stress_test.py 20 8 1000Results may vary slightly depending on hardware. The order of magnitude and relative improvements should remain consistent.