[DI Phase 1] Foundation + monorepo prep (tokenizers, daemon, SearchResult)

[bashandbone]

Phase 1: DI Foundation + Monorepo Prep

Parent Epic: #116
Target: Week 1 (5-7 days)
Risk Level: Low

Build core DI infrastructure while preparing for monorepo split by extracting ready packages and moving shared types.

Goals

• Core DI components (Container, Depends(), factories)
• Type-safe dependency resolution
• Override mechanism for testing
• Extract ready packages (tokenizers, daemon)
• Move SearchResult types to core
• 100% test coverage on DI
• Zero production changes

Why This Phase Matters

This phase creates the foundation that will eliminate 75% of circular dependencies in Phase 2. It also extracts packages that are already independent, giving us quick wins.

Implementation Checklist

Part A: Package Extraction (Days 1-2)

Extract immediately (already independent):

• Create packages/codeweaver-tokenizers/

  • Move src/codeweaver/tokenizers/ → package
  • Create pyproject.toml
  • Test independence
  • Update imports

• Create packages/codeweaver-daemon/

  • Move src/codeweaver/daemon/ → package
  • Create pyproject.toml
  • Test independence
  • Update imports

Move shared types:

• Create src/codeweaver/core/search_types.py
• Extract SearchResult from agent_api/find_code/types.py
• Update imports across codebase
• Validate no circular dependencies introduced

Part B: DI Infrastructure (Days 3-5)

Core components:

• Create codeweaver/di/ package structure
  • __init__.py - Public API
  • container.py - Container implementation
  • depends.py - Depends marker
  • providers.py - Provider factories
  • types.py - Type definitions

Container Implementation:

• Container class with registration mechanism
• Resolution with caching (singleton support)
• Override mechanism for testing
• Lifecycle hooks (startup/shutdown)
• Error handling and validation
• Async support

Provider Factories:

• get_embedding_provider() factory
• get_sparse_embedding_provider() factory (optional)
• get_reranking_provider() factory (optional)
• get_vector_store() factory
• Type aliases (EmbeddingDep, VectorStoreDep, etc.)

Key Pattern:

# Factory hides complexity
async def get_embedding_provider() -> EmbeddingProvider:
    """Factory auto-resolves from registry + settings."""
    from codeweaver.common.registry import get_provider_registry
    registry = get_provider_registry()
    # ... all the messy logic HERE, not in services
    return registry.get_embedding_provider_instance(...)

# Type alias for clean signatures
EmbeddingDep = Annotated[EmbeddingProvider, Depends(get_embedding_provider)]

Part C: Testing (Days 6-7)

Comprehensive testing:

• Container registration tests
• Container resolution tests
• Override mechanism tests
• Singleton behavior tests
• Lifecycle hooks tests
• Factory tests
• Integration tests
• 100% coverage on DI infrastructure

Test examples:

def test_container_override():
    container = Container()
    container.register(EmbeddingProvider, get_embedding_provider)
    container.override(EmbeddingProvider, MockEmbedding())
    result = await container.resolve(EmbeddingProvider)
    assert isinstance(result, MockEmbedding)

Part D: Documentation

• API documentation (docstrings)
• Usage examples
• Testing guide
• Architecture decision records
• Migration guide (draft)

Package Placement

DI infrastructure location:

• Initially: src/codeweaver/di/ (within main package)
• Phase 3: Move to packages/codeweaver-core/ (foundation layer)

Provider factories:

• Initially: src/codeweaver/di/providers.py
• Phase 3: Stay with or move to packages/codeweaver-providers/

Why Extract Packages Now?

Tokenizers & Daemon:

• ✅ Already independent (0 dependency violations)
• ✅ Perfect test cases for monorepo structure
• ✅ Quick wins that prove the approach
• ✅ Won't be affected by DI migration

SearchResult types:

• ✅ Breaks provider → agent_api dependency
• ✅ Makes core types truly foundational
• ✅ Enables clean package boundaries in Phase 3

Acceptance Criteria

• Container can register/resolve dependencies
• Override mechanism works for testing
• All tests passing (100% coverage on DI)
• Tokenizers extracted and building independently
• Daemon extracted and building independently
• SearchResult in core with no circular deps
• No changes to existing production code (except imports)
• Documentation complete
• Code review approved
• Performance baseline established

Deliverables

End of Week 1:

1. ✅ Working DI system (tested, documented)
2. ✅ 2 packages extracted (tokenizers, daemon)
3. ✅ SearchResult types moved to core
4. ✅ Zero production behavior changes
5. ✅ Ready for Phase 2 migration

Connection to Monorepo Strategy

This phase creates the foundation for Week 1 of the integrated strategy:

• DI foundation → Enables Phase 2 to break circular dependencies
• Package extraction → Proves monorepo structure works
• Type movement → Eliminates key coupling points

Success Metrics

Technical:

• 0 test failures
• 100% coverage on DI code
• 2 packages building independently
• SearchResult moved with 0 violations

Quality:

• Clear documentation
• Simple, obvious API design
• FastAPI-familiar patterns
• Easy to extend

Next Phase Preview

Phase 2 (#118) will use this foundation to:

• Migrate Indexer to DI → Eliminates 20+ violations
• Migrate search services → Eliminates 30+ violations
• Update tests with overrides → Simplifies testing by 80%

Reference

• Planning: INTEGRATED_DI_MONOREPO_STRATEGY.md (Week 1)
• Examples: plans/DI_PROVIDER_EXAMPLES.md
• Original plan: plans/dependency-injection-architecture-plan.md

[github-actions]

Claude finished Adam Poulemanos (@bashandbone)'s task —— View job

---

Issue Analysis: DI Phase 1 Infrastructure

Summary

This issue tracks Phase 1 of the Dependency Injection Epic (#116) - building core DI infrastructure without modifying production code. This is the foundation for a multi-phase migration from manual provider instantiation to FastAPI-inspired dependency injection.

Goal: 60-70% reduction in dependency boilerplate, improved testability, and scalability to 100+ providers.

---

Context

Parent Epic (#116)

The DI Epic outlines a comprehensive 5-phase plan to replace manual provider instantiation with dependency injection:

• Phase 1 (This Issue): Core DI infrastructure - Zero production changes
• Phase 2 ([DI Phase 2] Core migration - Breaking 75% of circular dependencies #118): Migrate core services (Indexer, search)
• Phase 3 ([DI Phase 4] pydantic-ai integration (agents + data sources) #119): Integrate pydantic-ai providers
• Phase 4 ([DI Phase 5] Advanced features (health, telemetry, plugins) #120): Advanced features (health checks, telemetry)
• Phase 5 ([DI Phase 6] Cleanup, registry simplification, and v0.4 breaking changes #121): Cleanup and deprecation

Current State

The codebase currently uses manual provider instantiation via registries:

• ProviderRegistry in src/codeweaver/common/registry.py (~1500 lines)
• Manual fetching for 40+ providers across 6 types
• No declarative dependencies
• Hard to test (manual mocking required)

Example of current pattern (from Epic #116):

class Indexer:
    def __init__(self):
        registry = get_provider_registry()
        provider = registry.get_embedding_provider()
        self.embedding = registry.get_embedding_provider_instance(provider, singleton=True)

Desired pattern after DI:

from codeweaver.di.providers import EmbeddingDep, VectorStoreDep

class Indexer:
    def __init__(self, embedding: EmbeddingDep, vector_store: VectorStoreDep):
        self.embedding = embedding
        self.vector_store = vector_store

---

Key Observations

1. Missing Planning Documents

The issue references planning documents that do not exist in the repository:

• ❌ plans/DI_IMPLEMENTATION_CHECKLIST.md
• ❌ plans/DI_PROVIDER_EXAMPLES.md
• ❌ plans/DI_ARCHITECTURE_SUMMARY.md
• ❌ plans/dependency-injection-architecture-plan.md
• ❌ plans/DI_ARCHITECTURE_DIAGRAMS.md

Current plans/ directory contains:

• flexible-extension-system-spec.md
• telemetry-integration-checklist.md
• IMPLEMENTATION_PLAN.md
• node_type_parser_implementation_spec.md
• telemetry-metrics-implementation-plan.md

2. Constitutional Compliance

This work aligns well with the Project Constitution:

✅ Principle II (Proven Patterns): Channels FastAPI's DI system
✅ Principle III (Evidence-Based): Incremental, testable phases
✅ Principle V (Simplicity): Hides complexity in factories
✅ Principle I (AI-First Context): Self-documenting dependencies

3. No Existing DI Infrastructure

No src/codeweaver/di/ directory exists yet. This Phase 1 work starts from scratch.

4. Dependency on External Libraries

The DI system will likely use FastAPI/pydantic patterns:

• pydantic>=2.12.3 (already in dependencies)
• pydantic-settings>=2.11.0 (already in dependencies)
• Patterns from fastmcp>=2.12.5 (dependency injection context)

---

Related Issues

Will Obsolete/Simplify

As noted in Epic #116, this work will significantly impact:

• Register default services in services registry #84: Register default services - DI will handle this
• Improve EmbeddingRegistry: configurable size, dynamic sizing, persistence #100: EmbeddingRegistry improvements - registries become thin layer
• Integrate Indexer with services registry and implement persistence loading #103: Indexer service integration - DI replaces current approach

Related Architecture Work

• Implement dynamic provider registration system #83: Implement dynamic provider registration system
• Implement provider system for reranker models (mirror embedding system) #109: Implement provider system for reranker models
• Convert provider TypedDicts to Pydantic models #93: Convert provider TypedDicts to Pydantic models

Part of DI Epic Series

• [EPIC] DI + Monorepo: Unified architecture transformation #116 (Parent): [EPIC] Replace registries with dependency injection system
• [DI Phase 1] Foundation + monorepo prep (tokenizers, daemon, SearchResult) #117 (This): Phase 1 - Core infrastructure
• [DI Phase 2] Core migration - Breaking 75% of circular dependencies #118: Phase 2 - Core service migration
• [DI Phase 4] pydantic-ai integration (agents + data sources) #119: Phase 3 - pydantic-ai integration
• [DI Phase 5] Advanced features (health, telemetry, plugins) #120: Phase 4 - Advanced features
• [DI Phase 6] Cleanup, registry simplification, and v0.4 breaking changes #121: Phase 5 - Cleanup and deprecation

---

Implementation Notes

Phase 1 Scope (From Checklist)

This issue covers only infrastructure building:

Infrastructure

• Create codeweaver/di/ package
• Modules: __init__.py, container.py, depends.py, providers.py, types.py

Container Implementation

• Registration mechanism
• Resolution with caching (singleton support)
• Override mechanism for testing
• Lifecycle hooks (startup/shutdown)
• Error handling and validation
• Async support

Provider Factories

• get_embedding_provider() factory
• get_sparse_embedding_provider() factory (optional)
• get_reranking_provider() factory (optional)
• get_vector_store() factory
• Type aliases (EmbeddingDep, VectorStoreDep, etc.)

Testing

• 100% coverage on DI infrastructure
• Container registration/resolution tests
• Override mechanism tests
• Singleton behavior tests
• Lifecycle hooks tests
• Factory tests
• Integration tests

Documentation

• API documentation (docstrings)
• Usage examples
• Testing guide
• Architecture decision records

Critical Success Criteria

From the issue acceptance criteria:

• ✅ All tests passing
• ✅ Zero changes to existing production code (key constraint)
• ✅ Documentation complete
• ✅ Code review approved
• ✅ Performance baseline established

---

Recommendations

1. Create Planning Documents First

Before implementation, create the referenced planning documents to ensure alignment:

• plans/DI_IMPLEMENTATION_CHECKLIST.md - Detailed implementation tracking
• plans/DI_PROVIDER_EXAMPLES.md - Code examples showing usage patterns
• plans/DI_ARCHITECTURE_SUMMARY.md - Quick overview for contributors

Rationale: Evidence-based development (Constitution Principle III) requires documented plans before implementation.

2. Start with Minimal Viable DI System

Focus on core functionality first:

1. Container with basic registration/resolution
2. Depends() marker
3. One provider factory (embedding) as proof of concept
4. Override mechanism for testing

Then expand to full feature set once patterns are proven.

3. Reference Existing FastMCP Patterns

CodeWeaver already uses fastmcp>=2.12.5. Review FastMCP's dependency injection patterns for consistency:

• See data/context/apis/complete_docs/fast-mcp-VERY-LARGE.md (23,000 lines)
• See data/context/apis/complete_docs/fast-mcp-summary.md (140 lines)

4. Consider pydantic-graph Integration

The codebase has pydantic-graph>=1.4.0 available. Consider if pydantic-graph's dependency management could complement or replace custom DI infrastructure.

• See data/context/apis/complete_docs/pydantic-graph.md (1052 lines)

5. Performance Baseline Early

Establish performance baseline measurements before building DI system to validate "within 5% of baseline" acceptance criterion.

---

Suggested Labels

Based on issue content and project structure:

• ✅ enhancement (already applied)
• ✅ architecture (already applied)
• ✅ dependency-injection (already applied)
• 🔵 phase-1 - Track phase progression
• 🔵 infrastructure - Foundation work
• 🔵 testing-required - 100% coverage requirement
• 🔵 v0.2-early - Target milestone

---

Questions for Clarification

1. Planning Documents: Should these be created as part of Phase 1 work, or do they exist elsewhere (e.g., in a branch)?

2. FastMCP vs Custom DI: Should the DI system integrate with FastMCP's existing DI patterns, or be independent?

3. pydantic-graph: Has pydantic-graph been evaluated as an alternative to building custom DI infrastructure?

4. Migration Strategy: The epic mentions "v0.4" as the breaking change timeline (12+ months). Is this still the plan?

---

Difficulty Estimate

Complexity: Medium-High

Rationale:

• Building production-grade DI system from scratch
• Must support async operations
• Requires lifecycle hooks and complex caching
• 100% test coverage requirement
• Must be generic enough for 40+ providers across 6 types

Estimated Effort: 2-4 weeks for experienced developer

• Week 1: Container + basic registration/resolution
• Week 2: Provider factories + type system
• Week 3: Testing infrastructure (100% coverage)
• Week 4: Documentation + edge cases

Impact: High

• Foundation for entire DI migration
• Will affect all future provider-related work
• Sets patterns for Phases 2-5

---

Next Steps

If you'd like me to proceed with implementation:

1. Create planning documents in plans/ directory
2. Implement core DI package at src/codeweaver/di/
3. Build comprehensive test suite in tests/di/
4. Document API and usage patterns

Should I proceed with implementation, or would you like to provide additional guidance first?

---