White Towers Architecture Pattern

Singleton services that maintain persistent state and respond to events.

The White Towers are centralized services in the Oculus system that act as the source of truth for specific domains. They follow a consistent pattern:

Core Principles

1. Single Instance (Singleton)

• ONE global instance per service type
• Created at API startup
• Shared across all requests
• No creating new instances on every operation

2. Persistent State

• State loaded from disk at startup
• State saved to disk on changes
• Survives restarts - the text doesn't change, why lose the index?
• Eventually backed by Mongo/DynamoDB, but for now: JSON on disk

3. Event-Driven Updates

• Services listen for events (node saved, node deleted, etc.)
• Updates are incremental - only process what changed
• No full rebuilds except explicit emergency requests
• Pattern: "I'm node X, I have Y" → service does the needful

4. Lazy Cleanup

• If data references something that no longer exists, remove on access
• Better to have stale pointers than missing valid data
• Eventual consistency is acceptable

Current White Towers

Fence Index (FenceIndexBuilder)

Purpose: Global index of all fences for fast ${fence:label} resolution

Location: services/oculus-api/oculus/fence_index_builder.py

State: ~/.local/share/oculus/.cache/fence-index/fence-index.json

Pattern:

# At startup (api.py)
fence_indexer = FenceIndexBuilder(registry, index_dir)
fence_indexer.load_from_disk()  # Load existing state

# On node save
fence_indexer.update_node(slug)  # Incremental update
fence_indexer.save_to_disk()     # Persist

# On fence access (variable_substitution.py)
fence_meta = fence_indexer.get_fence_by_label('pantry')
if not fence_meta:
    return "Fence not found"

# Lazy cleanup (if fence doesn't exist when accessed)
try:
    execute_fence(fence_meta)
except FenceNotFoundError:
    fence_indexer.remove_fence(fence_id)

Node Registry (Registry)

Purpose: Index of all nodes with metadata (tags, type, source)

Location: services/oculus-api/oculus/registry.py

State: ~/.local/share/oculus/.cache/registry.yaml

Current Issues:

• ❌ Calls reload() which rescans entire filesystem
• ❌ Should use incremental update_node(slug) instead
• 🔧 Needs refactor to match White Tower pattern

Target Pattern:

# At startup
registry = Registry(storage_dir)
registry.load_from_disk()  # Don't scan filesystem

# On node save
registry.update_node(slug, metadata)
registry.save_to_disk()

# On node delete
registry.remove_node(slug)
registry.save_to_disk()

Future White Towers

AST Cache

• Cache parsed AST tokens
• Invalidate on file modification
• Persist to disk for fast startup

Variable Resolution Cache

• Cache resolved ${var} expansions
• Invalidate when dependencies change
• Dependency tracking built-in

Search Index

• Full-text search across all nodes
• Incremental updates on save
• Eventually Elasticsearch/Algolia

Anti-Patterns to Avoid

❌ Creating New Instances

# BAD - creates new instance, reads stale disk state
indexer = FenceIndexBuilder(...)
fence = indexer.get_fence('foo')

# GOOD - use global singleton
fence = fence_indexer.get_fence('foo')

❌ Full Rebuilds on Every Change

# BAD - rebuilds entire index on every save
def on_save(slug):
    indexer.rebuild_index()

# GOOD - incremental update
def on_save(slug):
    indexer.update_node(slug)

❌ Not Persisting State

# BAD - loses state on restart
def __init__(self):
    self.index = {}

# GOOD - persist and reload
def __init__(self):
    self.index = self._load_from_disk()

❌ Blocking on Stale References

# BAD - errors if reference is stale
fence_meta = index.get_fence('foo')
if not fence_meta:
    index.rebuild_index()  # Expensive!
    fence_meta = index.get_fence('foo')

# GOOD - lazy cleanup
fence_meta = index.get_fence('foo')
if not fence_meta:
    return "Fence not found"
# On access, if it fails:
try:
    execute(fence_meta)
except NotFoundError:
    index.remove_fence(fence_id)  # Clean up stale ref

Implementation Checklist

When creating a new White Tower:

• Single global instance created at API startup
• load_from_disk() method - loads existing state
• save_to_disk() method - persists current state
• update_node(slug) method - incremental update
• remove_node(slug) method - incremental removal
• Event hook registration (e.g., on_save, on_delete)
• No reload() or rebuild() in hot path
• Lazy cleanup for stale references
• State persists across restarts

Benefits

Performance: No filesystem scanning, no full rebuilds Consistency: Single source of truth Resilience: State survives restarts Scalability: Ready for database backend (Mongo/Dynamo) Developer Experience: Predictable, fast, reliable

Migration Path

Current → Database-Backed:

# Phase 1: JSON files (current)
def save_to_disk(self):
    with open(self.state_file, 'w') as f:
        json.dump(self.state, f)

# Phase 2: MongoDB
def save_to_db(self):
    self.collection.replace_one(
        {'_id': 'singleton'},
        self.state,
        upsert=True
    )

# Phase 3: Redis/DynamoDB (hot path)
def save_to_cache(self):
    redis.set('state:fence_index', 
              json.dumps(self.state))

The file-based implementation is the foundation. Once it works, swapping in a database is trivial.

---

The White Towers stand eternal, unchanged by the chaos below. They persist.

Slots

North

slots:
- architecture

South

slots: []

East

slots: []

West

slots: []