lantern

wanderland-paper-implementation

Implementation

Technology Stack

Wanderland is implemented as a collection of services:

Service Purpose Technology
oculus-api Graph engine, fence execution, path resolution Python/FastAPI
stuffy Browser-based graph viewer/editor HTML/JS/marked.js
task-api Detective case management Python/FastAPI
knowledge-base-api Semantic search over graph Python/ChromaDB
mcp-proxy MCP tool exposure Python
ccp Service orchestration Bash/systemd

All services communicate via HTTP APIs. The MCP proxy exposes graph operations as tools that AI agents can invoke directly.

The AST Pipeline

Document processing follows the AST converter pipeline:

  • Tokenization: Markdown parsed via markdown-it-py (GFM-like mode with tables, strikethrough)
  • Fence Detection: State machine tracks fence boundaries (handles nested fences)
  • Section Extraction: Headers create hierarchical structure
  • Slot Parsing: Cardinal sections (N/S/E/W) parsed for graph edges
  • Provenance Injection: Auto-injects verification sections on save
  • Virtual Fence Detection: Scans for patterns (todo lists, tables) and annotates tokens

The fence detector uses a state machine to handle edge cases:

  • Bare fences (no language specifier)
  • Nested fences (fences within fences)
  • Fence attributes ([key=value] syntax)

Fence Execution

Fences execute based on language and attributes. The system supports multiple execution backends:

Language Execution Model Use Case
python exec() with injected context General computation, data transforms
yaml Parse and return as data Configuration, slot definitions
bash Subprocess with timeout Shell commands, scripts
sql Database query via connection string External database queries
aws boto3 API calls via service/method attrs AWS resource queries
http HTTP requests via url/method attrs External API integration

Python fences receive injected context: config (parameters), graph (API access), and result (output dict). Fences can query other nodes, execute sub-fences, and return structured data.

AWS fences use declarative YAML with attributes specifying service and method. A fence marked aws[service=ec2,method=describe_instances] containing filter YAML executes the boto3 call and returns instance data. This enables live infrastructure queries embedded in documentation.

HTTP fences fetch external APIs. Attributes specify url and method; fence content provides headers and body. Variable substitution (e.g., ${SSM_TOKEN}) injects secrets at execution time.

SQL fences query external databases via connection strings in attributes. The fence content is the query; results return as structured data for downstream processing or display.

Execution is sandboxed per-fence. Results cache at L3. Middleware transforms at L3.5 before final caching.

Virtual Fences

Beyond explicit fences, the system detects virtual fencesβ€”patterns in prose that expose stateful behavior through multiple interfaces:

Pattern Virtual Fence Actions
- [ ] Task todo-list add, toggle, check, uncheck, remove
<!-- mode: radio --> radio-list single-select variant
` Col Col
## Provenance provenance review, verify, check-drift

The same virtual fence is accessible through multiple interfaces:

MCP Tools: AI agents invoke oculus_execute_virtual_fence with action and parameters.

CLI (toodles): Human-friendly commands: toodles check my-node:Tasks 0

Web Components: Stuffy renders :::wonderland-todo-list as clickable checkboxes.

Shell Integration (snippy): Fish functions query radio lists as configuration selectors.

All clients are isomorphicβ€”they consume the same HTTP API with identical semantics. The CLI wraps the API. The web component polls the API. The MCP tool calls the API. The shell function curls the API. This means:

  • No client-specific logic in the server
  • Any new interface requires only API bindings
  • Behavior is identical regardless of access method
  • Testing one client validates all clients

The isomorphism extends beyond documents: the client layer exhibits the same structural equivalence as the document/compiler/database mapping. A checklist is simultaneously documentation, API, CLI, and widgetβ€”not through synchronization, but through architectural collapse.

The Graph Index

Nodes register in an in-memory index on startup:

GraphIndex:
  nodes: Dict[slug, GraphNode]
  fences: Dict[fence_id, FenceMetadata]  
  tags: Dict[tag, Set[slug]]
  edges: Dict[slug, Dict[direction, List[slot]]]

The fence index enables discovery: "find all fences with service=ec2" returns executable AWS operations across the entire graph.

MCP Integration

Every labeled fence becomes an MCP tool:

# In document: ```python[my-tool]
# Callable via: oculus_execute_fence("my-tool")

# The fence definition IS the tool definition
# No separate registration required

The MCP proxy exposes ~100 tools covering:

  • Navigation (goto, look, walk, back)
  • Reading (peek, scroll, edges, explore)
  • Writing (poke, node, link, tag)
  • Execution (execute fence, execute virtual fence)
  • Cases (open, pursue, clue, solved)

Provenance Hooks

Filesystem hooks trigger on document save:

  • Pre-save: Validate structure, check for conflicts
  • Post-save: Inject/update provenance section, reindex fences

The provenance injector:

  • Scans for existing Provenance section
  • Adds Document status (always present)
  • Adds Fence entries (one per fence with generated ID)
  • Initializes all as πŸ”΄ Unverified

Hash verification compares stored hash against current content hash. Drift detection enables "trust but verify" workflows.

North

slots:
- slug: wanderland-paper
  context:
  - Parent paper node
  - Paper parent to implementation section

East

slots:
- slug: edge-middleware
  context:
  - Edge middleware implementation
- slug: how-to-create-oculus-tool
  context:
  - Fence-as-tool pattern
- slug: wanderland-paper-evaluation
  context:
  - Section sequence

West

slots:
- slug: wanderland-paper-system-design
  context:
  - Previous section

Provenance

Document

  • Status: πŸ”΄ Unverified

Fences

wanderland-paper-implementation-fence-execution-fence-0

  • Status: πŸ”΄ Unverified

wanderland-paper-implementation-the-graph-index-fence-0

  • Status: πŸ”΄ Unverified

wanderland-paper-implementation-mcp-integration-fence-0

  • Status: πŸ”΄ Unverified

wanderland-paper-implementation-north-fence-0

  • Status: πŸ”΄ Unverified

wanderland-paper-implementation-east-fence-0

  • Status: πŸ”΄ Unverified

wanderland-paper-implementation-west-fence-0

  • Status: πŸ”΄ Unverified