lantern

cmd-poke

CLI Command: poke

API: POST /api/oculus/poke/{slug} Files: oculus/path_parser.py:write_path(), oculus/prose_provider.py

Write values to hierarchical sections with set/append/prepend modes. Supports prose content, YAML/JSON fences, and nested heading paths (H1.H2.H3.H4.H5).

Architecture (O(1) Graph Operations)

Poke uses an optimized architecture that writes directly to the graph cache:

  • Parse - Path agenda.part-4 is parsed to identify target section
  • Navigate - Graph walker finds target GraphNode via H1/H2 entry index
  • Create - If section doesn't exist, CREATE IT (competent computing)
  • Update - Tokens are updated in-place within the graph
  • Cache - Updated graph is written to RAW cache, higher levels invalidated
  • Dirty Track - Node is marked dirty for async flush to disk
  • Flush - Background worker writes dirty nodes to markdown files

This avoids re-parsing the entire document for each poke, making updates O(1) instead of O(n).

Competent Computing: Auto-Create Sections

If a section doesn't exist, poke creates it. This follows the competent-computing philosophy:

"I sent you data. I want the output to look like what I sent you. If I told you to do it wrong, that's on me."

# Poke to nonexistent section "metrics" - CREATES "## Metrics"
oculus poke my-node metrics "CPU: 42%"

# Deep path creates nested sections
oculus poke my-node config.database.settings "timeout: 30"
# Creates: ## Config β†’ ### Database β†’ #### Settings

Section levels are determined by path depth:

  • First path part β†’ H2 (standard entry point)
  • Each nested part β†’ parent level + 1

This is not heuristic-based guessing. It's deterministic: you said create it, we create it. Wrong path? That's on you. Rollback exists.

Cache Levels (Sprouting Pattern)

  • 🌱 seed/raw - Raw AST tokens (poke writes here)
  • 🌿 sprout/substituted - Variables resolved
  • πŸͺ΄ stalk/interpolated - Includes resolved
  • ☁️ clouds/rendered - Fences executed

When poke updates RAW, all higher levels are invalidated automatically.

Dirty Tracking & Async Flush

Pokes don't write to disk immediately. Instead:

  • Graph cache is updated in memory
  • Node is marked dirty (GET /api/oculus/cache/dirty)
  • Background FlushManager writes to disk periodically
  • Manual flush: POST /api/oculus/cache/flush

File-based dirty tracking supports multi-worker uvicorn deployments.

Usage

CLI Examples

# Set prose content at H1 section
oculus poke my-node agenda "New agenda content"

# Set content at nested H2 section
oculus poke my-node agenda.part-1 "Part 1 content"

# Navigate deep H1 > H2 > H3 > H4 > H5
oculus poke my-node config.database.credentials.primary.host "localhost"

# Append to existing content
oculus poke my-node notes --mode append "Additional note"

# Prepend to existing content
oculus poke my-node changelog --mode prepend "## v2.0.0 - Breaking changes"

API Examples

# Set prose content
curl -X POST "http://localhost:7778/api/oculus/poke/my-node" \
  -H "Content-Type: application/json" \
  -d '{"path": "agenda.part-1", "value": "New content"}'

# Write YAML as fenced block
curl -X POST "http://localhost:7778/api/oculus/poke/config" \
  -H "Content-Type: application/json" \
  -d '{"path": "database.yaml", "value": {"host": "localhost", "port": 5432}, "fence_type": "yaml"}'

# Append to list in YAML fence
curl -X POST "http://localhost:7778/api/oculus/poke/logs" \
  -H "Content-Type: application/json" \
  -d '{"path": "entries.yaml", "value": {"timestamp": "2025-11-24", "msg": "test"}, "operation": "append"}'

API Request Schema

{
  "path": "section.subsection",
  "value": "content or object",
  "operation": "set|append|prepend|replace",
  "fence_type": "yaml|json|python|...",
  "force": false,
  "create_if_missing": false,
  "context": "optional trace context"
}

API Response Schema

{
  "slug": "node-slug",
  "actual_slug": "expanded-slug",
  "path": "section.subsection",
  "value": "...",
  "operation": "set",
  "updated": true
}

Operations

This is the quick reference for common poke operations. Think of it as a cheat sheet.

1. Replace Section Content (set)

Replace the entire content of an existing section:

# Replace prose content
oculus poke my-node section-name "New content replaces everything"

# Replace with markdown (content will be normalized)
oculus poke my-node section-name "New paragraph.\n\nAnother paragraph."

2. Add Content to End of Section (append)

Add content after existing content in a section:

# Append to existing section
oculus poke my-node notes --mode append "This goes at the end"

# Append multiple paragraphs
oculus poke my-node notes --mode append "First new paragraph.\n\nSecond new paragraph."

3. Add Content to Beginning of Section (prepend)

Add content before existing content in a section:

# Prepend urgent notice
oculus poke my-node notes --mode prepend "⚠️ Important update!"

4. Add a New Top-Level Section (H2)

Key insight: To add a new section, poke to the PARENT with the header included.

# Add new H2 section to root of document
oculus poke my-node content --mode append "## New Section\n\nThis is the new section content."

# Using API
curl -X POST "http://localhost:7778/api/oculus/poke/my-node" \
  -H "Content-Type: application/json" \
  -d '{"path": "content", "value": "## New Section\n\nContent here", "operation": "append"}'

5. Add a Sub-Section (Child Section)

To add a child section (H3 under H2), poke to the parent section:

# Add H3 "### Details" under existing "## Overview" section
oculus poke my-node overview --mode append "### Details\n\nThe detailed information goes here."

# Add deeply nested section
oculus poke my-node overview.details --mode append "#### Sub-Details\n\nEven more specific info."

⚠️ Common Mistake: Don't try to poke directly to a section that doesn't exist with append:

# ❌ WRONG - section doesn't exist yet
oculus poke my-node new-section --mode append "## New Section\n\nContent"

# βœ… RIGHT - poke to parent to add new section
oculus poke my-node content --mode append "## New Section\n\nContent"

6. Create Nested Sections (Auto-Create)

For set operations, sections are auto-created if they don't exist:

# Auto-creates: ## Config β†’ ### Database β†’ #### Settings
oculus poke my-node config.database.settings "timeout: 30"

# Note: This works for 'set' but NOT for 'append' to non-existent sections

7. Add Structured Data (YAML/JSON)

# Add YAML fence to section
oculus poke my-node config --fence-type yaml '{"host": "localhost", "port": 5432}'

# Append to existing YAML list
oculus poke my-node logs.yaml --mode append '{"timestamp": "2025-01-01", "msg": "test"}'

Quick Reference Table

Goal Command
Replace section poke node section "content"
Append to section poke node section --mode append "content"
Prepend to section poke node section --mode prepend "content"
Add new H2 section poke node content --mode append "## New\n\nContent"
Add H3 under H2 poke node parent-section --mode append "### Child\n\nContent"
Auto-create nested poke node a.b.c "content" (set only)

Error: Section Not Found with Append

If you see this error:

❌ Section "my-section" not found in node

πŸ“ Looks like you're trying to add a new section with a header.

πŸ’‘ To add a new child section, poke to the PARENT section instead

This means you tried to append to a section that doesn't exist. The fix is to poke to the parent section and include the header in your content.

Path Navigation

Path Navigation

Paths use a proper tokenizer and recursive descent parser - no regex. The grammar is explicit and predictable.

Formal Grammar
path         := segment ('.' segment)* level_suffix?
segment      := identifier bracket_suffix? | 'fences' bracket_suffix? | 'meta'
bracket_suffix := '[' bracket_content ']'
bracket_content := number | '*' | query
query        := key '=' value (',' key '=' value)*
level_suffix := '@L' digit+
identifier   := (letter | digit | '_' | '-' | ':')+

Implementation: oculus/path_tokenizer.py (tokenizer + recursive descent parser)

Level Suffix (@L3, @L4, @L5)

The level suffix specifies which cache level to write to:

Suffix Level Effect
@L3 Sprout Write to substituted level
@L4 Rendered Materialize executed data
@L5 Clouds Write rendered output
# Poke and materialize at L4 (execute and cache)
oculus poke my-node config.yaml@L4 '{"host": "localhost"}'
Fence Addressing (DWIM)

Poke uses a Do What I Mean (DWIM) approach to fence addressing. The goal: if we can identify a single fence unambiguously, use it.

Address Components
Component Syntax Example
Section section.subsection config.database
Type .yaml, .json, .python config.yaml
Index [0], [1] config[0], config.yaml[1]
Label .my-label (if not a type) config.pantry
Data path .key.subkey config.yaml.host
Level @L3, @L4, @L5 config.yaml@L4
Addressing Examples
Path Meaning
data Section prose/content
data.location DWIM: single fence β†’ key location
data[0].location Fence #0 (any type), key location
data.yaml First yaml fence in section
data.yaml.location First yaml fence, key location
data.yaml[1].location Second yaml fence, key location
data.my-label Fence with label my-label
data.yaml@L4 First yaml fence, materialize at L4
DWIM Resolution

When addressing a fence without explicit type/index:

  • Navigate to section via graph
  • Collect all fences in section
  • Apply filters (type, index, label) if provided
  • If exactly 1 fence matches β†’ use it
  • If 0 matches β†’ error: fence not found
  • If N matches (N>1) β†’ error: ambiguous
Ambiguity Errors

If a section has multiple fences and you don't specify which one:

❌ Ambiguous path 'data.location'. Section 'data' has 3 fences:
  [0] prose  
  [1] yaml
  [2] python

Specify type or index:
  data.yaml.location      (by type)
  data[1].location        (by index)
  data.prose              (prose fence)

This forces explicit addressing when there's ambiguity, with helpful suggestions.

Section Index vs Type Index
  • section[n] β†’ nth fence in section (any type)
  • type[n] β†’ nth fence of that specific type
# Section has: prose, yaml, yaml, python
data[0]           # β†’ prose (first fence)
data[1]           # β†’ first yaml (second fence)
data.yaml[0]      # β†’ first yaml
data.yaml[1]      # β†’ second yaml
data[3]           # β†’ python (fourth fence)

Test Suite

Live test results for poke operations (executed via graphnode).

Test Results

TableConfig:
  array_path: tests
  columns:
    Test: test
    Operation: operation
    Path: path
    Status: status
    Details: details
  format: markdown

❌ Fence Execution Error: "'poke-test-data' - Curiouser and curiouser! That path seems to have vanished. Perhaps you meant: 'poke-test-fixture'?"

Summary

path: summary

❌ Fence Execution Error: "'poke-test-data' - Curiouser and curiouser! That path seems to have vanished. Perhaps you meant: 'poke-test-fixture'?"

Data Source: [[poke-test-data]]

Implementation Status

βœ… Completed (2025-11-25):

  • O(1) graph operations via HierarchicalGraph navigation
  • Dirty tracking with file-based multi-worker support
  • Async flush via FlushManager background worker
  • Cache level cascade (RAW update invalidates higher levels)
  • H1/H2 entry index for fast section lookup
  • collect_tokens() for graph β†’ token serialization
  • Auto-create sections - competent computing, no "section not found" errors
  • Unified poke() - single entry point in path_parser.py, legacy PokeEngine removed
  • Undo support - snapshots before each poke, see cmd-undo

πŸ“‹ TODO:

  • Graph-as-value normalization (poke markdown with headings, normalize levels)
  • Performance benchmarks for large documents
  • Conflict resolution for concurrent pokes to same section

πŸ§ͺ Requested Test: Prose/Fence Interleaving

Test that scope: full preserves the interleaved order of prose and fences. When content has:

Prose A
```fence1```
Prose B
```fence2```

After poke with scope: full, the sequence should remain Prose A β†’ fence1 β†’ Prose B β†’ fence2, not Prose A β†’ Prose B β†’ fence1 β†’ fence2.

Case: task-d5c44cec-b7c9-4409-9ffd-e9d9ece9a0d6

Slots

North

slots:
- slug: oculus-cli
  context: []
- slug: pattern-looking-glass-development
  context: []

South

slots:
- slug: poke-test-data
  context: []

East

slots:
- slug: pattern-embedded-tests
  context: []
- slug: cmd-undo
  context: []
- slug: poke-insert-after-troubleshooting
  context: []

West

slots:
- slug: pattern-embedded-tests
  context: []

Operations

Path Navigation

Provenance

Document

  • Status: πŸ”΄ Unverified

Fences

cmd-poke-competent-computing-auto-create-sections-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-cli-examples-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-api-examples-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-api-request-schema-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-api-response-schema-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-1-replace-section-content-set-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-2-add-content-to-end-of-section-append-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-3-add-content-to-beginning-of-section-prepend-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-4-add-a-new-top-level-section-h2-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-5-add-a-sub-section-child-section-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-5-add-a-sub-section-child-section-fence-1

  • Status: πŸ”΄ Unverified

cmd-poke-6-create-nested-sections-auto-create-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-7-add-structured-data-yamljson-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-error-section-not-found-with-append-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-formal-grammar-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-level-suffix-l3-l4-l5-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-ambiguity-errors-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-section-index-vs-type-index-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-test-results-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-summary-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-implementation-status-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-north-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-south-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-east-fence-0

  • Status: πŸ”΄ Unverified

cmd-poke-west-fence-0

  • Status: πŸ”΄ Unverified
πŸ“š creating-tools 3/7
↓ southpoke-test-data
← westpattern-embedded-tests