Loom Dev Log - 2025-01-12

Session Start

Continuing from yesterday's session. Core engine complete (6.7k lines, 215 tests). Today: index, layers, poke mechanics.

The Docker Mental Model

Layer 0 is immutable - the file on disk. Like a Dockerfile base image.

The ephemeral layer is the current attention surface - accumulated changes from the session.

Poke commits changes - like docker commit. Creates a new layer, doesn't mutate base.

Peek composes base + layers to produce the view.

Poke Targeting Discussion

Need to nail down how poke targets positions in the stream.

The K/Q/V model:

• V = stream position (where in the token stream)
• K = key for lookup (section, fence label, path)
• Q = query specificity (how precise the targeting is)

Current syntax: slug:section.fence.key

• slug resolves to node
• section walks to heading
• fence identifies which fence in section
• key walks into fence data

Questions:

• Is section.fence.key enough specificity?
• What about multiple fences of same type in a section?
• Position-based fallback? (slug:section.yaml[0] vs slug:section.yaml[1])

The op is mainly for agents - if it works for Claude, it scales to humans who will mostly inject at higher levels anyway.

Address Grammar v0.2 - With Cursor Semantics

(* Loom Address Grammar - cursor positioning for stream injection *)

address     = slug [ ":" locator ] ;
slug        = identifier ;

(* Locator finds the injection point *)
locator     = section_loc                     (* section-level targeting *)
            | stream_loc ;                    (* stream position targeting *)

(* Section locator - inject relative to section *)
section_loc = section_name [ "." fence_loc ] [ anchor ] ;
section_name = identifier ;

(* Fence locator - into structured data *)
fence_loc   = fence_type [ index ] [ "." key_path ] ;
fence_type  = identifier ;
index       = "[" selector "]" ;
key_path    = identifier { "." identifier } ;

(* Anchor - where in section to inject prose *)
anchor      = "@" position_ref ;
position_ref = "start"                        (* after section heading *)
             | "end"                          (* before next section *)
             | "after:" pattern               (* after matching text *)
             | "before:" pattern              (* before matching text *)
             | integer ;                      (* token position in section *)

(* Index selectors *)
selector    = integer                         (* position: 0, 1, -1 *)
            | identifier                      (* label lookup *)
            | attr_query ;                    (* key=value *)
attr_query  = identifier "=" value ;

(* Patterns for find semantics *)
pattern     = quoted_string                   (* literal match *)
            | "/" regex "/" ;                 (* regex match *)

(* Primitives *)
identifier  = letter { letter | digit | "-" | "_" } ;
integer     = [ "-" ] digit { digit } ;
quoted_string = '"' { any_char } '"' ;
value       = identifier | quoted_string ;

Examples - The Full Picture

Address	Meaning
Section-level
ana:traits	Section content
ana:traits@start	Inject after heading
ana:traits@end	Inject before next section
ana:traits@after:"## Quirks"	Inject after "## Quirks" line
ana:traits@before:"---"	Inject before horizontal rule
ana:traits@5	Token position 5 in section
Fence-level
ana:traits.yaml	First yaml fence
ana:traits.yaml[1]	Second yaml fence
ana:traits.yaml[personality]	Labeled fence
ana:traits.yaml.name	Key "name" in fence
Combined
ana:traits@after:"```yaml"	After first yaml fence opens

Injection Operations

The operation is separate from the address:

poke(address, value, operation=...)

Operations:
- "set"     → replace target entirely
- "append"  → add after target
- "prepend" → add before target
- "insert"  → splice at cursor (@ anchor required)

The Editor Model

Document as stream:
┌────────────────────────────────────────┐
│ # Section                              │ ← section-name targets this
│                                        │ ← @start inserts here
│ Some prose here.                       │ ← @after:"prose" inserts after
│                                        │
│ ```yaml                                │ ← .yaml targets this fence
│ key: value                             │ ← .yaml.key targets this
│ ```                                    │ ← @after:"```yaml" after open
│                                        │ ← @end inserts here
│ # Next Section                         │
└────────────────────────────────────────┘

Every position in the stream is addressable. The anchor (@) places the cursor.

Notes

CHECKPOINT: Event-Sourced Layer Stack Complete

297 tests passing. Committed and pushed: 753de45

Built the full event-sourced layer stack based on the Lebowski Corollary:

L0 Tokens (THE RUG - source of truth)
    ↓ [middleware: edge capture]
L1 Evolved (viruses resolved)
    ↓ [middleware: edge capture]
L2 Rendered (components executed)
    ↓ [middleware: edge capture]
L3 Projected (view-specific)
    ↓ [middleware: edge capture]
L4 Presented (final output)

New modules created:

• loom/layers/__init__.py - Package exports
• loom/layers/core.py - LayerStack, LayerView, Transformers, LayerCache
• loom/layers/events.py - Event, EventStore (System + Session scopes)
• loom/layers/middleware.py - MiddlewareChain, EdgeCapture, Validation, Transform
• loom/storage/edges.py - EdgeCapturingStorage wrapper

Key architecture:

# L0 is truth - everything else derives from it
stack.set_l0("dude", tokens)

# Get any layer - materialized from L0 + edges
view = stack.get_layer("dude", LayerLevel.L2_RENDERED)

# Replay from truth at any time
stack.replay_to("dude", LayerLevel.L3_PROJECTED, from_l0=old_tokens)

Two event scopes:

• System events (permanent) - data layer transitions, stored in JSONL
• Session events (ephemeral) - observer views, user-managed GC with TTL

Middleware at every transition:

chain = MiddlewareChain(LayerLevel.L0_TOKENS, LayerLevel.L1_EVOLVED)
chain.use(EdgeCaptureMiddleware(EventScope.SYSTEM))
chain.use(ValidationMiddleware(lambda x: len(x) > 0))
chain.use(TransformMiddleware(evolve_tokens))

The Lebowski Corollary in code:

The rug is the source of truth. All opinions derive from it. When opinions drift, reconcile to truth. Replay from L0 + edges. "That's just, like, your opinion, man."

L0 tokens are the rug. Layers are opinions. Edges capture every transition. The rug really ties the room together.

Diff Cache Implemented

266 tests passing.

Created loom/diff.py - Token diff cache.

# Poke now returns the diff (the edge)
diff = poke("node:section", "value")

# The diff captures:
diff.before_hash     # State before
diff.after_hash      # State after  
diff.edge_id         # hash(before + after) - the edge identity
diff.snipped         # Tokens removed (tiny)
diff.spliced         # Tokens inserted (tiny)
diff.address         # Where it happened
diff.operation       # How (set, append, etc.)

# Inverse for undo
inverse = diff.inverse()

# Cache queries
cache = get_cache()
cache.get_by_before(hash)  # All edges from state
cache.get_by_after(hash)   # All edges to state
cache.get_history(slug)    # All edges for node
cache.find_path(a, b)      # BFS path between states

Diffs are:

• Tiny (just changed tokens)
• Cheap (keep forever)
• Tagged with before/after hashes (edge identity)
• The join table for time-travel queries

Meaning is in the edges.

Grammar + Poke Implemented

262 tests passing.

Created:

• loom/address.py - Grammar v0.3 parser with full syntax support
• loom/poke.py - Stream write operations with vim-like semantics

Two basic operations on data streams:

• peek - read from address
• poke - write to address

Poke operations:

poke("node:section", "new content", Operation.SET)
poke("node:section", "more", Operation.APPEND)
poke("node:section.yaml.key", "value", Operation.SET)
poke("node:section@start", "at beginning")
poke("node:section@end", "at end")
poke("node:section@after:'marker'", "after pattern")
poke("node:section@1:3", "REPLACED")  # Range replace

LOCATE → SNIP → SPLICE → LIGATE. Like DNA transcription.

Next: snip caching (TTL-based), meaning in edges (provenance tracking).

Grammar v0.3 - Named Beats Indexed

Updated loom-fence-grammar with #label syntax:

slug:section.yaml#config      ← named fence (content matters)
slug:section.yaml[0]          ← positional (order matters)
slug:section.yaml[-1]         ← last item

Named beats indexed when the content matters more than the order.

The Docker mental model + K/Q/V mapping + formal EBNF = agents now have syntax for "put this HERE."

6.7k lines, 215 tests. Core engine complete. Now it's cursor semantics.

Progress Update

Completed:

• Address Grammar v0.2 - Created loom-fence-grammar as canonical source with:- EBNF grammar for address syntax
• Language mappings for treesitter (Python, YAML, JSON, Bash, EBNF, Markdown)
• K/Q/V construct definitions per language
• Python Handler - Created fence handler with introspection:- Without params → returns fence shape (AST analysis of config.get() calls)
• With params → executes with params as config dict
• loom/execute/handlers/python.py
• Params Pipeline - Full end-to-end working:- CLI: --params/-p flag accepts JSON
• API: peek() and peek_label() accept params dict
• Executor: FenceExecutor.execute() passes params via ctx.attrs['params']

Key Insight:

peek/poke are universal primitives. AST/treesitter processing is an evolution that uses those primitives internally. The system can read the graph while being part of the universe itself - self-referential introspection and modification.

Test:

# Introspection (no params)
loom peek @create-planfile
# Returns: parameter documentation with usage example

# Execution (with params)
loom peek @create-planfile -p '{"name": "my-plan"}'
# Returns: "Created plan-my-plan"

Poke Test Fences

Acceptance criteria: these fences work → poke works.

# Creates a new plan file with initial content
# Params: name, content
from loom.api import poke
from pathlib import Path

name = config.get('name', 'untitled-plan')
content = config.get('content', '# Plan\n\n## Tasks\n\n')

# Create the node file directly for now (layer 0)
nodes_dir = Path.home() / '.local/share/oculus/nodes'
plan_path = nodes_dir / f"plan-{name}.md"

if plan_path.exists():
    result = f"Plan '{name}' already exists"
else:
    plan_path.write_text(content)
    result = f"Created plan-{name}"

print(result)

❌ Fence Execution Error: No module named 'loom' Traceback (most recent call last): File "/app/oculus/providers/python_provider.py", line 468, in execute exec(fence_content, exec_globals, exec_locals) File "", line 3, in ModuleNotFoundError: No module named 'loom'

Plan 'untitled-plan' not found

These are layer-0 operations (direct file writes) for now. Once we have the layer system, they become:

# Layer-aware version (future)
poke(f"plan-{name}:tasks", content, operation="append")

Testing

loom peek @create-planfile   # needs params support
loom peek @append-planfile   # needs params support

Next: add --params to CLI, then test these fences.

Provenance

Document

• Status: 🔴 Unverified

Changelog

• 2026-01-11 19:07: Node created by mcp - Starting dev log for today's Loom session

South

slots:
- slug: loom-fence-grammar
  context:
  - Linking grammar to today's dev log