Lua API: function Meta / function Pandoc filters + Meta<->ConfigValue design (bd-2llqjsms, bd-a9g50za2)#404
Merged
Conversation
…2llqjsms, bd-a9g50za2)
Phases 1-2 of claude-notes/plans/2026-07-20-lua-meta-pandoc-filters.md.
New module crates/pampa/src/lua/config_value.rs — the Meta<->ConfigValue
design center:
- push_config_value / push_meta: ConfigValue -> native Lua shapes
(modern-pandoc marshaling: scalars stay scalars, PandocInlines/Blocks
become Inlines/Blocks userdata lists, Array -> List, Map -> plain
table with a "Meta"-named metatable at the top level). Path/Glob/Expr
become LuaConfigSpecial userdata (mutable .value, __tostring);
explicit null via LuaConfigNull.
- peek_config_value / peek_meta: Lua -> ConfigValue with
*reconciliation*: subtrees structurally equal to the original keep
their original nodes (source_info, merge_op, entry order, key_source
preserved byte-for-byte); changed/new nodes get By::filter
provenance; edited same-kind containers keep the container node's
YAML provenance. New map keys append in sorted order (Lua hash
iteration is nondeterministic).
- quarto.config.{str,md,path,glob,expr,null} constructors mirroring the
YAML tag system.
function Meta invocation (filter.rs): pandoc applyFully order —
typewise element-walk -> Meta, topdown Meta -> element-walk; nil return
keeps meta; non-table return is a Q-11-4 error. Q-11-6 "unimplemented"
warning narrowed to Pandoc/Doc (still uninvoked; Phase 4).
pandoc.read/write meta switched to the native shape (was the legacy
{t="MetaInlines"} tagged-table encoding — itself a parity bug); the
dead meta_value_to_lua/lua_to_meta_value family removed from types.rs.
Registered divergences (divergences.md): D-num (meta numbers never
stringified), D-null (null reads as nil; reconciliation preserves
null-valued keys across passthrough), D-order (map iteration order
unspecified).
Tests: 41 new config_value unit tests (push/peek/reconcile/constructors),
8 new Meta-handler integration tests (order both traversals, provenance,
Q-11-4), readwrite native-shape pins, Q-11-6 tests updated, smoke-all
E2E fixture filters/meta-handler.qmd (title replacement + new key +
typewise-order marker), manually verified via q2 render (<title> and
subtitle from the handler observed in output).
Verification: cargo build/nextest --workspace green (10330 tests),
clippy/fmt/custom lints green, hub-client build:all + test:ci +
test:wasm green with rebuilt WASM. cargo xtask verify's only failure
was ts-packages/quarto-hub-mcp "live:" tests, which timeout because
wss://sync.automerge.org is currently unreachable (environmental,
unrelated to this change).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…hase 3) New module crates/pampa/src/lua/pandoc_doc.rs: - pandoc.Pandoc(blocks, meta?) with peekMetaValue-style meta normalization via the ConfigValue round-trip; pandoc.Meta; the six Meta* coercion constructors (MetaBool strictly typed — mlua's bool coercion would have accepted any truthy value, caught red-first; MetaString renders numbers as an explicit stringification request; MetaList/MetaMap normalize elementwise, numbers stay numeric per D-num). - Shared "Pandoc" doc metatable: walk (element legs + Meta leg in applyFully order; Pandoc leg is Phase 4), deep clone, __concat (right-biased meta union per pandoc-types Semigroup), __eq (source-free structural equality). pandoc.read docs carry the same metatable, so read and constructed docs are interchangeable; readwrite.rs now delegates its doc conversions here. - apply_meta_function parameterized by new_source so the filter pass attributes to By::filter(path) while doc:walk attributes via the Lua stack. Conformance ratchet (TDD driver — xfails removed first, 12 unexpected failures observed, then implemented): 196 pass / 7 xfail. Remaining: normalize x2 (deferred follow-up strand), walk-order x2 (Phase 4 meta-value traversal + Pandoc leg), D-num numbers x1 + SimpleTable x2 (permanent DIVERGENCE). Also lands the Phase 4 differential spec: 3 new oracle-backed cases (doc-handlers-typewise-order, doc-handlers-topdown-order, meta-values-walked-typewise) pinning pandoc 3.9.0.2's applyFully semantics — meta cleared at case end so the deliberately different meta wire encodings never reach the comparison; currently xfailed in differential/xfail.txt until Phase 4 implements the Pandoc handler and meta-value traversal. Existing oracles regenerated byte-identical. Verification: workspace nextest 10334 passed; xtask verify Rust legs (lints/clippy/fmt/build/tests) green, ts leg failed only in the network-dependent quarto-hub-mcp live tests (sync.automerge.org still unreachable — environmental); hub-client build:all + test:ci green with rebuilt WASM. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… Phase 4) Completes the doc-level filter work of claude-notes/plans/2026-07-20-lua-meta-pandoc-filters.md. Element walk parity: pandoc's walkBlocksAndInlines traverses metadata — walk_meta_config_value (walk.rs) walks MetaInlines/MetaBlocks payloads with either walker; typewise_pandoc runs meta-then-blocks per pass (pandoc's field order), topdown_pandoc root-down. Container/scalar nodes keep provenance; only payload vectors are replaced. Element filters now see merged config metadata, same as pandoc/Q1 (a Str uppercaser uppercases the title). Pandoc/Doc handler: apply_pandoc_function (nil keeps the doc; doc-shaped return reconciles meta against the pre-handler meta; Q-11-4 on invalid returns). apply_full_filter is the shared applyFully implementation used by both the filter pass and doc:walk. Q-11-6 retired: warning path deleted, catalog entry removed, warning tests flipped to test_doc_level_handlers_run_without_diagnostics (invocation effects asserted in the filter_tests suites). Scoreboards: conformance 198 pass / 5 xfail (normalize x2 -> bd-62lppjuy follow-up strand; D-num + SimpleTable x2 permanent DIVERGENCE); differential 30/30 (the 3 Phase-4 spec cases flipped green). New tests: 7 Pandoc-handler/meta-walk integration tests, doc_level_handlers_wasm smoke in wasm_lua.rs (type-checked against wasm32 with -Zbuild-std locally; runs in Linux CI). Fixture change: smoke-all filters/meta-handler.qmd now asserts a nonzero Str count (seen-[1-9][0-9]*-strs) instead of an exact count — the walk traverses merged config metadata, whose size is not the fixture's to pin. filter_tests' typewise-order test updated 2 -> 3 (title Str now counted). Docs: new user guide docs/guides/authoring/lua-filters.qmd (element filters, Meta/Pandoc handlers, native meta representation with registered divergences, quarto.config.*), linked from guides index; render verified with q2. E2E: q2 render with a Str+Meta+Pandoc filter produced "summary: 7 strs, after meta" in the output HTML (Pandoc last, saw Meta's value, appended a block; count proves meta traversal). Verification: pampa 4250 tests green; workspace nextest 10346 green; clippy + custom lints clean; hub-client build:all + test:ci + test:wasm green with rebuilt WASM. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements doc-level Lua filter handlers and the Meta↔ConfigValue design, closing bd-2llqjsms and bd-a9g50za2 (parent epic bd-grkrb9nj). Design record + phased log:
claude-notes/plans/2026-07-20-lua-meta-pandoc-filters.md.What's here (4 commits, one per phase group)
function Meta(c4723a68) — newcrates/pampa/src/lua/config_value.rs: ConfigValue ↔ native-Lua marshaling following modern pandoc's conventions (scalars stay scalars, Inlines/Blocks userdata lists, List/plain-table compounds;!path/!glob/!exprbecome typed userdata with a mutable.value). The return path reconciles against the original tree: untouched metadata keepssource_info/merge_op/entry order byte-for-byte; changed/new nodes getBy::filterprovenance. Newquarto.config.{str,md,path,glob,expr,null}constructors mirror the YAML tag system.function Metais invoked in pandoc's applyFully order;pandoc.read/writemeta switched to the native shape (fixing a legacy tagged-table parity bug).75adf0b7) —pandoc.Pandoc(blocks, meta),pandoc.Meta, the sixMeta*coercion constructors, and a sharedPandocdoc metatable (walk, deepclone, right-biased__concat, structural__eq);pandoc.readdocs and constructed docs are interchangeable values.Pandoc/Dochandlers + full walk parity (fcbd359f) — element filter functions now traverse metadata values (per-pass, meta-first, matching pandoc's field order);apply_full_filteris the single applyFully implementation behind both the filter pass anddoc:walk. Interim Q-11-6 warning fully retired (code + catalog + tests). New user guidedocs/guides/authoring/lua-filters.qmd.8384575f).Deliberate divergences (registered in
crates/pampa/tests/lua-conformance/divergences.md)nil; reconciliation keeps null-valued keys across passthrough filters;quarto.config.null()writes one.pairs(meta)order unspecified (as in pandoc); reconciled new keys append sorted for deterministic output.Scoreboards & verification
doc:normalize()×2 → follow-up strand bd-62lppjuy; D-num ×1 + SimpleTable ×2 permanent DIVERGENCE).build:all+test:ci+test:wasmgreen with rebuilt WASM; newdoc_level_handlers_wasmsmoke test type-checked against wasm32 (runs in Linux CI).q2 renderverified twice with inspected output (Meta title/subtitle replacement; Pandoc-handler summary block proving order + meta traversal), plus permanent smoke-all fixturefilters/meta-handler.qmd.quarto-hub-mcplive tests time out becausesync.automerge.orgis currently unreachable — environmental, unrelated to this change.Behavioral note for reviewers
Element filters now see (and can transform) the merged config metadata, exactly as in pandoc/Q1 — e.g. a Str-uppercasing filter also uppercases the title. The smoke fixture asserts a nonzero meta-walk count rather than pinning merged-config size. This intersects with bd-uf4epv4w (machine-facing metadata exposure) if we later want to scope what user filters can touch.
🤖 Generated with Claude Code