Skip to content

cardano-rpc: Populate transaction bodies in FetchBlock responses#1247

Open
carbolymer wants to merge 9 commits into
masterfrom
mgalazyn/feature/fetchBlock-tx-bodies
Open

cardano-rpc: Populate transaction bodies in FetchBlock responses#1247
carbolymer wants to merge 9 commits into
masterfrom
mgalazyn/feature/fetchBlock-tx-bodies

Conversation

@carbolymer

@carbolymer carbolymer commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Context

Part of the node kernel access work (ADR-019): FetchBlock previously returned raw block bytes, the block header and the timestamp, but left body.tx empty.

This PR populates the transaction bodies. All 14 fields of the UTxO RPC Tx message are mapped: inputs, outputs, reference inputs, certificates, withdrawals, mint, witnesses, collateral, fee, validity, successful, auxiliary data, hash and governance proposals.

  • Transactions are converted at ledger level for every Shelley-based era; era-gated fields (reference inputs, mint, collateral, proposals) stay empty where the era predates them.
  • Spending, withdrawal and certificate redeemers are wired to their entries by script witness index, and successful reflects the phase-2 IsValid flag.
  • A new Certificate module maps all 19 certificate variants (Dijkstra-safe via concrete-era dispatch); a new Governance module maps all 7 governance actions including partial parameter updates.
  • Byron transactions are converted directly from the Byron ledger types, since cardano-api's Tx era has no Byron constructor. Byron fees are implicit in the UTxO and stay unset; epoch boundary blocks yield no transactions. This makes FetchBlock serve transaction data for the full mainnet history.

The first four commits are small cardano-api prerequisites, each with its own changelog fragment: re-export byronBlockRaw from Cardano.Api.Consensus, add ConwayEraPParams to EraCommonConstraints, export the per-era toScriptIndex variants, and make genTx sign the transaction body it returns.

Known limitations, deliberate and documented in code:

  • TxInput.as_output is unset (resolving it requires UTxO lookups).
  • Byron fee is unset (implicit in Byron; requires UTxO lookups).
  • The Byron positional witness-to-input pairing is not representable in the proto WitnessSet; native_bytes is the fidelity escape hatch.
  • Pre-existing bytes-field encoding divergences (TxOutput.address as text, inline Datum.hash carrying CBOR) are tracked separately in gRPC: bytes fields encoded as text or wrong payload (TxOutput.address, Datum.hash) #1246 and not changed here.

How to trust this PR

This PR is integrated and tested end-to-end in IntersectMBO/cardano-node#6579.

The conversion is covered at three levels:

  • Property tests (cardano-rpc-test, 77 tests): per-era field projections of generated transactions against the ledger values across all seven Shelley-based eras, certificate arm routing, injected-redeemer wiring (spending/rewarding/certifying at known indices), a wire-level protobuf encode/decode roundtrip forcing total evaluation across all eras, Byron golden tests decoding real block CBOR, and a forged-annotation guard pinning Byron transaction ids to the on-chain bytes rather than a re-serialisation.
  • E2E (gRPC: fetchBlock method cardano-node#6579): submits a payment transaction and a native-script minting transaction over gRPC, locates their blocks, fetches them via FetchBlock and asserts the mapped fields (hash, fee, inputs, outputs, validity, mint, witness set contents) against the submitted values, plus the NOT_FOUND and INVALID_ARGUMENT error paths.
  • The mapping was cross-checked against other UTxO RPC implementations (Dolos/pallas, Blink Labs/gouroboros) for the byte-encoding conventions of addresses, asset names, hashes and witnesses.

Review commit by commit; every commit builds and passes tests on its own. The conversion lives in Cardano.Rpc.Server.Internal.UtxoRpc.Type.{Tx,Certificate,Governance,Byron} and the handler wiring in Cardano.Rpc.Server.Internal.UtxoRpc.Sync.

Run the tests:

cabal test cardano-rpc-test
TASTY_PATTERN='/RPC FetchBlock/' cabal test cardano-testnet-test

Checklist

  • Commit sequence broadly makes sense and commits have useful messages
  • New tests are added if needed and existing tests are updated. See Running tests for more details
  • Self-reviewed the diff
  • Changelog fragment added in .changes/

@carbolymer
carbolymer changed the base branch from master to mgalazyn/feature/rpc-fetchblock-add-timestamp July 8, 2026 17:14
@carbolymer carbolymer changed the title Mgalazyn/feature/fetch block tx bodies cardano-rpc: Populate transaction bodies in FetchBlock responses Jul 8, 2026
Comment thread cardano-api/src/Cardano/Api/Block.hs Fixed
@carbolymer
carbolymer force-pushed the mgalazyn/feature/fetchBlock-tx-bodies branch 6 times, most recently from f9cb860 to 812cb1c Compare July 9, 2026 13:34
@carbolymer
carbolymer force-pushed the mgalazyn/feature/rpc-fetchblock-add-timestamp branch from 73e1a16 to 534dce1 Compare July 9, 2026 13:50
@carbolymer
carbolymer force-pushed the mgalazyn/feature/fetchBlock-tx-bodies branch from 812cb1c to b0dbce2 Compare July 9, 2026 13:53
@carbolymer
carbolymer changed the base branch from mgalazyn/feature/rpc-fetchblock-add-timestamp to mgalazyn/refactor/rpc-reorganise-modules July 9, 2026 13:54
@carbolymer
carbolymer force-pushed the mgalazyn/refactor/rpc-reorganise-modules branch from 2daffe7 to ecc1f35 Compare July 10, 2026 09:04
@carbolymer
carbolymer force-pushed the mgalazyn/feature/fetchBlock-tx-bodies branch from b0dbce2 to 0d12995 Compare July 10, 2026 09:04
@carbolymer
carbolymer force-pushed the mgalazyn/refactor/rpc-reorganise-modules branch 2 times, most recently from a5413ed to 9396d3b Compare July 11, 2026 01:36
Base automatically changed from mgalazyn/refactor/rpc-reorganise-modules to master July 11, 2026 02:01
@carbolymer
carbolymer force-pushed the mgalazyn/feature/fetchBlock-tx-bodies branch from 0d12995 to 2fec518 Compare July 17, 2026 12:12
@carbolymer
carbolymer marked this pull request as ready for review July 17, 2026 12:17
Copilot AI review requested due to automatic review settings July 17, 2026 12:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the cardano-rpc UTxO RPC implementation so FetchBlock returns fully populated transaction bodies (including Byron), backed by new ledger-to-protobuf conversion modules and expanded property/golden test coverage. It also includes supporting cardano-api exports/helpers needed to correctly wire redeemers and access Byron raw blocks.

Changes:

  • Populate FetchBlock responses with cardano.body.tx for Shelley-based eras and Byron, using new conversion modules (Type.Tx, Type.Byron, Type.Certificate, Type.Governance).
  • Refactor and extend UTxO RPC type mappings (scripts, protocol parameters, tx outputs) and add extensive property/golden tests for mapping correctness across eras.
  • Add cardano-api exports for per-era toScriptIndex* helpers and re-export byronBlockRaw for Byron block access.

Reviewed changes

Copilot reviewed 26 out of 28 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
cardano-rpc/test/cardano-rpc-test/Test/Cardano/Rpc/Type.hs Adds a property test for metadatum integer clamping behavior.
cardano-rpc/test/cardano-rpc-test/Test/Cardano/Rpc/FetchBlockTx.hs New property tests projecting txToUtxoRpcTx results against ledger facts across eras.
cardano-rpc/test/cardano-rpc-test/Test/Cardano/Rpc/ByronTx.hs New property/golden tests for Byron tx/block conversion.
cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/TxOutput.hs Extracts reusable multiasset conversion helper.
cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/Tx.hs New Shelley-based ledger-tx → UTxO RPC Tx conversion including redeemer wiring.
cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/Script.hs Factors out scriptToUtxoRpcScript for reuse.
cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/ProtocolParameters.hs Reworks protocol parameter mapping using HKD setters; adds protocol version conversion helper.
cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/Governance.hs New mapping for Conway governance proposals/actions to proto.
cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/Certificate.hs New mapping for Shelley/Conway/Dijkstra certificate variants to proto.
cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/Byron.hs New conversion for Byron-era txs and block tx extraction.
cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type.hs Re-exports newly introduced conversion entry points.
cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Sync.hs Wires FetchBlock to include populated transaction bodies.
cardano-rpc/src/Cardano/Rpc/Server/Internal/Orphans.hs Adds Inject instance for protocol version mapping.
cardano-rpc/README.md Updates feature matrix to mark FetchBlock as supported.
cardano-rpc/cardano-rpc.cabal Registers new modules and dependencies; wires new test modules.
cardano-api/src/Cardano/Api/Tx/Internal/Body.hs Exports toScriptIndexAlonzo/Conway/Dijkstra and generalises their types.
cardano-api/src/Cardano/Api/Tx.hs Re-exports the per-era toScriptIndex* helpers from the public API.
cardano-api/src/Cardano/Api/Experimental/Era.hs Adds ConwayEraPParams to EraCommonConstraints (changelog marked breaking).
cardano-api/src/Cardano/Api/Consensus/Internal/Reexport.hs Re-exports consensus byronBlockRaw.
cardano-api/src/Cardano/Api/Consensus.hs Exposes byronBlockRaw from the public consensus API.
cardano-api/gen/Test/Gen/Cardano/Api/Typed.hs Adjusts genTx to sign a single body; adds genTxWitnesses.
.changes/20260715_cardano_api_gen_tx_single_body.yml Changelog fragment for generator/testlib adjustment.
.changes/20260715_cardano_api_export_to_script_index_variants.yml Changelog fragment for new toScriptIndex* exports.
.changes/20260715_cardano_api_conway_pparams_common_constraints.yml Changelog fragment marking added constraint as breaking.
.changes/20260708_cardano_rpc_fetchblock_tx_bodies.yml Changelog fragment for FetchBlock tx body population feature.
.changes/20260708_cardano_api_byron_block_raw_reexport.yml Changelog fragment for byronBlockRaw re-export.

Comment thread cardano-rpc/test/cardano-rpc-test/Test/Cardano/Rpc/FetchBlockTx.hs
Comment thread cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/Tx.hs
Comment on lines +69 to +73
-- Byron transactions are not representable as cardano-api's 'Tx era',
-- so they are converted straight from the Byron ledger types
txs = case block of
ByronBlock consensusBlock ->
byronBlockTxs (byronBlockRaw consensusBlock)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracking this in: #1246

@carbolymer
carbolymer force-pushed the mgalazyn/feature/fetchBlock-tx-bodies branch from 2fec518 to 1cc20c7 Compare July 17, 2026 12:37
@carbolymer
carbolymer requested a review from a team as a code owner July 17, 2026 14:35
@carbolymer
carbolymer force-pushed the mgalazyn/feature/fetchBlock-tx-bodies branch from 09b776c to 8966469 Compare July 17, 2026 14:41
@carbolymer carbolymer self-assigned this Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants