feat(#22): decode Bytes<64> secp256k1 validators on ledger-9.1 - #38
Merged
Conversation
The Midnight multisig ISM enrolls validators as 64-byte secp256k1 public keys (SEC1 uncompressed body X_be || Y_be) rather than 20-byte addresses, and the current node serializes contract state with the contract-state[v8] wire tag (onchain-state 4.0.0) that the ledger-8 adapter could not read. This migrates the native-decode path and teaches it the new validator format. - Repoint the native-decode crates at the ledger-9.1 git tags (not published on crates.io), unified via [patch.crates-io]; the patch wire-matches the node's VerifierKey serialization (transient-crypto). - Toolchain -> 1.96.1: the ledger crates need >=1.95, but 1.97 breaks a transitive size-of 0.1.5, so size-of is vendored and patched to drop target-invalid fn-ptr ABIs. - state_decode / ism: read the validator set as Bytes<64> pubkeys and derive each 20-byte address as keccak256(pubkey)[12..] for the agent's signature-sorting bookkeeping. - mailbox: MAX_SIGNATURES 16 -> 4 to match the on-chain MAX_VALIDATORS cap; forward the quorum-sized signature set unpadded. - toolkit: configurable submit timeout via MIDNIGHT_SUBMIT_TIMEOUT_SECS. - Mark the state-decode fixture tests #[ignore] pending regeneration for the v8 / 64-byte registry format.
size-of 0.1.5 (a transitive Starknet build-dep) declares target-invalid fn-ptr ABIs that rustc >= 1.95 rejects (E0570); the ledger-9.1 decode graph forces rustc >= 1.95 (sysinfo 0.39 / cfg_select), so it must be patched. Replace the in-repo vendored copy with a Cargo [patch] pointing at equilibriumco/size-of (fork of 0.1.5 with only the target-invalid ABIs dropped). Removes the ~30 vendored files; same functional fix.
Mirror the contract-side removal of the on-chain merkle tree: MerkleTreeHook no longer reads an on-chain root. count/latest_checkpoint derive from the off-chain IncrementalMerkle reconstructed from dispatched_messages (the exact root the validator already signs), so the local-vs-anchor cross-check stays self-consistent (no false reorg panic). Re-derive state_decode paths for the new night layout (ISM [1,1-4], mailbox [1,8-10]; no more count/current_root slot) and regenerate night-state-dispatched.hex. Pre-removal state preserved on branch archive/onchain-merkle-tree.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Teach the Midnight chain adapter to decode the real-ECDSA multisig ISM
state on the current Midnight node. Two coupled changes forced this:
(
X_be || Y_be), not 20-byte addresses.contract-state[v8]wiretag (
onchain-state 4.0.0); the adapter was pinned to the ledger-8crates that only understand
[v6], so every ISM / message read failedto deserialize.
Changes
ledger-9.1git tags (not on crates.io), unified with a[patch.crates-io]block; the node serializesVerifierKeys withtransient-crypto, so that is wire-matched.>=1.95;1.97breaksa transitive
size-of 0.1.5, sosize-ofis vendored + patched todrop target-invalid fn-ptr ABIs.
Bytes<64>pubkey decode (state_decode.rs,ism.rs): read thevalidator set as 64-byte pubkeys; derive each 20-byte address as
keccak256(pubkey)[12..]for the agent's signature-sorting bookkeeping.MAX_SIGNATURES16 -> 4 (mailbox.rs), matching the on-chainMAX_VALIDATORS; forward the quorum-sized signature set unpadded.toolkit.rs) viaMIDNIGHT_SUBMIT_TIMEOUT_SECS.#[ignore]'d pending regeneration for the v8 / 64-byteregistry format.
Testing
Validated end-to-end against a local Midnight devnet: an inbound
EVM->Midnight delivery ran the real multisig ISM
handleproof(keccak256 digest +
secp256k1EcdsaVerify) to completion and creditedthe recipient, using agents built from this branch.
On-chain merkle tree removed (folded into this PR)
Mirrors the contract-side removal (
equilibriumco/hyperlane-midnight#89): theMerkleTreeHookno longer reads an on-chain root.count/latest_checkpointderive from the off-chainIncrementalMerklereconstructed fromdispatched_messages(the exact root the validator already signs), so the local-vs-anchor cross-check stays self-consistent (no false reorg panic). Thestate_decodepaths are re-derived for the newnightlayout (ISM[1,1-4], mailbox[1,8-10]; no more count/current_root slot) andnight-state-dispatched.hexis regenerated. Motivation: the on-chain tree had grown the outbound proving key to 2.63 GiB, trapping at the 2 GiB wasm32 wall so outbound never proved.Pre-removal state preserved on branch
archive/onchain-merkle-treein both repos (still hits the wall — outbound does not prove there).