Reduced Data Temporary Softfork, implemented as a modified BIP9 temporary deployment#238
Reduced Data Temporary Softfork, implemented as a modified BIP9 temporary deployment#238dathonohm wants to merge 34 commits into
Conversation
|
Let me suggest adding a note that OP_RETURN is deprecated in help texts, please. |
|
what's the timeline to get this merged so the signaling can use this implementation? |
|
There is no specific timeline to get this merged into Knots, as it is not confirmed that it will be eligible for merging, even when complete. However, I am aiming to have this draft ready for review in the next few days. Miner signaling can still use this deployment if the activation client is released after the start of the signaling period (which is today, so this will definitely happen). |
|
All comments from #234 are now addressed. Undrafting since the code is relatively stable now. Still needs rebase. |
OP_RETURN is not deprecated; it is merely limited to 83 bytes in consensus. |
df70e59 to
d699af3
Compare
|
Rebased on v29.2.knots20251110. Ready for review. |
|
Concept NACK There shouldn't be any emergency softfork to address spam without at least a sketeched out permanent solution |
|
@stackingsaunter Please keep conceptual discussion to the BIP PR. This PR is for code review only. |
luke-jr
left a comment
There was a problem hiding this comment.
Review not complete yet
994b2fe to
5a54af7
Compare
|
All review comments are now addressed. CI is now fully passing (except for the "test each commit" job). A tag has been created for the current version, RC2 (identical to this branch except for the last commit, which updates the UA string for BIP-110). Next I will make a release for RC2, and clean up the commit history here. |
5a54af7 to
cc089c4
Compare
datacarriersize is not deprecated, but OP_RETURN is. Documenting it is out of scope for this PR, though. |
| // at which all the in-chain inputs of the tx were included in blocks. | ||
| // Typical usage of GetPriority with chainActive.Height() will ensure this. | ||
| int heightDiff = currentHeight - cachedHeight; | ||
| int heightDiff = int(currentHeight) - int(cachedHeight); |
There was a problem hiding this comment.
Fixes UBSan unsigned integer overflow: 432 - 433 can't be represented in unsigned int (https://github.com/dathonohm/bitcoin/actions/runs/19997804442, ASan+UBSan job). The cast makes the subtraction signed so the dResult < 0 check on line 114 works correctly. Same pattern as line 70 in the same file (introduced in 28b7673).
There was a problem hiding this comment.
As the comment says, calling this with a currentHeight < cachedHeight is invalid...
There was a problem hiding this comment.
Removing this commit to expose the failure, then I'll work on a fix. This is an upstream bug.
f4045f3 to
1d3cdac
Compare
…height from reduced_data script validation rules Bugfix: validation: Do not cache the result of CheckInputScripts if flags_per_input is used (and avoid using it when unnecessary) Co-Authored-By: Dathon Ohm <dathonohm@proton.me> Co-Authored-By: Lőrinc <pap.lorinc@gmail.com> Github-Pull: #238 Rebased-From: 71722a2
…x8000009; adapt tests Squash of: - b662a23198 Define a service bit for BIP148 - 0db5282edd Add questionmark to end of BIP148 service bit string, and add to bitcoin-cli - b5a77cda16 Preferentially peer with nodes enforcing RDTS; rename BIP148 references to ReducedData; adapt tests - 85289a3ac3 net: ask DNS seed for x8000009 (original: ea8c1ee) Define a service bit for BIP148 Github-Pull: bitcoin#10532 Rebased-From: cd74a23fcf9588199e196ab31bc64972400c2027 Add questionmark to end of BIP148 service bit string, and add to bitcoin-cli Preferentially peer with nodes enforcing RDTS; rename BIP148 references to ReducedData; adapt tests Combines: preferential peering (367b344), BIP148->ReducedData rename (85a78b8), service bit test fixes (f98fe90, ebe821e, 5efab4a, 4b757d8, f648a21). net: ask DNS seed for x8000009 Co-Authored-By: Luke Dashjr <luke-jr+git@utopios.org> Co-Authored-By: 3c853b6299 <3c853b6299@pm.me> Co-Authored-By: Léo Haf <leohaf@orangepill.ovh> Github-Pull: #238 Rebased-From: 983555f
chrisguida
left a comment
There was a problem hiding this comment.
Post-merge review:
This is a fairly large change, but for a consensus change it's actually pretty small.
Consensus rule changes themselves are very simple and solid, and match the bip110 spec exactly. The most complicated part seems to be the UTXO grandfathering, glad reviewers caught a bug there in a previous cycle, and the fix + regression tests also look solid. (Note, grandfathering seems to only be active in consensus not policy?)
Tests also look very thorough, except for a few minor things I noted inline.
Been running this code in production for months now, so I'm happy to recommend it, let's activate this thing!
crACK/tACK f62f5fd
(note: the actual branch merged into the 0508 release of Knots is rdts_combined-29+knots (see spec). I have verified that that branch reproduces all of the commits on this branch exactly, except for the peering code which expands the limit on the number of non-bip110 outbound peers from 2 to 8.)
| tx_valid = self.create_test_transaction(script_opreturn_82, value=0) | ||
| result = node.testmempoolaccept([tx_valid.serialize().hex()])[0] | ||
| # OP_RETURN with value=0 may be rejected by standardness policy | ||
| self.log.info(f" ✓ OP_RETURN with {len(script_opreturn_82)} bytes: {result.get('allowed', False)}") |
There was a problem hiding this comment.
Should be an assert rather than just a log?
| if result is None: | ||
| self.log.info(" ✓ Generation tx with 83-byte OP_RETURN output accepted") | ||
| else: | ||
| self.log.info(f" Note: Generation tx OP_RETURN result: {result}") |
| result = node.testmempoolaccept([spending_tx.serialize().hex()])[0] | ||
| assert_equal(result['allowed'], False) | ||
| # Rejection happens during script verification | ||
| assert any(x in result['reject-reason'] for x in ['mempool-script-verify-flag', 'witness-program', 'bad-witness', 'discouraged']) |
There was a problem hiding this comment.
"discouraged" happens regardless of rdts rules
| if result2['allowed']: | ||
| self.log.info(f" DEBUG: Taproot spend with annex was unexpectedly accepted: {result2}") | ||
| assert_equal(result2['allowed'], False) | ||
| self.log.info(f" ✓ Taproot spend with annex: REJECTED ({result2['reject-reason']})") |
There was a problem hiding this comment.
assert reject reason (same for rules 4-7)
| # All undefined versions (v2-v16) are validated identically | ||
| self.log.info(" ✓ Witness versions v2-v16 are all similarly rejected") | ||
|
|
||
| def test_taproot_annex_rejection(self): |
There was a problem hiding this comment.
rules 4-7 only test policy enforcement, maybe add a submitblock test for consensus testing?
| assert 'non-mandatory-script-verify-flag' in result['reject-reason'] or 'Push value size limit exceeded' in result['reject-reason'] | ||
| self.log.info(" ✓ PUSHDATA with 257 bytes rejected in witness script") | ||
|
|
||
| # Test 2.3: P2SH redeemScript with 300-byte PUSHDATA (tests BIP16 exception boundary) |
There was a problem hiding this comment.
Should add a test case that tests >256B redeemScripts that don't have too-large pushes
| assert 'mempool-script-verify-flag' in result_normal['reject-reason'] | ||
| self.log.info(f" ✓ Normal testmempoolaccept correctly rejected: {result_normal['reject-reason']}") | ||
|
|
||
| # Test 2: Try to bypass with ignore_rejects=["non-mandatory-script-verify-flag"] |
There was a problem hiding this comment.
| # Test 2: Try to bypass with ignore_rejects=["non-mandatory-script-verify-flag"] | |
| # Test 2: Try to bypass with ignore_rejects=["mempool-script-verify-flag-failed"] |
| const auto reduced_data_start_height = DeploymentActiveAt(*pindex, m_chainman, Consensus::DEPLOYMENT_REDUCED_DATA) | ||
| ? m_chainman.m_versionbitscache.StateSinceHeight(pindex->pprev, params.GetConsensus(), Consensus::DEPLOYMENT_REDUCED_DATA) | ||
| : std::numeric_limits<int>::max(); | ||
|
|
||
| const CheckTxInputsRules chk_input_rules{DeploymentActiveAt(*pindex, m_chainman, Consensus::DEPLOYMENT_REDUCED_DATA) ? CheckTxInputsRules::OutputSizeLimit : CheckTxInputsRules::None}; |
There was a problem hiding this comment.
Could avoid creating flags_per_input when rdts is inactive (see also line 2956):
| const auto reduced_data_start_height = DeploymentActiveAt(*pindex, m_chainman, Consensus::DEPLOYMENT_REDUCED_DATA) | |
| ? m_chainman.m_versionbitscache.StateSinceHeight(pindex->pprev, params.GetConsensus(), Consensus::DEPLOYMENT_REDUCED_DATA) | |
| : std::numeric_limits<int>::max(); | |
| const CheckTxInputsRules chk_input_rules{DeploymentActiveAt(*pindex, m_chainman, Consensus::DEPLOYMENT_REDUCED_DATA) ? CheckTxInputsRules::OutputSizeLimit : CheckTxInputsRules::None}; | |
| const bool reduced_data_active = DeploymentActiveAt(*pindex, m_chainman, Consensus::DEPLOYMENT_REDUCED_DATA); | |
| const int reduced_data_start_height = reduced_data_active | |
| ? m_chainman.m_versionbitscache.StateSinceHeight(pindex->pprev, params.GetConsensus(), Consensus::DEPLOYMENT_REDUCED_DATA) | |
| : std::numeric_limits<int>::max(); | |
| const CheckTxInputsRules chk_input_rules{reduced_data_active ? CheckTxInputsRules::OutputSizeLimit : CheckTxInputsRules::None}; |
| flags_per_input.clear(); | ||
| for (size_t j = 0; j < tx.vin.size(); j++) { | ||
| prevheights[j] = view.AccessCoin(tx.vin[j].prevout).nHeight; | ||
| if (prevheights[j] < reduced_data_start_height) { |
There was a problem hiding this comment.
..continued from 2906:
| if (prevheights[j] < reduced_data_start_height) { | |
| if (reduced_data_active && prevheights[j] < reduced_data_start_height) { |
ReducedData Temporary Softfork (BIP-110/RDTS)
Implementation of BIP-110 for Bitcoin Knots.
Versionbits extensions
max_activation_height(mutually exclusive withtimeout) with BIP8/148-style mandatory signaling enforcementactive_durationfor temporary deployments (expiry after ~1 year) and a correspondingEXPIREDBIP9 statethreshold(55% for RDTS instead of global 95%)Consensus rules (when DEPLOYMENT_REDUCED_DATA is active)
SCRIPT_VERIFY_REDUCED_DATA), except BIP16 redeemScriptDISCOURAGE_UPGRADABLE_WITNESS_PROGRAM,DISCOURAGE_UPGRADABLE_TAPROOT_VERSION,DISCOURAGE_OP_SUCCESSenforced at consensus levelOP_IF/OP_NOTIFforbidden in TapscriptOther features
NODE_REDUCED_DATAservice bit (bit 27) with preferential peeringx8000009)Commit structure (34 commits)
The commits have been structured such that every commit passes all tests (except for the coin_age_priority UBSan issue which is fixed outside this PR). The BIP9 modifications are placed first, since the actual consensus validation changes depend on the existence of
DEPLOYMENT_REDUCED_DATA(originally a buried deployment).Two commits are intentionally large because their changes break a large amount of tests, so test adaptations that must land atomically are bundled:
45c004e30eand983555f940.