Merge bitcoin/bitcoin#35215: coins: use SipHash-1-3-UJ for CCoinsMap keys

3bfdcbd coins: reuse cache hasher for txid set (Lőrinc)
2beab94 coins: use SipHash-1-3-UJ for `CCoinsMap` (Lőrinc)
7ff55cc bench: add fixed-width SipHash benchmarks (Lőrinc)
3aea854 test: add SipHash-1-3-UJ coverage (Pieter Wuille)
a0ccd4a crypto: add fixed-width SipHash-1-3-UJ (Pieter Wuille)
c2d7931 crypto: add generic SipHash-1-3-UJ (Pieter Wuille)
25bfca0 refactor: simplify adding SipHash-1-3-UJ (Lőrinc)
af50ba8 test: add shared SipHash vectors (Lőrinc)

Pull request description:

  **Problem:** The in-memory UTXO cache hashes `COutPoint` keys containing a 32-byte txid and a 32-bit output index.
  SipHash-2-4 processes the txid as four independent 64-bit blocks, so its optimized 32-byte and 36-byte paths both take 14 SipRounds.
  This also matters for hash-prefix index work such as [#35531](bitcoin/bitcoin#35531): once a persisted key format chooses a hash function, changing it later requires reindexing.

  **Fix:** Add `SipHasher13UJ`, a custom block-oriented variant combining Pieter Wuille's jumbo-block suggestion with SipHash-1-3, the reduced-round variant discussed in the [SipHash analysis](https://eprint.iacr.org/2012/351.pdf).
  It provides inline `Hash` overloads for the fixed-width inputs used here.
  Use a dedicated `SaltedCoinsCacheHasher` for `CCoinsMap` and `CoinsViewOverlay`'s temporary earlier-txid set, while other outpoint tables remain on SipHash-2-4.
  The salted hash values vary between restarts and are never persisted or sent over the network.

  **Design:** `SipHasher13UJ` accepts normal 64-bit blocks and 256-bit jumbo blocks.
  For hash-table use, cryptographic hash outputs must make up all but a small bounded number of retained jumbo blocks.
  The construction mixes all four limbs around one SipRound, omits byte-oriented padding, and uses an `"unpadded"` finalizer distinct from standard SipHash-1-3.
  The fixed-width paths take four rounds for one `uint256` jumbo block and five when followed by one normal block.
  For outpoints, the 32-bit output index is zero-extended into a normal 64-bit block.

  Retained `CCoinsMap` entries identify real transaction outputs, so their keys contain computed txids.
  Missing-input validation may probe arbitrary claimed prevouts, but `FetchCoin()` immediately erases their temporary entries when the backend lookup fails, so non-hash keys cannot accumulate.
  The assumeutxo loader assumes snapshot txids are valid while loading and verifies the complete snapshot's content hash before activation.
  Every entry in the temporary earlier-txid set is a computed transaction hash, and the set is bounded by the block's transaction count.

  This construction is limited to local hash tables and is not a general-purpose or protocol SipHash replacement.
  Pieter discussed the construction with [SipHash co-author Jean-Philippe Aumasson](bitcoin/bitcoin#35215 (comment)), whose preliminary analysis did not find an easier collision construction and supported SipHash-1-3 for this hash-table use.

  <img width="2100" height="860" alt="siphash_compare_updated" src="https://github.com/user-attachments/assets/cefec6f8-5ec0-450a-a0a2-f946de9ef36d" />

  **Structure:** Shared vectors first cover the existing generic and fixed SipHash-2-4 paths in C++, the generic path in Python, and their randomized equivalence in the fuzzer.
  A behavior-neutral refactor then moves the round, compression, and finalization logic into inline `SipHashState` methods; assembly inspection shows that the fixed-width paths retain their instruction counts, while the generic byte loop retains its prior code generation through a local state copy.
  Three Pieter-authored commits add the generic UJ specification, fixed-width implementation, and shared correctness coverage.
  Benchmarks follow that coverage, then separate commits change `CCoinsMap`'s hasher and reuse it for the temporary earlier-txid set.

  **Tests:** The shared JSON supplies the same byte sequences to the generic C++ and Python SipHash-2-4 implementations, with applicable fixed-width paths checked against the same expected output.
  The SipHash-2-4 rows include the 64 official vectors for inputs from 0 to 63 bytes and cases that vary input chunking.
  The UJ outputs were generated by an independent implementation and are checked using normal blocks, equivalent zero-extended jumbo blocks, and applicable fixed-width `Hash` overloads.
  The integer fuzzer extends these comparisons to arbitrary values and mixed normal/jumbo block encodings.

  [Counting the dbcache buckets](https://gist.github.com/l0rinc/d68f56c3ed89f76f56da6632ef6f2d92) indicates the new outpoint hasher retains the uniform bucket distribution expected by `CCoinsMap`:
  <img width="1200" height="750" alt="ccoinsmap-collisions" src="https://github.com/user-attachments/assets/eeedec81-acdc-4adf-a9c8-bfce089700da" />

  **Benchmarks:** Fixed-width microbenchmarks compare SipHash-2-4 with SipHash-1-3-UJ for 32-byte hashes and inputs containing a 32-byte hash plus a 32-bit index.
  Reported aarch64 measurements and an [independent x86_64 run](bitcoin/bitcoin#35215 (comment)) show the outpoint path is about 2x faster.
  <details><summary>Benchmark runner</summary>

  ```bash
  for COMPILER in gcc clang; do \
    if [ "$COMPILER" = gcc ]; then CC=gcc; CXX=g++; else CC=clang; CXX=clang++; fi; \
    cmake -B "build-bench-$COMPILER" -DCMAKE_BUILD_TYPE=Release -DBUILD_BENCH=ON -DBUILD_TESTS=OFF -DBUILD_GUI=OFF -DENABLE_WALLET=OFF -DCMAKE_C_COMPILER="$CC" -DCMAKE_CXX_COMPILER="$CXX" >/dev/null 2>&1 && \
    cmake --build "build-bench-$COMPILER" --target bench_bitcoin -j"$(nproc)" >/dev/null 2>&1 && \
    echo "" && echo "$(date -I) | SipHash fixed-width microbench | $("$CXX" --version | head -1) | $(hostname) | $(uname -m) | $(lscpu | awk -F: '/Model name/{print $2; exit}' | xargs) | $(nproc) cores | $(free -h | awk '/^Mem:/{print $2}') RAM" && \
    "build-bench-$COMPILER/bin/bench_bitcoin" -filter='SipHash.*32b|SipHash.*36b' -min-time=10000; \
  done
  ```
  </details>

  A two-run GCC `-reindex-chainstate` comparison of the same `CCoinsMap` hot path through height 957,759 with `-dbcache=2000` on a Ryzen 7 3700X/SSD reduced mean wall time from 11,278 s to 10,759 s, a ~5% validation speedup.

ACKs for top commit:
  achow101:
    light ACK 3bfdcbd
  sipa:
    ACK 3bfdcbd (to the extent the code/ideas aren't my own)
  andrewtoth:
    ACK 3bfdcbd
  optout21:
    ACK 3bfdcbd

Tree-SHA512: c3c66051cb1ebdb0cddbc8b8bed2297c524842f92960531de23d9586c5bb950b302c33d06fc15c2320cbbf97273b22ec3f17fd0e7ddcc7df4a8132a47a606277

Author: achow101

src/bench/crypto_hash.cpp

@@ -190,10 +190,10 @@ static void SHA512(benchmark::Bench& bench)
     });
 }
 
-static void SipHash_32b(benchmark::Bench& bench)
+static void SipHash24_32b(benchmark::Bench& bench)
 {
     FastRandomContext rng{/*fDeterministic=*/true};
-    PresaltedSipHasher presalted_sip_hasher(rng.rand64(), rng.rand64());
+    PresaltedSipHasher presalted_sip_hasher{rng.rand64(), rng.rand64()};
     auto val{rng.rand256()};
     auto i{0U};
     bench.run([&] {
@@ -203,6 +203,49 @@ static void SipHash_32b(benchmark::Bench& bench)
     });
 }
 
+static void SipHash24_36b(benchmark::Bench& bench)
+{
+    FastRandomContext rng{/*fDeterministic=*/true};
+    PresaltedSipHasher presalted_sip_hasher{rng.rand64(), rng.rand64()};
+    auto val{rng.rand256()};
+    uint32_t extra{rng.rand32()};
+    auto i{0U};
+    bench.run([&] {
+        ankerl::nanobench::doNotOptimizeAway(presalted_sip_hasher(val, extra));
+        ++i;
+        val.data()[i % uint256::size()] ^= i & 0xFF;
+        extra += i;
+    });
+}
+
+static void SipHash13UJ_32b(benchmark::Bench& bench)
+{
+    FastRandomContext rng{/*fDeterministic=*/true};
+    SipHasher13UJ sip_hasher{rng.rand64(), rng.rand64()};
+    auto val{rng.rand256()};
+    auto i{0U};
+    bench.run([&] {
+        ankerl::nanobench::doNotOptimizeAway(sip_hasher.Hash(val));
+        ++i;
+        val.data()[i % uint256::size()] ^= i & 0xFF;
+    });
+}
+
+static void SipHash13UJ_36b(benchmark::Bench& bench)
+{
+    FastRandomContext rng{/*fDeterministic=*/true};
+    SipHasher13UJ sip_hasher{rng.rand64(), rng.rand64()};
+    auto val{rng.rand256()};
+    uint32_t extra{rng.rand32()};
+    auto i{0U};
+    bench.run([&] {
+        ankerl::nanobench::doNotOptimizeAway(sip_hasher.Hash(val, uint64_t{extra}));
+        ++i;
+        val.data()[i % uint256::size()] ^= i & 0xFF;
+        extra += i;
+    });
+}
+
 static void MuHash(benchmark::Bench& bench)
 {
     MuHash3072 acc;
@@ -273,7 +316,10 @@ BENCHMARK(SHA256_32b_STANDARD);
 BENCHMARK(SHA256_32b_SSE4);
 BENCHMARK(SHA256_32b_AVX2);
 BENCHMARK(SHA256_32b_SHANI);
-BENCHMARK(SipHash_32b);
+BENCHMARK(SipHash24_32b);
+BENCHMARK(SipHash24_36b);
+BENCHMARK(SipHash13UJ_32b);
+BENCHMARK(SipHash13UJ_36b);
 BENCHMARK(SHA256D64_1024_STANDARD);
 BENCHMARK(SHA256D64_1024_SSE4);
 BENCHMARK(SHA256D64_1024_AVX2);

src/coins.cpp

@@ -19,6 +19,13 @@ TRACEPOINT_SEMAPHORE(utxocache, add);
 TRACEPOINT_SEMAPHORE(utxocache, spent);
 TRACEPOINT_SEMAPHORE(utxocache, uncache);
 
+SaltedCoinsCacheHasher::SaltedCoinsCacheHasher(bool deterministic)
+    : m_hasher{
+          deterministic ? 0x8e819f2607a18de6 : FastRandomContext().rand64(),
+          deterministic ? 0xf4020d2e3983b0eb : FastRandomContext().rand64()}
+{
+}
+
 CoinsViewEmpty& CoinsViewEmpty::Get()
 {
     static CoinsViewEmpty instance;
@@ -35,7 +42,7 @@ std::optional<Coin> CCoinsViewCache::PeekCoin(const COutPoint& outpoint) const
 
 CCoinsViewCache::CCoinsViewCache(CCoinsView* in_base, bool deterministic) :
     CCoinsViewBacked(in_base), m_deterministic(deterministic),
-    cacheCoins(0, SaltedOutpointHasher(/*deterministic=*/deterministic), CCoinsMap::key_equal{}, &m_cache_coins_memory_resource)
+    cacheCoins(0, SaltedCoinsCacheHasher{/*deterministic=*/deterministic}, CCoinsMap::key_equal{}, &m_cache_coins_memory_resource)
 {
     m_sentinel.second.SelfRef(m_sentinel);
 }
@@ -331,7 +338,7 @@ void CCoinsViewCache::ReallocateCache()
     cacheCoins.~CCoinsMap();
     m_cache_coins_memory_resource.~CCoinsMapMemoryResource();
     ::new (&m_cache_coins_memory_resource) CCoinsMapMemoryResource{};
-    ::new (&cacheCoins) CCoinsMap{0, SaltedOutpointHasher{/*deterministic=*/m_deterministic}, CCoinsMap::key_equal{}, &m_cache_coins_memory_resource};
+    ::new (&cacheCoins) CCoinsMap{0, SaltedCoinsCacheHasher{/*deterministic=*/m_deterministic}, CCoinsMap::key_equal{}, &m_cache_coins_memory_resource};
 }
 
 void CCoinsViewCache::SanityCheck() const
@@ -376,7 +383,7 @@ CCoinsViewCache::ResetGuard CoinsViewOverlay::StartFetching(const CBlock& block
         // Loop through the block inputs and set their prevouts in the queue.
         // Filter inputs that spend outputs created earlier in the same block. These outputs will be created
         // directly in the cache from the tx that creates them, so they will not be requested from a base view.
-        std::unordered_set<Txid, SaltedTxidHasher> earlier_txids;
+        std::unordered_set<Txid, SaltedCoinsCacheHasher> earlier_txids;
         earlier_txids.reserve(block.vtx.size());
         for (const auto& tx : block.vtx | std::views::drop(1)) {
             for (const auto& input : tx->vin) {

src/coins.h

@@ -9,6 +9,7 @@
 #include <attributes.h>
 #include <compressor.h>
 #include <core_memusage.h>
+#include <crypto/siphash.h>
 #include <memusage.h>
 #include <primitives/transaction.h>
 #include <primitives/transaction_identifier.h>
@@ -18,7 +19,6 @@
 #include <util/check.h>
 #include <util/log.h>
 #include <util/overflow.h>
-#include <util/hasher.h>
 
 #include <cassert>
 #include <cstdint>
@@ -219,6 +219,43 @@ struct CCoinsCacheEntry
     }
 };
 
+/**
+ * SipHash-1-3-UJ based hasher for the coins cache and related coins containers.
+ *
+ * Retained entries identify real transaction outputs, so their keys contain computed txids.
+ * Missing-input lookups may contain arbitrary claimed prevouts, but FetchCoin() immediately
+ * erases their temporary entries when the backend lookup fails, so non-hash keys cannot
+ * accumulate.
+ *
+ * The assumeutxo loader assumes snapshot txids are valid while loading and verifies the
+ * complete snapshot's content hash before activation.
+ *
+ * Hash values are process-local and must not be persisted, serialized, or compared across
+ * processes.
+ *
+ * Having the hash noexcept lets libstdc++ recalculate it during rehash instead of storing it in
+ * each node.
+ */
+class SaltedCoinsCacheHasher
+{
+    const SipHasher13UJ m_hasher;
+
+public:
+    SaltedCoinsCacheHasher(bool deterministic = false);
+
+    /** Hash a transaction ID, itself a cryptographic hash, as one jumbo block. */
+    size_t operator()(const Txid& id) const noexcept
+    {
+        return m_hasher.Hash(id.ToUint256());
+    }
+
+    /** Hash an outpoint as its txid jumbo block followed by the zero-extended index as one normal block. */
+    size_t operator()(const COutPoint& id) const noexcept
+    {
+        return m_hasher.Hash(id.hash.ToUint256(), uint64_t{id.n});
+    }
+};
+
 /**
  * PoolAllocator's MAX_BLOCK_SIZE_BYTES parameter here uses sizeof the data, and adds the size
  * of 4 pointers. We do not know the exact node size used in the std::unordered_node implementation
@@ -229,7 +266,7 @@ struct CCoinsCacheEntry
  */
 using CCoinsMap = std::unordered_map<COutPoint,
                                      CCoinsCacheEntry,
-                                     SaltedOutpointHasher,
+                                     SaltedCoinsCacheHasher,
                                      std::equal_to<COutPoint>,
                                      PoolAllocator<CoinsCachePair,
                                                    sizeof(CoinsCachePair) + sizeof(void*) * 4>>;

src/crypto/siphash.cpp

@@ -6,64 +6,36 @@
 
 #include <uint256.h>
 
-#include <bit>
 #include <cassert>
 #include <span>
 
-#define SIPROUND do { \
-    v0 += v1; v1 = std::rotl(v1, 13); v1 ^= v0; \
-    v0 = std::rotl(v0, 32); \
-    v2 += v3; v3 = std::rotl(v3, 16); v3 ^= v2; \
-    v0 += v3; v3 = std::rotl(v3, 21); v3 ^= v0; \
-    v2 += v1; v1 = std::rotl(v1, 17); v1 ^= v2; \
-    v2 = std::rotl(v2, 32); \
-} while (0)
-
 CSipHasher::CSipHasher(uint64_t k0, uint64_t k1) : m_state{k0, k1} {}
 
 CSipHasher& CSipHasher::Write(uint64_t data)
 {
-    uint64_t v0 = m_state.v[0], v1 = m_state.v[1], v2 = m_state.v[2], v3 = m_state.v[3];
-
     assert(m_count % 8 == 0);
-
-    v3 ^= data;
-    SIPROUND;
-    SIPROUND;
-    v0 ^= data;
-
-    m_state.v[0] = v0;
-    m_state.v[1] = v1;
-    m_state.v[2] = v2;
-    m_state.v[3] = v3;
-
+    m_state.Compress2(data);
     m_count += 8;
     return *this;
 }
 
 CSipHasher& CSipHasher::Write(std::span<const unsigned char> data)
 {
-    uint64_t v0 = m_state.v[0], v1 = m_state.v[1], v2 = m_state.v[2], v3 = m_state.v[3];
-    uint64_t t = m_tmp;
-    uint8_t c = m_count;
+    SipHashState state{m_state.Copy()};
+    uint64_t t{m_tmp};
+    uint8_t c{m_count};
 
     while (data.size() > 0) {
         t |= uint64_t{data.front()} << (8 * (c % 8));
         c++;
         if ((c & 7) == 0) {
-            v3 ^= t;
-            SIPROUND;
-            SIPROUND;
-            v0 ^= t;
+            state.Compress2(t);
             t = 0;
         }
         data = data.subspan(1);
     }
 
-    m_state.v[0] = v0;
-    m_state.v[1] = v1;
-    m_state.v[2] = v2;
-    m_state.v[3] = v3;
+    m_state = state;
     m_count = c;
     m_tmp = t;
 
@@ -72,91 +44,46 @@ CSipHasher& CSipHasher::Write(std::span<const unsigned char> data)
 
 uint64_t CSipHasher::Finalize() const
 {
-    uint64_t v0 = m_state.v[0], v1 = m_state.v[1], v2 = m_state.v[2], v3 = m_state.v[3];
+    return m_state.Copy()
+                  .Compress2(m_tmp | (uint64_t{m_count} << 56))
+                  .Finalize4();
+}
 
-    uint64_t t = m_tmp | (((uint64_t)m_count) << 56);
+SipHasher13UJ& SipHasher13UJ::Write(uint64_t data) noexcept
+{
+    m_state.Compress1(data);
+    return *this;
+}
 
-    v3 ^= t;
-    SIPROUND;
-    SIPROUND;
-    v0 ^= t;
-    v2 ^= 0xFF;
-    SIPROUND;
-    SIPROUND;
-    SIPROUND;
-    SIPROUND;
-    return v0 ^ v1 ^ v2 ^ v3;
+SipHasher13UJ& SipHasher13UJ::WriteJumbo(const uint256& hash) noexcept
+{
+    m_state.Compress1Jumbo(hash);
+    return *this;
 }
 
-uint64_t PresaltedSipHasher::operator()(const uint256& val) const noexcept
+uint64_t SipHasher13UJ::Finalize() const noexcept
 {
-    uint64_t v0 = m_state.v[0], v1 = m_state.v[1], v2 = m_state.v[2], v3 = m_state.v[3];
-    uint64_t d = val.GetUint64(0);
-    v3 ^= d;
+    return m_state.Copy().Finalize3U();
+}
 
-    SIPROUND;
-    SIPROUND;
-    v0 ^= d;
-    d = val.GetUint64(1);
-    v3 ^= d;
-    SIPROUND;
-    SIPROUND;
-    v0 ^= d;
-    d = val.GetUint64(2);
-    v3 ^= d;
-    SIPROUND;
-    SIPROUND;
-    v0 ^= d;
-    d = val.GetUint64(3);
-    v3 ^= d;
-    SIPROUND;
-    SIPROUND;
-    v0 ^= d;
-    v3 ^= (uint64_t{4}) << 59;
-    SIPROUND;
-    SIPROUND;
-    v0 ^= (uint64_t{4}) << 59;
-    v2 ^= 0xFF;
-    SIPROUND;
-    SIPROUND;
-    SIPROUND;
-    SIPROUND;
-    return v0 ^ v1 ^ v2 ^ v3;
+uint64_t PresaltedSipHasher::operator()(const uint256& val) const noexcept
+{
+    return m_state.Copy()
+                  .Compress2(val.GetUint64(0))
+                  .Compress2(val.GetUint64(1))
+                  .Compress2(val.GetUint64(2))
+                  .Compress2(val.GetUint64(3))
+                  .Compress2(uint64_t{32} << 56)
+                  .Finalize4();
 }
 
-/** Specialized implementation for efficiency */
 uint64_t PresaltedSipHasher::operator()(const uint256& val, uint32_t extra) const noexcept
 {
-    uint64_t v0 = m_state.v[0], v1 = m_state.v[1], v2 = m_state.v[2], v3 = m_state.v[3];
-    uint64_t d = val.GetUint64(0);
-    v3 ^= d;
-    SIPROUND;
-    SIPROUND;
-    v0 ^= d;
-    d = val.GetUint64(1);
-    v3 ^= d;
-    SIPROUND;
-    SIPROUND;
-    v0 ^= d;
-    d = val.GetUint64(2);
-    v3 ^= d;
-    SIPROUND;
-    SIPROUND;
-    v0 ^= d;
-    d = val.GetUint64(3);
-    v3 ^= d;
-    SIPROUND;
-    SIPROUND;
-    v0 ^= d;
-    d = ((uint64_t{36}) << 56) | extra;
-    v3 ^= d;
-    SIPROUND;
-    SIPROUND;
-    v0 ^= d;
-    v2 ^= 0xFF;
-    SIPROUND;
-    SIPROUND;
-    SIPROUND;
-    SIPROUND;
-    return v0 ^ v1 ^ v2 ^ v3;
+    return m_state.Copy()
+                  .Compress2(val.GetUint64(0))
+                  .Compress2(val.GetUint64(1))
+                  .Compress2(val.GetUint64(2))
+                  .Compress2(val.GetUint64(3))
+                  .Compress2((uint64_t{36} << 56) | extra)
+                  .Finalize4();
 }