build: Added build support for jemalloc. by pdath · Pull Request #315 · bitcoinknots/bitcoin

pdath · 2026-06-21T01:15:26Z

Back in 2024 Bitcoin Core added support for an alternative memory allocator called jemalloc. bitcoin#25569
However, support was never added to the build environment.

This is primarily a cmake change to enable support to link in the jemalloc library, and a tiny documentation update. There are no source code changes. I have modelled this change off WITH_ZMQ. Keys steps to use jemalloc are:

sudo sudo apt install libjemalloc-dev
cmake -B build-jemalloc-on -DWITH_JEMALLOC=ON -DCMAKE_BUILD_TYPE=Release -DRDTS_CONSENT=IMPLICIT
cmake --build build-jemalloc-on -j$(nproc)

You can confirm jemalloc has been built into the executables with:

ldd build-jemalloc-on/bin/bitcoind | grep jemalloc

You can enable memory allocation performance information with a command like:

MALLOC_CONF="stats_print:true" bitcoind

This will cause a lot of data to be displayed after the executable exits.

I have run the following test suite on Linux, and everything passed.

cd ~/bitcoin/build
ctest --output-on-failure

Problem Being Solved

The problem I am trying to solve is how to improve Bitcoin Knots' performance for miners. Pools frequently invoke RPC functions that involve extensive JSON processing, such as getblocktemplate. UniValue often performs many thousands of memory allocations/frees per block.

Why is jemalloc better than the glibc allocator for Bitcoin Knots?

Per thread memory pools. This means less mutex contention for memory allocations between threads.
Per thread cache of frequently alocations, for lock free allocations..
Size class bins to reduce memory fragmentation.
Uses madvise() on Linux to give the kernel hints about how the memory will be used

Benchmark Results (on my machine)

Performance improvements are across the board. The only regression I have been able to find is BlockToJsonVerboseWrite.

Benchmark	`WITH_JEMALLOC=OFF`	`WITH_JEMALLOC=ON`	Improvement
`AssembleBlock`	636,476.43	605,815.89	+4.82%
`BlockAssemblerAddPackageTxns`	1,413,593,628.00	1,301,676,538.50	+7.92%
`BlockToJsonVerboseWrite`	36,673,658.35	42,366,423.95	-15.52%
`BlockToJsonVerbosity1`	558,989.43	469,013.92	+16.10%
`BlockToJsonVerbosity2`	78,972,618.43	69,016,445.29	+12.61%
`BlockToJsonVerbosity3`	78,990,756.59	68,406,736.46	+13.40%
`CCoinsCaching`	915.62	813.13	+11.19%
`MempoolCheck`	167,298,038.94	149,698,390.81	+10.52%
`MempoolCheckEphemeralSpends`	283,963.22	238,437.33	+16.03%
`RpcMempool`	12,907,460.40	11,443,657.15	+11.34%

Reproducing Benchmark Results

To reproduce the benchmark results, use the following commands. Note that each benchmark will take 5 minutes to run.

Build Without jemalloc

cmake -B build-jemalloc-off -DCMAKE_BUILD_TYPE=Release -DBUILD_BENCH=ON -DRDTS_CONSENT=IMPLICIT
cmake --build build-jemalloc-off -j$(nproc) -t bench_bitcoin 

./build-jemalloc-off/bin/bench_bitcoin \
  -filter="AssembleBlock|Block[AT].*|CCoinsCaching|MempoolC.*|RpcMempool" \
  -min-time=30000 | \
  awk -F'|' '/,/{print $2,$NF}'

Build With jemalloc

cmake -B build-jemalloc-on -DWITH_JEMALLOC=ON -DCMAKE_BUILD_TYPE=Release -DBUILD_BENCH=ON -DRDTS_CONSENT=IMPLICIT
cmake --build build-jemalloc-on -j$(nproc) -t bench_bitcoin 

./build-jemalloc-on/bin/bench_bitcoin \
  -filter="AssembleBlock|Block[AT].*|CCoinsCaching|MempoolC.*|RpcMempool" \
  -min-time=30000 | \
  awk -F'|' '/,/{print $2,$NF}'

The above commands will output "ns/op" followed by the test name.
Compare the results to verify the performance improvements shown in the benchmark table above.

Retropex · 2026-07-02T11:05:20Z

tACK 4b23cf1820a23e982b253e8fd5fc8b8036ce54e8

Tested on macOS.

brew install jemalloc

It might be good to update the macOS doc too.

pdath · 2026-07-02T11:18:42Z

brew install jemalloc
It might be good to update the macOS doc too.

I have updated the build-osx.md file with the dependency.

Back in 2024 Bitcoin Core added support for an alternative memory allocator called jemalloc. bitcoin#25569 However, support was never added to the build environment. This is primarily a cmake change to enable support to link in the jemalloc library, and a tiny documentation update. There are no source code changes. I have modelled this change off WITH_ZMQ. Keys steps to use jemalloc are: sudo sudo apt install libjemalloc-dev cmake -B build -DWITH_JEMALLOC=ON cmake --build build The problem I am trying to solve is how to improve the performance of Bitcoin Knots for miners. Pool's frequently exercise RPC functions that involve a lot of JSON processing, such as getblocktemplate. When I use the following benchmark: bench_bitcoin -filter="RpcMempool" -min-time=15000 I get a 30% performance improvement when jemalloc is used. When I run the entire benchmark suite there are no performance regressions of note (two performed very slightly slower, but so little I consider it within test error margins). You can confirm jemalloc has been built into the executables with: ldd build/bin/bitcoind | grep jemalloc ldd build/bin/bitcoin-cli | grep jemalloc You can enable memory allocation performance information with a command like: MALLOC_CONF="stats_print:true" bitcoind This will cause a lot of data to be displayed after the executable exits. I have run the following test suite on Linux, and everything passed. cd ~/bitcoin/build ctest --output-on-failure

kwsantiago

A few CMake issues in FindJemalloc.cmake. On a typical Linux box find_library finds libjemalloc directly so the build works, but the pkg-config fallback and config-mode paths are effectively non-functional as written. Converging the whole module on the FindZeroMQ.cmake IMPORTED_TARGET pattern would fix comments 1-3 at once. No security blockers found: the secure-allocator path uses mmap+memset and bypasses malloc, so jemalloc doesn't affect key zeroization, and the option is OFF-by-default.

kwsantiago · 2026-07-02T23:53:21Z

+    find_library(JEMALLOC_LIBRARY NAMES jemalloc)
+    find_path(JEMALLOC_INCLUDE_DIR NAMES jemalloc/jemalloc.h jemalloc.h)
+  else()
+    set(JEMALLOC_LIBRARY ${PC_Jemalloc_LIBRARIES})


PC_Jemalloc_LIBRARIES holds library names (e.g. jemalloc), not absolute paths, but it feeds IMPORTED_LOCATION at line 55, which requires a full path to the library file. This breaks the pkg-config path. Use pkg_check_modules(... IMPORTED_TARGET ...) + add_library(Jemalloc::Jemalloc ALIAS PkgConfig::PC_Jemalloc) as FindZeroMQ does, instead of the hand-rolled imported target.

kwsantiago · 2026-07-02T23:53:21Z

+  find_package_handle_standard_args(Jemalloc
+    REQUIRED_VARS JEMALLOC_LIBRARY JEMALLOC_INCLUDE_DIR
+  )
+


Trailing whitespace here and on lines 38, 47 (repo whitespace lint will flag these).

kwsantiago · 2026-07-02T23:53:21Z


+#### jemalloc Dependencies
+
+Support for jemalloc requires the following dependency:


Question, not something I have verified: does linking libjemalloc actually override malloc on macOS? @Retropex reported a successful build/test on macOS, which confirms it compiles and links, but not necessarily that jemalloc is serving allocations at runtime. On macOS the two-level namespace + zone allocator generally means a plain link does not interpose malloc without DYLD_INSERT_LIBRARIES or a je-prefixed build. If the allocator is not actually replaced on macOS, this doc would be misleading. Worth confirming (e.g. vmmap or a MALLOC_CONF stats check) that jemalloc handles allocations before documenting it as supported.

Looks like you are right, macOS still uses malloc:

==== Summary for process 39264 ReadOnly portion of Libraries: Total=595.2M resident=32.2M(5%) swapped_out_or_unallocated=563.0M(95%) Writable regions: Total=603.7M written=31.0M(5%) resident=424.7M(70%) swapped_out=0K(0%) unallocated=179.0M(30%) VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced) =========== ======= ======== ===== ======= ======== ====== ===== ======= Activity Tracing 256K 16K 16K 0K 0K 16K 0K 1 CoreMedia Capture Data 498.5M 422.7M 422.7M 0K 0K 0K 0K 3363 CoreMedia Capture Data (reserved) 640K 0K 0K 0K 0K 0K 0K 8 reserved VM address space (unallocated) Kernel Alloc Once 32K 16K 16K 0K 0K 0K 0K 1 MALLOC guard page 3232K 0K 0K 0K 0K 0K 0K 4 MALLOC metadata 928K 384K 384K 0K 0K 0K 0K 5 MALLOC_NANO metadata 32K 16K 16K 0K 0K 0K 0K 1 see MALLOC ZONE table below MALLOC_SMALL 8192K 32K 32K 0K 0K 0K 0K 2 see MALLOC ZONE table below MALLOC_TINY 4096K 32K 32K 0K 0K 0K 0K 1 see MALLOC ZONE table below Memory Tag 22 64.0M 16K 16K 0K 0K 0K 0K 1 STACK GUARD 56.5M 0K 0K 0K 0K 0K 0K 34 Stack 25.6M 704K 704K 0K 0K 0K 0K 36 VM_ALLOCATE 256K 256K 256K 0K 0K 0K 0K 1 __AUTH 39K 34K 0K 0K 0K 0K 0K 13 __AUTH_CONST 230K 177K 52K 0K 0K 0K 0K 46 __DATA 1369K 327K 206K 0K 0K 0K 0K 40 __DATA_CONST 306K 306K 67K 0K 0K 0K 0K 48 __DATA_DIRTY 167K 167K 132K 0K 0K 0K 0K 36 __LINKEDIT 577.4M 17.8M 0K 0K 0K 0K 0K 6 __OBJC_RO 79.1M 56.1M 0K 0K 0K 0K 0K 1 __OBJC_RW 2598K 1542K 22K 0K 0K 0K 0K 1 __TEXT 17.8M 14.4M 0K 0K 0K 0K 0K 51 __TPRO_CONST 128K 48K 48K 0K 0K 0K 0K 2 mapped file 120.9M 120.9M 0K 0K 0K 0K 0K 4 page table in kernel 561K 561K 561K 0K 0K 0K 0K 1 shared memory 48K 48K 48K 0K 0K 0K 0K 2 unused but dirty shlib __DATA 47.3M 11.6M 8001K 0K 0K 0K 0K 1248 =========== ======= ======== ===== ======= ======== ====== ===== ======= TOTAL 1.5G 648.1M 433.1M 0K 0K 16K 0K 4957 TOTAL, minus reserved VM space 1.5G 648.1M 433.1M 0K 0K 16K 0K 4957 VIRTUAL RESIDENT DIRTY SWAPPED ALLOCATION BYTES DIRTY+SWAP REGION MALLOC ZONE SIZE SIZE SIZE SIZE COUNT ALLOCATED FRAG SIZE % FRAG COUNT =========== ======= ========= ========= ========= ========= ========= ========= ====== ====== DefaultMallocZone_0x1056fc000 12.8M 304K 304K 0K 184 10K 294K 97% 6 DefaultPurgeableMallocZone_0x1057c4000 0K 0K 0K 0K 0 0K 0K 0% 0 =========== ======= ========= ========= ========= ========= ========= ========= ====== ====== TOTAL 12.8M 304K 304K 0K 184 10K 294K 97% 6

That's said, I don't know why CoreMedia Capture Data is involved here.

I don't know anything about MacOS. If you run this command (setting the MALLOC_CONF environmental variable):

MALLOC_CONF="stats_print:true" bitcoind

And then CTRL-C, do you get a lot of statistics dumped?

No => jemalloc definitely not working
Yes -> jemalloc is likely to be working

Yes it dump a lot of statistics! I have compiled again without jemalloc and it did not dump statistics this time.

that's said vmmap does not report anything about using jemalloc.

Well, if it is reporting jemalloc statistics ... I would say it is using jemalloc.

If you run the benchmark:
bench_bitcoin -filter="RpcMempool" -min-time=15000

Do you see a performance increase? If so, it is using jemalloc for sure.

Yes it dump a lot of statistics! I have compiled again without jemalloc and it did not dump statistics this time.

that's said vmmap does not report anything about using jemalloc.

ps. Those statistics it dumped prompted what will be my next pull request to replace the UniValue engine. I have a yyjson version in testing, and it is a lot faster.

Look pretty similar, within error margin.

With:

% ./bench_bitcoin -filter="RpcMempool" -min-time=15000 | ns/op | op/s | err% | total | benchmark |--------------------:|--------------------:|--------:|----------:|:---------- | 2,175,005.12 | 459.77 | 0.2% | 16.55 | `RpcMempool`

Without:

% ./bench_bitcoin -filter="RpcMempool" -min-time=15000 | ns/op | op/s | err% | total | benchmark |--------------------:|--------------------:|--------:|----------:|:---------- | 2,193,857.14 | 455.82 | 0.3% | 16.53 | `RpcMempool`

Hmm. Need someone to test on other platforms as well then.
It could also be that the memory allocator on macOS, with whatever compiler you are using, is simply better than on Ubuntu with GCC.

@Retropex , could I ask you to re-test this on MacOS? I think I have corrected a build issue. Note that I have also added a section on "Reproducing Benchmark Results".

Ari4ka approved these changes Jun 21, 2026

View reviewed changes

pdath force-pushed the feature-jemalloc branch from 4b23cf1 to 85415c3 Compare July 2, 2026 11:17

pdath closed this Jul 2, 2026

pdath reopened this Jul 2, 2026

pdath force-pushed the feature-jemalloc branch from 85415c3 to e128fe8 Compare July 2, 2026 11:22

kwsantiago reviewed Jul 2, 2026

View reviewed changes

Update jemalloc detection in cmake.

b6accde

pdath force-pushed the feature-jemalloc branch from 1a102b4 to b6accde Compare July 20, 2026 07:00

pdath requested a review from kwsantiago July 20, 2026 09:05

Ari4ka approved these changes Jul 20, 2026

View reviewed changes


		#### jemalloc Dependencies

		Support for jemalloc requires the following dependency:

Uh oh!

Conversation

pdath commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem Being Solved

Benchmark Results (on my machine)

Reproducing Benchmark Results

Build Without jemalloc

Build With jemalloc

Uh oh!

Retropex commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pdath commented Jul 2, 2026

Uh oh!

kwsantiago left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kwsantiago Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pdath commented Jun 21, 2026 •

edited

Loading

Retropex commented Jul 2, 2026 •

edited

Loading

kwsantiago Jul 2, 2026 •

edited

Loading