Skip to content

build: Added build support for jemalloc. #315

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
pdath wants to merge 2 commits into
base: 29.x-knots
Choose a base branch
from
Open

build: Added build support for jemalloc. #315

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e128fe8
build: Added build support for jemalloc.
pdath Jun 20, 2026
b6accde
Update jemalloc detection in cmake.
pdath Jul 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,11 @@ if(WITH_USDT)
find_package(USDT MODULE REQUIRED)
endif()

option(WITH_JEMALLOC "Enable jemalloc support." OFF)
if(WITH_JEMALLOC)
find_package(Jemalloc MODULE REQUIRED)
endif()

option(ENABLE_EXTERNAL_SIGNER "Enable external signer support." ON)

option(ENABLE_TOR_SUBPROCESS "Enable support for executing a dedicated Tor node." ON)
Expand Down Expand Up @@ -818,6 +823,7 @@ message(" external signer ..................... ${ENABLE_EXTERNAL_SIGNER}")
message(" tor subprocess ...................... ${ENABLE_TOR_SUBPROCESS}")
message(" port mapping using UPnP ............. ${WITH_MINIUPNPC}")
message(" ZeroMQ .............................. ${WITH_ZMQ}")
message(" jemalloc ............................ ${WITH_JEMALLOC}")
message(" USDT tracing ........................ ${WITH_USDT}")
message(" QR code (GUI) ....................... ${WITH_QRENCODE}")
message(" DBus (GUI) .......................... ${WITH_DBUS}")
Expand Down
46 changes: 46 additions & 0 deletions cmake/module/FindJemalloc.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Copyright (c) 2026-present The Bitcoin Knots developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or https://opensource.org/license/mit/.

#[=======================================================================[
FindJemalloc
-----------

Finds the jemalloc headers and library.

This is a wrapper around find_package()/pkg_check_modules() commands that:
- facilitates searching in various build environments
- prints a standard log message

#]=======================================================================]

include(FindPackageHandleStandardArgs)

find_package(Jemalloc ${Jemalloc_FIND_VERSION} NO_MODULE QUIET)
Comment thread
pdath marked this conversation as resolved.
if(Jemalloc_FOUND)
find_package_handle_standard_args(Jemalloc
REQUIRED_VARS Jemalloc_DIR
VERSION_VAR Jemalloc_VERSION
)
if(TARGET jemalloc)
add_library(Jemalloc::Jemalloc ALIAS jemalloc)
elseif(TARGET jemalloc-static)
add_library(Jemalloc::Jemalloc ALIAS jemalloc-static)
endif()
mark_as_advanced(Jemalloc_DIR)
else()
find_package(PkgConfig REQUIRED)
if(Jemalloc_FIND_VERSION)
set(_jemalloc_pc_arg "jemalloc>=${Jemalloc_FIND_VERSION}")
else()
set(_jemalloc_pc_arg "jemalloc")
endif()
pkg_check_modules(PC_Jemalloc QUIET IMPORTED_TARGET ${_jemalloc_pc_arg})
find_package_handle_standard_args(Jemalloc
REQUIRED_VARS PC_Jemalloc_FOUND
VERSION_VAR PC_Jemalloc_VERSION
)
if(Jemalloc_FOUND)
add_library(Jemalloc::Jemalloc ALIAS PkgConfig::PC_Jemalloc)
endif()
endif()
10 changes: 10 additions & 0 deletions doc/build-osx.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,16 @@ For more information on ZMQ, see: [zmq.md](zmq.md)

---

#### jemalloc Dependencies

Support for jemalloc requires the following dependency:

@kwsantiago kwsantiago Jul 2, 2026 •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question, not something I have verified: does linking libjemalloc actually override malloc on macOS? @Retropex reported a successful build/test on macOS, which confirms it compiles and links, but not necessarily that jemalloc is serving allocations at runtime. On macOS the two-level namespace + zone allocator generally means a plain link does not interpose malloc without DYLD_INSERT_LIBRARIES or a je-prefixed build. If the allocator is not actually replaced on macOS, this doc would be misleading. Worth confirming (e.g. vmmap or a MALLOC_CONF stats check) that jemalloc handles allocations before documenting it as supported.

@Retropex Retropex Jul 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like you are right, macOS still uses malloc:

==== Summary for process 39264
ReadOnly portion of Libraries: Total=595.2M resident=32.2M(5%) swapped_out_or_unallocated=563.0M(95%)
Writable regions: Total=603.7M written=31.0M(5%) resident=424.7M(70%) swapped_out=0K(0%) unallocated=179.0M(30%)

                                   VIRTUAL RESIDENT    DIRTY  SWAPPED VOLATILE   NONVOL    EMPTY   REGION
REGION TYPE                           SIZE     SIZE     SIZE     SIZE     SIZE     SIZE     SIZE    COUNT (non-coalesced)
===========                        ======= ========    =====  ======= ========   ======    =====  =======
Activity Tracing                      256K      16K      16K       0K       0K      16K       0K        1
CoreMedia Capture Data              498.5M   422.7M   422.7M       0K       0K       0K       0K     3363
CoreMedia Capture Data (reserved)     640K       0K       0K       0K       0K       0K       0K        8         reserved VM address space (unallocated)
Kernel Alloc Once                      32K      16K      16K       0K       0K       0K       0K        1
MALLOC guard page                    3232K       0K       0K       0K       0K       0K       0K        4
MALLOC metadata                       928K     384K     384K       0K       0K       0K       0K        5
MALLOC_NANO metadata                   32K      16K      16K       0K       0K       0K       0K        1         see MALLOC ZONE table below
MALLOC_SMALL                         8192K      32K      32K       0K       0K       0K       0K        2         see MALLOC ZONE table below
MALLOC_TINY                          4096K      32K      32K       0K       0K       0K       0K        1         see MALLOC ZONE table below
Memory Tag 22                        64.0M      16K      16K       0K       0K       0K       0K        1
STACK GUARD                          56.5M       0K       0K       0K       0K       0K       0K       34
Stack                                25.6M     704K     704K       0K       0K       0K       0K       36
VM_ALLOCATE                           256K     256K     256K       0K       0K       0K       0K        1
__AUTH                                 39K      34K       0K       0K       0K       0K       0K       13
__AUTH_CONST                          230K     177K      52K       0K       0K       0K       0K       46
__DATA                               1369K     327K     206K       0K       0K       0K       0K       40
__DATA_CONST                          306K     306K      67K       0K       0K       0K       0K       48
__DATA_DIRTY                          167K     167K     132K       0K       0K       0K       0K       36
__LINKEDIT                          577.4M    17.8M       0K       0K       0K       0K       0K        6
__OBJC_RO                            79.1M    56.1M       0K       0K       0K       0K       0K        1
__OBJC_RW                            2598K    1542K      22K       0K       0K       0K       0K        1
__TEXT                               17.8M    14.4M       0K       0K       0K       0K       0K       51
__TPRO_CONST                          128K      48K      48K       0K       0K       0K       0K        2
mapped file                         120.9M   120.9M       0K       0K       0K       0K       0K        4
page table in kernel                  561K     561K     561K       0K       0K       0K       0K        1
shared memory                          48K      48K      48K       0K       0K       0K       0K        2
unused but dirty shlib __DATA        47.3M    11.6M    8001K       0K       0K       0K       0K     1248
===========                        ======= ========    =====  ======= ========   ======    =====  =======
TOTAL                                 1.5G   648.1M   433.1M       0K       0K      16K       0K     4957
TOTAL, minus reserved VM space        1.5G   648.1M   433.1M       0K       0K      16K       0K     4957

                                          VIRTUAL   RESIDENT      DIRTY    SWAPPED ALLOCATION      BYTES DIRTY+SWAP          REGION
MALLOC ZONE                                  SIZE       SIZE       SIZE       SIZE      COUNT  ALLOCATED  FRAG SIZE  % FRAG   COUNT
===========                               =======  =========  =========  =========  =========  =========  =========  ======  ======
DefaultMallocZone_0x1056fc000               12.8M       304K       304K         0K        184        10K       294K     97%       6
DefaultPurgeableMallocZone_0x1057c4000         0K         0K         0K         0K          0         0K         0K      0%       0
===========                               =======  =========  =========  =========  =========  =========  =========  ======  ======
TOTAL                                       12.8M       304K       304K         0K        184        10K       294K     97%       6

That's said, I don't know why CoreMedia Capture Data is involved here.

@pdath pdath Jul 7, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know anything about MacOS. If you run this command (setting the MALLOC_CONF environmental variable):

MALLOC_CONF="stats_print:true" bitcoind

And then CTRL-C, do you get a lot of statistics dumped?

No => jemalloc definitely not working
Yes -> jemalloc is likely to be working

@Retropex Retropex Jul 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it dump a lot of statistics! I have compiled again without jemalloc and it did not dump statistics this time.

that's said vmmap does not report anything about using jemalloc.

@pdath pdath Jul 7, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, if it is reporting jemalloc statistics ... I would say it is using jemalloc.

If you run the benchmark:
bench_bitcoin -filter="RpcMempool" -min-time=15000

Do you see a performance increase? If so, it is using jemalloc for sure.

@pdath pdath Jul 7, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it dump a lot of statistics! I have compiled again without jemalloc and it did not dump statistics this time.

that's said vmmap does not report anything about using jemalloc.

ps. Those statistics it dumped prompted what will be my next pull request to replace the UniValue engine. I have a yyjson version in testing, and it is a lot faster.

@Retropex Retropex Jul 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look pretty similar, within error margin.

With:

% ./bench_bitcoin -filter="RpcMempool" -min-time=15000

|               ns/op |                op/s |    err% |     total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
|        2,175,005.12 |              459.77 |    0.2% |     16.55 | `RpcMempool`

Without:

% ./bench_bitcoin -filter="RpcMempool" -min-time=15000

|               ns/op |                op/s |    err% |     total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
|        2,193,857.14 |              455.82 |    0.3% |     16.53 | `RpcMempool`

@pdath pdath Jul 7, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. Need someone to test on other platforms as well then.
It could also be that the memory allocator on macOS, with whatever compiler you are using, is simply better than on Ubuntu with GCC.

@pdath pdath Jul 20, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Retropex , could I ask you to re-test this on MacOS? I think I have corrected a build issue. Note that I have also added a section on "Reproducing Benchmark Results".


``` bash
brew install jemalloc
```

---

#### Test Suite Dependencies

There is an included test suite that is useful for testing code changes when developing.
Expand Down
4 changes: 4 additions & 0 deletions doc/build-unix.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,10 @@ ZMQ-enabled binaries are compiled with `-DWITH_ZMQ=ON` and require the following

sudo apt-get install libzmq3-dev

jemalloc-enabled binaries are compiled with `-DWITH_JEMALLOC=ON` and require the following dependency:

sudo apt-get install libjemalloc-dev

User-Space, Statically Defined Tracing (USDT) dependencies:

sudo apt install systemtap-sdt-dev
Expand Down
2 changes: 2 additions & 0 deletions src/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -350,6 +350,7 @@ if(BUILD_DAEMON)
core_interface
bitcoin_node
$<TARGET_NAME_IF_EXISTS:bitcoin_wallet>
$<TARGET_NAME_IF_EXISTS:Jemalloc::Jemalloc>
)
install_binary_component(bitcoind HAS_MANPAGE)
endif()
Expand All @@ -363,6 +364,7 @@ if(WITH_MULTIPROCESS AND BUILD_DAEMON)
bitcoin_node
bitcoin_ipc
$<TARGET_NAME_IF_EXISTS:bitcoin_wallet>
$<TARGET_NAME_IF_EXISTS:Jemalloc::Jemalloc>
)
install_binary_component(bitcoin-node)
endif()
Expand Down
1 change: 1 addition & 0 deletions src/bench/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ target_link_libraries(bench_bitcoin
$<TARGET_NAME_IF_EXISTS:bitcoinconsensus>
bitcoin_node
Boost::headers
$<TARGET_NAME_IF_EXISTS:Jemalloc::Jemalloc>
)

if(ENABLE_WALLET)
Expand Down
2 changes: 2 additions & 0 deletions src/qt/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -418,6 +418,7 @@ target_link_libraries(bitcoin-qt
core_interface
bitcoinqt
bitcoin_node
$<TARGET_NAME_IF_EXISTS:Jemalloc::Jemalloc>
)

import_plugins(bitcoin-qt)
Expand All @@ -436,6 +437,7 @@ if(WITH_MULTIPROCESS)
bitcoinqt
bitcoin_node
bitcoin_ipc
$<TARGET_NAME_IF_EXISTS:Jemalloc::Jemalloc>
)
import_plugins(bitcoin-gui)
install_binary_component(bitcoin-gui)
Expand Down
2 changes: 2 additions & 0 deletions src/test/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,7 @@ target_link_libraries(test_bitcoin
secp256k1
Boost::headers
libevent::extra
$<TARGET_NAME_IF_EXISTS:Jemalloc::Jemalloc>
)

if(ENABLE_WALLET)
Expand All @@ -166,6 +167,7 @@ if(WITH_MULTIPROCESS)
PRIVATE
core_interface
univalue
$<TARGET_NAME_IF_EXISTS:Jemalloc::Jemalloc>
)

target_sources(test_bitcoin
Expand Down