Skip to content

Building.md

Latest commit

 

History

History
400 lines (225 loc) · 17.7 KB

Building.md

File metadata and controls

400 lines (225 loc) · 17.7 KB
Raw

Building

This document is intended for mediasoup developers.

NPM scripts

The package.json file in the main folder includes the following scripts:

npm run typescript:build

Compiles mediasoup TypeScript code (node/src folder) JavaScript and places it into the node/lib directory.

npm run typescript:watch

Compiles mediasoup TypeScript code (node/src folder) JavaScript, places it into the node/lib directory an watches for changes in the TypeScript files.

npm run worker:build

Builds the mediasoup-worker binary. It invokes invokebelow.

npm run worker:prebuild-name

Prints the name of the corresponding mediasoup-worker prebuild tar file.

npm run worker:prebuild

Creates a prebuilt of mediasoup-worker binary in the worker/prebuild folder.

npm run lint

Runs both lint:node and lint:worker tasks.

npm run lint:node

Validates mediasoup TypeScript files using ESLint, Prettier and Knip.

npm run lint:worker

Validates mediasoup worker C++ files using clang-format. It invokes invoke lint below.

See Install clang-format for requirements.

npm run format

Runs both format:node and format:worker tasks.

npm run format:node

Format TypeScript and JavaScript code using Prettier.

npm run format:worker

Rewrites mediasoup worker C++ files using clang-format. It invokes invoke format below.

See Install clang-format for requirements.

npm run tidy:worker

Runs clang-tidy and performs C++ code checks following worker/.clang-tidy rules. It invokes invoke tidy below.

See Install clang-tidy for requirements.

npm run tidy:worker:fix

Same as npm run tidy:worker but it also applies fixes.

npm run flatc

Runs both flatc:node and flatc:worker tasks.

npm run flatc:node

Compiles FlatBuffers .fbs files in worker/fbs to TypeScript code.

npm run flatc:worker

Compiles FlatBuffers .fbs files in worker/fbs to C++ code.

npm run test

Runs both test:node and test:worker tasks.

npm run test:node

Runs Jest test units located at node/test folder.

Jest command arguments can be given using -- as follows:

npm run test:node -- --testPathPatterns "node/src/test/test-Worker.ts" --testNamePattern "createWorker"

npm run test:worker

Runs Catch2 test units located at worker/test folder. It invokes invoke test below.

npm run coverage

Runs coverage:node task.

npm run coverage:node

Same as test:node task but it also opens a browser window with TypeScript coverage results.

npm run release:check

Runs linters and tests in Node and C++ code. Also verifies that CHANGELOG.md has an entry matching the mediasoup version in package.json.

npm run release x.y.z

Prepares and triggers the release of a new version "x.y.z" of the mediasoup NPM package. The actual GitHub release and NPM publish are done by GitHub Actions (mediasoup-npm-publish.yaml) once the pushed tag arrives. It:

  • Performs checks (lint + test + build + publish dry-run + CHANGELOG.md entry check). It runs before the version bump, so the CHANGELOG check validates the previous version's entry (still in package.json), which is harmless.
  • Bumps the version to "x.y.z" in package.json and package-lock.json with npm version x.y.z --no-git-tag-version, and sets the top ### NEXT heading of CHANGELOG.md to ### x.y.z.
  • Commits the bump (with a "release x.y.z [no-ci]" message), creates the "x.y.z" tag, and pushes the branch and the tag.

Requirements for it to work:

  • Must be called with a SEMVER version as single argument.
  • Must be in the main branch.
  • Work tree must be clean.
  • Changes for the new version must be under the ### NEXT heading in CHANGELOG.md.

npm run release:rust:check

Runs linters and tests in Rust code (cargo fmt, cargo clippy, cargo test and cargo doc). Also verifies that rust/CHANGELOG.md has an entry matching the mediasoup crate version in rust/Cargo.toml. Finally, when that version is not yet published on crates.io (i.e. it has been bumped and is about to be released), it also runs the publish dry-run (cargo publish --dry-run for the three crates as a group).

The publish dry-run is skipped when the mediasoup crate version is already published, because Cargo would then resolve the dependencies among the three crates against the already-published copies on crates.io and any schema/API change made since the last release would fail verification spuriously even though nothing is being published.

npm run release:rust <crate> x.y.z

Prepares and triggers the release of a new version "x.y.z" of a mediasoup Rust crate (mediasoup, mediasoup-sys or mediasoup-types). The actual GitHub release (if any) and crates.io publish are done by GitHub Actions (mediasoup-crate-publish.yaml) once the pushed commit/tag arrives. It:

  • Performs checks (lint + test + build + publish dry-run, plus the rust/CHANGELOG.md entry check when releasing the mediasoup crate). They run before the version bump, so the CHANGELOG check validates the previous version's entry (still in the manifest), which is harmless.
  • Bumps the crate version to "x.y.z" in its Cargo.toml (rust/Cargo.toml, worker/Cargo.toml or rust/types/Cargo.toml) and reflects it in the (workspace root) Cargo.lock. When releasing mediasoup-sys / mediasoup-types, it also bumps the matching version requirement of that dependency in the mediasoup crate's rust/Cargo.toml (committed together with the release), so the mediasoup crate keeps depending on the just-released version.

Then, depending on the crate:

  • For mediasoup: it also sets the top ### NEXT heading of rust/CHANGELOG.md to ### x.y.z, commits the bump (with a release rust-x.y.z [no-ci] message), creates the rust-x.y.z tag and pushes the branch and the tag. The tag triggers mediasoup-crate-publish.yaml, which creates the GitHub release from rust/CHANGELOG.md and publishes the crate.
  • For mediasoup-sys / mediasoup-types: it commits the bump with a <crate> x.y.z [crate-publish] [no-ci] message and pushes the branch (no tag, no CHANGELOG change). The [crate-publish] marker is what mediasoup-crate-publish.yaml detects on the branch push to publish that crate (without a GitHub release).

Since mediasoup depends on mediasoup-sys and mediasoup-types, when several crates need a new version publish the dependencies first (mediasoup-types / mediasoup-sys) and mediasoup last, so each crate's dependencies are already on crates.io. The releases can be run back to back without waiting for the dependencies' GitHub Actions runs to complete: mediasoup-crate-publish.yaml runs are serialized through a single concurrency queue, so the mediasoup one stays queued until the sibling crates are published (see Rust-crates.md). Requirements for it to work:

  • Must be called with a crate name and a SEMVER version as the two arguments.
  • Must be in the main branch.
  • Work tree must be clean.
  • Cargo.lock must be in sync (run cargo build and commit it if needed), otherwise the release aborts before doing anything irreversible.
  • When releasing the mediasoup crate, the changes for the new version must be under the ### NEXT heading in rust/CHANGELOG.md.

Rust

The only special feature in Rust case is special environment variable "MEDIASOUP_LOCAL_DEV", that when set to "true" will allow incremental recompilation of changed C++ sources during hacking on mediasoup.

It is not necessary for normal usage of mediasoup as a dependency.

Other than that and normal cargo commands, see npm run release:rust:check and npm run release:rust above.

Python Invoke and tasks.py file

mediasoup uses Python Invoke library for managing and organizing tasks in the worker folder (mediasoup worker C++ subproject). Invoke is basically a replacemente of make + Makefile written in Python. mediasoup automatically installs Invoke in a local custom path during the installation process (in both Node and Rust) so the user doesn't need to worry about it.

Tasks are defined in worker/tasks.py. For development purposes, developers or contributors can install Invoke using pip3 install invoke and run tasks below within the worker folder.

See all the tasks by running invoke --list within the worker folder.

NOTE: For some of these tasks to work, npm dependencies of worker/scripts/package.json must be installed:

npm ci --prefix worker/scripts

invoke (default task)

Alias of invoke mediasoup-worker task below.

invoke meson-ninja

Installs meson and ninja into a local custom path.

invoke clean

Cleans built objects and binaries.

invoke clean-build

Cleans built objects and other artifacts, but keeps mediasoup-worker binary in place.

invoke clean-pip

Cleans meson and ninja installed in local prefix with pip.

invoke clean-subprojects

Cleans subprojects downloaded with Meson.

invoke clean-all

Cleans built objects and binaries, meson and ninja installed in local prefix with pip and all subprojects downloaded with Meson.

invoke check-wrap-status

Check the status of the Meson subprojects. It also prints whether there are updates available.

invoke update-wrap-file [subproject]

Updates the wrap file of a Meson subproject (those in worker/subprojects folder). After updating it, invoke setup must be called by passing MESON_ARGS="--reconfigure" environment variable. Usage example:

cd worker
invoke update-wrap-file openssl
MESON_ARGS="--reconfigure" invoke setup

invoke mediasoup-worker

Builds the mediasoup-worker binary at worker/out/Release.

If the "MEDIASOUP_MAX_CORES" environment variable is set, the build process will use that number of CPU cores. Otherwise it will auto-detect the number of cores in the machine.

"MEDIASOUP_BUILDTYPE" environment variable controls build types, "Release" and "Debug" are presets optimized for those use cases. Other build types are possible too, but they are not presets and will require "MESON_ARGS" use to customize build configuration.

Check the meaning of useful macros in the worker/include/Logger.hpp header file if you want to enable tracing or other debug information.

Binary is built at worker/out/MEDIASOUP_BUILDTYPE/build.

In order to instruct the mediasoup Node.js module to use the "Debug" mediasoup-worker` binary, an environment variable must be set before running the Node.js application:

MEDIASOUP_BUILDTYPE=Debug node myapp.js

If the "MEDIASOUP_WORKER_BIN" environment variable is set (it must be an absolute file path), mediasoup will use the it as mediasoup-worker binary and won't compile the binary:

MEDIASOUP_WORKER_BIN="/home/xxx/src/foo/mediasoup-worker" node myapp.js

invoke libmediasoup-worker

Builds the libmediasoup-worker static library at worker/out/Release.

"MEDIASOUP_MAX_CORES"` and "MEDIASOUP_BUILDTYPE" environment variables from above still apply for static library build.

invoke xcode

Builds a Xcode project for the mediasoup worker subproject.

invoke lint

Validates mediasoup worker C++ files using clang-format and rules in worker/.clang-format. It also validates Python code within worker folder (i.e. worker/tasks.py file).

Requirements:

  • A specific version of clang-formatis required. See Install clang-format.
  • clang-format-VERSION or clang-format (corresponding to the required version) must be in the PATH. If not, add it before running the command.

invoke format

Rewrites mediasoup worker C++ files using clang-format.

Requirements:

  • A specific version of clang-formatis required. See Install clang-format.
  • clang-format-VERSION or clang-format (corresponding to the required version) must be in the PATH. If not, add it before running the command.

invoke tidy

Runs clang-tidy and performs C++ code checks following worker/.clang-tidy rules.

Requirements:

  • invoke clean must have been called first.
  • A specific version of clang-tidyis required. See Install clang-tidy.
  • clang-tidy-VERSION or clang-tidy (corresponding to the required version) must be in the PATH. If not, add it before running the command. Same for other clang-tidy related executables such as run-clang-tidy and clang-apply-replacements,

Environment variables:

  • "MEDIASOUP_TIDY_CHECKS": Optional. Comma separated list of checks. Overrides the checks defined in worker/.clang-tidy file.
  • "MEDIASOUP_TIDY_FILES": Optional. Space separated source file paths to process. All .cpp files will be processes by default.
    • File paths must be relative to worker/ folder.
    • File paths can use glob syntax. Example: "src/RTC/SCTP/**/*.cpp".

Usage example in macOS:

PATH="/opt/homebrew/opt/llvm/bin/:$PATH" invoke tidy

It may happens that clang-tidy doesn't know where C++ standard libraries are so it shows lot of warnings about them. Depending on your local setup this may work:

PATH="/opt/homebrew/opt/llvm/bin/:$PATH" CPATH=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1 invoke tidy

invoke tidy-fix

Same as invoke tidy but it also applies fixes.

invoke test

Builds and runs the mediasoup-worker-test binary at worker/out/Release (or at worker/out/Debug if the "MEDIASOUP_BUILDTYPE" environment variable is set to "Debug"), which uses Catch2 to run test units located at worker/test folder.

invoke test-asan-address

Run test with Address Sanitizer with -fsanitize=address.

invoke test-asan-undefined

Run test with Address Sanitizer with -fsanitize=undefined.

invoke fuzzer

Builds the mediasoup-worker-fuzzer binary (which uses libFuzzer) at worker/out/Release (or at worker/out/Debug/ if the "MEDIASOUP_BUILDTYPE" environment variable is set to "Debug").

Requirements:

  • Linux with fuzzer capable clang++.
  • "CC" environment variable must point to clang.
  • "CXX" environment variable must point to clang++.

Read the Fuzzer documentation for detailed information.

invoke fuzzer-run-all

Runs all fuzzer cases.

invoke docker

Builds a Linux Ubuntu Docker image with fuzzer capable clang++ and all dependencies to run mediasoup.

invoke docker-run

Runs a container of the Ubuntu Docker image created with invoke docker. It automatically executes a bash session in the mediasoup directory, which is a Docker volume that points to the mediasoup root folder.

NOTE: To install and run mediasoup in the container, previous installation (if any) must be properly cleaned by entering the worker directory and running invoke clean-all.

invoke docker-alpine

Builds a Linux Alpine Docker image with all dependencies to run mediasoup.

invoke docker-alpine-run

Runs a container of the Alpine Docker image created with invoke docker-alpine. It automatically executes an ash session in the mediasoup directory, which is a Docker volume that points to the mediasoup root folder.

NOTE: To install and run mediasoup in the container, previous installation (if any) must be properly cleaned by entering the worker directory and running invoke clean-all.

invoke docker-386

Builds a 386 Linux Debian (32 bits arch) Docker image with all dependencies to run mediasoup.

invoke docker-alpine-386

Runs a container of the 386 Linux Debian (32 bits arch) Docker image created with invoke docker-386. It automatically executes an ash session in the mediasoup directory, which is a Docker volume that points to the mediasoup root folder.

NOTE: To install and run mediasoup in the container, previous installation (if any) must be properly cleaned by entering the worker directory and running invoke clean-all. NOTE: Due to the very old Node v18 in this image, in order to run mediasoup Node tests, npm ci must be executed with --ignore-scripts --engine-strict=false arguments.

Makefile

The worker folder contains a Makefile file for the mediasoup worker C++ subproject. It acts as a proxy to the Invoke tasks defined in tasks.py. The Makefile file exists to help developers or contributors that prefer keep using make commands.

All tasks defined in tasks.py (see above) are available in Makefile. There is only one exception:

  • The update-wrap-file needs a "SUBPROJECT" environment variable indicating the subproject to update. Usage example: 
    cd worker
make update-wrap-file SUBPROJECT=openssl

Install clang-format

A specific clang-format version is required to be installed in the system, which is defined in clang-scripts.mjs.

macOS:

brew install clang-format@VERSION

Linux:

apt-get install clang-format-VERSION

Install clang-tidy

A specific clang-tidy version is required to be installed in the system, which is defined in clang-scripts.mjs.

macOS:

brew install clang-tidy@VERSION

Linux:

apt-get install clang-tidy-VERSION