test: rewrite integration tests in Typescript, using new test harness

[lewisdaly]

Motivation

As we bring TigerBeetle into Mojaloop, I've found the current integration test setup (where the harness lives outside of the runner, dependent on docker, coordinated across a number of shell scripts) to be quite brittle and hard to debug.

Since we want to switch to the native nodejs test runner anyway, we decided it was best to bring this work in now instead of waiting before we bring in TigerBeetle. This improves the speed of local deveopment, and makes the integration tests on CI less brittle.

Additionally, this brings the time it takes to run the integration tests in CI from 17 minutes -> ~6 minutes, with more improvements to be made by increasing the parallelism and tightening timeouts later.

How it Works

Each *.int.ts file is an integration test file, and is resonsible for running it's own infrastructure with Harness. (currently mysql, kafka, redis). Actually it's mariadb and redpanda since they give much better startup times.

For example:

import Harness from './harness'
const harness = harness.getInstance()

// Then, in the before() and after() hooks: 
before(async () => {
  await harness.up()
  await harness.setupGlobals()
})

after(async () => {
  await harness.teardownGlobals()
  await harness.down()
})

Since each test file has it's isolated infrastructure, we don't have to code around existing database state, or run the tests in a given order to ensure the database has the expected state.

Other Bits And Pieces

Api Helpers

I've also introduced a api-helpers.ts file, which gives us higher level abstractions for doing common things inside of Mojaloop, with sensible defaults. For example, to create and prepare a payment, we can do:

await ApiHelpers.buildPayment()
      .deps(harness, handlerAll)
      .parties('dfsp_a', 'dfsp_b')
      .transferId('1000001')
      .build()
      .prepare()

Internally, the harness tracks the kafka messages being produced and consumed, and waits until the required messages are produced and consumed, or fails if the incorrect number of messages were produced.

Snapshot Testing

At TigerBeetle, we use snapshot testing extensively as it helps both easily test assertions in our tests, but also give additional context to the API contract to the reader.

For example, it's much easier and more meaningful to the reader to write a snapshot assertion like so:

const messages = harness.spoolLastPayload(1)
Snapshot.from(`[
  {
    "errorInformation": {
      "errorCode": "3106",
      "errorDescription": "Modified request",
      "extensionList": {
        "extension": [
          {
            "key": "cause",
            "value": :ignore
          }
        ]
      }
    }
  }
]`).checkUnwrap(messages)

Than the equivalent assertions:

assert.equal(messages.length, 1)
assert(messages[0].errorInformation)
assert.equal(messages[0].errorInformation.errorCode, "3106")
assert.equal(messages[0].errorInformation.errorDescription, "Modified request")
assert(messages[0].errorInformation.extensionList)
assert(messages[0].errorInformation.extensionList.extension)
assert.equal(messages[0].errorInformation.extensionList.extension.length, 1)
assert.equal(messages[0].errorInformation.extensionList.extension[0].key, "cause")
assert(messages[0].errorInformation.extensionList.extension[0].key.value)

When we make changes to the API, we can then rerun the tests with SNAP_UPDATE=1 node --test path/to/file, and the snapshot testing library will update the snapshot for us.

AI Assistance Disclosure
AI tools were used in producing part of this contribution.
Tools used: Claude Code
Scope: Claude was used to help analyse existing integration tests, and trace the expected test data passed to different components. All code was handwritten by @lewisdaly.
All AI-generated content has been reviewed, understood, and validated by the author.

[sonarqubecloud]

Quality Gate failed

Failed conditions
8.3% Duplication on New Code (required ≤ 3%)
C Reliability Rating on New Code (required ≥ A)
D Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

💡 Need a hand with PR review? Try Gitar by Sonar!