-
Notifications
You must be signed in to change notification settings - Fork 7
Indexer TCP protocol
All values are in little endian exception for network addresses and version field that are encoded in big endian.
Length fields usually encoded as variable length interger and below is marked by VarInt.
Every message has the following format:
| Field | Type | Size | Description |
|---|---|---|---|
| Type | uint32 | VarInt | Message ID that defines format of payload |
| Length | uint32 | VarInt | Size of payload in bytes |
| Payload | ubyte[] | Length | Rest part of the message |
Maximum payload size is 10 megabytes. If message doesn't have payload, length field is missing.
Message types:
| Message ID | Name |
|---|---|
| 0 | Version |
| 1 | Version ACK |
| 2 | Filters Request |
| 3 | Filters Response |
| 4 | Filter Event |
| 5 | Peer Request |
| 6 | Peer Response |
| 7 | Fee Request |
| 8 | Fee Response |
| 9 | Introduce Peer |
| 10 | Reject |
| 11 | Ping |
| 12 | Pong |
| 13 | Rates Request |
| 14 | Rates Response |
Current version is 2.0.0.
When client connects to indexer it has to immediately send version message. The same does indexer.
Version message:
| Field | Type | Size | Description |
|---|---|---|---|
| Version | uint32 | 4 | Version number (encoding see below) |
| Time | uint64 | 8 | UNIX timestamp in seconds |
| Nonce | uint64 | 8 | Node random nonce, randomly generated every time a version message is sent. This nonce is used to detect connections to self. |
| Currencies | uint32 | VarInt | Amount of currencies blocks following the field. For clients it is 0. |
| ScanBlocks | ScanBlock | Currencies*sizeof(ScanBlock) | Information about each supported currency |
Version is encoded as big endian 32 bit word as 10 bits major version, 10 bits minor version and 10 bits patch version, last 2 are reserved. Example: 1.2.4 is encoded as 0b0000000100_0000000010_0000000001_00 word and then encoded in big endian order. Version format is same as for ScanBlock and for Version message.
ScanBlock:
| Field | Type | Size | Description |
|---|---|---|---|
| Currency | uint32 | VarInt | Currency tag |
| Version | uint32 | 4 | Version of filters format |
| ScanHeight | uint64 | VarInt | Last height scanned by the server |
| Height | uint64 | VarInt | Actual height known by the server |
Currencies:
| Tag | Code | Currency |
|---|---|---|
| 0 | BTC | Bitcoin |
| 1 | TBTC | Testnet Bitcoin |
| 2 | ERGO | Ergo |
| 3 | TERGO | Testnet Ergo |
| 4 | USDTO | Omni USDT |
| 5 | TUSDTO | Testnet Omni USDT |
| 6 | LTC | Litecoin |
| 7 | TLTC | Testnet Litecoin |
| 8 | ZEC | ZCash |
| 9 | TZEC | Testnet ZCash |
| 10 | CPR | Cypra token |
| 11 | TCPR | Testnet Cypra |
| 12 | DASH | Dash |
| 13 | TDASH | Testnet Dash |
No payload. Sent in response for version message if the version is accepted by the peer.
When peer need a range of filters it has to send the message. Server should return from 0 to min (server_max_amount, amount) of filters in filters response message.
| Field | Type | Size | Description |
|---|---|---|---|
| Currency | uint32 | VarInt | Currency tag |
| Start | uint64 | VarInt | Start height of filters |
| Amount | uint32 | VarInt | Amount of filters to expect at most |
Maximum amount of filters is 2000.
Indexer can truncate response to the maximum amount of filters. Minimum maximum is 100 and recommended is 1000 for BTC.
| Field | Type | Size | Description |
|---|---|---|---|
| Currency | uint32 | VarInt | Currency tag |
| Amount | uint32 | VarInt | Amount of filters in response |
| Filters | Filters | ? | GZIP encoded bytes of Filters |
Length of filters block is deducted from total length of message.
Filters block format after decompression consists of many Filter records:
| Field | Type | Size | Description |
|---|---|---|---|
| BlockId | ubyte[] | Currency specific | Block hash of the filter |
| Length | uint32 | VarInt | Size in bytes of filter |
| Filter | ubyte[] | Length | Raw bytes of filter |
Length of block id (block hash) is currency specific. For BTC it is 32 bytes.
When indexer scan new block it announce it to all connected peers with the following message:
| Field | Type | Size | Description |
|---|---|---|---|
| Currency | uint32 | VarInt | Currency tag |
| Height | uint64 | VarInt | Height of the filter |
| BlockId | ubyte[] | Currency specific | Block hash of the filter |
| Length | uint32 | VarInt | Size in bytes of filter |
| Filter | ubyte[] | Length | Raw bytes of filter |
Length of block id (block hash) is currency specific. For BTC it is 32 bytes.
| Field | Type | Size | Description |
|---|---|---|---|
| Type | ubyte | 1 | Tag of address type (IPV6, IPV4, Onion) |
| Address | ubyte[] | AddressSize(Type) | Address bytes which format depends on the Type tag |
| Port | uint16 | 2 | Port number of the address. Big endian. |
Note that IPV4 and IPV6 address is encoded in big endian.
Type tag:
| Tag | Name | AddressSize |
|---|---|---|
| 0 | IPV4 | 4 |
| 1 | IPV6 | 16 |
| 2 | Onion V3 | 56 |
Port number is missing in onion addresses.
When peer want to know a list of other known peers it should send the message. It has no payload.
Indexer responds to the peer request with the following message:
| Field | Type | Size | Description |
|---|---|---|---|
| Amount | uint32 | VarInt | Amount of addresses |
| Addresses | Address[] | Amount*sizeof(Address) | Peer addresses |
Note that sizeof(Address) is runtime dependent. You can deduct length of whole addresses array by length of the message in message header.
The message is sent by client when client desires to know which fee predictions are for specific currencies.
| Field | Type | Size | Description |
|---|---|---|---|
| CurrencyLen | uint32 | VarInt | Amount of currencies |
| Currencies | Currency[] | CurrencyLen * sizeof(Currency) | Fee level code |
The message is sent as response to fee request.
| Field | Type | Size | Description |
|---|---|---|---|
| Amount | uint32 | VarInt | Amount of FeeResps |
| Fees | FeeResp[] | variable | Fee responses |
FeeResp. Exact message layout depends on the currency tag. First 4 bytes is always the currency tag
Message for BTC/TBCT
| Field | Type | Size | Description |
|---|---|---|---|
| Currency | uint32 | VarInt | Currency tag |
| FeeFastCons | uint64 | VarInt | satoshi/vbyte for Fast level, conservative estimation |
| FeeFastEcon | uint64 | VarInt | satoshi/vbyte for Fast level, economical estimation |
| FeeModerateCons | uint64 | VarInt | satoshi/vbyte for Moderate level, conservative estim. |
| FeeModerateEcon | uint64 | VarInt | satoshi/vbyte for Moderate level, economical estimation |
| FeeCheapCons | uint64 | VarInt | satoshi/vbyte for Cheap level, conservative estimation |
| FeeCheapEcon | uint64 | VarInt | satoshi/vbyte for Cheap level, economical estimation |
At the moment we have specific format only for BTC/TBTC. Any other currency uses generic message
| Field | Type | Size | Description |
|---|---|---|---|
| Currency | uint32 | VarInt | Currency tag |
| FeeFast | uint64 | VarInt | Amount of currency for Fast level |
| FeeModerate | uint64 | VarInt | Amount of currency for Moderate level |
| FeeCheap | uint64 | VarInt | Amount of currency for Cheap level |
Levels description:
| Name | Description |
|---|---|
| Fast | The fastest confirmation times. Higher fees than others. |
| Moderate | Target confirmation blocks (10 for BTC). Medium speed. |
| Cheap | The slowest confirmation times but cheapest fees. |
The way to advertise indexer address across other indexers.
| Field | Type | Size | Description |
|---|---|---|---|
| Amount | uint32 | VarInt | Amount of addresses |
| Addresses | Address[] | Amount*sizeof(Address) | Peer addresses |
Note that sizeof(Address) is runtime dependent. You can deduct length of whole addresses array by length of the message in message header.
If some error occurs the message is sent by other side to inform peer about error:
| Field | Type | Size | Description |
|---|---|---|---|
| Type | uint32 | VarInt | Message ID that triggered the error |
| Code | uint32 | VarInt | Error code unique for each type of errors |
| Length | uint32 | VarInt | Size of error message |
| Message | ubyte[] | Length | Message about the error. UTF8 |
| Data | ubyte[] | ? | Additional fields that are specific to error codes. |
Length of the rest data that is specific to error is deducted from length of whole message in the message header.
Error codes:
| Code | Name | Messages | Description |
|---|---|---|---|
| 0 | MessageHeaderParsing | Any | Error when parsing header of message |
| 1 | MessageParsing | Any | Error when parsing body of message |
| 2 | InternalServerError | Any | Error that is on server side |
| 3 | ZeroBytesReceived | Any | Got zero bytes when expected a header or payload |
| 4 | VersionNotSupported | Version | Server doesn't support given version of protocol |
Ping message to test aliveness of the connection. Server should respond with corresponding pong message.
| Field | Type | Size | Description |
|---|---|---|---|
| Nonce | ubyte[] | 8 | Random bytes for nonce |
Pong message to test aliveness of the connection should contain the same nonce from corresponding ping message.
| Field | Type | Size | Description |
|---|---|---|---|
| Nonce | ubyte[] | 8 | Bytes from ping message |
Rates Request
| Field | Type | Size | Description |
|---|---|---|---|
| Amount | uint32 | VarInt | Amount of elements |
| RateReqEls | RateReqEl[] | Variable | Request currencies |
Field types:
RateReqEl
| Field | Type | Size | Description |
|---|---|---|---|
| Currency | uint32 | VarInt | Currency tag |
| Amount | uint32 | VarInt | Amount of fiats |
| Fiats | Fiat[] | Amount*sizeof(Fiat) | Peer addresses |
Fiat
| Field | Type | Size | Description |
|---|---|---|---|
| Fiat | uint32 | VarInt | Fiat tag |
Fiat tags:
| Value | Name |
|---|---|
| 0 | USD |
| 1 | EUR |
| 2 | RUB |
Rates Response
| Field | Type | Size | Description |
|---|---|---|---|
| Amount | uint32 | VarInt | Amount of elements |
| RatesRespEls | RatesRespEl[] | Variable | Rates |
RatesRespEl
| Field | Type | Size | Description |
|---|---|---|---|
| Currency | uint32 | VarInt | Currency tag |
| Amount | uint32 | VarInt | Amount of pairs |
| Rates | FiatRate[] | Amount * 12 | Rates for the Currency |
FiatRate
| Field | Type | Size | Description |
|---|---|---|---|
| Fiat | uint32 | VarInt | Fiat tag |
| Rate | fixed64 | 8 | The rate as fixed point |
For all currently supported fiats we use fixed point with 2 decimals after point. Exchange rate 65000.24 is presented as 6500024 decimal.