Storage Id prefix structure and resolving

[tshemsedinov]

Id structure

• Low 24 bit - system Id
• Root system Id 0
• Systems from 1 to 8388607 are reserved
• Systems from 8388608 to 16777215 will be assigned to registered systems
• High 40 bit - entities inside system

High 40 bit also contains:

• Server suffix (for horizontal scaling)
• NodeId can be calculated (Id - server suffix) % nodes and NideId can change each restart, based on node count

Terms and Definitions

System - is a separate data storage having its own user accounts and granted permissions, own horizontal sharding with suffix tree, own routing map with registered servers. System can have one way replication with other systems on selected categories. Replicated objects will not change Ids. System contains 1 or more servers.

Server - is a single device running multiple nodes (2 or more, one for server monitoring and orchestration, second for business-logic, storage and serving requests). Each server have its own storage accessible for all nodes, but nodes will read/write just entities bound to certain nodeId.

Node - is a process holding in memory multiple entities. Id % nodeCount = nodeId

Entity - storage unit (record of certain category with its multiple detail records of other categories, relations and constraints). Each entity (with details) should be stored and processed in one certain
Server and inside it in one certain node.

To be implemented

• Id generation service
• Id routing API
• Replication mechanism between systems
• Sharding inside system (including server cloning)
• Storage redundancy for persistence instead of backups
• Client-side cache
• Locking service
• Distributed transactions

Examples

System registry example:

{
  0: { host: '10.1.1.10', port: 4000 }, // root server
  8388607: { host: '10.18.8.20', port: 4000 } // system 8388607
}

System infrastructure example:

{
  // system 8388607 servers and ports
  0: { host: '10.18.8.20', ports: [4000, [4]] },
  2: { host: '10.18.8.21', ports: [2000, [20]] },
  1: { host: '10.18.8.22', ports: [2000, [20]] },
  5: { host: '10.18.8.23', ports: [2000, [40]] },
  3: { host: '10.18.8.24', ports: [2000, [30]] },
  7: { host: '10.18.8.25', ports: [2000, [30]] }
}

Globalstorage Id API

Ids will be generated and stored in special service. It will have following configurable parameters with default values:

{
  MINIMAL_RESERVED: 1000,
  GENERATE_BUNCH: 100,
}

• gs.parseId(Id: BigInt): { system: Number, server: Number, node: Number }
• gs.route(Id: BigInt): { system: { host, port }, server: { host, ports }, node: Number } - find target
• gs.generateId(callback(err, id))
• gs.generateBunch(count, callback(err, ids: Array of BigInt))
• gs.systems: Map // this systems collection keyed by system Id: Number
• gs.servers: Map // this system servers collection keyed by server Id: Number
• gs.system: Number // this system Id
• gs.server: Number // this server suffix (not including system Id)
• gs.serverBits: Number // number of bits for this server
• gs.serverMask: Number // bit mask (2 ** serverBits) - 1 to extract server suffix
• gs.node: Number // this node Id
• gs.nodeCount: Number // Node count on this server