Skip to content

An index rebuilt from its own definition still loses its full-text and geospatial configuration at four sites #5742

Description

@lvca

Split out of #5723 (PR #5734), which fixed one of five sites and added the accessor the other four need.

What happens

Any site that rebuilds an index from its own configuration has to read that configuration back and feed it through the creation path. Four of them read it with IndexInternal.getMetadata(), which for the WRAPPER index types answers the underlying LSM-Tree's plain IndexMetadata:

  • LSMTreeFullTextIndex keeps its analyzers, query-parser options and BM25 parameters in its own FullTextIndexMetadata
  • LSMTreeGeoIndex keeps its geohash resolution and storage layout as plain fields
  • LSMSparseVectorIndex keeps its dimensionality, scoring modifier and weight quantization in its own LSMSparseVectorIndexMetadata

None of that is reachable through the underlying index, so each of these sites silently recreates the index with the DEFAULT configuration. For a full-text index that means a different analyzer and different ranking; for a geospatial one, a different cell size. Nothing logs, and the index keeps working.

The four sites

Site Reads Loses
TruncateTypeStatement.IndexDefinition.from index.getMetadata() full-text, geospatial config
DatabaseChecker (CHECK DATABASE FIX) nothing at all full-text, geospatial, and vector config
LocalDocumentType.addBucketInternal idx.getMetadata() full-text, geospatial config on the new bucket's sub-index
LocalDocumentType.addSuperType index.getMetadata() same, on the propagated sub-index

DatabaseChecker is the worst of the four: it passes no metadata whatsoever, so CHECK DATABASE FIX on a vector index would recreate it with dimensions 0. It is also the site whose whole purpose is repair.

RebuildIndexStatement is already correct - it reconstructs the typed metadata from the index's persisted JSON - and is only listed here because it could now drop that round-trip in favour of the accessor below.

Fix

PR #5734 added IndexInternal.getMetadataForNewFile(), the companion of the getPageSizeForNewFile() from #5713, plus a polymorphic IndexMetadata.copy(typeName, propertyNames, bucketId) across all five metadata classes. Each site becomes:

final IndexMetadata metadata = index.getMetadataForNewFile().copy(typeName, properties, bucketId);

Each is a small change, but each is a distinct user-visible operation (TRUNCATE TYPE, CHECK DATABASE FIX, ALTER TYPE ... BUCKET, ALTER TYPE ... SUPERTYPE) and deserves its own regression test asserting the configuration survives - a behavioural one, the way #5734 asserts an English stem still matches after the carry-over rather than comparing an attribute.

Also worth covering here

copyType() is deliberately non-atomic: the records commit, then each index builds in its own transaction, because the build cannot see the records otherwise. A failure on either side is caught and drops the copy, but a hard crash between the record commit and the index build leaves a populated-but-unindexed copy behind. That is consistent with the documented non-atomicity and there is no cheap way to close it, but it is worth a CHECK DATABASE story or at least an explicit note in the user documentation.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions