Summary
As a node operator, I want harvest to fail loudly when a file_ref path is not converted to a web URL by replacePrefix, so that local filesystem paths are never silently stored in the registry as product references.
Motivation
When fileInfo/fileRef is configured with a replacePrefix but a product's file path does not match any configured prefix, harvest currently stores a raw local file path (e.g., /path/to/archive/data.xml or a file:// URI) as the file_ref value in the registry. File references in the registry must be publicly accessible HTTP/HTTPS URLs — local paths are meaningless to end users and downstream services that resolve those references.
This silent failure can go undetected, leading to registry data that cannot be dereferenced. Harvest should detect this condition and fail for affected products so operators are alerted immediately.
Acceptance Criteria
Additional Context
Relevant code:
- Replacement logic:
FileMetadataExtractor.getFileRef() in registry-common — if no rule prefix matches, the raw file URI path is returned unchanged.
file_ref is set for both Label_File_Info/file_ref and Data_File_Info/file_ref.
- The
-f/--force flag is already defined in HarvestCli and propagated to RegistryManager — it should also suppress this new validation.
Example config (expected usage):
<fileRef replacePrefix="/path/to/archive" with="https://url/to/archive/" />
With this config, any product whose label path does not begin with /path/to/archive would fail validation unless --force is supplied.
Summary
As a node operator, I want harvest to fail loudly when a
file_refpath is not converted to a web URL byreplacePrefix, so that local filesystem paths are never silently stored in the registry as product references.Motivation
When
fileInfo/fileRefis configured with areplacePrefixbut a product's file path does not match any configured prefix, harvest currently stores a raw local file path (e.g.,/path/to/archive/data.xmlor afile://URI) as thefile_refvalue in the registry. File references in the registry must be publicly accessible HTTP/HTTPS URLs — local paths are meaningless to end users and downstream services that resolve those references.This silent failure can go undetected, leading to registry data that cannot be dereferenced. Harvest should detect this condition and fail for affected products so operators are alerted immediately.
Acceptance Criteria
fileRefrules are evaluated, if the resultingfile_refvalue does not start withhttp://orhttps://, harvest must log a clear error message identifying the affected product (LID/VID) and file path, and the product must not be loaded into the registry.-f/--forceflag is provided, harvest logs a warning instead of an error and continues loading the product (preserving existing behavior for operators who explicitly opt in).replacePrefixis correctly converted to an HTTP URL → product loads successfully.replacePrefix→ validation error, product is not loaded (no--force).replacePrefixwith--force→ warning logged, product is loaded.Additional Context
Relevant code:
FileMetadataExtractor.getFileRef()inregistry-common— if no rule prefix matches, the raw file URI path is returned unchanged.file_refis set for bothLabel_File_Info/file_refandData_File_Info/file_ref.-f/--forceflag is already defined inHarvestCliand propagated toRegistryManager— it should also suppress this new validation.Example config (expected usage):
With this config, any product whose label path does not begin with
/path/to/archivewould fail validation unless--forceis supplied.