VSI: surface CALIBRATION lookup table as named slope/origin/final keys#4427
Conversation
Tag 20051 (CALIBRATION) sub-volumes carry a piecewise-linear lookup table mapping a raw axis (e.g. uint16 pixel value) to a calibrated axis (e.g. Z position in micrometres for DSX EFI height maps, or intensity in counts for deconvolved fluorescence). PR ome#4417 named the tag but the underlying value is currently surfaced (when it makes it through the metadataIndex/VALUE filter at all) as a single comma-separated string of every (raw, calibrated) pair, which is unwieldy and silently dropped for some file types (e.g. Olympus DSX-2000 collapsed-EFI height maps). Since the LUT is linear by construction, expose the same information as four scalar named keys instead: Calibration Function Slope : (cal_last - cal_first)/(raw_last - raw_first) Calibration Function Origin : cal value at raw_first Calibration Function Final : cal value at raw_last Calibration Function NPoints : number of (raw, cal) pairs Origin and Final together work for both monotonic directions (DSX EFI height maps decrease, deconvolved fluorescence increases). Validated against ten DSX-2000 EFI height-map .vsi acquisitions and the public CellSens samples on downloads.openmicroscopy.org/images/ CellSens/. Each file with a properly-formed CALIBRATION sub-volume now produces the four-scalar summary; files without the tag remain silent. Refs: ome#4398, ome#4417 Signed-off-by: Brenden Ferland <brendenferland@gmail.com>
|
Spotted in #4417's diff: the existing Want me to switch to |
|
Thanks, @PEEKPerformer. Before we proceed with review, please do submit the relevant dataset to Zenodo. Per OME's contributing policy, we do also need a signed Contributor License Agreement: |
|
Sounds good, I submitted both yesterday. Thanks! |
|
Thanks, @PEEKPerformer. Submitted dataset is now here: https://zenodo.org/records/19893921. |
melissalinkert
left a comment
melissalinkert
left a comment
There was a problem hiding this comment.
Apologies for the delay, @PEEKPerformer. This approach is generally fine, and has been included in tests without issue for some time. I only have one minor requested change based on code review; once that is addressed and we have one more passing build, I am happy to merge.
| double rawLast = doubleValues[2 * (nPairs - 1)]; | ||
| double calLast = doubleValues[2 * (nPairs - 1) + 1]; | ||
| double dRaw = rawLast - rawFirst; | ||
| if (dRaw != 0) { |
There was a problem hiding this comment.
This check may yield unexpected results; best practice when comparing double values is not to use == or !=. https://cclc.mtu.edu/java-doubles-and-floating-point-numbers/ has a concise explanation with additional references.
In other readers, we would use something like Math.abs(dRaw - 0) > Constants.EPSILON to check that dRaw is non-zero.
Comparing the raw-axis span dRaw against zero with != is unreliable for doubles. Use Math.abs(dRaw) > Constants.EPSILON, matching the float comparison idiom used elsewhere in the readers (per review feedback).
melissalinkert
left a comment
melissalinkert
left a comment
There was a problem hiding this comment.
Thanks, @PEEKPerformer. Just waiting on a passing nightly build with this latest commit included and then will merge.
2 participants
Refs: #4398, #4417.
Tag 20051 contains a linear
(raw, calibrated)lookup table. PR #4417 named the tag but didn't extract anything from it. A linear LUT is a slope and an offset. Two numbers. This patch writes them out as scalar keys.Four new keys:
Calibration Function Slope:(cal_last - cal_first) / (raw_last - raw_first)Calibration Function Origin: cal at the first raw pointCalibration Function Final: cal at the last raw pointCalibration Function NPointsFor DSX-2000 EFI height maps, Slope is the µm-per-uint16-count factor that converts a height-map pixel value to absolute Z. Without it, height-map TIFFs out of
bfconvertare uncalibrated.Code
+31 lines, 0 deletions. Single case in
CellSensReader.java::readTags, in the existingDOUBLE_2branch. Gated on:Doesn't run on files without the tag. Doesn't modify the existing string-form output.
Validation
README pull-request testing checklist:
develop(verified viagit merge-tree).ant clean jars tools: BUILD SUCCESSFUL, 31s.ant test: 1833 tests across all phases (unit, long-running, no-ome-xml, no-exif, no-jhdf, spec). 0 failures, 0 skips. 3m35s.if/elsewith primitives andString.equals).showinfend-to-end on representative.vsifiles (see below).showinfon the public CellSens samples atdownloads.openmicroscopy.org/images/CellSens/:Slope=1.290, Origin=47.23, Final=84606, NPoints=2.Slope=0.1, Origin=-3276.8(16-bit display LUT).showinfon 10 DSX-2000 EFI height-map.vsifiles from a polymer-composite study I'm running. Each produces the four-scalar quartet. For the 40× porestack:EFI scan range = 192 µm. Cross-checked against a from-scratch binary parser of the same file. Agrees to 6 decimals.
Notes
I don't have access to the curated data repo, so
ant test-automatedagainst the full corpus is what I need from you.Happy to upload a representative DSX file to the Bio-Formats Zenodo community. Polymer composite, 314 KB
.vsiplus 6 MB.ets, no biological content.The existing string-form
Calibration Function Valuekey is still silently dropped on some files by the filter around line 1997. The summary keys here come through unconditionally because they calladdGlobalMetadirectly. Fixing the underlying filter touches a hotter code path. Out of scope.cc @melissalinkert (touched the same code in #4417)