Merge pull request #130 from ocean-uhh/126-fix-nortek-raw-reader-works-for-aqd-files-from-gen1-instruments-but-not-for-files-from-gen2

fix: Nortek raw reader works for .aqd files from Gen1 instruments but…

Author: ysorge

docs/source/developers_guide.rst

@@ -174,13 +174,17 @@ The ``SbeCnvReader`` handles SeaBird CNV files, commonly used for CTD profile da
    dataset = ssl.read("rbr_export.mat", file_format='rbr-matlab')
    dataset = ssl.read("rsktools_export.mat", file_format='rbr-matlab-rsktools')
 
+   # RBR HEX exports require an explicit format key because .hex is also used
+   # by SeaBird HEX files
+   dataset = ssl.read("rbr_logger.hex", file_format='rbr-hex')
+
 *Legacy API:*
 
 The ``RbrRskReader`` family handles RBR RSK files from moored instruments:
 
 .. code-block:: python
 
-   from seasenselib.readers import RbrRskAutoReader, RbrMatlabReader
+   from seasenselib.readers import RbrRskAutoReader, RbrMatlabReader, RbrHexReader
    
    # Auto-detect RSK format version
    reader = RbrRskAutoReader("solo_temp.rsk")
@@ -190,6 +194,10 @@ The ``RbrRskReader`` family handles RBR RSK files from moored instruments:
    reader = RbrMatlabReader("rbr_export.mat")
    dataset = reader.data
 
+   # HEX format reader
+   reader = RbrHexReader("rbr_logger.hex")
+   dataset = reader.data
+
 **Nortek Aquadopp Instruments**
 
 *Modern API:*
@@ -685,4 +693,4 @@ The SeaSenseLib repository includes example data files in the ``examples/`` dire
 * ``DSE18_013889_20180827_1349.mat``: RBR TR1050 MATLAB export
 * ``DSE18_SBE05608482_2018_08_27.cnv``: SeaBird SBE56 thermistor
 
-These files let you test the data processing with different file formats.
+These files let you test the data processing with different file formats.

docs/source/reader_notes.rst

@@ -33,11 +33,17 @@ anchors and channel metadata are preserved in ``raw_metadata``.
 Nortek raw
 ----------
 
-The ``nortek-raw`` reader decodes classic Nortek raw binary files, currently
-advertised for Aquadopp ``.aqd``, Vector ``.VEC`` and AWAC ``.wpr`` files,
-with MHKiT DOLfYN. Nortek Signature ``.ad2cp`` files use a separate DOLfYN
-decoder and are not part of this reader. This reader is marked experimental
-because support is still being validated across Nortek raw variants.
+The ``nortek-raw`` reader decodes Nortek raw binary files with MHKiT DOLfYN.
+It dispatches classic Aquadopp ``.aqd``, Vector ``.VEC`` and AWAC ``.wpr``
+files to DOLfYN's classic Nortek decoder, and dispatches AD2CP/Gen2-style
+``.aqd`` files to DOLfYN's Nortek2 decoder based on the binary file header.
+Nortek2 averaged ``*_avgd.aqd`` products with ID 38 packets are decoded by a
+small SeaSenseLib fallback because the DOLfYN Nortek2 indexer does not include
+that packet family.
+
+Nortek Signature ``.ad2cp`` files use the same DOLfYN decoder family but are
+not advertised by this reader yet. This reader is marked experimental because
+support is still being validated across Nortek raw variants.
 
 Velocity is preserved as vector variable ``vel``. If ``coord_sys`` is
 ``earth``, the decoded ``dir`` coordinate usually identifies east, north and

docs/source/supported_formats.rst

@@ -37,16 +37,16 @@ Format keys can be used with ``ssl.read(filename, file_format='key')`` when auto
      - ``.dat`` + ``.hdr``
      - ``nortek-ascii``
      - Nortek ASCII format (requires header file)
-   * - Nortek
-     - Aquadopp
-     - ``.csv``
-     - ``nortek-csv``
-     - Nortek AquaPro CSV export format
-   * - Nortek
-     - Aquadopp, Vector, AWAC
-     - ``.aqd``, ``.VEC``, ``.wpr``
-     - ``nortek-raw``
-     - Nortek raw binary files via MHKiT DOLfYN (experimental)
+   * - Nortek
+     - Aquadopp
+     - ``.csv``
+     - ``nortek-csv``
+     - Nortek AquaPro CSV export format
+   * - Nortek
+     - Aquadopp, Aquadopp Gen2, Vector, AWAC
+     - ``.aqd``, ``.VEC``, ``.wpr``
+     - ``nortek-raw``
+     - Nortek raw binary files; Gen2 ``.aqd`` and averaged ``*_avgd.aqd`` products are detected from the binary header (experimental)
    * - RBR
      - Solo T
      - ``.rsk``
@@ -77,19 +77,19 @@ Format keys can be used with ``ssl.read(filename, file_format='key')`` when auto
      - ``.mat``
      - ``rbr-matlab-rsktools``
      - RBR MATLAB RSKtools export
-   * - RBR
-     - Various
-     - ``.txt``, ``.dat``
-     - ``rbr-ascii``
-     - RBR ASCII format
-   * - RBR
-     - TR1050
-     - ``.hex``
-     - ``rbr-hex``
-     - RBR binary HEX format (explicit format key required)
-   * - SeaBird
-     - SBE37 MicroCAT
-     - ``.cnv``
+   * - RBR
+     - Various
+     - ``.txt``, ``.dat``
+     - ``rbr-ascii``
+     - RBR ASCII format
+   * - RBR
+     - TR1050
+     - ``.hex``
+     - ``rbr-hex``
+     - RBR binary HEX format (explicit format key required)
+   * - SeaBird
+     - SBE37 MicroCAT
+     - ``.cnv``
      - ``sbe-cnv``
      - SeaBird CNV format (time series)
    * - SeaBird
@@ -153,10 +153,10 @@ SeaSenseLib can automatically detect format for files with unique extensions:
    dataset = ssl.read('your_file.cnv')     # SeaBird CNV files
    dataset = ssl.read('your_file.hex')     # SeaBird SBE37 HEX files
    dataset = ssl.read('logger_data.rsk')   # RBR RSK files (auto-selects modern/legacy)
-   dataset = ssl.read('grid_data.nc')      # NetCDF files
-   dataset = ssl.read('sensor_data.csv')   # CSV files
-   dataset = ssl.read('tob_data.tob')      # Seasun TOB files
-   dataset = ssl.read('aquadopp.aqd')      # Nortek raw files
+   dataset = ssl.read('grid_data.nc')      # NetCDF files
+   dataset = ssl.read('sensor_data.csv')   # CSV files
+   dataset = ssl.read('tob_data.tob')      # Seasun TOB files
+   dataset = ssl.read('aquadopp.aqd')      # Nortek raw files
 
 **Explicit Format Specification:**
 
@@ -165,13 +165,13 @@ SeaSenseLib can automatically detect format for files with unique extensions:
    # When automatic detection fails
    dataset = ssl.read('data.txt', file_format='sbe-ascii')
    
-   # For ambiguous extensions like .mat
-   rbr_data = ssl.read('logger_export.mat', file_format='rbr-matlab')
-   adcp_data = ssl.read('current_data.mat', file_format='adcp-matlab-uhhds')
-
-   # RBR HEX files also require an explicit key because .hex auto-detects
-   # as SeaBird SBE37 HEX
-   rbr_hex_data = ssl.read('logger.hex', file_format='rbr-hex')
+   # For ambiguous extensions like .mat
+   rbr_data = ssl.read('logger_export.mat', file_format='rbr-matlab')
+   adcp_data = ssl.read('current_data.mat', file_format='adcp-matlab-uhhds')
+
+   # RBR HEX files also require an explicit key because .hex auto-detects
+   # as SeaBird SBE37 HEX
+   rbr_hex_data = ssl.read('logger.hex', file_format='rbr-hex')
 
 **Multi-file Formats:**
 
@@ -182,20 +182,20 @@ SeaSenseLib can automatically detect format for files with unique extensions:
                           file_format='nortek-ascii',
                           header_file='current_meter.hdr')
 
-   # Nortek AquaPro CSV exports use an explicit format key because .csv is
-   # also used by the generic CSV reader
-   nortek_csv_data = ssl.read('Average Velocity DF3.csv',
-                              file_format='nortek-csv')
-
-   # Nortek raw Aquadopp, Vector and AWAC data is decoded by MHKiT DOLfYN
-   # (experimental)
-   nortek_raw_data = ssl.read('DS-FDA01.aqd',
-                              file_format='nortek-raw')
-
-   # RDI raw ADCP data is decoded by MHKiT DOLfYN
-   rdi_data = ssl.read('DS2_2025_recovery.000',
-                       file_format='rdi-raw',
-                       nens=100)
+   # Nortek AquaPro CSV exports use an explicit format key because .csv is
+   # also used by the generic CSV reader
+   nortek_csv_data = ssl.read('Average Velocity DF3.csv',
+                              file_format='nortek-csv')
+
+   # Nortek raw Aquadopp, Vector and AWAC data is decoded by MHKiT DOLfYN
+   # (experimental)
+   nortek_raw_data = ssl.read('DS-FDA01.aqd',
+                              file_format='nortek-raw')
+
+   # RDI raw ADCP data is decoded by MHKiT DOLfYN
+   rdi_data = ssl.read('DS2_2025_recovery.000',
+                       file_format='rdi-raw',
+                       nens=100)
 
    # Automatic detection also works for supported RDI raw suffixes:
    rdi_data = ssl.read('DS2_2025_recovery.000', nens=100)
@@ -212,17 +212,17 @@ Format Detection Summary
 - ``.hex`` → ``sbe-hex`` (SeaBird SBE37 HEX files)
 - ``.rsk`` → ``rbr-rsk`` (RBR RSK files - automatically selects modern/legacy reader)
 - ``.nc`` → ``netcdf`` (NetCDF files)
-- ``.csv`` → ``csv`` (CSV files)
-- ``.tob`` → ``seasun-tob`` (Sea & Sun TOB files)
-- ``.aqd/.VEC/.wpr`` → ``nortek-raw`` (Nortek raw binary files via MHKiT DOLfYN, experimental)
-- ``.000/.PD0/.ENR/.ENS/.ENX`` → ``rdi-raw`` (RDI raw ADCP files via MHKiT DOLfYN)
-
-**Requires explicit format keys** (ambiguous extensions):
-
-- RBR ``.hex`` files: ``rbr-hex`` (``.hex`` otherwise auto-detects as ``sbe-hex``)
-- ``.mat`` files: ``rbr-matlab``, ``rcm-matlab``, ``adcp-matlab-uhhds``, ``adcp-matlab-rdadcp``
-- ``.txt/.dat`` files: ``rbr-ascii``, ``sbe-ascii``, ``nortek-ascii``
-- Nortek AquaPro ``.csv`` exports: ``nortek-csv`` (``.csv`` otherwise auto-detects as generic ``csv``)
+- ``.csv`` → ``csv`` (CSV files)
+- ``.tob`` → ``seasun-tob`` (Sea & Sun TOB files)
+- ``.aqd/.VEC/.wpr`` → ``nortek-raw`` (Nortek raw binary files via MHKiT DOLfYN, experimental)
+- ``.000/.PD0/.ENR/.ENS/.ENX`` → ``rdi-raw`` (RDI raw ADCP files via MHKiT DOLfYN)
+
+**Requires explicit format keys** (ambiguous extensions):
+
+- RBR ``.hex`` files: ``rbr-hex`` (``.hex`` otherwise auto-detects as ``sbe-hex``)
+- ``.mat`` files: ``rbr-matlab``, ``rcm-matlab``, ``adcp-matlab-uhhds``, ``adcp-matlab-rdadcp``
+- ``.txt/.dat`` files: ``rbr-ascii``, ``sbe-ascii``, ``nortek-ascii``
+- Nortek AquaPro ``.csv`` exports: ``nortek-csv`` (``.csv`` otherwise auto-detects as generic ``csv``)
 - Multi-file formats: ``nortek-ascii`` (requires both ``.dat`` and ``.hdr`` files)
 
 When to Use Format Keys