SWOPP 3

.github/copilot-instructions.md

@@ -0,0 +1,45 @@
+# Copilot Instructions for routetools
+
+This repository implements weather-routing optimization with JAX and CMA-ES. Keep changes focused, reproducible, and validated.
+
+Write and reason in English.
+
+## Required Workflow
+
+1. Read relevant source files and matching tests before editing.
+2. Make the smallest change that satisfies the request.
+3. Add or update tests when behavior changes.
+4. After each update, run both commands:
+   - `make hooks`
+   - `make test`
+5. If either command fails, fix the issues and rerun both commands.
+6. Do not finalize work until both commands pass.
+
+## Code Conventions
+
+- Use the repository toolchain (`uv` via `make` targets).
+- Preserve existing public APIs unless the request explicitly requires a breaking change.
+- Follow `ruff` and `pytest` settings in `pyproject.toml`.
+- Keep docstrings in NumPy style for public functions.
+- Prefer vectorized/JAX-friendly implementations in performance-sensitive code paths.
+
+## Testing Conventions
+
+- Place tests in `tests/` near the relevant domain file.
+- For bug fixes, add a regression test first whenever practical.
+- Keep tests deterministic and lightweight unless a larger benchmark is explicitly requested.
+
+## Data and Artifacts
+
+- Do not commit large generated outputs.
+- Treat `data/` contents as potentially large and optional in local environments.
+- Fail with clear error messages when optional datasets are missing.
+
+## Permissions
+
+- Make sure you have the necessary permissions to push to the repository. If you do not have permissions, stop and ask for them, guiding the user to the appropriate process to gain access.
+- You can add, commit and push changes to this repository. Never commit to 'main' or 'swopp' branches directly.
+- If you are on 'main' or 'swopp', create a new branch for your changes and open a pull request for review.
+- Create tests before implementing new features or fixing bugs. Tests should be in the `tests/` directory and follow existing patterns.
+- Make sure to run all tests and hooks before pushing your changes. If you encounter any issues, please fix them before pushing.
+- Do small commits, preferably one per logical change. This makes it easier to review and understand the history of changes.

.github/workflows/pre-commit-pr.yml

@@ -1,14 +1,14 @@
-name: pre-commit-pr
+name: post-commit-push
 
 on:
-  pull_request:
+  push:
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
 
 jobs:
-  pre-commit:
+  post-commit:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
@@ -30,5 +30,5 @@ jobs:
           uv pip install pip
       - uses: pre-commit/action@v3.0.1
         with:
-          # Ejecutar solo en ficheros que hayan cambiado https://github.com/pre-commit/action/issues/7
-          extra_args: --color=always --from-ref ${{ github.event.pull_request.base.sha }} --to-ref ${{ github.event.pull_request.head.sha }}
+          # Run hooks after commits are pushed
+          extra_args: --color=always --all-files --hook-stage post-commit

.gitignore

@@ -167,3 +167,16 @@ output*
 # User files
 nohup.out
 *.zip
+output/
+
+# CodaBench competition — large data & build artifacts
+codabench/reference_data/*.nc
+codabench/reference_data/*.shp
+codabench/reference_data/*.shx
+codabench/reference_data/*.dbf
+codabench/reference_data/*.prj
+codabench/logo/
+codabench/test_submission/
+
+# Separate repo
+routingviz/

README.md

@@ -22,6 +22,10 @@ Tools:
 - [prettier](https://prettier.io/): format YAML and Markdown
 - [codespell](https://github.com/codespell-project/codespell): check spelling in source code
 
+## Documentation
+
+- [Parametric Performance Model](docs/parametric_model.md) — closed-form RISE model for ship power prediction (hull, wind, wave, and wingsail components).
+
 ## Installation
 
 ### Application
@@ -38,7 +42,18 @@ Install package and pinned dependencies with the [`uv`](https://docs.astral.sh/u
    uv sync
    ```
 
-4. Run any command or Python script with `uv run`, for instance:
+4. (Optional) Install the SWOPP3 performance model:
+
+   ```{bash}
+   uv sync --extra swopp3 --find-links release_package/wheels
+   ```
+
+   If the pre-built wheels are available locally in `release_package/wheels/`,
+   `uv` will resolve `swopp3-performance-model` from that directory. Wheels are
+   available for Python 3.10 – 3.13 on Linux (manylinux) and Windows
+   (win_amd64).
+
+5. Run any command or Python script with `uv run`, for instance:
 
    ```{bash}
    uv run routetools/cmaes.py
@@ -50,63 +65,6 @@ Install package and pinned dependencies with the [`uv`](https://docs.astral.sh/u
    source .venv/bin/activate
    ```
 
-### Git credentials for VCS dependencies
-
-When `uv` installs a package from a git repository (VCS dependency), Git may need credentials to fetch the remote. On non-interactive environments this commonly fails with:
-
-```bash
-fatal: could not read Username for 'https://github.com': terminal prompts disabled
-```
-
-Use one of the following approaches to make VCS fetches non-interactive.
-
-**Option A: SSH (preferred)**
-
-Generate an SSH key (WSL / Linux):
-
-```bash
-ssh-keygen -t ed25519 -C "your_email@example.com" -f ~/.ssh/id_ed25519 -N ""
-eval "$(ssh-agent -s)"
-ssh-add ~/.ssh/id_ed25519
-cat ~/.ssh/id_ed25519.pub
-```
-
-Add the printed public key to GitHub (Settings → SSH and GPG keys). Test access:
-
-```bash
-ssh -T git@github.com
-git ls-remote git@github.com:Weather-Routing-Research/weather-routing-benchmarks.git refs/heads/main
-```
-
-Then use an SSH pip URL when adding or declaring the dependency:
-
-```bash
-uv add 'git+ssh://git@github.com/Weather-Routing-Research/weather-routing-benchmarks.git@main#egg=wrr_bench'
-uv sync
-```
-
-If you run `uv` from PowerShell on Windows, ensure the Windows SSH agent is running and the key is loaded (Start-Service ssh-agent; ssh-add $env:USERPROFILE\\.ssh\\id_ed25519), or run `uv` from WSL where the key was created.
-
-**Option B: HTTPS with credentials (fallback)**
-
-Use Git Credential Manager or GitHub CLI to cache credentials so Git won't prompt:
-
-PowerShell (Windows):
-
-```powershell
-git config --global credential.helper manager-core
-gh auth login --hostname github.com --git-protocol https
-```
-
-WSL / Linux (use gh or configure a credential helper that works in your environment):
-
-```bash
-gh auth login --hostname github.com --git-protocol https
-# or configure `git config --global credential.helper cache` for short-term caching
-```
-
-After configuring credentials, retry the `uv add` / `uv sync` command.
-
 ### Library
 
 Install a specific version of the package with `pip` or `uv pip`:
@@ -157,6 +115,63 @@ If your computer does not have a GPU, you can force JAX to use the CPU with `JAX
 JAX_PLATFORMS=cpu uv run scripts/single_run.py
 ```
 
+### ERA5 weather data pipeline
+
+The `routetools.era5` module provides real-world ERA5 wind and wave fields
+for weather routing. Two download backends are available:
+
+- **GCS** (default) — Google Cloud archive, no API key required.
+- **CDS** — Copernicus Climate Data Store (requires `cdsapi` + API key).
+
+**1. Download ERA5 data** for the Atlantic corridor (USNYC ↔ DEHAM):
+
+```bash
+uv run scripts/download_era5.py --corridor atlantic --year 2023
+```
+
+This creates `data/era5/era5_wind_atlantic_2023.nc` and
+`data/era5/era5_waves_atlantic_2023.nc`.
+
+**2. Run the real-world benchmark** (New York → Hamburg, Jan 8 2023):
+
+```bash
+uv run scripts/era5_benchmark.py --departure 2023-01-08T00:00:00
+```
+
+See `scripts/download_era5.py --help` and `scripts/era5_benchmark.py --help`
+for all available options.
+
+### SWOPP3 ERA5 pipeline
+
+The default SWOPP3 competition pipeline is:
+
+```bash
+uv run scripts/download_era5.py
+uv run scripts/swopp3_run.py
+```
+
+These two commands line up without extra path flags. The downloader writes the
+four default 2024 files that `scripts/swopp3_run.py` expects:
+
+```text
+data/era5/era5_wind_atlantic_2024.nc
+data/era5/era5_waves_atlantic_2024.nc
+data/era5/era5_wind_pacific_2024.nc
+data/era5/era5_waves_pacific_2024.nc
+```
+
+`scripts/swopp3_run.py` validates these files before running any case. If one
+or more inputs are missing, it exits immediately with a precise error message
+instead of silently substituting a great-circle route or running without
+weather data. This is intentional:
+
+- GC cases still require wind and wave data for SWOPP3 energy evaluation.
+- Optimised cases require wind data for the CMA-ES vectorfield and wind/wave
+  data for the final SWOPP3 energy evaluation.
+
+If you download a different year or only one corridor, pass matching
+`--wind-path*` and `--wave-path*` options to `scripts/swopp3_run.py`.
+
 ## Reproduce the results (paper)
 
 To reproduce the results from the paper, run the following command:

codabench/README.md

@@ -0,0 +1,121 @@
+# SWOPP3 Weather Routing Benchmark — CodaBench Setup
+
+This directory contains everything needed to host the SWOPP3 Weather Routing
+Benchmark on [CodaBench](https://www.codabench.org/).
+
+## Directory Structure
+
+```
+codabench/
+├── README.md                 ← You are here
+├── competition.yaml          ← Competition configuration (title, phases, leaderboard)
+├── scoring_program/
+│   ├── scoring.py            ← Evaluates submissions (validation + scoring)
+│   └── metadata.yaml         ← CodaBench scoring program metadata
+├── pages/
+│   ├── overview.md           ← Competition description
+│   ├── data.md               ← Data download & API instructions
+│   ├── submission.md         ← Submission format specification
+│   ├── evaluation.md         ← How submissions are scored
+│   └── terms.md              ← Terms and conditions
+├── starting_kit/
+│   └── starting_kit.py       ← Great-circle baseline (example submission)
+└── logo.png                  ← Competition logo
+```
+
+## Step-by-Step Setup
+
+### 1. Create the Competition Bundle
+
+CodaBench requires a `.zip` bundle containing the competition definition.
+Build it from this directory:
+
+```bash
+cd codabench
+bash build_bundle.sh
+```
+
+This creates `scoring_program.zip`, `starting_kit.zip`, `reference_data.zip`,
+and a combined `competition_bundle.zip`. See `build_bundle.sh` for details.
+
+### 2. Create the Competition on CodaBench
+
+1. Go to **[codabench.org](https://www.codabench.org/)** and sign in.
+2. Click **Benchmarks/Competitions** → **Create**.
+3. Fill in the **Details** tab:
+   - **Title:** `SWOPP3 Weather Routing Benchmark`
+   - **Logo:** Upload a logo (PNG)
+   - **Description:** Copy from `pages/overview.md` or write a summary
+   - **Competition Docker Image:** Use `fjsuarez/swopp3-scorer:latest`
+   - **Competition Type:** Competition
+4. **Pages** tab:
+   - Add pages from the `pages/` directory (Overview, Data, Submission, Evaluation, Terms)
+5. **Phases** tab:
+   - Create one phase ("Main Phase")
+   - Upload `scoring_program.zip` as the Scoring Program
+   - Upload `reference_data.zip` as the Reference Data
+   - Set start/end dates, max submissions per day (3), total max (100)
+6. **Leaderboard** tab:
+   - Add columns matching the keys in `competition.yaml` → `leaderboard`
+   - Primary ranking column: `total_energy_mwh` (ascending)
+7. **Publish** the competition.
+
+### 3. Reference Data
+
+The `reference_data/` directory must contain the 6-hourly ERA5 NetCDF files
+(~3.1 GB total) and the Natural Earth land shapefile before building the
+bundle. See `build_bundle.sh` for the full list of required files.
+
+The scoring program is **self-contained** — it uses only `numpy`, `netCDF4`,
+`pyshp`, `shapely`, and `matplotlib` (listed in `scoring_program/requirements.txt`).
+No `routetools` or JAX installation is needed on the CodaBench worker.
+The Docker image `fjsuarez/swopp3-scorer:latest` has all dependencies pre-installed.
+
+### 4. Starting Kit
+
+Upload `starting_kit.zip` to CodaBench so participants can download a working
+baseline. The `starting_kit.py` script generates a valid submission using
+great-circle routes.
+
+## Scoring
+
+The scoring program validates submission format and **re-evaluates every
+route** using the RISE performance model with the official ERA5 data from
+`reference_data/`. This guarantees all energy values are computed with the
+same model and weather data.
+
+If the ERA5 files are not present in `reference_data/`, the scorer falls back
+to self-reported energy values from the participants' CSVs.
+
+## Testing Locally
+
+Test the scoring program locally before deploying:
+
+```bash
+# Create a mock submission using the starting kit
+cd starting_kit
+python starting_kit.py --output-dir /tmp/test_submission
+
+# Simulate CodaBench's invocation
+mkdir -p /tmp/codabench_input/res /tmp/codabench_input/ref /tmp/codabench_output
+cp -r /tmp/test_submission/* /tmp/codabench_input/res/
+cp reference_data/config.json /tmp/codabench_input/ref/
+
+cd ../scoring_program
+python scoring.py /tmp/codabench_input /tmp/codabench_output
+
+# Check results
+cat /tmp/codabench_output/scores.json
+cat /tmp/codabench_output/scoring_log.txt
+```
+
+## Key CodaBench Concepts
+
+| Concept             | Meaning in SWOPP3                                      |
+| ------------------- | ------------------------------------------------------ |
+| **Phase**           | Single evaluation phase (all 366 departures × 8 cases) |
+| **Scoring Program** | `scoring.py` — validates and scores submissions        |
+| **Reference Data**  | ERA5 NetCDF files + Natural Earth shapefile (~3.1 GB)  |
+| **Input Data**      | The submission zip uploaded by participants            |
+| **Starting Kit**    | `starting_kit.py` — great-circle baseline code         |
+| **Leaderboard**     | Ranked by total energy (MWh), lower = better           |