Oetc support (#50)

* Add oetc configuration support

Co-authored-by: Fabian Hofmann <fab.hof@gmx.de>

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Fabian Hofmann <fab.hof@gmx.de>

Author: 3 people

doc/configtables/solving.csv

@@ -41,5 +41,12 @@ solver,,,
 -- name,--,"One of {'gurobi', 'cplex', 'highs', 'cbc', 'glpk'}; potentially more possible",Solver to use for optimisation problems in the workflow; e.g. clustering and linear optimal power flow.
 -- options,--,Key listed under ``solver_options``.,Link to specific parameter settings.
 solver_options,,dict,Dictionaries with solver-specific parameter settings.
+oetc,,,Configuration options for Open Energy Transition Computing (OETC) cluster support.
+-- name,--,str,Name identifier for the OETC job.
+-- authentication_server_url,--,str,URL of the OETC authentication server for job submission.
+-- orchestrator_server_url,--,str,URL of the OETC orchestrator server for job management.
+-- cpu_cores,--,int,Number of CPU cores to request for the OETC job. (includes RAM amount at the moment with a factor of 8)
+-- disk_space_gb,GB,int,Amount of disk space in gigabytes to request for the OETC job.
+-- delete_worker_on_error,bool,"{'true','false'}",Whether to delete the worker instance when an error occurs during job execution.
 mem,MB,int,Estimated maximum memory requirement for solving networks.
 mem_logging_frequency,s,int,Interval in seconds at which memory usage is logged.

doc/oetc.rst

@@ -0,0 +1,92 @@
+..
+  SPDX-FileCopyrightText: Contributors to PyPSA-Eur <https://github.com/pypsa/pypsa-eur>
+
+  SPDX-License-Identifier: CC-BY-4.0
+
+##########################################
+OETC Integration
+##########################################
+
+The OETC platform allows PyPSA-Eur to leverage cloud-based resources for
+solving energy system models. This integration enables solving of optimization problems
+using remote compute resources when local computational capacity is insufficient.
+
+.. _oetc_overview:
+
+Overview
+========
+
+OETC provides a cloud-based optimization service that can handle PyPSA networks by:
+
+- Offloading optimization tasks to remote compute resources
+- Supporting various solvers (Highs, Gurobi, etc.) on cloud infrastructure
+- Automatically managing authentication and resource allocation
+- Providing configurable compute resources (CPU cores, disk space)
+
+The integration is implemented through the ``linopy.oetc`` module and is configured
+within the PyPSA-Eur solving configuration.
+
+.. _oetc_setup:
+
+Setup and Authentication
+========================
+
+To use OETC with PyPSA-Eur, you need to:
+
+1. **Set up environment variables** for authentication based on your OETC credentials:
+
+   .. code:: bash
+
+       export OETC_EMAIL="your-email@example.com"
+       export OETC_PASSWORD="your-password"
+
+2. **Configure OETC settings** in your configuration file (see :ref:`oetc_config`).
+
+.. _oetc_config:
+
+Configuration
+=============
+
+OETC integration is configured within the ``solving`` section of your configuration file.
+Add an ``oetc`` subsection to enable cloud-based optimization:
+
+.. code:: yaml
+
+    solving:
+      # ... other solving options ...
+      oetc:
+        name: "job-name" # Arbitrary human readable job identifier
+        authentication_server_url: "https://auth.oetc.example.com"
+        orchestrator_server_url: "https://orchestrator.oetc.example.com"
+        compute_provider: "GCP"  # Currently only GCP is supported
+        cpu_cores: 4 # This also sets the amount of RAM by a factor 8
+        disk_space_gb: 20
+
+.. note::
+
+   The ``solver`` and ``solver_options`` are automatically passed from the main
+   solving configuration and do not need to be specified in the OETC section.
+
+.. _oetc_usage:
+
+Usage
+=====
+
+Once configured, OETC integration works transparently with the standard PyPSA-Eur
+workflow. If a oetc configuration is present, PyPSA-Eur will automatically use
+the OETC platform for the model optimization during the solve network rule.
+
+An example of a log entry when using OETC:
+.. code::
+    INFO:linopy.oetc:OETC - Signing in...
+    INFO:linopy.oetc:OETC - Signed in
+    INFO:linopy.oetc:OETC - Fetching user GCP credentials...
+    INFO:linopy.oetc:OETC - Fetched user GCP credentials
+    INFO:linopy.oetc:OETC - Submitting compute job...
+    INFO:linopy.oetc:OETC - Compute job 992616b6-0f24-4ab5-8675-ba8098d795fa started
+    INFO:linopy.oetc:OETC - Waiting for job 992616b6-0f24-4ab5-8675-ba8098d795fa to complete...
+    INFO:linopy.oetc:OETC - Job 992616b6-0f24-4ab5-8675-ba8098d795fa status: PENDING, checking again in 30 seconds...
+    INFO:linopy.oetc:OETC - Job 992616b6-0f24-4ab5-8675-ba8098d795fa status: PENDING, checking again in 45 seconds...
+    INFO:linopy.oetc:OETC - Job 992616b6-0f24-4ab5-8675-ba8098d795fa completed successfully!
+    INFO:linopy.oetc:OETC - Model solved successfully. Status: ok
+    INFO:linopy.oetc:OETC - Objective value: 3.11e+07

doc/release_notes.rst

@@ -43,9 +43,13 @@ Upcoming Release
 
 * Updated `build_osm_network` and `clean_osm_data` to handle voltage levels below 220 kV (down to 63 kV). When `base_network` is set to `osm_raw`, an electricity grid from voltage levels AC 63 kV to 750 kV can be created (experimental feature). For an example, see the example configuration in `config/examples/config-distribution-grid-experimental.yaml`.
 
+
+* Added integration with the OETC platform
+
 * Fix `retrieve_eurostat_data` and `retrieve_eurostat_household_data` on Windows by avoiding a double access to a temporary file.
   (https://github.com/PyPSA/pypsa-eur/pull/1825)
 
+
 PyPSA-Eur v2025.07.0 (11th July 2025)
 =====================================
 

envs/linux-64.lock.yaml

@@ -305,7 +305,7 @@ dependencies:
   - libxslt=1.1.43=h7a3aeb2_0
   - libzip=1.11.2=h6991a6a_0
   - libzlib=1.3.1=hb9d3cd8_2
-  - linopy=0.5.5=pyhd8ed1ab_0
+  - linopy=0.5.7=pyhd8ed1ab_0
   - locket=1.0.0=pyhd8ed1ab_0
   - lxml=6.0.1=py312h70dad80_0
   - lz4=4.4.4=py312hf0f0c11_0

envs/osx-64.lock.yaml

@@ -270,7 +270,7 @@ dependencies:
   - libxslt=1.1.43=h59ddae0_0
   - libzip=1.11.2=h31df5bb_0
   - libzlib=1.3.1=hd23fc13_2
-  - linopy=0.5.5=pyhd8ed1ab_0
+  - linopy=0.5.7=pyhd8ed1ab_0
   - llvm-openmp=21.1.0=hf4e0ed4_0
   - locket=1.0.0=pyhd8ed1ab_0
   - lxml=6.0.1=py312ha108ba6_0

envs/osx-arm64.lock.yaml

@@ -270,7 +270,7 @@ dependencies:
   - libxslt=1.1.43=h429d6fd_0
   - libzip=1.11.2=h1336266_0
   - libzlib=1.3.1=h8359307_2
-  - linopy=0.5.5=pyhd8ed1ab_0
+  - linopy=0.5.7=pyhd8ed1ab_0
   - llvm-openmp=21.1.0=hbb9b287_0
   - locket=1.0.0=pyhd8ed1ab_0
   - lxml=6.0.1=py312h4144cae_0

envs/win-64.lock.yaml

@@ -250,7 +250,7 @@ dependencies:
   - libxslt=1.1.43=h25c3957_0
   - libzip=1.11.2=h3135430_0
   - libzlib=1.3.1=h2466b09_2
-  - linopy=0.5.5=pyhd8ed1ab_0
+  - linopy=0.5.7=pyhd8ed1ab_0
   - llvm-openmp=21.1.0=hfa2b4ca_0
   - locket=1.0.0=pyhd8ed1ab_0
   - lxml=6.0.1=py312hc85b015_0

scripts/solve_network.py

@@ -40,6 +40,7 @@
 import pypsa
 import xarray as xr
 import yaml
+from linopy.remote.oetc import OetcCredentials, OetcHandler, OetcSettings
 from pypsa.descriptors import get_activity_mask
 from pypsa.descriptors import get_switchable_as_dense as get_as_dense
 
@@ -1335,6 +1336,17 @@ def solve_network(
     kwargs["assign_all_duals"] = cf_solving.get("assign_all_duals", False)
     kwargs["io_api"] = cf_solving.get("io_api", None)
 
+    oetc = solving.get("oetc", None)
+    if oetc:
+        oetc["credentials"] = OetcCredentials(
+            email=os.environ["OETC_EMAIL"], password=os.environ["OETC_PASSWORD"]
+        )
+        oetc["solver"] = kwargs["solver_name"]
+        oetc["solver_options"] = kwargs["solver_options"]
+        oetc_settings = OetcSettings(**oetc)
+        oetc_handler = OetcHandler(oetc_settings)
+        kwargs["remote"] = oetc_handler
+
     kwargs["model_kwargs"] = cf_solving.get("model_kwargs", {})
     kwargs["keep_files"] = cf_solving.get("keep_files", False)