Skip to content

Merge changes from upstream - #54

Closed
daniel-rdt wants to merge 8 commits into
masterfrom
merge-upstream-20-08
Closed

Merge changes from upstream#54
daniel-rdt wants to merge 8 commits into
masterfrom
merge-upstream-20-08

Conversation

@daniel-rdt

@daniel-rdt daniel-rdt commented Aug 20, 2025

Copy link
Copy Markdown
Member

Changes proposed in this Pull Request

This PR brings in upstream changes. No conflicting changes.

Checklist

  • I tested my contribution locally and it works as intended.
  • Code and workflow changes are sufficiently documented.
  • Changed dependencies are added to envs/environment.yaml.
  • Changes in configuration options are added in config/config.default.yaml.
  • Changes in configuration options are documented in doc/configtables/*.csv.
  • OET license identifier is added to all edited or newly created code files.
  • Sources of newly added data are documented in doc/data_sources.rst.
  • A release note doc/release_notes.rst is added.

blakeldridge and others added 8 commits August 11, 2025 08:47
@blakeldridge
Fix issue PyPSA#1792 : updated colours in plot_balance_map (PyPSA#1793)
91eb24f 
Co-authored-by: Blake Eldridge <blake.eldridge@fnc.co.uk>
@TomKae00 @pre-commit-ci
refactor: update TES standing loss rates to DEA data (PyPSA#1791)
23c0b84 
* refactor: use standing_losses from cost file instead of tes_tau

* fix: delete widow code

* refactor: add release_notes

* refactor: mention new costs file version in release notes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refactor: remove tes_tau from sector.csv config

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@lkstrp
fix: update v1 flag for deprecations (PyPSA#1797)
5f7ce2b
@dependabot
chore(deps): bump the github-actions group across 1 directory with 2 …
2dee4bd 
…updates (PyPSA#1798)

Bumps the github-actions group with 2 updates in the / directory: [actions/checkout](https://github.com/actions/checkout) and [actions/download-artifact](https://github.com/actions/download-artifact).


Updates `actions/checkout` from 4 to 5
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v4...v5)

Updates `actions/download-artifact` from 4 to 5
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](actions/download-artifact@v4...v5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: actions/download-artifact
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@aleks-g
bugfix typo id to ID (PyPSA#1800)
c7b728d
@lkstrp
fix: update v1 flag for rc (PyPSA#1802)
4e11346
@euronion
doc: Remove retrieve_natural_earth reference (PyPSA#1805)
1750b4e
@daniel-rdt
Merge remote-tracking branch 'upstream/master' into merge-upstream-20-08
20c206f
@daniel-rdt
daniel-rdt requested a review from coroa August 20, 2025 08:55
@daniel-rdt daniel-rdt self-assigned this Aug 20, 2025
@daniel-rdt

daniel-rdt commented Aug 20, 2025 •
edited
Loading

Copy link
Copy Markdown
Member Author

Hi @coroa, here is the upstream merge (as of August 20th). There are no conflicting changes with our OET fork. If you approve the PR, I can quickly merge it :)

@tgilon

tgilon commented Sep 1, 2025

Copy link
Copy Markdown
Member

@daniel-rdt I synced upstream this morning, before reading this PR. I never create a PR for OET/pyspa-eur, as no specific project relies on it. Projects always have their own fork where they need to track changes (and refuse some of them if needed). Does this mean that this PR is no longer relevant?

@tgilon

tgilon commented Sep 18, 2025

Copy link
Copy Markdown
Member

@daniel-rdt sounds like this got lost

@daniel-rdt

daniel-rdt commented Sep 18, 2025

Copy link
Copy Markdown
Member Author

@daniel-rdt sounds like this got lost

sorry, I missed this. Yes indeed, as upstream was already synced, I will close this PR

@daniel-rdt daniel-rdt closed this Sep 18, 2025
coroa added a commit that referenced this pull request Nov 14, 2025
@tgilon @daniel-rdt
@pre-commit-ci @coroa
feat: introduce offshore wind hubs topology (#54)
d34afb8 
* feat: skip tyndp_renewable_profile technologies in workflow and build_renewable_profiles

* feat: clean pecd data for years 2030, 2040, 2050 and build_renewable_profiles_pecd

* feat: add pecd profiles as input to prepare_sector_network and add_brownfield rules

* fix: add hotfix for add_existing_baseyear existing renewable capacities and fix name of profiles in add_brownfield rule

* fix: fix pylint

* fix: fix pylint better

* feat: introduce clean_tyndp_offshore_hubs

* doc: document newly added functions

* feat: improve configuration based on master

* feat: define outputs

* feat: enable offshore hubs by default for tyndp configurations

* fix: add electricity param for add_existing_baseyear rule in perfect foresight workflow

* feat: improve workflow implementation for more efficient execution of clean_pecd_data with handling of not available pecd years

* fix: hotfix for add_brownfield p_max_pu until tyndp generators are added. Needs to be reverted later

* doc: add description of new configuration option to configtables

* doc: add release note

* feat: add retrieval rule for PECD data from google drive until data on Zenodo

* fix: fix ruleorder statement

* fix: remove unneeded inputs to build_renewable_profiles_pecd and update download link for pecd data

* fix: set file outputs for retrieve_tyndp_bundle

* feat: assume non extendable links for missing data

* fix: integrate new config for rule

* refactor: rename clean_tyndp_offshore_hubs to build_tyndp_offshore_hubs to match conventions

* doc: update license identifier

* feat: update pecd retrieval to use gcp storage

* feat: WIP introduce offshore reference grid in the network (assuming absolute investment cost)

* feat: switch to relative costs (EUR/kW and EUR/kW/a) and define a specific function

* feat: map AC nodes to H2 Z2 nodes for H2 offshore grid

* doc: add basic release note

* refactor: rename function to be tyndp specific

* revert: revert change in default config

* feat: build electrolysers data

* feat: attach offshore electrolysers

* refactor: switch functions

* feat: build generators data

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* feat: overwrite default lifetime assumptions with tyndp specific values

* feat: generalize more make_index

* fix: adjust offshore radial locations

* doc: update units info

* feat: add offshore generators to the network (without potentials and pecd data)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* doc: detail GENERATOR approach

* WIP feat: extend GENERATOR processing to take discrepancies into account

* WIP feat: refactor everything

* feat: extract both existing and potentials from layer and trajectories from zone

* doc: improve documentation of load_offshore_generators

* feat: resolve discrepancy in DEOH002

* feat: filter for selected countries

* fix: take efficiency into account for h2-generating wind farms

* feat: attach PECD profiles to offwind generators

* refactor: use tyndp specific renewable carriers and differentiate between tyndp renewable carriers and pecd renewable profiles

* bugfix: include missing offshore node PECD data as empty columns and replace UK with GB naming convention

* feat: improve myopic approach, removing previous assets

* bugfix: fix pylint and remove hydro from test config

* bugfix: add existing res dependent on renewable carriers list

* feature: add 2050 pecd renewable profiles explicitly and filter for snapshots when cleaning data

* feat: introduce a constraint to limit expansion of collocated technologies that use the same potential

* bugfix: fix params for add_existing_baseyear solve_perfect

* feat: add zone potential constraint

* bugfix: change year in renewable profile pecd back to integer

* fix: address patches used to develop parallel features

* fix: avoid adding an empty constraint

* doc: adjust license identifier

* fix: address pecd profiles mapping in add_brownfield

* refactor: refactor myopic approach for offshore assets

* refactor: adjust nomenclature of offshore grid capacity

* refactor: filter radial nodes out of buses

* refactor: assign radially connected electrolysers to home market nodes

* fix: use tyndp data for electrolysers capital cost

* refactor: improve radial nodes exclusions

* fix: prevent filtering of grid data from expansion candidates

* refactor: remove location from electrolysers

* refactor: change approach for nodes and locations in generators

* feat: read pecd data using location

* refactor: use bus instead of bus0 for generators

* fix: correct typo in h2 offshore grid

* fix: add country to rename for electrolysers

* feat: improve constraint formulation to account for refactoring

* bugfix: fix year dimension for 2035 an 2045 steps

* fix: adjust layer constraint to account for existing capacities

* refactor: rename variables to align with the constraints standards

* feat: introduce bin in index for compatibility with add_brownfield

* feat: add colors for new carriers

* fix: remove OH nodes from the list used for distribution

* Apply suggestions from code review

Co-authored-by: Daniel Rüdt <117752024+daniel-rdt@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Apply suggestions from code review

Co-authored-by: Daniel Rüdt <117752024+daniel-rdt@users.noreply.github.com>

* feat: update the set of colors for the new technologies

* refactor: change config name for offshore hubs (apply suggestion)

* doc: improve release note

* refactor: use unpack to define offshore inputs in prepare_sector_network

* refactor: make filter more explicit to exclude offshore grid in add_brownfield

* fix: account for the shared potential of hydrogen- and electricity-generating wind farms

* fix: remove double counting of existing capacities

* fix: account for efficiency in existing for zonal constraint

* feat: resolve DEOH002 capacity mismatch in existing capacities

* ci: revert temporary CO2 budget increase - tests now pass with original values

* doc: refine documentation of discrepancy in DEOH002

* doc: remove outdated documention from load_offshore_hubs

* refactor: replace generic RuntimeError with ValueError for invalid input

* refactor: assign nyears before use

* fix: set offshore_zone_trajectories across all foresight scripts

* doc: change position of comment in add_brownfield

* feat: remove reference to config/config.yaml, as it is not used

* fix: set offshore_zone_trajectories across all foresight scripts

* refactor: add empty input for offshore files in prepare_sector_network

* fix: set offshore_zone_trajectories across all foresight scripts

* fix: add empty input for offshore files in prepare_sector_network

* refactor: refactor existing plotting routines to align conventions

* feat: introduce offshore plotting routines

* refactor: match new naming convention for plots

* refactor: remove the default reference to base network for offshore plotting routine

* refactor: split offshore plotting routine using a carrier wildcard

* fix: attach h2 piplines to the correct buses

* fix: remove previous build year from efficiency calculations

* fix: group links in plot_offshore_network

* feat: add a offshore transmission capacity limits

* ci: fix run name keyword

* Apply suggestions from code review

Co-authored-by: Thomas Gilon <thomas.gilon@openenergytransition.org>
Co-authored-by: Jonas Hörsch <coroa@posteo.de>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: import chain from itertools

* refactor: generalize get tyndp renewable carriers

* ci: fix run name keyword

* Revert "ci: fix run name keyword"

This reverts commit 5c2de44.

* fix: fix merge conflict in add_brownfield regarding tyndp_renewable_carriers

* feat: make legend optional in plot_offshore_network

* feat: modularize link width plotting functionality

* feat: add hubs_only parameter to filter offshore hub in visualization

* fix: remove placeholder for renewable technology max capacities in build_renewable_profiles_pecd

* doc: add information about PECD Version 3.1 to documentation

* fix: remove unnecessary params from pecd rules

* feat: change fn_pecd to dir_pecd and add comment on skiprows

* refactor: change to if not continue style to reduce diff

* feat: improve pecd profile configuration by making profile names explicit and improving documentation

* refactor: simplify tyndp_renewable_carriers also in smk files

* feat: improve mapping from tyndp techs to PECD profile names

* Apply suggestions from code review

Co-authored-by: Daniel Rüdt <117752024+daniel-rdt@users.noreply.github.com>

* fix: fix merge conflicts

* refactor: remove logger as argument

* feat: make the retrieve outputs more explicit

* fix: fix outputs in retrieve_tyndp_bundle

* refactor: use explicit list of tyndp carriers in constraint

* refactor: apply code suggestion

* refactor: improve code

* doc: adjust licensing

* doc: make README more explicit about make tyndp

* refactor: define a specific tyndp all function

* refactor: prefer branch over lambda formulation

* refactor: apply code suggestions

* fix: fix file name in all rule

* refactor: remove unused function arguments

* refactor: update carrier naming to explicitly separate offshore hubs and DRES

* fix: fix: restrict max_capacity condition to inter-hubs connections and correct unit

* fix: fix dimension issue in add_brownfield

* refactor: convert to explicit hydrogen buses sooner

* feat: add plotting threshold for plot_offshore_network

* fix: use mainland GB coordinates instead of Northern Ireland

* fix: remove Offshore Hubs from hydrogen plots

* feat: introduce time-varying efficiencies for offwind generators (#68)

* feat: introduce time-varying efficiencies for offwind generators

* feat: use fixed 2020 efficiency values to prevent solver infeasibility

* refactor: rename efficiency to efficiency_dc_to_b0 for offshore generators

* refactor: move if outside of extend in Snakefile

* refactor: remove redundant formulation in add_brownfield

* refactor: move input_offshore_hubs in file

* refactor: test before using input_offshore_hubs

* refactor: improve minor code inconsistencies

---------

Co-authored-by: daniel-rdt <daniel.ruedt@openenergytransition.org>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Rüdt <117752024+daniel-rdt@users.noreply.github.com>
Co-authored-by: Jonas Hörsch <coroa@posteo.de>
brynpickering added a commit that referenced this pull request Dec 4, 2025
@brynpickering
Merge pull request #54 from open-energy-transition/feat/fes-workbook-…
f621a1f 
…download-and-extract

Add FES workbook extractor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@coroa coroa Awaiting requested review from coroa

Assignees

@daniel-rdt daniel-rdt

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@daniel-rdt @tgilon @blakeldridge @TomKae00 @lkstrp @aleks-g @euronion