Conversation
The three Belgian Pro League play-offs (Champions', Europe, Relegation) were absent because their Transfermarkt competition IDs were not in config.yml and the competitions are not listed in Transfermarkt's confederation hierarchy (so tfmkt competitions never returned them). Add EJPL, POBE, BPO4 to config.yml, introduce data/supplemental_competitions.json to hand-register competitions that sit outside the confederation index, and update acquire_competitions() to merge supplemental records into data/competitions.json after the regular scrape. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…nality - Add EJPL, POBE, BPO4 to competition_codes in dbt_project.yml so base_appearances filters them in (they were scraped but silently dropped) - Fix base_national_teams.sql: use parent.country_name fallback when name is empty - Relax expected_number_of_first_tier_leagues test to accept 31-32 (COL1 is seasonally absent in June-Aug during the Finalización off-season) - Add 2026 World Cup edition to tournament_editions.json Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Restores max_value from 650,000 → 850,000; the prod dataset has grown past 650K and this bound was previously 850K before #367 reverted it. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Plans are ephemeral; DuckDB files are build artifacts. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
data/supplemental_competitions.json: hand-crafted JSONL for competitions outside Transfermarkt's confederation hierarchy (Belgian play-offs can't be discovered by the regulartfmkt competitionsscrape)scripts/acquiring/transfermarkt-scraper.py:acquire_competitions()now mergessupplemental_competitions.jsonintodata/competitions.jsonafter the regular scrapeconfig.yml: addedEJPL,POBE,BPO4to the competition allowlistdbt/dbt_project.yml: addedEJPL,POBE,BPO4tocompetition_codessobase_appearancesno longer filters them outdbt/models/base/transfermarkt_scraper/base_national_teams.sql: fall back toparent.country_namewhennameis empty (fixes 123/124 national teams with blank names)dbt/tests/expected_number_of_first_tier_leagues.sql: relax bound toBETWEEN 31 AND 32— COL1 (Colombia Finalización) is seasonally absent Jun–Augdata/tournament_editions.json: added 2026 World Cup (season 2025) entrydocs/developer-guide.md+CLAUDE.md: documented thesupplemental_competitions.jsonpattern andcompetition_typeURL-routing rulesCloses
Closes #372
Test plan
96 PASS, 1 WARN, 0 ERROR)EJPL/POBE/BPO4appear indev.competitionswithsub_type = play_offtournament_editions.jsonentry is in place, re-runacquire_local --asset tournament_gamesafter the group stage concludes🤖 Generated with Claude Code