Skip to content

Commit ca63586

Browse files
jairus-mjairus-m
authored
Merge pull request #19 from jairus-m/config/dbt-docs
Restructure dbt YMLs + add 'compute_kind' metadata
2 parents 0496515 + 0599b36 commit ca63586

13 files changed

Lines changed: 327 additions & 31 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ While the short-term goal is to learn these tools, the greater goal is to unders
5050

5151
## TODO:
5252
- Beef up the ML pipeline with `dagster-mlflow` for experiment tracking, model versioning, better model observability, etc
53-
- Add new Strava end points
53+
- Utilize Snowflake Cloning/dbt Slim CI for CI
5454
- Implement partitions/backfilling with dlt/Dagster
5555

5656
# Getting Started:

‎analytics_dbt/models/reporting/_dim_activities.yml‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
version: 2
2+
3+
models:
4+
- name: dim_activities
5+
description: Model derived from src_activities with additional time-based columns
6+
columns:
7+
- name: id
8+
description: Primary key for the activity
9+
data_tests:
10+
- unique
11+
- not_null
12+
13+
- name: name
14+
description: '{{ doc("name") }}'
15+
16+
- name: has_heartrate
17+
description: '{{ doc("has_heartrate") }}'
18+
19+
- name: private
20+
description: '{{ doc("private") }}'
21+
22+
- name: visibility
23+
description: '{{ doc("visibility") }}'
24+
25+
- name: all_sport_types
26+
description: values=['Run', 'VirtualRide', 'Ride', 'MountainBikeRide', 'Swimming', 'Rock Climb', etc]
27+
28+
- name: sport_type
29+
data_tests:
30+
- accepted_values:
31+
values: ['Cycling', 'Running', 'Other']
32+
33+
- name: week_start
34+
description: Start date of the week for this activity
35+
36+
- name: day_of_week_num
37+
description: Numeric representation of the day of the week (1-7)
38+
39+
- name: day_of_week
40+
description: Name of the day of the week
41+
42+
- name: week_num
43+
description: Week number within the year
44+
45+
- name: month_num
46+
description: Numeric representation of the month (1-12)
47+
48+
- name: month
49+
description: Name of the month

‎analytics_dbt/models/reporting/_fct_activities.yml‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
version: 2
2+
3+
models:
4+
- name: fct_activities
5+
description: Clean Strava activities
6+
columns:
7+
- name: id
8+
description: Unique ID for each Strava activity
9+
data_tests:
10+
- unique
11+
- not_null
12+
13+
- name: distance_miles
14+
description: '{{ doc("distance_miles") }}'
15+
data_tests:
16+
- not_null
17+
18+
- name: moving_time_minutes
19+
description: '{{ doc("moving_time_minutes") }}'
20+
data_tests:
21+
- not_null
22+
23+
- name: elapsed_time_minutes
24+
description: '{{ doc("elapsed_time_minutes") }}'
25+
26+
- name: total_elevation_gain_feet
27+
description: '{{ doc("total_elevation_gain_feet") }}'
28+
29+
- name: achievement_count
30+
description: '{{ doc("achievement_count") }}'
31+
32+
- name: kudos_count
33+
description: '{{ doc("kudos_count") }}'
34+
35+
- name: comment_count
36+
description: '{{ doc("comment_count") }}'
37+
38+
- name: athlete_count
39+
description: '{{ doc("athlete_count") }}'
40+
41+
- name: average_speed_mph
42+
description: '{{ doc("average_speed_mph") }}'
43+
44+
- name: max_speed_mph
45+
description: '{{ doc("max_speed_mph") }}'
46+
47+
- name: pr_count
48+
description: '{{ doc("pr_count") }}'
49+
50+
- name: average_cadence
51+
description: '{{ doc("average_cadence") }}'
52+
53+
- name: average_temp
54+
description: '{{ doc("average_temp") }}'
55+
56+
- name: average_watts
57+
description: '{{ doc("average_watts") }}'
58+
59+
- name: max_watts
60+
description: '{{ doc("max_watts") }}'
61+
62+
- name: weighted_average_watts
63+
description: '{{ doc("weighted_average_watts") }}'
64+
65+
- name: kilojoules
66+
description: '{{ doc("kilojoules") }}'
67+
68+
- name: average_heartrate
69+
description: '{{ doc("average_heartrate") }}'
70+
71+
- name: max_heartrate
72+
description: '{{ doc("max_heartrate") }}'
73+
74+
- name: elev_high_feet
75+
description: '{{ doc("elev_high_feet") }}'
76+
77+
- name: elev_low_feet
78+
description: '{{ doc("elev_low_feet") }}'

‎analytics_dbt/models/reporting/_reporting_schema.yml‎

Lines changed: 0 additions & 25 deletions
This file was deleted.

‎analytics_dbt/models/reporting/_sport_type_weekly_totals.yml‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
version: 2
2+
3+
models:
4+
- name: sport_type_weekly_totals
5+
description: Aggregated metrics by sport type per week
6+
columns:
7+
- name: week_start
8+
description: The start date of the week for which the activities are summarized
9+
10+
- name: sport_type
11+
description: The type of sport or activity (e.g. Running, Cycling, Other)
12+
13+
- name: num_activities
14+
description: The total number of activities for the given week and sport type
15+
16+
- name: distance_miles
17+
description: The total distance covered in miles for all activities in the week
18+
19+
- name: moving_time_minutes
20+
description: The total time spent moving (active time) in minutes for all activities in the week
21+
22+
- name: elapsed_time_minutes
23+
description: The total elapsed time in minutes for all activities in the week, including stops
24+
25+
- name: total_elevation_gain_feet
26+
description: The total elevation gain in feet for all activities in the week
27+
28+
- name: achievement_count
29+
description: The total number of achievements earned across all activities in the week
30+
31+
- name: kudos_count
32+
description: The total number of kudos received across all activities in the week
33+
34+
- name: comment_count
35+
description: The total number of comments received across all activities in the week
36+
37+
- name: pr_count
38+
description: The total number of personal records (PRs) set across all activities in the week

analytics_dbt/models/source/_src_docs.md renamed to analytics_dbt/models/source/__src_docs.md

File renamed without changes.

analytics_dbt/models/source/_src_schema.yml renamed to analytics_dbt/models/source/_src_activities.yml

File renamed without changes.

‎analytics_dbt/models/source/_src_stats.yml‎

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
version: 2
2+
3+
models:
4+
- name: src_stats
5+
description: This model contains various athletic activity statistics, including totals for rides, runs, and swims, with unit conversions applied. It also includes recent 4-week totals.
6+
columns:
7+
- name: all_ride_totals__count
8+
description: Total count of all rides
9+
10+
- name: all_ride_totals__distance_miles
11+
description: Total distance of all rides in miles
12+
13+
- name: all_ride_totals__elapsed_time_days
14+
description: Total elapsed time of all rides in days
15+
16+
- name: all_ride_totals__elevation_gain_miles
17+
description: Total elevation gain of all rides in miles
18+
19+
- name: all_ride_totals__moving_time_days
20+
description: Total moving time of all rides in days
21+
22+
- name: all_run_totals__count
23+
description: Total count of all runs
24+
25+
- name: all_run_totals__distance_miles
26+
description: Total distance of all runs in miles
27+
28+
- name: all_run_totals__elapsed_time_days
29+
description: Total elapsed time of all runs in days
30+
31+
- name: all_run_totals__elevation_gain_miles
32+
description: Total elevation gain of all runs in miles
33+
34+
- name: all_run_totals__moving_time_days
35+
description: Total moving time of all runs in days
36+
37+
- name: all_swim_totals__count
38+
description: Total count of all swims
39+
40+
- name: all_swim_totals__distance_miles
41+
description: Total distance of all swims in miles
42+
43+
- name: all_swim_totals__elapsed_time_hours
44+
description: Total elapsed time of all swims in hours
45+
46+
- name: all_swim_totals__elevation_gain_feet
47+
description: Total elevation gain of all swims in feet
48+
49+
- name: all_swim_totals__moving_time_hours
50+
description: Total moving time of all swims in hours
51+
52+
- name: biggest_climb_elevation_gain_feet
53+
description: Elevation gain of the biggest climb in feet
54+
55+
- name: biggest_ride_distance_miles
56+
description: Distance of the biggest ride in miles
57+
58+
- name: recent_ride_totals__achievement_count
59+
description: Achievement count for rides in the last 4 weeks
60+
61+
- name: recent_ride_totals__count
62+
description: Count of rides in the last 4 weeks
63+
64+
- name: recent_ride_totals__distance_miles
65+
description: Total distance of rides in the last 4 weeks in miles
66+
67+
- name: recent_ride_totals__elapsed_time_hours
68+
description: Total elapsed time of rides in the last 4 weeks in hours
69+
70+
- name: recent_ride_totals__elevation_gain_feet
71+
description: Total elevation gain of rides in the last 4 weeks in feet
72+
73+
- name: recent_ride_totals__moving_time_hours
74+
description: Total moving time of rides in the last 4 weeks in hours
75+
76+
- name: recent_run_totals__achievement_count
77+
description: Achievement count for runs in the last 4 weeks
78+
79+
- name: recent_run_totals__count
80+
description: Count of runs in the last 4 weeks
81+
82+
- name: recent_run_totals__distance_miles
83+
description: Total distance of runs in the last 4 weeks in miles
84+
85+
- name: recent_run_totals__elapsed_time_hours
86+
description: Total elapsed time of runs in the last 4 weeks in hours
87+
88+
- name: recent_run_totals__elevation_gain_feet
89+
description: Total elevation gain of runs in the last 4 weeks in feet
90+
91+
- name: recent_run_totals__moving_time_hours
92+
description: Total moving time of runs in the last 4 weeks in hours
93+
94+
- name: recent_swim_totals__achievement_count
95+
description: Achievement count for swims in the last 4 weeks
96+
97+
- name: recent_swim_totals__count
98+
description: Count of swims in the last 4 weeks
99+
100+
- name: recent_swim_totals__distance_miles
101+
description: Total distance of swims in the last 4 weeks in miles
102+
103+
- name: recent_swim_totals__elapsed_time_minutes
104+
description: Total elapsed time of swims in the last 4 weeks in minutes
105+
106+
- name: recent_swim_totals__elevation_gain_feet
107+
description: Total elevation gain of swims in the last 4 weeks in feet
108+
109+
- name: recent_swim_totals__moving_time_minutes
110+
description: Total moving time of swims in the last 4 weeks in minutes
111+
112+
- name: ytd_ride_totals__count
113+
description: Count of year-to-date rides
114+
115+
- name: ytd_ride_totals__distance_miles
116+
description: Total distance of year-to-date rides in miles
117+
118+
- name: ytd_ride_totals__elapsed_time_days
119+
description: Total elapsed time of year-to-date rides in days
120+
121+
- name: ytd_ride_totals__elevation_gain_feet
122+
description: Total elevation gain of year-to-date rides in feet
123+
124+
- name: ytd_ride_totals__moving_time_days
125+
description: Total moving time of year-to-date rides in days
126+
127+
- name: ytd_run_totals__count
128+
description: Count of year-to-date runs
129+
130+
- name: ytd_run_totals__distance_miles
131+
description: Total distance of year-to-date runs in miles
132+
133+
- name: ytd_run_totals__elapsed_time_days
134+
description: Total elapsed time of year-to-date runs in days
135+
136+
- name: ytd_run_totals__elevation_gain_feet
137+
description: Total elevation gain of year-to-date runs in feet
138+
139+
- name: ytd_run_totals__moving_time_days
140+
description: Total moving time of year-to-date runs in days
141+
142+
- name: ytd_swim_totals__count
143+
description: Count of year-to-date swims
144+
145+
- name: ytd_swim_totals__distance_miles
146+
description: Total distance of year-to-date swims in miles
147+
148+
- name: ytd_swim_totals__elapsed_time_minutes
149+
description: Total elapsed time of year-to-date swims in minutes
150+
151+
- name: ytd_swim_totals__elevation_gain_feet
152+
description: Total elevation gain of year-to-date swims in feet
153+
154+
- name: ytd_swim_totals__moving_time_minutes
155+
description: Total moving time of year-to-date swims in minutes

analytics_dbt/models/stage/_stg_schema.yml renamed to analytics_dbt/models/stage/__stg_schema.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,12 @@ sources:
55
schema: strava_data
66
tables:
77
- name: activities
8-
description: Source data containing all activities and fields
8+
description: Raw source data containing all activities and fields
99
columns:
1010
- name: id
1111
description: Unique ID for each Strava activity
1212
tests:
1313
- unique
1414
- not_null
1515
- name: stats
16-
description: Athlete stats data for YTD and monthly totals
16+
description: Raw athlete stats data for YTD and monthly totals

‎analytics_dbt/target/manifest.json‎

Lines changed: 1 addition & 1 deletion
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)